Smart NICs

Jyothi
7 min readJul 28, 2024

--

Why Smart NICs

  • For achieving seamless video conferencing and streaming services which rely on massive data transfer.
  • For high-speed internet access, data processing demands and IoT development
  • SmartNICs offer acceleration, improved network performance, security, and storage.

SmartNICs are specialized NICs with programmable features for customization. SmartNIC is a Programmable Accelerator, similar to GPUs, it offloads tasks to improve data center efficiency. It offload tasks from servers, enhancing performance and reducing latency.

Internal architecture of a Smart NIC, utilizing an FPGA (Field-Programmable Gate Array) chip

This diagram represents the internal architecture of a Smart NIC, utilizing an FPGA (Field-Programmable Gate Array) chip to enhance its capabilities. Here’s a detailed explanation of each component in the context of Smart NIC architecture:

  • Network Interface (e.g., QSFP28): This is the physical interface through which the Smart NIC connects to the network. It handles the reception and transmission of data packets.
  • Ethernet Controller (PHY/MAC/PCS): The Ethernet controller handles the physical layer (PHY), media access control (MAC), and physical coding sublayer (PCS). It processes incoming and outgoing Ethernet frames, ensuring proper encoding/decoding and framing.
  • Packet Interface: Acts as an intermediary between the Ethernet controller and the internal data processing components of the Smart NIC. It routes incoming packets to the FIFO buffer and outgoing packets to the Ethernet controller.
  • FIFO Buffer: This buffer temporarily holds incoming packets. It ensures a smooth flow of data by handling bursts of incoming packets.
  • Header Parser: Extracts and interprets the headers of incoming packets to determine their type, source, destination, and other relevant information. This information is crucial for subsequent processing stages.
  • Flow Engine: Manages and processes the flow of packets based on the parsed header information. It can apply policies, perform load balancing, and ensure that packets are routed correctly within the Smart NIC.
  • Rules Engine: Applies predefined rules to incoming packets. These rules can include security checks, filtering, and other policy-based actions to control the handling of network traffic.
  • Customer Logic: This block represents customizable logic defined by the user or vendor. It can include specialized processing tasks such as encryption, compression, deep packet inspection, or other application-specific functions.
  • Direct Memory Access (DMA): Facilitates high-speed data transfer between the Smart NIC and the server’s memory without burdening the CPU. It allows efficient movement of packet data to and from the Smart NIC.
  • PCIe Express Generation 5×16: It provides a high-speed interface between the Smart NIC and the server, allowing for rapid data transfer and communication with the host system.
  • GDDR6 Memory: High-speed memory used for temporary storage of packet data and processing results. It supports the Smart NIC’s high-performance data processing requirements.
  • Server PCIe Gen5×16: Represents the connection to the server, facilitating communication between the Smart NIC and the host system.

This illustrates another variant of SmartNIC architecture:
Traffic Control: consists of traffic manager, Tx/RX ports, and DMA engine. The Traffic Manager/NIC Switch handles incoming and outgoing network packets. It’s responsible for routing traffic efficiently. TX/RX Ports represent transmit and receive channels for data. The DMA Engine (Direct Memory Access) facilitates efficient data transfer between the NIC and system memory.

  • Computing Units: The NIC Core performs packet processing tasks, such as checksum validation, encryption, and decryption. L1 Cache provides fast access to frequently used data. Packet Processing (Domain-Specific) is where specialized tasks related to network protocols or applications occur.
  • Multi-Queue Host Communication enables parallel communication with the host system. Another DMA Engine assists in data movement within the NIC.
  • Packet Buffer Memory: Scratchpad and L2/DRAM collectively form the packet buffer memory. Scratchpad is a small, fast-access memory area. L2/DRAM provides larger storage for packets.
  • PCIe Interface: The arrow connecting the Computing Units to the PCIe interface indicates data flow or connectivity.

Capabilities of SmartNICs

Hardware offload of network, security and storage functions allows for better utilisation of host servers to run application workloads.

Network Functions: SmartNIC’s Programmable Accelerator handles multiple networking tasks like load balancing, telemetry, and routing. It Improves efficiency by offloading tasks from the host server’s CPU. Network Overlays and Tunneling enable packet routing between overlay networks for containerized or virtualized systems, perform tunneling to facilitate network communication.

Network Security Functions: SmartNICs dynamically drop packets to protect servers, offloading DDoS detection to SmartNICs safeguards the host server CPU. SmartNICs manage both inbound and outbound packet filtering. It reduces the load on host server CPU by offloading traffic filtering. SmartNICs operates independently of the host system which helps to provide robust, secure, and reliable network services without being impacted by the host system’s state. Multi-Tenant Isolation enable isolation in shared physical data centers.

Encryption and Storage Functions: Smart NICs can support Hardware-based encryption procedures. Smart NIC can be a secure tunnel endpoint. Smart NICs provide storage offloading features such as remote storage protocols and vendor-specific solutions.

Computing Functions of SmartNICs: CPU intensive tasks can be offloaded to Smart NICs. Eg. Transcoding video using adaptive-bitrate compression, blockchain hashing processes. Smart NICs can be programmed by offloading network applications functionality such as handling in-memory databases and accelerating DNS query processing

It helps in boosting server efficiency, frees up server CPU for other tasks, and helps in improving overall hardware performance.

SmartNICs in Data Center Networking

This diagram illustrates the differences between a conventional NIC and a Smart NIC in terms of packet processing and communication with virtual machines (VMs):

Conventional NIC: The kernel layer handles interrupts and context switching. When a packet is sent to a conventional NIC, it involves interrupt handling and context switching within the kernel layer. The packet moves from the NIC to the VM through the kernel layer, involving additional processing steps and potential latency.

Smart NIC: In a Smart NIC, packets can be sent directly to the VM without kernel-level interruption handling. The Smart NIC can perform data processing tasks independently, functioning like a server inside a server. This eliminates the need for reverse context switching and handling interrupts in the kernel layer, leading to more efficient and faster communication.

SmartNICs in data center operations

DPUs, NPUs, and IPUs are often grouped as xPUs. They share functional principles but differ in capabilities. xPUs are considered as evolved SmartNICs. They enhance networking, storage, and security. SmartNICs can run the network software processes and freeup server processing. Additional SmartNIC applications include packet capture, intrusion detection, network management, telemetry, data decompression and deduplication.

OpenStack and OVN on SmartNIC DPUs

DPU is high-performance network controller with hardware acceleration and Multi-core CPU for OS and control plane applications. The DPU network controller is presented to the hypervisor OS as a set of Physical Functions (PFs) and Virtual Functions (VFs). PFs and VFs serve for control plane communication and provide hardware-accelerated virtual network controllers. The DPU NIC has an embedded switch (NIC switch) that can be programmed by the general-purpose CPU. It is useful in managing traffic within the DPU. The DPU’s CPU programs PFs and VFs exposed to the hypervisor host. Port representors are used to connect virtual switches offloaded to the NIC. However, in the off-path SmartNIC DPU scenario, representors are programmed on a different host from the hypervisor.

Architecture Change with Off-Path SmartNIC DPUs:

Off-path SmartNIC DPUs introduce a significant change. Network agents responsible for NIC switch configuration and representor interface plugging now run on a separate System-on-Chip (SoC). This SoC has its own CPU, memory, and runs a separate OS kernel. Control plane mechanisms are needed to orchestrate accelerated port allocation between the DPU and the hypervisor.

References

https://ubuntu.com/blog/ubuntu-on-smartnics-drive-data-centre-efficiency

https://ubuntu.com/blog/what-is-a-smartnic-and-how-is-the-technology-shaping-modern-data-centres

https://ubuntu.com/blog/data-centre-networking-smartnics

https://www.techtarget.com/searchnetworking/tip/An-introduction-to-smart-NICs-and-their-benefits

https://www.5gtechnologyworld.com/what-is-a-smartnic-and-its-underlying-architecture/

https://codilime.com/glossary/smartnic/

https://community.fs.com/blog/smartnic-vs-nic-how-to-choose.html

Introducing Jyo Services — Your Go-To Partner for Tech Solutions

Are you looking to boost your career or bring your tech ideas to life? Jyo Services offers a range of specialized solutions tailored just for you:

  • One-on-One Interview Preparation: Tailored coaching for freshers and professionals with up to 7 years of software development experience.
  • Custom Software Development: Expertise in embedded technologies, AI/ML, Gen AI, Cyber Security, Linux-based applications, and network protocol developments.
  • Patent Assistance: Comprehensive support for drafting, filing and maintaining patents in India.
  • College Projects: Professional guidance for Engineering students on academic projects.

Unlock your potential with Jyo Services! Contact us today to get started.

DM me at: www.linkedin.com/in/jyothivs or jyos.v.s@gmail.com

--

--