DPDK

What is DPDK

DPDK is the Data Plane Development Kit that consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures. Designed to run on x86, POWER and ARM processors. Polling-mode drivers skips packet processing from the operating system kernel to processes running in user space. This offloading achieves higher computing efficiency and higher packet throughput than is possible using the interrupt-driven processing provided in the kernel.

Why DPDK for voipmonitor

Sniffing packets by kernel linux is driven by IRQ interrupts - every packet (or if driver supports every set of packets) needs to be handled by interrupt which has limitation around 3Gbit on 10Gbit cards (it depends on CPU). DPDK allows to read pacekts directly in userspace not using interrupts which allows faster packet reading (so called poll-mode reading). It needs some tweaks to the operating system (cpu affinity / NOHZ kernel) as the reader thread is sensitive to any scheduler delays which can occur on overloaded or misconfigured system. For 6Gbit packet rate with 3 000 000 packets / second any slight delays can cause packet drops.

installation

Version >= DPDK 21.08.0 is requried - download the latest version from:

https://core.dpdk.org/download/

How it works

On supported NIC cards (https://core.dpdk.org/supported/) the ethernet port needs to be unbinded from kernel and binded to DPDK, the command for it is:

• no special driver is needed - debian 10/11 already has support for this out of the box
• bind/unbind means that when you undind NIC port from the kernel you cannot use it within the operating system - the port dissapears (you will not see eth1 for example)
• you can unbind from dpdk and bind back to kernel so eth1 can be used again

list of available network devices:

dpdk-devbind.py -s

Network devices using kernel driver
===================================
0000:0b:00.0 'NetXtreme II BCM5709 Gigabit Ethernet 1639' if=enp11s0f0 drv=bnx2 unused= *Active*
0000:0b:00.1 'NetXtreme II BCM5709 Gigabit Ethernet 1639' if=enp11s0f1 drv=bnx2 unused=
0000:1f:00.0 'Ethernet Controller 10-Gigabit X540-AT2 1528' if=ens3f0 drv=ixgbe unused=
0000:1f:00.1 'Ethernet Controller 10-Gigabit X540-AT2 1528' if=ens3f1 drv=ixgbe unused=

bind both 10gbit ports to vfio-pci driver (this driver is available by default on >= debian10)

modprobe vfio-pci
dpdk-devbind.py -b vfio-pci 0000:1f:00.0 0000:1f:00.1

bind B port back to kernel:

dpdk-devbind.py -b ixgbe 0000:1f:00.1

On some systems vfio-pci does not work for 10Gbit card - instead igb_uio (for Intel cards) needs to be loaded alongside with special kernel parameters:

/etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="iommu=pt intel_iommu=on"

Loading igb_uio for IBM internal X540-AT2 4 port 10Gbit card:

modprobe igb_uio

More information about drivers:

https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html

dpdk is now ready to be used by voipmonitor