Voipmonitor and hugepages

Modifications before build

define.h

source file of voipmonitor sniffers needs to be altered first

Modify

#define MAX_PRE_PROCESS_PACKET_NEXT_THREADS 16
#define MAX_PROCESS_RTP_PACKET_HASH_NEXT_THREADS 16

Add

#define EXPERIMENTAL_T2_DETACH_X_MOD true
#define EXPERIMENTAL_T2_DIRECT_RTP_PUSH true
#define EXPERIMENTAL_T2_QUEUE_FULL_STAT true
#define EXPERIMENTAL_SUPPRESS_AUDIOCODES true
#define EXPERIMENTAL_SUPPRESS_KAMAILIO true
#define EXPERIMENTAL_SUPPRESS_CALL_CONFIRMATION_FOR_RTP_PROCESSING true
#define EXPERIMENTAL_PRECREATION_RTP_HASH_INDEX true
#define EXPERIMENTAL_SUPPRESS_AST_CHANNELS true
#define EXPERIMENTAL_LITE_RTP_MOD true

Makefile

after configure you need to modify the Makefile produced (in 2022-03-14)

architecture set to native

Change architecture to native

ifeq ($(GCCARCH),x86_64)
  GCCARCH = native
  MTUNE = -mtune=native
endif

change optimizers flag

Change from -O2 to -O3

CXXFLAGS +=  -Wall -fPIC -g3 -O3 -march=$(GCCARCH) ${MTUNE} ${INCLUDES} ${FBSDDEF} ${MYSQL_WITHOUT_SSL_SUPPORT}

Modifications before service start

create init_hugepages script

Create the /etc/init.d/init_hugepages script. Don't forget to set executable flag (chmod +x init_hugepages) Following script is dedicated only for voipmonitor service that is configured to run only on 1 NUMA (node1)

Use single numa node

makes sense only if voipmonitor is configured to run only on single numa node

#!/bin/bash
echo 1 > /proc/sys/vm/drop_caches
echo 1 > /proc/sys/vm/compact_memory
echo 0 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
echo 60000 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

Without limitation on numa node

#!/bin/bash
echo 1 > /proc/sys/vm/drop_caches
echo 1 > /proc/sys/vm/compact_memory
echo 0 > /proc/sys/vm/nr_hugepages
echo 60000 > /proc/sys/vm/nr_hugepages

• note that the amount of hugepages(60k here) depends on amount of RAM you need to allocate to voipmonitor (max_buffer_mem option), and size of the single hugepage. (Example is for 120GB)

Find the size of hugepage

 cat /proc/meminfo | grep -i hugepagesize

Details on hupages in system

 cat /proc/meminfo | grep -i huge

voipmonitor.service

service file for systemd located in /etc/systemd/system

• note that the bellow init scripts uses numactl binary that can be installed with

apt-get install numactl
yum install numactl

napatech related

you need to start voipmonitor with -k and set Type=simple in init script.

[Unit]
Description=VoIPmonitor sniffer

[Service]
Type=simple
PIDFile=/var/run/voipmonitor.pid
Environment=TCMALLOC_MEMFS_MALLOC_PATH=/dev/hugepages/test
ExecStartPre=/usr/local/sbin/init_hugepages
ExecStart=/usr/bin/numactl --cpunodebind=1 --membind=1 /usr/local/sbin/voipmonitor --config-file=/etc/voipmonitor.conf -k -v1,pcap_stat_period=5

[Install]
WantedBy=multi-user.target

without naptech

[Unit]
Description=VoIPmonitor sniffer

[Service]
Type=forking
PIDFile=/var/run/voipmonitor.pid
Environment=TCMALLOC_MEMFS_MALLOC_PATH=/dev/hugepages/test
ExecStartPre=/usr/local/sbin/init_hugepages
ExecStart=/usr/bin/numactl --cpunodebind=1 --membind=1 /usr/local/sbin/voipmonitor --config-file=/etc/voipmonitor.conf -v1,pcap_stat_period=5

[Install]
WantedBy=multi-user.target

Start service

Then enabling the service(start afterbootup) and starting the service can be done:

systemctl enable voipmonitor.service
systemctl start voipmonitor.service

Coredump file create change

Choose location where is enough space for credumps (coredumps may be huge depends on how much RAM were allocated by the sniffer process)

echo "/media/voipmonitor/sniffer01/core.%e.%p.%h.%t" > /proc/sys/kernel/core_pattern