Sniffer configuration: Difference between revisions
Line 108: | Line 108: | ||
= destination_number_mode = | = destination_number_mode = | ||
take number from INVITE URI or To: SIP header. If destination_number_mode = 1 It will always save number from To: header. if destination_number_mode = | take number from INVITE URI or To: SIP header. If destination_number_mode = 1 It will always save number from To: header. if destination_number_mode = 2 it will take number from INVITE URI. | ||
default: | default: |
Revision as of 17:46, 10 December 2013
Configuration file has only one section named [general] where all configuration directives belongs. List of directives will now follow with their description and recommendation values. Name in [ ] brackets is equivalent for command line which takes precendence over configuration file. in case of running more voipmonitor instances on the same or another servers configured to save to one database and the same cdr table it is possible to differentiate CDR by id_sensor column. If you set id_sensor >= 0 the number will be saved in cdr.id_sensor column.
id_sensor
This specifies on which interface will voipmonitor listen. It can listen on one interface or on all interfaces. To listen on all interfaces use interface = any The number is between 1 - 65535 (16bit number)
id_sensor = 1
interface
listening interface. Can be 'any' which will listen on all interfaces - NOTE that "any" will not put intefaces into promiscuous mode and you have to do it with "ifconfig eth0 promisc". You can also sniff on multiple interfaces by providing list of them delimited by ',' example: interface = eth0,eth1 command line option is -i
#interface = eth0,eth1 #interface = any interface = eth0
threading_mod
- default threading_mod = 1 uses one thread for reading from interface doing deduplication at once* threading_mod = 2 (which is automatically set if you have multiple interfaces (interface = eth0,eth1,...)) reads from each interface in separate thread which is better option on multi core systems than interface = any * threading_mod = 3 will do deduplication (if enabled) in separate thread which is needed for high traffic* threading_mod = 4 will do deduplication in more than one threads - use this option if you enable deduplication and your traffic is over 100Mbit
threading_mod = 1
mirror
since version 8 sniffer implements new mirroring option. Sender is packing data to compressed stream over the TCP to remote sniffer. if you are going to use this sniffer only as a mirroring sniffer all you need is to set interface, packetbuffer_* set compression on and set packetbuffer_file_* so in case the connection to remote sniffer will die or will be temporarily slow the sender will not loose single packet. The mirroring is trying to reconnect in case of failure. Packets are mirrored including the the original timestamp and headers. this mirroring hopefully replaces pcapscandir feature which will be probably removed in favor of this approach.
The sender needs to set only interface, ringbuffer, packetbuffer_*, filter and this two folling mirror_destination_*
mirror_destination_ip = mirror_destination_port =
On receiver set mirror_bind_ip and mirror_bind_port. Do not forget to set firewall so no other except the sender will be able to connect to the receiver
mirror_bind_ip = mirror_bind_port =
scanpcapdir
scan pcap files folder and read file by file. This is in conjunction with running tcpdump which creates pcap file each 5 seconds (-G 5) storing pcap files named by UNIX_TIMESTAMP to /dev/shm/voipmonitor folder (do not forget create it) using 1GB ring buffer to avoid losing packets (-B500000 - you can lower it but not higher) filtering udp packets (udp parameter whcih you can change to your needs). voipmonitor then reads created files (and delete it after processing. This approach can be used for testing throughput or for very high voip traffic (>500Mbit). If the sniffer is able to process pcap files in realtime - there will be in /dev/shm/voipmonitor folder only one or two pcap files. If the sniffer is not able to process in realtime (blocking by I/O or by CPU) number of pcap files will grow faster then the sniffer is able process.
scanpcapdir = /dev/shm/voipmonitor
WARNING: libpcap < 1.1 contains memory leak when pcap filter is set - do not set filter in this config or upgrade libpcap to the latest (debian 6 libpcap contains the leak) static compiled voipmonitor from voipmonitor.org contains the latest libpcap
tcpdump example command: nice -n -20 tcpdump -B500000 -i eth2 udp -G 5 -w /dev/shm/voipmonitor 2>/dev/null 1>/dev/null &
ringbuffer
ringbuffer is circular memory queue directly in kernel memory space. libpcap is reading from this queue and delivers packets to voipmonitor. If the network rate is > 100 Mbit we recommend to set ringbuffer to at least 500. Maximum value is 2000 MB. ringbuffer = 50 [ --ring-buffer ]
packetbuffer
packet buffer is new voipmonitor buffering architecture (since version 8). If enabled new threads are created which raads packets from kernel ringbuffer and queues them into dynamically allocated memory. Packets are dequeued and passed to next threads which reads the content. This will ensure that kernel ringbuffer will not overrun due to CPU or disk I/O spikes. packet buffer will dynamically grow until packetbuffer_total_maxheap is reached. Compression can be enabled (packetbuffer_compress) which compress the buffer with fast snappy algorythm with 50% compression ratio thus doubling the time when the buffer gets filled (600Mbit traffic consumes ~30% one one core Xeon E5-2620).
It is also possible to use disk buffer if the packet buffer memory is filled by enabling packetbuffer_file_totalmaxsize which is usefull when sniffer is only mirroring data over TCP to another sniffer - if the connection brakes or slowed down and packet buffer gets filled it will start using file buffer until the connection reestablishes so no single packet is lost. Enabling file buffer in non mirroring mode to the same disk as spooldir will get not much benefit because if the process is blocked mainly due to disk I/O it has no benefit to add more I/O by caching unprocessed packets to the same I/O layer.
packetbuffer_enable = yes packetbuffer_total_maxheap = 2000 #in MB packetbuffer_compress = no #enable compression if you have >2 CPU cores packetbuffer_file_totalmaxsize = 0 #MB. Default is disabled. packetbuffer_file_path = /var/spool/voipmonitor/packetbuffer
rtpthreads
number of threads to process RTP packets. If not specified it will be number of available CPUs - 1. If equal to zero RTP threading is turned off. Each thread allocates default 20MB for buffers (increase to 100 on very high loads). This buffer can be controlled with rtpthread-buffer. For < 150 concurrent calls you can turn it off.
rtpthreads = 3
sdp_reverse_ipport
since 8.4RC17 if voip SIP device (with source IP 1.1.1.1) sends in SDP that it wants RTP for example to 10.0.0.1 and port 10000 the call also sniff RTP from 1.1.1.1:10000 or to 1.1.1.1:10000 which does more problems than it solves. Now this behaviour is changed and if you need this option back you can enable it
default no
sdp_reverse_ipport = yes
natalias
in case the SIP(media) server is behind public IP (1.1.1.1) NATed to private IP (10.0.0.3) to sniff all traffic correctly you can specify alias for this case. You can specify more netaliases duplicating rows. In most cases this is not necessary because voipmonitor is able to track both RTP streams based on the other side IP. But if the stream is incoming from another IP then SIP source signalization and also from another IP than the SIP device which is also behind NAT its impossible to track the correct IP. Please note that this is for case where the SIP server is behind NAT and also the client is behind NAT. If your SIP server has public IP do not bother with this.
natalias = 1.1.1.1 10.0.0.3 natalias = 1.1.1.2 10.0.0.4
managerip
define bind address for manager interface. Default is 127.0.0.1 it is not recommended to change this unless really needed due to security. If you need it on some other IP make sure you set firewall and change the standard port for better security
managerip = 127.0.0.1
managerport
This specifies TCP port which will voipmonitor listen for incoming connections which controls voipmonitor or for getting information about calls. *reload configuration echo reload | nc localhost 5029 *get number of calls echo totalcalls | nc localhost 5029 *get list of calls in json format echo listcalls | nc localhost 5029
managerport = 5029 [ --manager-port <port number> ]
managerclient
connects to server and listen for commands.
managerclient = serverip or hostname managerclientport = 1234
destination_number_mode
take number from INVITE URI or To: SIP header. If destination_number_mode = 1 It will always save number from To: header. if destination_number_mode = 2 it will take number from INVITE URI.
default:
destination_number_mode = 1
sipport
define SIP ports wihch will voipmonitor listen. For each port make new line with sipport = port (multiple lines)
sipport = 5060 sipport = 5061 sipport = 5062
rtptimeout
rtptimeout is important value which specifies how much seconds from the last SIP packet or RTP packet is call closed and writen to database. It means that if you need to monitor ONLY SIP you have to set this to at leat 2 hours = 7200 assuming your calls is not longer than 2 hours. Take in mind that seting this to very large value will cause to keep call in memory in case the call lost BYE and can consume all memory and slows down the sniffer - so do not set it to very high numbers. Default is 300 seconds.
rtptimeout = 300
jitterbuffer
By default voipmonitor uses three types of jitterbuffer simulators to compute MOS score. First variant is saved into cdr.[ab]_f1 and represents MOS score for devices which has only fixed 50ms jitterbuffer. Second variant is same as first but for fixed 200ms and is saved to cdr.[ab]_f2 Third varinat is adaptive jitterbuffer simulator up to 500ms Jitterbuffer simulator is the most CPU intensive task which is voipmonitor doing. If you are hitting CPU 100% turn off some of the jitterbuffer simulator. Recommended for higher loads is to use only fixed 200ms (f1)
jitterbuffer_f1 = yes jitterbuffer_f2 = yes jitterbuffer_adapt = yes
callslimit
callslimit will limit maximum numbers of calls processed by voipmonitor at the same time. If calls are over limit it will be ignored (INVITE)
callslimit = 0
cdrproxy
in case SIP session travels accross several proxies (and Call-ID header DOES not change) and you would like to track all sip proxies and make them searchable in GUI / database. If disabled cdr will store to destination sip column destination IP from the first INVITE. If enabled there will be destination IP from the latest invite and all proxy ip will be stored in cdr_proxy table.
default = yes
cdrproxy = yes
cdr_ua_enable
this option allows to skip storing cdr.a_ua and cdr.b_ua - this is workaround for those who has extreme cdr rate and number of user agents in database is over 1000 and CPU is not powerfull enough to store cdr in real time. In future this option will be removed once we optimize this rutine. default = yes
cdr_ua_enable = yes
rtp-firstleg
this is important option if voipmonitor is sniffing on SIP proxy like kamailio or openser and sees both RTP leg of CALL. In that case use this option. It will analyze RTP only for the first LEG and not each 4 RTP streams which will confuse voipmonitor. Drawback of this switch is that voipmonitor will analyze SDP only for SIP packets which have the same IP and port of the first INVITE source IP and port. It means it will not work in case where phone sends INVITE from a.b.c.d:1024 and SIP proxy replies to a.b.c.d:5060.
rtp-firstleg = no [ --rtp-firstleg ]
allow-zerossrc
- since 8.0.2
- default = no
SSRC in RTP headers must not equal zero according to RFC so voipmonitor is ignoring such RTP by default. If you still need to parse such packets enable it
allow-zerossrc = yes
deduplicate
duplicate check do md5 sum for each packet and if md5 is same as previous packet it will discard it. WARNING: md5 is expensive function (slows voipmonitor 3 times) so use it only if you have enough CPU or for pcap conversion only. Default is no.
deduplicate = yes
deduplicate_ipheader
- since 8.1
prior verison 8.0.1 deduplicate was comparing only data without ip header and udp header so duplicate packets was matched also in case the IP addresses differes. This was good for some cases but it leads to completely ignore RTP streams in other cases. Now default option is to check duplicates based on not only data but ip headers too. To change this set deduplicate_ipheader = no. default = yes.
deduplicate_ipheader = yes
sipoverlap
enable/disable updating called number from To: header from each caller INVITE. Default is enabled so it supports overlap dialing (RFC 3578)if you want to disable this behaviour and see always number only from the first INVITE set sipoverlap = no
sipoverlap = yes
sip-register
Enable parsing of SIP REGISTER message. SQL register table stores active SIP registrations. Once it expires it is removed from the table to new sql register_state table. The register state table is used to store changes in registrations. SQL table register_failed is used to store all failed sip register. To not overload this table there is counter column which adds +1 for each failed register from the same source.
sip-register = no [ -R ]
sip-register-active-nologbin
if mysql binlog is enabled, skip binlog inserts into active table (which is MEMORY type) if you still want to replicate this too (huge I/O impact) set it to = no sip-register-active-nologbin = yes
skipdefault
if yes, all SIP calls will be ignored unless capture rules set skip flag based on IP or Tel. numbers (mysql.filter_*)
skipdefault = yes
nocdr
if yes, voipmonitor will not save CDR to MySQL
nocdr = no [ -c ]
maxpcapsize
limit pcap file size (in MB). default disabled. since 8.4
maxpcapsize = 500
savesip
Store SIP packets to pcap file.
savesip = [ --sip-register ]
savertp
save RTP packets to pcap file. savertp = yes automatically saves RTCP packets you can also save only RTP header without AUDIO: savertp = header if save RTP is aneblad it will also save UDPTL packets (used for T.38) you can also set savertp = no and control what calls will record RTP in mysql table filter_ip or filter_tel which is controled in GUI -> Capture rules. Sending reload command will reload configuration from filter_* table. You can also set savertp = yes but denies recording RTP based on rules in filter_* table.
savertp = yes | header [ -R ]
pcapsplit
voipmonitor by default splits SIP and RTP packets to individual files (in case spooldiroldschema = no) which are located in SIP and RTP directories. This feature allows instance cleaning RTP streams differently then SIP packets to join two pcap files SIP+RTP use mergecap command line utility which is included in wireshark package default = yes | spooldiroldschema must be set to no
pcapsplit = yes
savertcp
Store RTCP packets to pcap file.
savertcp = yes [ --save-rtcp ]
saveudptl
save UDPTL packets (T.38). If savertp = yes the udptl packets are saved automatically. If savertp = no and you want to save only udptl packets enable saveudptl = yes and savertp = no
saveudptl = yes
savegraph
This is usefull only if you have commercial WEB GUI which uses graph files for ploting graph
savegraph = plain [ -G or --save-graph=[gzip|plain] ]
saveaudio
save RTP payload to audio file. Choose 'wav' for WAV PCM or 'ogg' for OGG 25kbps format. please note that this has great impact on I/O and can overload your storage leading to lose packets. Better way is to store only sip+rtp and convert wav files on demand.
saveaudio = wav
convert_dlt_sll2en10
- since 8.0.2
in case you need to have ethernet encapsulation and you are sniffing on interface = any set this to = yes. this is needed only in case you need to merge pcap files with different encapsulations. default is no.
convert_dlt_sll2en10 = no
keycheck
default path to WEB GUI used to construct path to key check for codecs
default paths: #keycheck = /var/www/voipmonitor/php/keycheck.php #keycheck = /var/www/html/voipmonitor/php/keycheck.php
saverfc2833
in case you are not saving RTP at all but you still want to save DTMF carried over RTP packets (RFC2833) you can enable this option. This feature slows down a bit processing RTP packets in main read thread in casse voipmonitor runs in threads. default = 0
saverfc2833 = 0
dtmf2db
Enable storing DTMF (SIP INFO or RFC2833) to cdr_dtmf database. It will store DTMF time and key then it will be shown in SIP history in the GUI
dtmf2db = 0
norecord-header
if any of SIP message during the call contains header X-VoipMonitor-norecord call will be not converted to wav and pcap file will be deleted.
norecord-header = yes
= norecord-dtmf =if any of SIP message during the call contains DTMF INFO sequence "*0" call will be not converted to wav and pcap file will be deleted. default: disabled
norecord-dtmf = yes
pauserecordingdtmf
enable pausing RTP/WAV recording if DTMF sequence detected. default: disabled
pauserecordingdtmf = *9
dumpallpackets
dump all packets to /tmp/voipmonitor-[UNIX_TIMESTAMP].pcap
dumpallpackets = yes
mos_g729
enable MOS score for G.729 codec. If enabled, all cdr with 0 packet loss and stable delays will have maximum MOS of 3.92 and for loss and unstable delay MOS will be calculated according to ITU-T objective PESQ method for G.729 codec. if you want to use MOS as good search value which corellates loss and delay into single value leave it disabled (which is by default). If set to no, all calls will be calculated like it is G.711. Recommended value = no
mos_g729 = no
custom_headers
Since 7.0RC7.
enable storing custom sip headers to database column cdr_next.custom_header_headername. You can specify more headers delimited by ";". WARNING - when you enable this feature voipmonitor will autoupgrade cdr_next table which can take hours depending on how large the table is. In GUI there is new section Settings#CDR_Custom_header.
custom_headers = X-asterisk-Info ; X-myheader
analogical for SIP message is custom_headers_message
custom_headers_message = X-asterisk-Info ; X-myheader
match_header
enable saving content of custom header (typicaly in-reply-to) to cdr_next.match_header this header is used in related CDR GUI for matching legs to onen call
match_header = in-reply-to
pcapcommand
pcapcommand will run command after pcap file is closed (after call ends). %pcap% is substitution for real pcap file name. execution is guaranteed to run in serialized way (not in parallel)WARNING - pcapcommand is implemented by forking program which is very expensive and is causing TLB shootouts on multicore system which can generate 500 000 interrupts / sec causing system to drop packets. Watch the performance carefuly (with "vmstat 1" column "in"). Gziping pcap files will be implemented as native function directly in C++ to obey TLB shootdowns.
pcapcommand = gzip %pcap%
filtercommand
- since 8.3
filtercommand will run command after each call which matches script == 1 in filter_ip or filter_telnum (capture rules in GUI) WARNING - filtercommand is implemented by forking program which is very expensive and is causing TLB shootouts on multicore system which can generate 500 000 interrupts per seconds causing system to drop packets. Watch the performance carefuly (with "vmstat 1" column "in").
all non alphanum characters except '/' '#' ' ' '+' ':' '-' '.' and '@' in callid, dirname, caller, called and calldate are substituted to '_'
default is disabled
filtercommand = myscript '%basename%' '%dirname%' '%caller%' '%called%' '%calldate%'
filter
libpcap tcpdump style filter. Voipmonitor listens in default only for UDP packets. Unfortunatly filtering UDP packets will filter all VLAN tagged packets which means that you cannot filter only UDP if you want to listen to VLAN tagged packets.
WARNING - if you need to sniff IPinIP (like mirrored packets from voipmonitor) filter = udp will filter all those packets. In this case just disable filter.
filter = udp or (vlan and udp) [ -f ]
convertchar
list characters that should be converted to underscore (_) in filenames if you want to include space, put it between other characters, like ': :' (will convert ':' and ' ' to '_') defaults to nore
# example - avoid ':' when Call-Id contains port number convertchar = :
spooldir
This is directory where all pcap/graph/wav files are stored.
spooldir = /var/spool/voipmonitor [ -d ]
spooldiroldschema
new spooldir schema stores all files to year-mon-day/hour/minute/[ALL|SIP|RTP|AUDIO] directories if you need to have the old schema year-mon-day/* enable spooldiroldschema = yes. default = no
spooldiroldschema = no
cachedir
store pcap and graph file to <cache/dir> and move it after call ends to spool directory. Moving all files are guaranteed to be serialized which solves slow random write I/O on magnetic or other media. Typical cache directory is /dev/shm/voipmonitor which is in RAM and grows automatically or /mnt/ssd/voipmonitor which is mounted to SSD disk or some very fast SAS/SATA disk where spool can be network storage or raid5 etc. wav files are not implemented yet
cachedir = /dev/shm/voipmonitor
cleandatabase_cdr
Removes cdr* partitions older then set number of days. If set to 0 it is disabled (default)
cleandatabase_cdr = 0
cleandatabase_register_state
Removes register_state partitions older then set number of days. If set to 0 it is disabled (default)
cleandatabase_register_failed
Removes cleandatabase_register_failed partitions older then set number of days. If set to 0 it is disabled (default)
cleandatabase
Sets cleandatabase_cdr and cleandatabase_register_state and cleandatabase_register_failed to the same values. Configuration first look at cleandatabase parameter then it looks for other cleandatabase_* parameters.
default is disabled
cleandatabase = 0
cleanspool_interval
clean spool directory every two hours from oldest file until spooldir size = cleanspool_size (in MB) WARNING - if you have milions files set cleanspool_interval to 24 hour (clean once per day) cleaning is very slow and can overload I/O (in seconds). Default is disabled.
cleanspool_interval = 7200
cleanspool_size
set target size of the spool directory for cleaning script (in MB). Default is disabled.
cleanspool_size = 50000 #usefull command to clean all RTP files older 7 days # find /var/spool/voipmonitor -maxdepth 1 -type d -mtime +7 -name '20*' | (while read d; do rm -rf $d/*/*/RTP; done)
promisc
This option is only relevant if you are mirroring traffic to your network card/cards. This will not work if interface = any - in this case, use ifconfig to put your desired interfaces to promis mode. Default value is yes and you want to turn it of on command line ues -n which will turn it off.
promisc = yes [ -n ]
database
sqldriver
sqldriver = mysql #sqldriver = odbc #odbcdriver = mssql #odbsdsn = voipmonitor #odbcuser = root #odbcpass =
voipmonitor can connect to mysql server or odbc driver. connecting voipmonitor to msssql please refer to README.mssql
cdr_partition
use partitioning cdr* tables by day. If you have schema without partitioning, you MUST start with new database. default is = yes
cdr_partition = yes
disable_partition_operations (since 8.4RC9)
disable partition creation which runs every 12 hours. If you have multiple sensors storing to one database it is redundant to create partitions by all sensors. default = no
disable_partition_operations = yes
mysqlhost
mysql server, default is 127.0.0.1 (localhost). Command line option is -h
mysqlhost = 127.0.0.1
mysqldb
mysql database, default is voipmonitor. Command line option is -b
mysqldb = voipmonitor
mysqlusername
mysql username, default is root
mysqlusername = root
mysqlpassword
mysql password, default is no password
mysqlpassword =
upgrade_try_http_if_https_fail
if https failes when upgrading the sniffer try also http
upgrade_try_http_if_https_fail = yes