fix missing RTP packets caused by SIP/SDP Packet Handling at Higher Traffic Rates: Previously, when t2_boost = yes (default setting), VoIPmonitor relied on the block of packets filling up quickly to trigger processing. For traffic rates below 200 Mb/s, an additional delay detection mechanism ensured blocks with a time difference exceeding 100ms between the first and last packet would be forwarded to the next thread. At higher traffic rates (>200 Mb/s), this delay detection was disabled, as it was assumed blocks would fill quickly. However, under certain conditions (e.g., low SIP traffic between threads), SIP/SDP packets could be delayed, causing RTP packets to be processed before the corresponding SIP/SDP packets. [VS-1669]
suppress the use of autocleanmingb if the disk size is smaller than this parameter
suppress log entry 'Critical low disk space in spool...' if _maxpoolsize_autoclean is greater than the used space
fix function SqlDb_mysql::getInsertId for client/server and cloud mode (fixes Configuration values in mysql have a higher weight) [VG-2911]
pb thread optimisation in case voipmonitor is sniffing from multiple interfaces or is in receiver mode.
use time in ms in active calls and registrations if there are cdr/register calldate columns for ms and even if the option time_precision_in_ms=yes is not specified
2024.11.2 02.12.2024
fix mirror receiver crash if mirror sender disconnects
add dpdk parameter dpdk_mtu [VS-1665]
2024.11.1 27.11.2024
fix cleanup calls in t2_boost=no [VS-1664]
fix srtp: completing the srtp key to the correct length, adding the srtp_use_all_keys (NO/yes) parameter to force the application of all known keys from all directions [VS-1661]
limiting the number of attempts when creating a test table to verify compression algorithms in mariadb [VS-1660]
2024.11.0 25.11.2024
increase SIP processing by optimization of sip process split between threads (callid search and call packet processing), reduce locking [VS-1651]
increase RTP processing by thread optimisations:
optimization of packetbuffer output threads, decomposition of detach2 thread into main thread and subthreads [VS-1652]
decomposition of defrag thread into main thread and subthreads [VS-1652]
optimise distribution of t2 threads to subthreads
fix potential crash in adjustSipResponse if set cdr_sip_response_number_max_length [VS-1655]
fix issue with parse spd if media record contains zero port [VS-1648]
automatic change of row_format parameter to dynamic if alter table fails with error ER_TOO_BIG_ROWSIZE (1118) [VG-2827]
rcc fraud alert - 'time period rules' table - apply new gui checkbox to specify whether to convert time zone to gui time [VG-2898]
add support for Cisco FabricPath [VS-1650]
add CDR filtering by TCP/UDP/both protocol [VS-1609]
fix creating sipcallerip_encaps column when creating new db [VS-1653]
add parameter server_cp_store_simple_connect_response = yes/NO for enforce compatibility in client/server mode between the new server and the extremely old client [VG-2908]
application of the cdr_force_primary_index_in_all_tables parameter to the cdr_stat_values and cdr_stat_sources tables [VS-1656]
fix potential crashses when voipmonitor terminates
extension of TcpReassembly::cleanup_simple to clean up infinite streams (applied when new option sip_tcp_reassembly_stream_max_length is set) [VS-1662]
2024.10.2 22.10.2024
better detection of rtp/audiograf using CDR.flags
add multithread support without locks for fftw - can be enabled with audiograph_fftw_multithread_mode = yes
dpdk: fix undefined reference to `rte_vdev_init'
add option to allow write empty header's value if header is missing in first packet and is present in next packets [VG-2175]
fix data availability detection in chunkbuffer if internal compression is enabled (tar_internalcompress_...) [VS-1642]
add parameter dpdk_nb_mbufs_strictly=yes/NO to suppress the increase of the specified number of mbufs by a multiple of 1024
fix potential crash in restart sensor in client mode
allow to use %_ wildcards in conditions for numbers in fraud alerts [VG-1724]
rcc fraud alert - time period rules table - apply new gui checkbox to specify whether to convert time zone to gui time [VG-2898]
2024.10.1 02.10.2024
fix crash when there are multiple c threads spawn (broken since 2024.08
fix crashes in fftw library [VS-1639]
fix compatibility with wireshark >= 4.2.3
dpdk: detection of binding of interfaces and numa nodes, automatic creation of necessary hugepages, automatic assignment of rte threads to cpu cores
add manager command for modify ssl_ipport - lock optimisations [VG-2866]
adding values for displaying mos/jitter/loss items to active calls in gui [VG-2871]
add manager command for modify ssl_ipport list,set,add,del which allows to change ssl_ipport without sniffer restart [VG-2866]
2024.09.4 20.9.2024
set usleep_minimal to 50us when vmware is detected (lowers CPU usage signifficantly)
change default values for dpdk_read_thread and dpdk_worker_thread parameters from 'std' to 'rte'
fix crashes in fftw library (initialization for multithreads use)
2024.09.3 18.9.2024
implement separate audio graphs files to spool/.../AUDIOGRAPH/ (spectrogram + wave) which allows to show it in GUI without needs to store RTP packets. New opeionts save_audiograph, audiograph*, maxpoolaudiographsize, maxpoolaudiographdays [VG-2874]
fix race in transcribe calls
fix potential stack overflow in sRtcpXrStreams::findAB
adding buffer overflow protection for hep data [VG-2882]
2024.09.2 10.9.2024
fix 2024.09.1 crashing when SIP REGISTER stora to pcap is enabled
2024.09.1 04.9.2024
add support for call information / sip history in active calls
fixed-jitterbuffer modification - add skew tracking to evaluate whether to drop or resync in resync_jb function [VS-1627]
use rtpc_xr even when there are no rtp streams [VS-552]
use rtpc sr / rr even if there are no rtp streams [VS-1608]
process the custom headers before send call info processes first invite [VG-2108]
allow specifying a replace pattern as part of a regular expression in the custom header definition [VS-1623]
fix saving time in custom header tables for option/notify/subscribe if saving times in ms is enabled (fix detecting presence of column for ms) [VG-2859]
fix sip dst ip detection if there are reverse in-dialog invites with port change
add nat alias to capture rules [VS-1624]
add support for ./configure --exec_prefix [VS-1628]
use sip reason if missing cause [VS-1630]
optimization of the decision whether to use receiver or sender id in mirror modes [VS-1632]
automatic enable of opt_sip_msg_save_ua if not present in the configuration and if cdr_ua_normalisation is enabled [VS-1631]
2024.08.1 06.8.2024
dpdk: fix support for multiple dpdk interfaces, support for virtual interfaces (example settings: dpdk_vdev=net_pcap_1:iface=eth0;net_pcap_2:iface=eth1; interface=net_pcap_1,net_pcap_2)
rework the activation of the cleandatabase process so that it can be used for database_backup. If cleandatabase is to be used for database_backup, database_backup_cleandatabase = yes [VS-1619]
fix the same number of columns when storing multiple records in the cdr_next_branches table [VS-1621]
support for groups of numbers in the registration filter [VG-1124]
enhance watchdog
store statistics from PUBLISH even if we don't have rtp [VS-552]
ssl keylogger - use exit instead of abort if initialization fails [VS-1622]
2024.07.1 16.7.2024
new query_cache_max_mb_utf parameter to limit the length of utf characters [VS-1602]
new configuration option mysql_security_invoker_routines = YES/no to allow creation of functions and procedures without dependency on the definer
tls: add support for session resumption in keylogger decryption mode [VS-1600]
better monitoring of fraud-seq alert intervals [VG-2835]
remove watchog script file after it has been run [VG-2824]
application of the usleep_minimal parameter also in the usleep function without progressive usleep increase
extend sip flood alert - sip packet type option, condition for destination ip addres [VG-1184]
ensure that in quick_save_cdr=quick mode call cleanup_calls is not triggered more often than once every 100ms
fix table list in SqlDb_mysql::getSourceTables (table cdr_audio_transcribe is not 'child') [VS-1616]
fix crash if dpdk is enabled in configuration but not supported in build
2024.06.2 19.06.2024
fix illegal instruction introduced in 2024.06.1
2024.06.1 18.06.2024
Add support for audio transcription using OpenAI Whisper https://openai.com/index/whisper/ [VS-1603]
Add support to change country assignment for existing CDR [VS-1130]
Add configuration parameter srtp_rtp_dtmf to decrypt only RTP-DTMF [VG-2820]
Add TOP countries charts in chart templates [VG-2808]
Add definition of ignored errors for creating partitions for tables cdr_next_N
tar_move: Enable using the destination path on a remote sensor to get PCAP/graph from GUI. Enable spool cleanup in the destination path (if spooldir_2 is set to the same path as tar_move_destination_path and if maxpoolsize_2 is set)
Fix rtp->avg_ptime calculation [VS-1598]
Fix detection if mysqlstore_max_threads_cdr is set
Reduce the limit for starting the next saving (to MySQL) thread from 1000 to 100
cleanspool: Fix printspool manager command for when there is only one hour of data in the spool
Compilation fix if libgcrypt is missing [VS-1312]
Fix possible crash when using reverse_iterator in CleanSpool::cSpoolData::removeLastDateHours
Add 'refresh' sub-parameter to printspool manager command
New disable_rtp_seq_check parameter which ignores rtp.seq duplication check [VS-1604]
Fix false seq rotation credit if disable_rtp_seq_probation is enabled
Fix writing MOS charts if jitterbuffer configuration options are set to 'null'
Anonymization fix (TCP/UDP protocol detection was inconsistent) [VG-2816]
fix saving values for rrd graph 'sql cache files' in server/client mode on client sid
add custom headers into send call information [VG-2108]
2024.05.2 04.05.2024
fix crash in manager command listcalls (if parameters contain ip_map:yes without set limit)
2024.05.1 02.05.2024
ptime implementation - parsed from SDP and analysed from RTP (store ptime for both caller/called) - database alter is required (GUI -> tools -> check mysql schema) [VG-1091]
new fast deduplication algorithm - murmur
added to active calls: number of calls by rtp ip src / dst, number of calls by caller / called numbers [VG-2129]
delay between INVITE and TRYING was measured from the first INVITE - this is now changed to the corresponding INVITE and not the first INVITE. The old behaviour can be changed by new option response_time_from_first_invite [VG-2798]
add support for cleandatabase by size [VS-1586]
create RTCP payload from HEP3. Fix IP defrag in hep client emulation [VS-1594]
number of required connections between gui and sniffer was optimised
HEP3: accept protocol_id ESP, enable all TCP ports in client emaulation [VS-1597]
ssl keylogger - add define option BOTH_METHODS to allow simultaneous use of SSL_CTX_set_keylog_callback and SSL function hooking [VS-1588]
suppress load query cache files if set query_cache=no [VS-1585]
backup/migration - skip non-existing tables in the source, adding a parameter database_backup_slave_record_safe_gap to allow specifying (or disabling) safe gaps for slave records
fix problem with the availability of the last pcap in the zstd archive before it is closed
fix typo in constraint creating in sip_msg table (case no db partition) [VS-1587]
fix tcp reassemble for some corner cases
fix crash in hep3/ipfix client emulation
2024.04.1 02.04.2024
for each sniffing interface do not use ethtool but kernel api instead. Number of interrupt queues are lowered to 8 automatically (or less depending on cpu cores). New parameter eth_max_channels controls it.
adding the ability to join reference crd tables when backing up (or migrating) the database (database_backup_merge_reference_tables = yes/NO)
fix keyloger compilation (add dl lib) [VS-1574]
using kernel api instead of ethtool and automatically set channels to 8 if set more
checking every minute whether cleaning should occur due to autoclean parameters [VS-1576]
fix problem with manager encryption if opt_nocdr=yes is enabled
added support for column ordering in active calls [VG-2782]
sensor_id in cdr is assigned by receiver sensor id and not sender (fixes pcap downloads) [VG-2784]
added new cdr flag when RTP duplication is detected (so it can be searched in DB / GUI) [VS-1582]
normalize text in sip responses, s850 reason and user agents (removes unique strings) - helps to maintain database smaller. This can be controlled by new options cdr_sip_response_normalisation, cdr_reason_normalisation, cdr_ua_normalisation - if you want to enable it - set them to yes
2024.03.1 10.03.2024
allow registrations in live sniffer even if sip-register=no, disable query_cache for live sniffer [VG-2771]
fix deadlock on some qemu virtualisations
2024.02.6 29.02.2024
fix slow communication on TCP manager between sniffers (1 second delay per 1 MB)
fix ssl keylogger compilation
do not call dns_lookup_common_hostnames when reading file with -r
2024.02.5 26.02.2024
reset errno before call recv on tcp manager socket - fixes stuck in manager or 100% cpu usage
2024.02.4 26.02.2024
fix issue with nocdr vs mirror sender
set the manager command 'totalcalls' as a command for which encryption is not necessary
2024.02.3 23.02.2024
fix upgrading / enabling encryption on tcp manager socket - in some situations the socket hangs with 100% CPU
encryption is disabled when old mirroring sender is used - please use the newer client/server for packet mirroring
auto enable t2_boost_direct_rtp_delay_ms=2000 if t2_boost=high_traffic is enabled
2024.02.2 20.02.2024
Adding encryption support for communication between php and sniffer manager which is now by default (port 5029). New GUI is required for activate it. [VG-2751]
Update systemd service file.
fix crash in read rtp without sip (with use --rtp-nosig) [VS-1560]
fix ssl keylog compilation, fix for wolfssl
fix crash in decompress tar to file (which is used when upgrading to the develop version)
-flto is now disabled by default in configure
fix SSL/TLS detection for Mysql 8.x [VS-1562]
limit log ether_type 0xFFFF only if this condition occurs more than 100 times per minute [VS-1563]
create separate sql queue for sip_msg (options/subscribe/notify)
set 3 threads by default to store cdr, messages, sip_msg and registrations; the next threads only start when 1000 requests are not completed
modify rrd to show the sip_msg queue graph (creating an alter to add a data source)
change sip-subscribe=no and sip-notify=no as default option (it needs to be enabled in voipmonitor.conf to store them)
read configuration from db only if --config-file is specified
fix crash in decompress tar to file
2024.02.1 02.02.2024
fix killing old watchdog scripts after running sniffer after crash/abort
2024.01.6 31.01.2024
fix no traffic passing through when max_buffer_mem is set to low values [VS-1552]
2024.01.5 30.01.2024
since 31.9 deduplication was enabled by default with fast CRC32 but there exist collisions which skips some SIP messages leading to ignore some calls. Deduplication is now disabled by default again and md5 hash is used instead of crc if deduplication is enabled. Faster hashes will be added in the future.
fix memory leak for all > 31.9 versions when short / non RTP packet arrives to the IP/port combination of some call [VS-1551]
fix ipfix and hep client emulation if pcap contains ipv6 [VS-1550]
2024.01.4 29.01.2024
TCP manager security fix (if you can not control access to the voipmonitor manager TCP port 5029 - upgrade is strongly advised)
change default value for option detect_alone_bye to no. This option can cause excessive load to the database - more details was documented in the voipmonitor.conf [VS-1547]
add 'force' option for 'enable_semicolon_in_number' - enable semicolon in a number even if it is not followed by [0-9] [VS-1538]
fix threading_expanded = high_traffic mode (some early RTP pakcets were missing due to packet reorder/race in queue) - high_traffic option is not enabled by default [VS-1545]
fix HEAP_HASHTABLE on 32bit sniffer
2024.01.3 24.01.2024
discard defective packets for which a negative datalength is calculated (might cause crash)
adding an unlimited number of attempts to connect client to server [VS-1542]
2024.01.2 23.01.2024
add support for option cdr_check_exists_callid in cloud mode [VS-1534]
fix spool dir check if additional spool directories are defined [VG-2743]
fix popen deadlock in findMysqlProcess on some configurations [VG-2743]
correct proxy detection if source ip is same as source ip proxy but with different source port [VS-1539]
added gzip to pcap_dump_zip_sip [VS-1541]
added length condition len <= 0xFFFFF to deduplicate
add packet lenght sanitization in packet deduplication [VS-1540]
2024.01.1 05.01.2024
modify proxy and dst ip detection - use via.branch and confirmed invite matches [VS-1536]
2024.01 04.01.2024
fix potential race condition / crash in tar if data entry is extremely delayed
added support for upgrading sniffer from develop branch
fixed minor issues detected by cppcheck
if pcap limit is set store it to the cdr.flag [VS-1514]
2023.12.3 20.12.2023
fix tls decode for some cases and cypher suites (messages were shortened) broken since 2012.11.0
2023.12.2 15.12.2023
force connection close on server in client/server which shortnes socket timeout which fixes too many mysql connections issue (all sniffer versions)
2023.12.1 11.12.2023
fix crc sse invalid instruction some servers
better filter for server/client traffic in packtebuffer_sender mode [VS-1524]
fix -flto warning
reducing the minimum required version of zstd to 1.4.4
dequeu queue modification on packetbuffer output add 20% heap limit for queue, add 'deq[N1/N2]' statement to log, N1 is the number of percent of the heap the dequeu queue occupies, N2 is the current amount of time contained in the queue [VS-1521]
2023.12.0 06.12.2023
force heap cleanup if the size is greater than 20%
fix potential crash in setSchedPolPriority
suppress call createMysqlPartitionsBillingAgregation if is_client() is true [VS-1514]
log to syslog if ether_type is 0xFFFF (happens on old vmware nic drivers) and add testing command: voipmonitor --check_bad_ether_type eth0
2023.11.8 30.11.2023
change gcc options for static builds to -march=x86-64 -mtune=generic which fixes support for all 64bit architectures. Optimised versions with SSE3 and SSE4 will come later.
2023.11.7 30.11.2023
change DNS resolver to c-ares in static builds which fixes compatibility issue on old sysmtes preventing auto upgrade (download.voipmonitor.org resolve fails)
2023.11.6 28.11.2023
fix crash in illegal instruction in crc32 when server does not have SSE 4.2 and sniffer is in client mode forced to crc32 by the sniffer server
2023.11.5 28.11.2023
fix crash in destroySession if SslDsslSession does not contain a valid session
change zstd default level compression for sip and graph from 3 to 1 which lowers memory foot print by half with almost same compression ratio
remove options tar_sip_level, tar_rtp_level, tar_graph_level and replace them with *_gzip, *_lzma, *_zstd variants
2023.11.4 25.11.2023
fix illegal instruction in libcurl 2023.11.X on < haswell when sniffer upgrades itself. Manual upgrade is needed when on version >= 2023.11.1 and <= 2023.11.3 on < haswell CPU
2023.11.3 24.11.2023
fix illegal instruction in zstd 2023.11.X on < haswell
2023.11.2 24.11.2023
fix illegal instruction in snappy 2023.11.1
2023.11.1 24.11.2023
static builds uses gcc12 with the latest libc. Minimum kernel version is now >= 3.2.0
2023.11.0 21.11.2023
voipmonitor.conf overhaul
speed optimisations in threading processing (reduce locks and contentions, automatic add/remove threads in t2 processes)
static builds is now compiled with -O3 and -flto (increases memory throughput)
t2_boost renamed to threading_expanded with new option high_traffic (high traffic will be default in near future)
max_buffer_mem is et to 2000MB if the max_buffer_mem parameter is not present in the configuration
new scheduler thread prioroity - sched_* options in voipmonitor.conf. By default sniffer sets NICE -20 to all important threads.
automatically enable hashtable_heap_size=64 (64MB) if max_buffer_mem is at least 1000MB
replace gzip with zstd compression which has the same compression ratio but is 5x faster (the latest GUI version is REQUIRED!) [VS-1496]
add support for LZ4 compression for pcap compression (slightly faster than lzo but worse compression ratio)
replace packet deduplication md5 with crc32 (including hw support on supported intel xeons) and enable deduplication by default - on high traffic 1-3 cpu cores are spared [VS-1497]
./configure script now compiles with -O3 and supports static linking with the latest mariadb
fix ss7 configure script
support CFLAGS and CXXFLAGS for ./configure params
limit maximum number of possible INVITE packets within one call to prevent sniffer CPU overload [VS-1488]
fix issue with audio creation if stream contains 0 received packets (due to probation test). New option disable_rtp_seq_probation = yes/NO [VS-1494]
fix storing loss to rtp_stat [VG-2683]
ignore_rtp_after_bye_confirmed is now yes by default
detect_alone_bye is now yes by default
query_cache is now yes by default
mysql_queue_limit = 1000000 is now default
dscp is now yes by default
fix early destroy_call_at for multibranch calls [VS-1504]
fix realtime concurrent call alert if the call changes between local and international states during processing [VS-1506]
fix problem with seq synchronization in tls 1.3 after restoring session from ssl_sesions table [VS-1505]
limit the number of 'to much INVITEs' messages in the syslog [VS-1507]
fix crash in sll to en10 conversion [VS-1508]
fix custom headers reload [VS-1509]?
31.9 05.09.2023
new threads were spawned too late on high traffic servers - thread CPU usage is now performed in shorter intervals (was 10seconds) [VG-2672]
lower memory usage for cdr trends
fix dst SIP ip for some cases (broken in 31.8) [VS-1492]
fix audio decode for some cases (enable the use of a stream with zero received packets in the ab stream selection) [VS-1494]
31.8 24.08.2023
added Session Establishment Effectiveness Ratio (SEER) metric (https://www.rfc-editor.org/rfc/rfc6076#page-19) [VG-2659]
add register last sip response code 699 as failed state [VS-1459]
apply new option 'only one' after skip/trim prefixes in country rules [VS-2658]
add support for voipmonitor / wireshark mode switcher in rtp analyzer [VS-1474]
diameter - add option diameter_ignore_prefix = yes/NO, add relation between public_identity and sip_from [VS-1420]
set MYSQL_OPT_RECONNECT so the mysql connection is reconnected automatically within the mysql library
fix livesniffer for long uid [VS-1455]
fix time overflow when packets are out of order [VS-1461]
correctly parse header even if it is delimited only by '\r' [VS-1460]
fix some low MOS rare cases [VS-1465]
fix issue with detection of registration expiration with multiple contact information [VS-1462]
fix issue with client restart in cloud or client mode (force stop client service) [VS-1463]
fix write to tar file when there is no disk space and cleanup is performed [VS-1440]
accept cleanup_calls_period parameter even if quick_save_cdr is enabled [VS-1471]
better proxy detection (fixes some cases not detected as proxy) [VS-1475]
revert default config/voipmonitor.conf to rtp tar setting: lzo/no-gzip
fix partition deletion for cdr_stat - force to keep active partition (for case of low cleandatabase_cdr_stat parameter) [VS-1477]
fix problem with concurrent creation of partitions from multiple sensors [VS-1477]
automatically enable mysql_enable_new_store=per_query (if 'no') if mysql_enable_set_id is enabled and if there are no conflicts in the configuration
fix value limitation in a/b_rtcp_loss columns, fix parsing a/b_rtcp_loss from rtcp. Fixes 'Out of range value for column 'b_rtcp_loss' at row' [VS-1480]
realtime concurrent calls fraud fixes - faster detach of calls from fraud logs in cleanup_calls [VS-1482]
fix automatic cleaning when disk space is low
31.7.5 22.06.2023
fix race in handle_dtmf leading to crash [VS-1456]
31.7.4 22.06.2023
fix some cleaning ssl issues [VS-1452]
fix live sniffer saving for older databases [VS-1454]
fix issue when there are same call-id INVITEs without any responses [VS-1440]
31.7.3 09.06.2023
cdr trends (special aggregation table) now stores src/dst MOS score with respect of the direction. MOS will now represents sender or receiver only and not MIN(src/dst)
