Manual export of pcap files from spooldir: Difference between revisions

From VoIPmonitor.org
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
== Notes ==
== Notes ==
'''RTP format:''' With default config shipped with latest voipmonitor sensor, is RTP compression enabled into LZO in time of capture - those LZOed files are tared into RTP archives based on date-hourminute of a call start and its call's call-id.
'''RTP format:''' With default voipmonitor.conf RTP pcap chunks are compressed by LZO which are tared and archived in directory in date-hourminute
 
  option '''pcap_dump_zip_rtp = lzo'''
  option '''pcap_dump_zip_rtp = lzo'''


'''SIP format:''' With default config shipped with latest voipmonitor sensor, is SIP compression enabled after tar archive was created:
'''SIP format:''' With default voipmonitor.conf SIP compression uses gzip
  option '''tar_compress_sip = gzip'''
  option '''tar_compress_sip = gzip'''




== Export pcap file with default config used ==
== Export pcap file with default config used ==
=== precondition ===
call needs to be captured with sensor's compression settings like in default voipmonitor.conf (no change to compression options)
pcap_dump_zip_rtp = lzo
option tar_compress_sip = gzip


=== information needed from CDR detail for export ===
=== Get information about CDR from database ===
You will need:
You will need:
  1.CDR.id (103)
  1.cdr.id (103)
  2.Date time of a call start (2016-08-23 16:37:38)
  2.Date time of call start (2016-08-23 16:37:38)
  3.Call-ID (CwA8j-SNSN)
  3.Call-ID (CwA8j-SNSN)
  4.Location of your spooldir ('spooldir' option is defined in /etc/voipmonitor.conf)
  4.Location of your spooldir (spooldir=X)  
====If your GUI is working====
 
example : [[File: cdr_detail_for_export_pcap_default.jpg]]
example : [[File: cdr_detail_for_export_pcap_default.jpg]]


====If your GUI is not working====
SQL Query:  
You can ask database for those CDR's values with a query like this ( it will list calls of a caller starting with '+222' in date 2017-02-01:
  SELECT cdr.calldate,cdr.caller,cdr.called,cdr.id as cdrID,cdr_next.fbasename as callID from cdr,cdr_next where cdr.id=cdr_next.cdr_ID and cdr.calldate >= '2017-02-01 00:00:00' and cdr.calldate <= '2017-02-01 23:59:59' and cdr.caller like '+222%';
  mysql> select cdr.calldate,cdr.caller,cdr.called,cdr.id as cdrID,cdr_next.fbasename as callID from cdr,cdr_next where cdr.id=cdr_next.cdr_ID and cdr.calldate >= '2017-02-01 00:00:00' and cdr.calldate < '2017-02-02 00:00:00' and cdr.caller like '+222%';


You MUST use cdr.calldate condition otherwise database will be overloaded by searching in all partitions


=== export SIP pcap ===
=== export SIP pcap ===
From spooldir location (by default its '/var/spool/voipmonitor' and calldate start '2016-08-23 16:37:38' in example and from CALL-ID header 'CwA8j-SNSN' you can write command:
 
  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/16/37/SIP/sip_2016-08-23-16-37.tar.gz' 'CwA8j-SNSN.pcap*' > /tmp/expsip.pcap
  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/16/37/SIP/sip_2016-08-23-16-37.tar.gz' 'CwA8j-SNSN.pcap*' > /tmp/expsip.pcap


Line 39: Line 36:




=== export RTP pcap II ===
=== export RTP ===
(harder way for old sniffers)
 
First we will need to get '''lzo positions''' from database (calldate start '2016-08-23 16:37:38'in example and from CALL-ID header 'CwA8j-SNSN' you can write a query), type=2 (means RTP filetype):
Get RTP positions
 
  mysql> SELECT pos FROM voipmonitor.cdr_tar_part where cdr_id = 103 and type = 2 and calldate = '2016-08-23 16:37:38';
  mysql> SELECT pos FROM voipmonitor.cdr_tar_part where cdr_id = 103 and type = 2 and calldate = '2016-08-23 16:37:38';


Line 51: Line 49:
  4 rows in set (0,00 sec)
  4 rows in set (0,00 sec)


Second we use positions returned from db to '''export RTP and unLZO''' using voipmonitor binary:
use positions returned from db and extract pcap  
/usr/local/sbin/voipmonitor -kc -d /var/spool/voipmonitor/ --untar-gui='/var/spool/voipmonitor//2016-08-23/16/37/RTP/rtp_2016-08-23-16-37.tar CwA8j-SNSN.pcap 0,164352,328704,493056 /tmp/exprtp.pcap'


=== merge SIP and RTP into one file ===
/usr/local/sbin/voipmonitor -kc -d /var/spool/voipmonitor/ --untar-gui='/var/spool/voipmonitor//2016-08-23/16/37/RTP/rtp_2016-08-23-16-37.tar CwA8j-SNSN.pcap 0,164352,328704,493056 rtp.pcap'
  mergecap -w /tmp/export.pcap /tmp/exportSIP.pcap /tmp/exportRTP.pcap
 
'''Alternative RTP extraction without knowing positions from database - this will consume more IO reads as tar file has to be fully scanned'''
 
  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > rtp.pcap




'''if LZO compression for RTP pcaps is enabled '''


== Export pcap file when LZO compression disabled for RTP in config ==
=== preconditions ===
call captured when sensor's compression settings changed from default voipmonitor.conf
pcap_dump_zip_rtp = '''no'''
option tar_compress_sip = gzip


=== information needed to collect from CDR ===
If LZO compression for RTP pcas is enabled you have to unLZO it first before merging with SIP
From picture in section above you will need:
2.Date time of a call start
3.Call-ID


=== export SIP pcap ===
tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/SIP/sip_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > ./exportSIP.pcap


=== export RTP pcap ===
  voipmonitor -kc --unlzo-gui='rtp.pcap rtp-uncompressed.pcap'
  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > ./exportRTP.pcap


=== merge SIP and RTP into one file ===
=== merge SIP and RTP into one file ===
  mergecap -w /tmp/export.pcap /tmp/exportSIP.pcap /tmp/exportRTP.pcap
(apt-get install tshark | yum install wireshark)
 
  mergecap -w rtp.pcap sip.pcap final.pcap

Revision as of 11:38, 27 November 2019

Notes

RTP format: With default voipmonitor.conf RTP pcap chunks are compressed by LZO which are tared and archived in directory in date-hourminute 

option pcap_dump_zip_rtp = lzo

SIP format: With default voipmonitor.conf SIP compression uses gzip 

option tar_compress_sip = gzip


Export pcap file with default config used

Get information about CDR from database

You will need: 

1.cdr.id (103)
2.Date time of call start (2016-08-23 16:37:38)
3.Call-ID (CwA8j-SNSN)
4.Location of your spooldir (spooldir=X)

example :

SQL Query: 

SELECT cdr.calldate,cdr.caller,cdr.called,cdr.id as cdrID,cdr_next.fbasename as callID from cdr,cdr_next where cdr.id=cdr_next.cdr_ID and cdr.calldate >= '2017-02-01 00:00:00' and cdr.calldate <= '2017-02-01 23:59:59' and cdr.caller like '+222%';

You MUST use cdr.calldate condition otherwise database will be overloaded by searching in all partitions

export SIP pcap

tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/16/37/SIP/sip_2016-08-23-16-37.tar.gz' 'CwA8j-SNSN.pcap*' > /tmp/expsip.pcap


export RTP pcap

voipmonitor -kc --unlzo-gui='input.pcap output.pcap'
#if path to file is not absolute (/...) it is relative to the spooldir directory
(GUI decompress it on the fly and serve it as gzip)


export RTP

Get RTP positions 

mysql> SELECT pos FROM voipmonitor.cdr_tar_part where cdr_id = 103 and type = 2 and calldate = '2016-08-23 16:37:38';

Returned: 

pos: 0
pos: 164352
pos: 328704
pos: 493056
4 rows in set (0,00 sec)

use positions returned from db and extract pcap 

/usr/local/sbin/voipmonitor -kc -d /var/spool/voipmonitor/ --untar-gui='/var/spool/voipmonitor//2016-08-23/16/37/RTP/rtp_2016-08-23-16-37.tar CwA8j-SNSN.pcap 0,164352,328704,493056 rtp.pcap'

Alternative RTP extraction without knowing positions from database - this will consume more IO reads as tar file has to be fully scanned 

tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > rtp.pcap


if LZO compression for RTP pcaps is enabled


If LZO compression for RTP pcas is enabled you have to unLZO it first before merging with SIP 


voipmonitor -kc --unlzo-gui='rtp.pcap rtp-uncompressed.pcap'

merge SIP and RTP into one file

(apt-get install tshark | yum install wireshark) 

mergecap -w rtp.pcap sip.pcap final.pcap
Retrieved from "https://www.voipmonitor.org//doc/index.php?title=Manual_export_of_pcap_files_from_spooldir&oldid=2905"