Manual export of pcap files from spooldir: Difference between revisions

From VoIPmonitor.org
Jump to navigation Jump to search
No edit summary
 
(11 intermediate revisions by 2 users not shown)
Line 20: Line 20:


SQL Query:  
SQL Query:  
  SELECT cdr.calldate,cdr.caller,cdr.called,cdr.id as cdrID,cdr_next.fbasename as callID from cdr,cdr_next where cdr.id=cdr_next.cdr_ID and cdr.calldate >= '2017-02-01 00:00:00' and cdr.calldate <= '2017-02-01 23:59:59' and cdr.caller like '+222%';
  SELECT cdr.calldate,cdr.caller,cdr.called,cdr.id as cdrID,cdr_next.fbasename as callID  
FROM cdr,cdr_next  
WHERE cdr.id=cdr_next.cdr_ID AND cdr.calldate >= '2017-02-01 00:00:00' AND cdr.calldate <= '2017-02-01 23:59:59' AND cdr.caller like '+222%';


You MUST use cdr.calldate condition otherwise database will be overloaded by searching in all partitions  
You MUST use cdr.calldate condition otherwise database will be overloaded by searching in all partitions  
=== flush data to tar file===
Before you ask tar binary or voipmonitor for file extraction, you need to flush data from sniffer's cache first ( if the tarball file is still opened ) send api command to sniffer service
echo "flush_tar '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar'" |nc 127.0.0.1


=== export SIP pcap ===
=== export SIP pcap ===


  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/16/37/SIP/sip_2016-08-23-16-37.tar.gz' 'CwA8j-SNSN.pcap*' > /tmp/expsip.pcap
  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/16/37/SIP/sip_2016-08-23-16-37.tar.gz' 'CwA8j-SNSN.pcap*' > /tmp/expsip.pcap
=== export RTP pcap ===
voipmonitor -kc --unlzo-gui='input.pcap output.pcap'
#if path to file is not absolute (/...) it is relative to the spooldir directory
(GUI decompress it on the fly and serve it as gzip)




Line 52: Line 50:


  /usr/local/sbin/voipmonitor -kc -d /var/spool/voipmonitor/ --untar-gui='/var/spool/voipmonitor//2016-08-23/16/37/RTP/rtp_2016-08-23-16-37.tar CwA8j-SNSN.pcap 0,164352,328704,493056 rtp.pcap'
  /usr/local/sbin/voipmonitor -kc -d /var/spool/voipmonitor/ --untar-gui='/var/spool/voipmonitor//2016-08-23/16/37/RTP/rtp_2016-08-23-16-37.tar CwA8j-SNSN.pcap 0,164352,328704,493056 rtp.pcap'
#rtp.pcap is already decompressed (no unLZO needed)


'''Alternative RTP extraction without knowing positions from database - this will consume more IO reads as tar file has to be fully scanned'''
==Alternative RTP extraction without knowing positions from database==
this will consume more IO reads as tar file has to be fully scanned, additionally you need to have tar file closed or you need to ask sniffer to flush cashed data if the file is still opened.


  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > rtp.pcap
  tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > rtp.pcap
# if LZO compression for RTP pcaps is enabled '''
voipmonitor -kc --unlzo-gui='/path/to/rtp.pcap /path/to/rtp-uncompressed.pcap'
#if path to file is not absolute (/...) it is relative to the spooldir directory
=== flush partially written data into tar file ===
Before you ask tar binary for file extraction you need to flush data from sniffer first ( if the pcap file is still opened ) using sniffer's api like:
echo "flush_tar '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar'" |nc 127.0.0.1


=== merge SIP and RTP into one file ===
(apt-get install tshark | yum install wireshark)
Note: the rtp.pcap needs to be de-lzoed first (lzo is default compression method used by sniffer service when rtp stored to spooldir)


'''if LZO compression for RTP pcaps is enabled '''
mergecap -w rtp.pcap sip.pcap final.pcap




If LZO compression for RTP pcas is enabled you have to unLZO it first before merging with SIP
=Create the AUDIO using pcap file=
after you get/have SIP+RTP in a single pcap file you can ask sniffer's binary for create audio file without need to have GUI installed/ db used (works only for g711 calls without the GUI)
==prepare voipmonitor-audio.conf==
Change following options copy of the sniffer's config file - sniffer needs to be set to analyze traffic (not mirror senders):
spooldir=/myAudio
savesip=no
savertp=no
savertcp=no
savegraph=no
saveaudio=ogg
#saveaudio=wav
saveaudio_stereo = yes
nocdr=yes
#for possible to create audio also from other then g711 codecs( g711 is supported with sniffer directly) you need to have valid license for GUI and enable following option with path to gui's keycheck file
#keycheck = /var/www/html/php/lib/keycheck.php


==call sniffer's binary==
Call the sniffer's binary with file.pcap as argument to create audio from it in spooldir defined above:
voipmonitor --config-file=/etc/voipmonitor-audio.conf -k -v1 -r /tmp/final.pcap


voipmonitor -kc --unlzo-gui='rtp.pcap rtp-uncompressed.pcap'
==Results location==
location of created audio files (it's names is CALL-ID observed in INVITE packet) depends on option **spooldiroldschema**


=== merge SIP and RTP into one file ===
===spooldiroldschema = no===
(apt-get install tshark | yum install wireshark)  
By default, the audio file will be created in directory /${SPOOLDIR}/DATE/HOUR/MINUTE/AUDIO/ (based on timestamp of packets
/myAudio/2020-07-31/19/52/AUDIO/sSCCXfB1Pa.ogg: Ogg data, Vorbis audio, stereo, 8000 Hz, ~36400 bps, created by: Xiph.Org libVorbis I (1.3.2)


  mergecap -w rtp.pcap sip.pcap final.pcap
===spooldiroldschema = yes===
the audio will be made in DATE dir (based on timestamp of packets
  /myAudio/2020-07-31/sSCCXfB1Pa.ogg: Ogg data, Vorbis audio, stereo, 8000 Hz, ~36400 bps, created by: Xiph.Org libVorbis I (1.3.2)

Latest revision as of 16:26, 28 February 2023

Notes

RTP format: With default voipmonitor.conf RTP pcap chunks are compressed by LZO which are tared and archived in directory in date-hourminute

option pcap_dump_zip_rtp = lzo

SIP format: With default voipmonitor.conf SIP compression uses gzip

option tar_compress_sip = gzip


Export pcap file with default config used

Get information about CDR from database

You will need:

1.cdr.id (103)
2.Date time of call start (2016-08-23 16:37:38)
3.Call-ID (CwA8j-SNSN)
4.Location of your spooldir (spooldir=X) 

example :

SQL Query:

SELECT cdr.calldate,cdr.caller,cdr.called,cdr.id as cdrID,cdr_next.fbasename as callID 
FROM cdr,cdr_next 
WHERE cdr.id=cdr_next.cdr_ID AND cdr.calldate >= '2017-02-01 00:00:00' AND cdr.calldate <= '2017-02-01 23:59:59' AND cdr.caller like '+222%';

You MUST use cdr.calldate condition otherwise database will be overloaded by searching in all partitions

flush data to tar file

Before you ask tar binary or voipmonitor for file extraction, you need to flush data from sniffer's cache first ( if the tarball file is still opened ) send api command to sniffer service

echo "flush_tar '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar'" |nc 127.0.0.1

export SIP pcap

tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/16/37/SIP/sip_2016-08-23-16-37.tar.gz' 'CwA8j-SNSN.pcap*' > /tmp/expsip.pcap


export RTP

Get RTP positions

mysql> SELECT pos FROM voipmonitor.cdr_tar_part where cdr_id = 103 and type = 2 and calldate = '2016-08-23 16:37:38';

Returned:

pos: 0
pos: 164352
pos: 328704
pos: 493056
4 rows in set (0,00 sec)

use positions returned from db and extract pcap

/usr/local/sbin/voipmonitor -kc -d /var/spool/voipmonitor/ --untar-gui='/var/spool/voipmonitor//2016-08-23/16/37/RTP/rtp_2016-08-23-16-37.tar CwA8j-SNSN.pcap 0,164352,328704,493056 rtp.pcap'
#rtp.pcap is already decompressed (no unLZO needed) 

Alternative RTP extraction without knowing positions from database

this will consume more IO reads as tar file has to be fully scanned, additionally you need to have tar file closed or you need to ask sniffer to flush cashed data if the file is still opened.

tar --wildcards -xOf '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar' 'R3YqlN7pnY.pcap*' > rtp.pcap
# if LZO compression for RTP pcaps is enabled 
voipmonitor -kc --unlzo-gui='/path/to/rtp.pcap /path/to/rtp-uncompressed.pcap'
#if path to file is not absolute (/...) it is relative to the spooldir directory

flush partially written data into tar file

Before you ask tar binary for file extraction you need to flush data from sniffer first ( if the pcap file is still opened ) using sniffer's api like:

echo "flush_tar '/var/spool/voipmonitor/2016-08-23/15/27/RTP/rtp_2016-08-23-15-27.tar'" |nc 127.0.0.1

merge SIP and RTP into one file

(apt-get install tshark | yum install wireshark) Note: the rtp.pcap needs to be de-lzoed first (lzo is default compression method used by sniffer service when rtp stored to spooldir)

mergecap -w rtp.pcap sip.pcap final.pcap


Create the AUDIO using pcap file

after you get/have SIP+RTP in a single pcap file you can ask sniffer's binary for create audio file without need to have GUI installed/ db used (works only for g711 calls without the GUI)

prepare voipmonitor-audio.conf

Change following options copy of the sniffer's config file - sniffer needs to be set to analyze traffic (not mirror senders):

spooldir=/myAudio
savesip=no
savertp=no
savertcp=no
savegraph=no
saveaudio=ogg
#saveaudio=wav
saveaudio_stereo = yes
nocdr=yes
#for possible to create audio also from other then g711 codecs( g711 is supported with sniffer directly) you need to have valid license for GUI and enable following option with path to gui's keycheck file
#keycheck = /var/www/html/php/lib/keycheck.php

call sniffer's binary

Call the sniffer's binary with file.pcap as argument to create audio from it in spooldir defined above:

voipmonitor --config-file=/etc/voipmonitor-audio.conf -k -v1 -r /tmp/final.pcap

Results location

location of created audio files (it's names is CALL-ID observed in INVITE packet) depends on option **spooldiroldschema**

spooldiroldschema = no

By default, the audio file will be created in directory /${SPOOLDIR}/DATE/HOUR/MINUTE/AUDIO/ (based on timestamp of packets

/myAudio/2020-07-31/19/52/AUDIO/sSCCXfB1Pa.ogg: Ogg data, Vorbis audio, stereo, 8000 Hz, ~36400 bps, created by: Xiph.Org libVorbis I (1.3.2)

spooldiroldschema = yes

the audio will be made in DATE dir (based on timestamp of packets

/myAudio/2020-07-31/sSCCXfB1Pa.ogg: Ogg data, Vorbis audio, stereo, 8000 Hz, ~36400 bps, created by: Xiph.Org libVorbis I (1.3.2)