Recipe 9.18 Searching for Strings in Network Traffic
9.18.1 Problem
You want to watch
network traffic, searching for strings in the transmitted data.
9.18.2 Solution
Use ngrep.To search for packets containing data that matches a
regular expression and
protocols that match a filter expression:
# ngrep [grep-options] regular-expression [filter-expression]
To search instead for a sequence of binary data:
# ngrep -X hexadecimal-digits [filter-expression]
To sniff packets and save them in a file:
# ngrep -O filename [-n count] [-d interface] [-s snap-length] regular-expression [filter-expression]
To read and display the saved network trace data:
$ ngrep -I filename regular-expression [filter-expression]
9.18.3 Discussion
ngrep is
supplied with SuSE but not Red Hat; however, it is easy to obtain and
install if you need it. Download it from http://ngrep.sourceforge.net and unpack it:
$ tar xvpzf ngrep-*.tar.gz
compile it:
$ cd ngrep
$ ./configure --prefix=/usr/local
$ make
and install it into /usr/local as
root:[8]
[8] We explicitly install in
/usr/local, because otherwise the
configure script would install into
/usr, based on the location of
gcc. We recommend /usr/local
to avoid clashes with vendor-supplied software in
/usr; this recommendation is codified in the
Filesystem Hierarchy Standard (FHS), http://www.pathname.com/fhs. The
configure script used for ngrep
is unusualsuch scripts typically install into
/usr/local by default, and therefore do not need
an explicit prefix option. We also create
the installation directories if they don't already
exist, to overcome deficiencies in the make
install command.
# mkdir -p /usr/local/bin /usr/local/man/man8
# make install
Sometimes we are interested in observing the data delivered by
network packets, known as the
payload. Tools like
tcpdump
[Recipe 9.16] and especially Ethereal [Recipe 9.17] can display the payload, but they are
primarily designed for protocol analysis, so their ability to select
packets based on arbitrary data is limited.[9]
[9] The
concept of a packet's payload is subjective. Each
lower-level protocol regards the higher-level protocols as its
payload. The highest-level protocol delivers the user data; for
example, the files transferred by FTP.
The ngrep command
searches network traffic for data that matches
extended regular expressions, in the same
way that the egrep command (or grep
-E) searches files. In fact, ngrep
supports many of the same command-line options as
egrep, such as -i
(case-insensitive), -w (whole words), or
-v (nonmatching). In addition,
ngrep can select packets using the same filter
expressions as tcpdump. To use
ngrep as an ordinary packet sniffer, use the
regular expression ".", which
matches any nonempty payload.ngrep is handy for detecting the use of insecure
protocols. For example, we can
observe FTP transfers to or from a server, searching for FTP request
command strings to reveal usernames, passwords, and filenames that
are transmitted as clear text:
$ ngrep -t -x 'USER|PASS|RETR|STOR' tcp port ftp and host server.example.com
interface: eth0 (10.44.44.0/255.255.255.0)
filter: ip and ( tcp port ftp )
match: USER|PASS|RETR|STOR
#############
T 2003/02/27 23:31:20.303636 10.33.33.33:1057 -> 10.88.88.88:21 [AP]
55 53 45 52 20 6b 61 74 69 65 0d 0a USER katie..
#####
T 2003/02/27 23:31:25.315858 10.33.33.33:1057 -> 10.88.88.88:21 [AP]
50 41 53 53 20 44 75 6d 62 6f 21 0d 0a PASS Dumbo!..
#############
T 2003/02/27 23:32:15.637343 10.33.33.33:1057 -> 10.88.88.88:21 [AP]
52 45 54 52 20 70 6f 6f 68 62 65 61 72 0d 0a RETR poohbear..
########
T 2003/02/27 23:32:19.742193 10.33.33.33:1057 -> 10.88.88.88:21 [AP]
53 54 4f 52 20 68 6f 6e 65 79 70 6f 74 0d 0a STOR honeypot..
###############exit
58 received, 0 dropped
The -t option adds
timestamps; use -T instead for relative times
between packets. The -x option prints hexadecimal
values in addition to the ASCII strings.ngrep prints a hash character (#) for each packet
that matches the filter expression: only those packets that match the
regular expression are printed in detail. Use the
-q option to suppress the hashes.
To search for binary
data, use the -X option with a sequence of
hexadecimal digits (of any length) instead of a regular expression.
This can detect some kinds of
buffer overflow attacks, characterized by
known signatures of fixed binary data.
|
ngrep can write and read
libpcap-format network trace files, using the
-O and -I options. [Recipe 9.16] This is especially convenient when running
ngrep repeatedly to refine your search, using data
captured previously, perhaps by another program. Usually
ngrep captures packets until killed, or it will
exit after recording a maximum number of packets requested by the
-n option. The -d option
selects a specific interface, if your machine has several. By
default, ngrep captures entire packets (in
contrast to tcpdump and
ethereal), since ngrep is
interested in the payloads. If your data of interest is at the
beginning of the packets, use the -s option to
reduce the snapshot and gain efficiency.When ngrep finds an interesting packet, the
adjacent packets might be of interest too, as context. The
ngrep -A option prints a specified
number of extra (not necessarily matching) packets for trailing
context. This is similar in spirit to the grep -A
option, but ngrep does not support a corresponding
-B option for leading context.
|
9.18.4 See Also
ngrep(8), egrep(1), grep(1), tcpdump(8). The home page for
ngrep is
http://ngrep.sourceforge.net, and
the tcpdump home page is http://www.tcpdump.org.Learn more about extended regular expressions in the
O'Reilly book Mastering Regular
Expressions .