Ngrep is a very user friendly packet sniffer, basically the “grep” equivalent at the network layer.

Here is a quick way of figuring out the http connections your browser is making even if you are browsing to a secure site, make sure that is the only site you are visiting as the command will capture all port 80 connections.

Installation –

apt-get install ngrep

Let us redirect all traffic ngrep captured to a file –

ngrep -d any -W byline port 80 | tee  /tmp/net_output

Now visit a secure site, say https://cnet.com, you will see nicely formated output

root@lindell:~# ngrep -d any -W byline port 80 | tee  /tmp/output
interface: any
filter: (ip or ip6) and ( port 80 )
####
T 17.31.198.19:33954 -> 72.21.91.29:80 [AP]
POST / HTTP/1.1.
Host: ocsp.digicert.com.
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0.
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8.
Accept-Language: en-US,en;q=0.5.
Accept-Encoding: gzip, deflate.
Content-Length: 83.
Content-Type: application/ocsp-request.
Connection: keep-alive.
..

From here, you can parse the /tmp/output file.

Similarly, you can parse the output file for the type of web server your favorite sites are using. Keep the ngrep command running, and visit all your favorite sites. Note, this works for http only, as https traffic is encrypted, for https only destination IP and port are shown.

In this case, I searched for the ‘Server:’ field in the HTTP response header from the web server. Apparently, nginx seems to be most popular, it is also interesting to see that AmazonS3 storage being used for hosting static content –

root@lindell:~# awk '/Server:/ {print $2}' /tmp/output |sort | uniq -c |sort -nr
    155 nginx.
     40 Apache.
     36 Apache-Coyote/1.1.
     20 Apache/2.2.3
     14 nginx/1.8.1.
      7 AmazonS3.
      6 Akamai
      5 ECS
      5 cloudflare-nginx.
      4 Omniture
      4 ESF.
      3 sffe.
      3 nginx/1.10.2.
      2 Microsoft-IIS/7.5.
      2 gws.
      2 AkamaiGHost.
      1 WildFly/8.
      1 Varnish.
      1 openresty.
      1 NetDNA-cache/2.2.
      1 Cowboy.
      1 ATS.
      1 Apache/2.2.14

References –
http://ngrep.sourceforge.net/usage.html
https://wiki.christophchamp.com/index.php?title=Ngrep