Curl – network performance stats

Curl is a tool commonly used with transferring data from or to a client/server. Its portability has allowed it to present in most OS installations by default, and thus making as a standard tool for troubleshooting and debugging network or server connectivity issues.

Here you will see some Curl features to gather network connectivity performance metrics, all exposed with -w flag.

-w, --write-out <format>
Make curl display information on stdout after a completed transfer.

time_namelookup
 The time, in seconds, it took from the start until the name resolving was completed.

time_connect
 The time, in seconds, it took from the start until the TCP connect to the remote host  (or  proxy)  was completed.

time_pretransfer
 The  time, in seconds, it took from the start until the file transfer was just about to begin.

time_starttransfer
  The  time,  in  seconds,  it took from the start until the first byte was just about to be transferred.

time_total
  The total time, in seconds, that the full operation lasted.

With the above 5 options, we can get a decent amount of information on the culprit for network issue. Is it DNS resolution delay? Or delay in TCP 3 way handshake? etc

Examples

On first run –

1
2
3
4
5
6
7
8
$ curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null https://www.fifa.com
 
Lookup time:    0.062836
Connect time:   0.078382
PreXfer time:   0.123689
StartXfer time: 0.155934
 
Total time: 0.182570

Second run –

1
2
3
4
5
6
7
8
$ curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null https://www.fifa.com
 
Lookup time:    0.022473
Connect time:   0.036173
PreXfer time:   0.087416
StartXfer time: 0.134248
 
Total time: 0.171503

On second run, the dns look up time is lower, as it is coming from cache. Let us flush dns cache –

1
2
3
4
5
6
7
8
9
10
$ sudo systemd-resolve --flush-caches
 
$ curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null https://www.fifa.com
 
Lookup time:    0.072109
Connect time:   0.086815
PreXfer time:   0.141417
StartXfer time: 0.185936
 
Total time: 0.227121

As expected the look up time is higher posed dns cache clear.

Additional information

You can get additional information by reading the manual pages for curl or even use curl to get information about curl –

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ curl cheat.sh/curl
 cheat.sheets:curl
# curl
# Command-line tool for transferring data with URL syntax
 
# Process a single GET request, and show its output on stdout.
curl http://path.to.the/file
 
# Download a file and specify a new filename.
curl http://example.com/file.zip -o new_file.zip
 
# Download multiple files.
curl -O URLOfFirstFile -O URLOfSecondFile
 
# Download all sequentially-numbered files (1-24).
curl http://example.com/pic[1-24].jpg
 
# Download a file and follow redirects.
curl -L http://example.com/file
....