Debug connection performance with curl
Posted by danielFeb 11
Curl – network performance stats
Curl is a tool commonly used with transferring data from or to a client/server. Its portability has allowed it to present in most OS installations by default, and thus making as a standard tool for troubleshooting and debugging network or server connectivity issues.
Here you will see some Curl features to gather network connectivity performance metrics, all exposed with -w flag.
-w, --write-out <format>
Make curl display information on stdout after a completed transfer.
time_namelookup
The time, in seconds, it took from the start until the name resolving was completed.
time_connect
The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
time_pretransfer
The time, in seconds, it took from the start until the file transfer was just about to begin.
time_starttransfer
The time, in seconds, it took from the start until the first byte was just about to be transferred.
time_total
The total time, in seconds, that the full operation lasted.
With the above 5 options, we can get a decent amount of information on the culprit for network issue. Is it DNS resolution delay? Or delay in TCP 3 way handshake? etc
Examples
On first run –
$ curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null https://www.fifa.com
Lookup time: 0.062836
Connect time: 0.078382
PreXfer time: 0.123689
StartXfer time: 0.155934
Total time: 0.182570
Second run –
$ curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null https://www.fifa.com
Lookup time: 0.022473
Connect time: 0.036173
PreXfer time: 0.087416
StartXfer time: 0.134248
Total time: 0.171503
On second run, the dns look up time is lower, as it is coming from cache. Let us flush dns cache –
$ sudo systemd-resolve --flush-caches
$ curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null https://www.fifa.com
Lookup time: 0.072109
Connect time: 0.086815
PreXfer time: 0.141417
StartXfer time: 0.185936
Total time: 0.227121
As expected the look up time is higher posed dns cache clear.
Additional information
You can get additional information by reading the manual pages for curl or even use curl to get information about curl –
$ curl cheat.sh/curl
cheat.sheets:curl
# curl
# Command-line tool for transferring data with URL syntax
# Process a single GET request, and show its output on stdout.
curl http://path.to.the/file
# Download a file and specify a new filename.
curl http://example.com/file.zip -o new_file.zip
# Download multiple files.
curl -O URLOfFirstFile -O URLOfSecondFile
# Download all sequentially-numbered files (1-24).
curl http://example.com/pic[1-24].jpg
# Download a file and follow redirects.
curl -L http://example.com/file
....
