Using curl to get help on Linux commands, programming languages and more. The most comprehensive cheat sheet.

If you are looking for a Linux and programming cheat sheet, please check
https://github.com/chubin/cheat.sh

It provides nicely colored help page, with plenty of examples in a CLI. Here are some sample runs I did.

Curl cheat sheet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
daniel@hidmo:/tmp$ curl cheat.sh/curl
# Download a single file
curl http://path.to.the/file
 
# Download a file and specify a new filename
curl http://example.com/file.zip -o new_file.zip
 
# Download multiple files
curl -O URLOfFirstFile -O URLOfSecondFile
 
# Download all sequentially numbered files (1-24)
curl http://example.com/pic[1-24].jpg
 
# Download a file and follow redirects
curl -L http://example.com/file
 
# Download a file and pass HTTP Authentication
curl -u username:password URL 
 
# Download a file with a Proxy
curl -x proxysever.server.com:PORT http://addressiwantto.access
 
# Download a file from FTP
curl -u username:password -O ftp://example.com/pub/file.zip
 
# Get an FTP directory listing
curl ftp://username:password@example.com
 
# Resume a previously failed download
curl -C - -o partial_file.zip http://example.com/file.zip
 
# Fetch only the HTTP headers from a response
curl -I http://example.com
 
# Fetch your external IP and network info as JSON
curl http://ifconfig.me/all/json
 
# Limit the rate of a download
curl --limit-rate 1000B -O http://path.to.the/file
 
# POST to a form
curl -F "name=user" -F "password=test" http://example.com
 
# POST JSON Data
curl -H "Content-Type: application/json" -X POST -d '{"user":"bob","pass":"123"}' http://example.com
 
# POST data from the standard in / share data on sprunge.us
curl -F 'sprunge=<-' sprunge.us

Python lists cheat list

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
daniel@hidmo:/tmp$ curl cheat.sh/python/list
#  python - Why does += behave unexpectedly on lists?
#  
#  The general answer is that += tries to call the __iadd__ special
#  method, and if that isn't available it tries to use __add__ instead.
#  So the issue is with the difference between these special methods.
#  
#  The __iadd__ special method is for an in-place addition, that is it
#  mutates the object that it acts on. The __add__ special method returns
#  a new object and is also used for the standard + operator.
#  
#  So when the += operator is used on an object which has an __iadd__
#  defined the object is modified in place. Otherwise it will instead try
#  to use the plain __add__ and return a new object.
#  
#  That is why for mutable types like lists += changes the object's
#  value, whereas for immutable types like tuples, strings and integers a
#  new object is returned instead (a += b becomes equivalent to a = a +
#  b).
#  
#  For types that support both __iadd__ and __add__ you therefore have to
#  be careful which one you use. a += b will call __iadd__ and mutate a,
#  whereas a = a + b will create a new object and assign it to a. They
#  are not the same operation!
 
>>> a1 = a2 = [1, 2]
>>> b1 = b2 = [1, 2]
>>> a1 += [3]          # Uses __iadd__, modifies a1 in-place
>>> b1 = b1 + [3]      # Uses __add__, creates new list, assigns it to b1
>>> a2
[1, 2, 3]              # a1 and a2 are still the same list
>>> b2
[1, 2]                 # whereas only b1 was changed
 
#  For immutable types (where you don't have an __iadd__) a += b and a =
#  a + b are equivalent. This is what lets you use += on immutable types,
#  which might seem a strange design decision until you consider that
#  otherwise you couldn't use += on immutable types like numbers!
#  
#  [Scott Griffiths] [so/q/2347265] [cc by-sa 3.0]

Golang concurrency cheat sheet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
daniel@hidmo:/tmp$ curl cheat.sh/go/concurrency
/*
 * go - When should I use concurrency in golang?
 * 
 * Not an expert in Go (yet) but I'd say:
 * 
 * Whenever it is easiest to do so.
 * 
 * The beauty of the concurrency model in Go is that it is not
 * fundamentally a multi-core architecture with checks and balances where
 * things usually break - it is a multi-threaded paradigm that not only
 * fits well into a multi-core architecture, it also fits well into a
 * distributed system architecture.
 * 
 * You do not have to make special arrangements for multiple goroutines
 * to work together harmoniously - they just do!
 * 
 * Here's an example of a naturally concurrent algorithm - I want to
 * merge multiple channels into one. Once all of the input channels are
 * exhausted I want to close the output channel.
 * 
 * It is just simpler to use concurrency - in fact it doesn't even look
 * like concurrency - it looks almost procedural.
 */
 
/*
  Multiplex a number of channels into one.
*/
func Mux(channels []chan big.Int) chan big.Int {
    // Count down as each channel closes. When hits zero - close ch.
    var wg sync.WaitGroup
    wg.Add(len(channels))
    // The channel to output to.
    ch := make(chan big.Int, len(channels))
 
    // Make one go per channel.
    for _, c := range channels {
        go func(c <-chan big.Int) {
            // Pump it.
            for x := range c {
                ch <- x
            }
            // It closed.
            wg.Done()
        }(c)
    }
    // Close the channel when the pumping is finished.
    go func() {
        // Wait for everyone to be done.
        wg.Wait()
        // Close.
        close(ch)
    }()
    return ch
}
 
/*
 * The only concession I have to make to concurrency here is to use a
 * sync.WaitGroup as a counter for concurrent counting.
 * 
 * Note that this is not purely my own work - I had a great deal of help
 * with this here (https:stackoverflow.com/q/19192377/823393).
 * 
 * [OldCurmudgeon] [so/q/19747950] [cc by-sa 3.0]
 */

Please check
https://github.com/chubin/cheat.sh for more information on installation and using its comprehensive features.

curl – use variables to show response times and other parameters

---

curl is a tool to interact with a server for transferring data. Although it supports various protocols, it is most commonly used with HTTP/S. It is sort of a browser for CLI folks and a go to tool when writing scripts to interact with servers.

---

In addition to transferring data, how do we show request and response parameters with curl. The answer is using variables, the complete list of variables can be found here.

Example – use “time_total” to show the total time, in seconds, that the full operation lasted.

1
2
$ curl  -w %{time_total} https://www.gcplinux.com
1.149143

It is best to add the variables in a file and use curl to reference the file for better formatting. Here I have added several http request and response variables I am interested in, such as num_connects, size_download, size_header, time_namelookup, time_pretransfer etc.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
daniel@hidmo:/tmp$ cat ccurl.txt 
      url_effective:  %{url_effective}\n
       content_type:  %{content_type}\n
          http_code:  %{http_code}\n
       http_version:  %{http_version}\n
       num_connects:  %{num_connects}\n
      num_redirects:  %{num_redirects}\n
          remote_ip:  %{remote_ip}\n
      size_download:  %{size_download}\n
        size_header:  %{size_header}\n
    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n
 
 
daniel@hidmo:/tmp$ curl -H 'Cache-Control: no-cache' -L -w "@ccurl.txt" -o /dev/null -s https://www.gcplinux.com
      url_effective:  https://gcplinux.com/
       content_type:  text/html; charset=UTF-8
          http_code:  200
       http_version:  1.1
       num_connects:  2
      num_redirects:  1
          remote_ip:  162.247.79.246
      size_download:  71273
        size_header:  537
    time_namelookup:  0.008585
       time_connect:  0.082511
    time_appconnect:  0.264110
   time_pretransfer:  0.264293
      time_redirect:  1.287257
 time_starttransfer:  3.077526
                    ----------
         time_total:  3.177939

As far as time related parameters, listed below are the ones you will most likely use –

• time_appconnect The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0)
• time_connect The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
• time_namelookup The time, in seconds, it took from the start until the name resolving was completed.
• time_pretransfer The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
• time_redirect The time, in seconds, it took for all redirection steps including name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3)
• time_starttransfer The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
• time_total The total time, in seconds, that the full operation lasted.

References –

---

https://curl.haxx.se/docs/manpage.html

https://stackoverflow.com/questions/18215389/how-do-i-measure-request-and-response-times-at-once-using-curl