Advanced curl for Debugging: A Practical Guide

When a service is down or an API is misbehaving, curl is often the first tool developers and SREs reach for. It's the Swiss Army knife for anything HTTP. But most usage stops at curl example.com. To truly diagnose complex issues, you need to go deeper.

This guide provides a set of advanced curl patterns for debugging performance bottlenecks, SSL/TLS failures, API authentication issues, and more.

Analyzing Performance with Timing Breakdowns

Is your website slow? Is it DNS, the network connection, or the server processing time? The curl -w or --write-out flag lets you pinpoint the bottleneck by providing detailed timing metrics for a request's lifecycle.

The -w flag accepts a format string with variables that curl will replace upon completion. To get a comprehensive breakdown, combine it with -s (silent mode) and -o /dev/null (discarding the body) so you only see the timing data.

curl -s -o /dev/null -w "\
dns_lookup:     %{time_namelookup}s\n\
tcp_connect:      %{time_connect}s\n\
tls_handshake:    %{time_appconnect}s\n\
pretransfer:      %{time_pretransfer}s\n\
start_transfer:   %{time_starttransfer}s\n\
--------------------------\n\
total_time:       %{time_total}s\n" https://api.serverinspector.com/healthz

This command will output something like:

dns_lookup:     0.001234s
tcp_connect:      0.034567s
tls_handshake:    0.078910s
pretransfer:      0.079000s
start_transfer:   0.123456s
--------------------------
total_time:       0.123500s

Key Timing Metrics Explained

By analyzing these metrics, you can quickly determine if a slowdown is caused by your network, your DNS provider, your TLS setup, or your application backend.

Inspecting Headers, Redirects, and Authentication

Status codes don't tell the whole story. Response headers, redirect chains, and authentication flows are critical for debugging application logic.

Viewing Response Headers

To see response headers along with the body, use -i (--include). To see only the headers (using a HEAD request), use -I (--head).

# Get headers only for a URL
curl -I https://google.com

# Output shows the 301 Moved Permanently redirect
# HTTP/2 301
# location: https://www.google.com/
# ...

For a more detailed view of the entire transaction, including request headers and the full TLS handshake, use -v (--verbose).

# Verbose output for debugging an API endpoint
curl -v https://api.github.com/zen

Following Redirects

By default, curl does not follow redirects. If you request a URL that returns a 3xx status code, curl will simply output the response and stop. To make it automatically follow the Location header, use the -L (--location) flag.

Combining -vL is a powerful way to debug a redirect chain or loop. You'll see each request and its 3xx response until you reach the final destination.

Debugging SSL/TLS Handshake Issues

SSL/TLS errors are common and disruptive. curl provides powerful options for diagnosing them.

Forcing a Specific Server IP with --resolve

The --resolve flag is one of curl's most powerful features. It allows you to bypass DNS and force curl to connect to a specific IP address for a given hostname and port.

Syntax: curl --resolve <HOST>:<PORT>:<IP_ADDRESS> <URL>

This is incredibly useful for:

1. Testing a new server before a DNS cutover: Point your domain to the new server's IP to ensure it's configured correctly.
2. Bypassing a CDN/Load Balancer: Connect directly to an origin server to rule out issues with the proxy layer.
3. Reproducing location-specific issues: If a user in a specific region is having trouble, you can test against the same server IP they are resolving to.

# Test the website on a new server at 198.51.100.25 before changing DNS
curl -vI --resolve example.com:443:198.51.100.25 https://example.com/

The verbose (-v) output will show you the full TLS handshake with the specified IP, allowing you to verify the correct certificate is being served.

Other SSL/TLS Debugging Flags

• --tlsv1.2: Force the use of TLS version 1.2. Useful for checking if your server supports modern security protocols.
• -k or --insecure: Ignore certificate validation. Use with caution! This is only for debugging environments with self-signed certificates, never in production scripts.

Manually running these curl commands is great for one-off debugging, but to ensure your services are consistently working, a dedicated monitoring tool is essential. ServerInspector can automate advanced checks, such as connecting to specific IPs, sending custom headers, and asserting on response bodies from multiple regions, alerting you the moment a problem occurs.

curl Debugging Checklist

Here is a quick reference for mapping common problems to curl commands.

Frequently Asked Questions

Q1: How can I see the exact data being sent and received, including request bodies? A1: Use the --trace-ascii <file> flag. To dump the trace directly to your terminal, use a hyphen for the filename: curl --trace-ascii - -d '{"key":"value"}' -H "Content-Type: application/json" https://httpbin.org/post. This shows everything, including headers and data, in plain text.

Q2: What's the difference between curl -i and curl -I? A2: curl -i (--include) includes the response headers in the output along with the body. It uses the method you specify (e.g., GET, POST). curl -I (--head) performs a HEAD request to fetch only the headers. This is faster as no body is transferred, but not all servers properly support HEAD requests.

Q3: How do I test an API that requires a cookie for authentication? A3: You can pass cookies using the -b or --cookie flag. You can provide them as a string (-b "session=abc123") or read them from a file (-b cookies.txt).

Q4: Can I use curl to submit a form? A4: Yes. Use the -d or --data flag for application/x-www-form-urlencoded data (standard form posts), like -d "name=daniel&project=curl". For multipart/form-data, typically used for file uploads, use the -F or --form flag, like -F "file=@/path/to/local/file".