TLS Web Scraping

httpcloak tutorial: Bypass TLS Fingerprinting (2026)

Your scraper works perfectly on test sites. Then you hit Cloudflare and get blocked instantly, even with a spoofed User-Agent.

The problem isn't your headers. It's your TLS fingerprint.

httpcloak is a Go HTTP client that produces browser-identical TLS fingerprints. Anti-bot systems that inspect your connection see Chrome 143, not Go's standard library or Python's requests.

This guide covers installing httpcloak, bypassing TLS fingerprinting, handling sessions and proxies, and the mistakes that get scrapers detected.

What is httpcloak? (TL;DR)

httpcloak is an HTTP client library that matches real browser fingerprints at the TLS, HTTP/2, and HTTP/3 protocol levels. It produces JA3/JA4 hashes identical to Chrome, Firefox, or Safari, so your requests look like real browser traffic.

The library works in Go (native), Python, Node.js, and C#.

Cloudflare, Akamai, and PerimeterX fingerprint your TLS handshake before you send a single HTTP header. If your fingerprint screams "bot," you're blocked. httpcloak fixes this at the protocol level.

httpcloak vs curl-impersonate vs tls-client: Which Should You Use?

Let's compare httpcloak to the alternatives so you can decide if it's the right tool for your project.

Feature httpcloak curl-impersonate tls-client requests
TLS Fingerprint (JA3/JA4) Yes (Chrome, Firefox, Safari) Yes (Chrome, Firefox) Yes (Chrome) No (Detectable)
HTTP/2 Fingerprint Yes (Full match) Yes (Full match) Yes (Full match) No
HTTP/3 (QUIC) Yes No No No
Post-Quantum TLS Yes (X25519MLKEM768) No No No
ECH (Encrypted SNI) Yes No No No
Native Python API Yes (requests-like) Subprocess only Yes Yes (Native)
Connection Pooling Yes (Automatic) No (Manual) Yes Yes
Session Persistence Yes (Built-in) No (Manual) Yes (Built-in) Yes (Built-in)
Proxy Support HTTP, SOCKS5, MASQUE HTTP, SOCKS5 HTTP, SOCKS5 HTTP, SOCKS5
Speed (req/sec) ~850 ~400 ~600 ~1200 (but blocked)

When to Use Each Tool

Use httpcloak when:

  • You need HTTP/3 fingerprinting (many CDNs now check this)
  • The site uses post-quantum TLS (Chrome 131+ default)
  • You need ECH support for sites requiring encrypted SNI
  • You want a clean Python/Node.js/Go API without subprocess calls

Use curl-impersonate when:

  • You're already using curl in shell scripts
  • You need maximum compatibility with existing tooling
  • HTTP/2 fingerprinting is sufficient for your targets

Use tls-client when:

  • You're working exclusively in Python
  • HTTP/2 is enough (no HTTP/3 needed)
  • You want the simplest possible API

Use standard requests when:

  • You're scraping sites without bot detection
  • Speed matters more than stealth
  • You're hitting APIs that don't fingerprint connections

Performance Benchmarks: httpcloak vs Alternatives

I ran benchmarks against a Cloudflare-protected test endpoint. Here's what I found:

Test Setup

  • Target: Cloudflare Enterprise site with JS challenge disabled
  • Requests: 1,000 sequential GET requests
  • Proxy: Same residential IP for all tests
  • Metric: Successful responses (not blocked)

Results

Library Success Rate Avg Response Time Blocked After
httpcloak (chrome-143) 100% 245ms Never
curl-impersonate 98.7% 312ms ~800 requests
tls-client 96.2% 287ms ~500 requests
Python requests 0% N/A Immediately

Key Findings

  1. httpcloak was the only library with 0 blocks in this test. HTTP/3 and post-quantum TLS support likely explain the gap for modern Cloudflare deployments.
  2. curl-impersonate started getting intermittent 403s around request 800, likely due to missing HTTP/3 fingerprint rotation.
  3. tls-client performed well initially but degraded faster, possibly because it lacks the newest Chrome fingerprint presets.
  4. Python requests was blocked on the first request. The TLS fingerprint is trivially identified by any modern WAF.

Step 1: Install httpcloak

Choose your language:

Python Installation

pip install httpcloak

The Python bindings provide a requests-compatible API. If you're familiar with requests, the API is nearly identical.

Go Installation

go get github.com/sardanioss/httpcloak

This adds httpcloak to your go.mod and downloads all dependencies.

Node.js Installation

npm install httpcloak

Both sync and async methods are available.

C# Installation

dotnet add package HttpCloak

Step 2: Make Your First Request (Verify It Works)

Let's hit Cloudflare's trace endpoint. This returns your connection details, including the TLS version and key exchange algorithm.

Python Example

import httpcloak

r = httpcloak.get("https://www.cloudflare.com/cdn-cgi/trace")

print(f"Status: {r.status_code}")
print(f"Protocol: {r.protocol}")
print(r.text)

Go Example

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/sardanioss/httpcloak/client"
)

func main() {
    c := client.NewClient("chrome-143")
    defer c.Close()

    resp, err := c.Get(context.Background(), "https://www.cloudflare.com/cdn-cgi/trace", nil)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Status: %d\n", resp.StatusCode)
    fmt.Printf("Protocol: %s\n", resp.Protocol)
    fmt.Println(resp.Text())
}

What You Should See

fl=283f39
h=www.cloudflare.com
ip=xxx.xxx.xxx.xxx
ts=1767716387.683
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36...
colo=LAX
http=http/3
tls=TLSv1.3
kex=X25519MLKEM768

Critical lines to check:

  • http=http/3 - Confirms HTTP/3 negotiation
  • kex=X25519MLKEM768 - Confirms post-quantum key exchange

If you see kex=X25519 (without MLKEM768), you're using an older fingerprint preset. Update to chrome-143.

Step 3: POST Requests with JSON

Most scraping involves form submissions or API calls. Here's how to POST JSON:

Python

import httpcloak

r = httpcloak.post("https://api.example.com/login", json={
    "username": "demo",
    "password": "secret123"
})

print(r.status_code)
print(r.json())

Go

body := []byte(`{"username": "demo", "password": "secret123"}`)

resp, err := c.Do(context.Background(), &client.Request{
    Method:  "POST",
    URL:     "https://api.example.com/login",
    Body:    body,
    Headers: map[string]string{
        "Content-Type": "application/json",
    },
})

When scraping authenticated content, cookies must persist between requests.

Python Session

import httpcloak

with httpcloak.Session(preset="chrome-143") as session:
    # Login - cookies saved automatically
    session.post("https://example.com/login", json={
        "user": "myuser",
        "pass": "mypass"
    })
    
    # Subsequent requests include session cookies
    r = session.get("https://example.com/dashboard")
    print(r.json())

Go Session

session := client.NewSession("chrome-143")
defer session.Close()

ctx := context.Background()

// Login
session.Post(ctx, "https://example.com/login",
    []byte(`{"user":"myuser","pass":"mypass"}`),
    map[string]string{"Content-Type": "application/json"})

// Authenticated request
resp, _ := session.Get(ctx, "https://example.com/dashboard", nil)
fmt.Println(resp.Text())

Pro Tip: Session Warming

Cloudflare sites share TLS infrastructure. Warming up your session on any Cloudflare site helps with subsequent requests:

session = httpcloak.Session(preset="chrome-143")

# Warm up on a low-security Cloudflare site
session.get("https://cloudflare.com/")

# Now hit your actual target
r = session.get("https://protected-target.com/")

Step 5: Proxy Configuration

For large-scale scraping, you need rotating proxies. httpcloak supports HTTP, SOCKS5, and MASQUE proxies.

Python with Proxy

import httpcloak

httpcloak.configure(
    preset="chrome-143",
    proxy="http://user:[email protected]:8080",
    timeout=30
)

r = httpcloak.get("https://target-site.com/data")

Go with Proxy

c := client.NewClient("chrome-143",
    client.WithProxy("http://user:[email protected]:8080"),
    client.WithTimeout(30*time.Second),
)
defer c.Close()

HTTP/3 Proxy Gotcha

HTTP proxies can't tunnel HTTP/3 traffic because HTTP/3 uses QUIC (UDP), and HTTP proxies only handle TCP.

Options:

  1. Use SOCKS5 proxies - They support UDP and work with HTTP/3
  2. Use MASQUE proxies - RFC 9298 tunnels UDP over HTTP/3 (premium providers only)
  3. Force HTTP/2 - client.WithForceHTTP2() if your proxy doesn't support UDP
# SOCKS5 with UDP support (works with HTTP/3)
session = httpcloak.Session(proxy="socks5://user:pass@proxy:1080")

# MASQUE (if your provider supports it)
session = httpcloak.Session(proxy="masque://proxy:443")

Step 6: Available Browser Presets

Preset Browser Post-Quantum HTTP/2 HTTP/3
chrome-143 Chrome 143 X25519MLKEM768 Yes Yes
chrome-143-windows Chrome 143 (Windows) X25519MLKEM768 Yes Yes
chrome-143-linux Chrome 143 (Linux) X25519MLKEM768 Yes Yes
chrome-143-macos Chrome 143 (macOS) X25519MLKEM768 Yes Yes
chrome-131 Chrome 131 X25519MLKEM768 Yes Yes
firefox-133 Firefox 133 X25519 Yes No
safari-18 Safari 18 X25519 Yes No
Recommendation: Use chrome-143 unless you specifically need Firefox or Safari fingerprints. Chrome has the widest compatibility.

Troubleshooting: Common Errors and Fixes

Error: "connection refused" or "dial tcp: connection timed out"

Cause: Network issue or blocked IP.

Fix:

# Add retry logic
session = httpcloak.Session(
    preset="chrome-143",
    retry=3,
    retry_on_status=[429, 500, 502, 503, 504]
)

Error: "tls: handshake failure"

Cause: The site doesn't support the cipher suites httpcloak is offering.

Fix: Try a different preset or force HTTP/2:

r = httpcloak.get(url, preset="firefox-133")  # Different cipher preference

Error: Still getting 403 Forbidden with httpcloak

Cause: The site is checking more than just TLS fingerprint. Common culprits:

  • Missing or mismatched cookies
  • JavaScript challenge not solved
  • IP reputation (datacenter IPs get flagged)
  • Behavioral analysis (too fast, too regular)

Fix:

  1. Use residential proxies instead of datacenter
  2. Add random delays between requests
  3. Warm up your session (see Step 4)
  4. Check if the site requires JS rendering (use browser automation instead)

Error: "module 'httpcloak' has no attribute 'get'"

Cause: Wrong package installed or import conflict.

Fix:

pip uninstall httpcloak
pip install httpcloak --no-cache-dir

Error: Response shows kex=X25519 instead of kex=X25519MLKEM768

Cause: Using an outdated preset that doesn't include post-quantum TLS.

Fix: Update to chrome-143 or newer:

r = httpcloak.get(url, preset="chrome-143")

Common Pitfalls to Avoid

Pitfall 1: Forgetting to Close Clients

Every client holds network connections. Failing to close causes resource leaks.

Bad:

c := client.NewClient("chrome-143")
resp, _ := c.Get(ctx, url, nil)
// Connection never closed — leaks resources

Good:

c := client.NewClient("chrome-143")
defer c.Close()
resp, _ := c.Get(ctx, url, nil)

Pitfall 2: Mismatched User-Agent and Fingerprint

If you override the User-Agent but it doesn't match the fingerprint preset, detection systems notice the inconsistency.

Bad:

# Chrome fingerprint with Firefox User-Agent — instant red flag
r = httpcloak.get(url, 
    preset="chrome-143",
    headers={"User-Agent": "Mozilla/5.0 Firefox/120.0"}
)

Good: Let httpcloak set the User-Agent automatically.

Pitfall 3: Using HTTP Proxies for HTTP/3 Traffic

HTTP proxies tunnel TCP only. HTTP/3 uses UDP.

Solution: Use SOCKS5 proxies or force HTTP/2 when using HTTP proxies.

When to Use httpcloak vs Browser Automation

Scenario httpcloak Playwright/Puppeteer
High-volume scraping (1000s req/min) Best choice Too slow
No JavaScript required Best choice Overkill
JS rendering required Not suitable Best choice
CAPTCHA solving needed Not suitable Best choice
SPAs with client-side routing Not suitable Best choice
API scraping Best choice Overkill
Form submissions Works well Works well

Best strategy: Use httpcloak for initial reconnaissance and API endpoints. Fall back to browser automation only when JavaScript rendering is required.

Integrating httpcloak with Proxy Rotation

For production scraping, combine httpcloak with rotating proxies. Here's a pattern that works:

import httpcloak
import random

PROXIES = [
    "http://user:[email protected]:8080",
    "http://user:[email protected]:8080",
    "http://user:[email protected]:8080",
]

def scrape_with_rotation(urls):
    results = []
    
    for url in urls:
        proxy = random.choice(PROXIES)
        
        try:
            r = httpcloak.get(url, 
                preset="chrome-143",
                proxy=proxy,
                timeout=30
            )
            results.append(r.json())
        except Exception as e:
            print(f"Failed {url}: {e}")
            continue
    
    return results

httpcloak FAQ

Does httpcloak work with Cloudflare-protected sites?

It clears the transport-layer checks. httpcloak emits browser-identical JA3/JA4 and HTTP/2 fingerprints, so a request no longer looks like a Go or Python client at the handshake. Cloudflare stacks other signals on top of that: IP reputation, request rate, cookie state, and interactive JavaScript challenges. A managed challenge page still needs a real browser or a solving service, and a datacenter IP with a bad reputation will get blocked no matter how clean the fingerprint is.

Why does my JA3 hash change on every request?

GREASE. Chrome inserts random placeholder values into the ClientHello, and httpcloak reproduces that behavior, so the JA3 hash differs run to run by design. A fixed JA3 across requests would itself be a tell. JA4 and the HTTP/2 (Akamai) fingerprint stay constant, so compare those when you verify your setup against a fingerprint-echo endpoint. Matching a JA3 string byte for byte is the wrong test.

Which browser preset should I use?

Match the preset to the traffic you want to blend into, which for most targets means current Chrome on desktop. Presets age: recent coverage uses chrome-146 as the current Chrome fingerprint, so the version numbers in the table earlier in this post will drift as browsers ship. Pin chrome-latest if you want the maintainer's tracking of the newest build, or check the repository for the current preset list before hardcoding a version. Running a preset from a browser release that has already been phased out is its own anomaly.

How often are presets updated?

The maintainer ships new presets when a browser release changes its fingerprint, which in Chrome's case happens often enough that a pinned version goes stale within a few release cycles. Watch the GitHub repository, and use describe_preset to dump the full spec of whatever preset you are running so you can see exactly what is being sent.

Is matching TLS enough on its own?

No. A server can fingerprint a request at several layers before it reads a single byte of your payload: the TLS ClientHello, the HTTP/2 SETTINGS frame, QUIC transport parameters, and TCP options such as TTL, MSS, and window size. Clients that only patch TLS still stand out everywhere else, and detectors that read more than one layer flag the mismatch between them. httpcloak borrows a real browser's values across the whole stack instead of one layer.

Why does HTTP/3 matter for fingerprinting?

HTTP/3 runs over QUIC, which carries its own fingerprintable surface in the transport parameters and GREASE frames, separate from TLS. A site that negotiates HTTP/3 can profile the QUIC connection independently. httpcloak matches Chrome's QUIC fingerprint when HTTP/3 is in use, and it can switch protocol mid-session so a connection can move between HTTP/1.1, HTTP/2, and HTTP/3 and re-handshake on the new transport.

Does httpcloak run JavaScript?

No. It is an HTTP client, so a client-rendered page comes back as an empty shell with the data still sitting behind a script that never executes. Check whether the content you want is in the raw HTML before you build around httpcloak; if it is not, you need browser automation or a rendering service, and httpcloak solves only the fingerprint half of the problem.

Can httpcloak solve CAPTCHAs?

No, for the same reason. CAPTCHAs need a rendered page and interaction, which means Playwright, Puppeteer, or a solving service. httpcloak reduces how often you get served one, since a clean fingerprint keeps you out of the challenge path on many sites, but it cannot answer a challenge once it appears.

Can I use httpcloak with existing Go HTTP middleware?

Not directly. httpcloak exposes its own client interface rather than wrapping net/http, because the fingerprint control lives below the level the standard library exposes. Middleware built around http.RoundTripper has to be adapted to httpcloak's request and response types. Budget for that when retrofitting an existing codebase rather than a greenfield scraper.

Does httpcloak support connection pooling and sessions?

Yes. Sessions pool and reuse connections to the same host, which matters because a fresh handshake per request is both slower and a behavioral signal in itself. Beyond pooling, a session can drop live connections while keeping TLS session tickets the way a browser tab does on reload, persist tickets and cookies to disk so a run resumes across processes, and perform a multi-hop warmup request before the real one to pre-populate cookies and session state.

Is httpcloak faster than curl-impersonate?

In my benchmarks, httpcloak averaged 245ms per request against 312ms for curl-impersonate. Native connection pooling and the absence of subprocess overhead account for the gap. The difference compounds on large jobs, but on a few hundred requests either option is fine and you should pick on ergonomics instead.

Can I use httpcloak outside Go?

Yes. The Go core is exposed to Python, Node.js, and .NET through a shared library, and the API is the same across all four. Install with pip install httpcloak, npm install httpcloak, go get github.com/sardanioss/httpcloak, or dotnet add package HttpCloak.

Do I still need proxies?

Fingerprinting and IP reputation are separate checks, and httpcloak only fixes the first. If your target rate-limits per IP or blocks datacenter ranges outright, you still need rotation. httpcloak proxies through HTTP CONNECT, SOCKS5, and MASQUE, so the proxy layer sits under the same session that carries your fingerprint.

Final Thoughts

httpcloak solves a specific problem: getting blocked despite having "perfect" headers. If your TLS fingerprint exposes you as a bot, no amount of header spoofing will help.

Start here:

  1. Install httpcloak (pip install httpcloak)
  2. Hit Cloudflare's trace endpoint
  3. Verify you see kex=X25519MLKEM768 and http=http/3

If those two values appear, you're mimicking Chrome 143 at the protocol level. That's as stealthy as an HTTP client can get without running a full browser.

For sites requiring JavaScript rendering, use httpcloak for API endpoints and fall back to browser automation for rendered pages. That hybrid approach gives you speed where you can afford it and stealth where you need it.