Your scraper runs clean on test pages. Then you point it at a real target and everything falls apart.
The response comes back 403. Or you get an endless "Checking your browser" screen. Or a Turnstile widget you can't click.
That's Cloudflare. It sits in front of roughly 20% of the web, and if you scrape retail, travel, jobs, or finance sites, you hit it constantly.
Here's the good news, and the reason this guide exists: you can get past Cloudflare without a single paid API or managed service. Every method below is something you build and control yourself.
I'll show you 7 methods, ordered easiest to hardest, with working Python. I'll also tell you which popular tools are dead in 2026 so you don't waste a weekend on them.
How do you bypass Cloudflare?
Bypassing Cloudflare means matching every signal a real browser sends: a residential IP, a real TLS fingerprint, and a browser that runs JavaScript. Use curl_cffi for TLS-only checks, or a stealth browser like Nodriver for JavaScript challenges and Turnstile. Rotate residential proxies to clear IP reputation blocks.
The trap that catches most people: they fix one layer and assume the rest follows.
You can't. Cloudflare scores every request across several checks at once and combines them into one trust number.
A Chrome User-Agent riding on top of a Python TLS handshake fails instantly, because those two signals don't belong to the same client. Consistency across layers is the whole game.
What Cloudflare actually checks
Cloudflare Bot Management runs your request through five checks before your scraper ever sees HTML. Fail any one badly enough and you're blocked.
Here's the flow every request goes through:
flowchart LR
A[Your request] --> B[IP reputation]
B --> C[TLS / JA3 fingerprint]
C --> D[HTTP/2 + header order]
D --> E[JavaScript challenge]
E --> F[Behavior + Turnstile]
F -->|trust score OK| G[Origin HTML]
F -->|trust score low| H[403 / challenge loop]
IP reputation. Datacenter IP ranges are flagged on sight. Residential and mobile IPs carry far higher trust.
TLS fingerprint (JA3/JA4). The order of cipher suites and extensions in your TLS handshake identifies your client. Python's default requests has an unmistakable fingerprint that screams "not a browser", see how TLS fingerprinting works for the mechanics.
HTTP/2 and header order. Real browsers send headers in a specific order over HTTP/2. Scripts usually don't, and the mismatch is a giveaway.
JavaScript challenge. The "Checking your browser" page runs JS your client has to execute. No JS engine, no pass.
Behavior and Turnstile. Cloudflare watches timing, mouse movement, and navigation paths, then may drop a Turnstile CAPTCHA on anything that looks scripted.
Raw HTTP libraries fail the last three outright. That's why "just add a User-Agent" stopped working years ago.
Cloudflare also ships different protection levels. A blog on the free plan is trivial; a bank running Bot Management with Turnstile is a real fight. Match your effort to the target.
The 7 methods at a glance
Start at the top and only move down when you're still blocked. Simpler methods are faster, cheaper, and easier to maintain.
| # | Method | Difficulty | Cost | Best for | Beats |
|---|---|---|---|---|---|
| 1 | curl_cffi TLS impersonation | Easy | Free | High-volume, no JS needed | TLS + header checks |
| 2 | Complete header set | Easy | Free | Light protection | Basic header checks |
| 3 | Residential proxy rotation | Easy | $ | IP reputation blocks | 1020 / IP bans |
| 4 | Nodriver stealth browser | Medium | Free | JS challenges, SPAs | JS + fingerprint |
| 5 | SeleniumBase UC Mode | Medium | Free | Turnstile CAPTCHAs | Turnstile |
| 6 | Camoufox (Firefox) | Hard | Free | Aggressive fingerprinting | Deep fingerprint checks |
| 7 | Session reuse + pacing | Medium | Free | Scaling any of the above | Rate limits, re-challenges |
Quick recommendation: try Method 1 first. It clears more sites than you'd expect, and it's a hundred times cheaper than a browser. Escalate to a stealth browser (Methods 4–6) only when JavaScript is in the way.
Basic methods (start here)
Method 1: Match your TLS fingerprint with curl_cffi
Difficulty: Easy
Cost: Free
Beats: TLS and header fingerprinting
Most scrapers die at the TLS handshake, before headers even matter. curl_cffi fixes that by borrowing a real browser's fingerprint.
It swaps Python's OpenSSL for Chrome's BoringSSL, so your JA3/JA4 signature matches an actual Chrome build. The curl_cffi package on PyPI is a near drop-in for requests.
Install it, then impersonate a browser:
from curl_cffi import requests
# impersonate="chrome" picks the latest supported Chrome fingerprint
r = requests.get(
"https://example.com",
impersonate="chrome", # matches TLS, ciphers, and header order to Chrome
timeout=15,
)
print(r.status_code)
print(r.text[:500])
The impersonate argument is the whole trick. It aligns your TLS fingerprint, cipher order, and HTTP/2 settings with a real Chrome so the handshake looks legitimate.
The catch: curl_cffi isn't a browser and can't run JavaScript. If the site throws a JS challenge or Turnstile, this alone won't clear it.
Pair it with residential proxies (Method 3) for TLS-only targets, or jump to a stealth browser when JS is involved.
Method 2: Send a complete, consistent header set
Difficulty: Easy
Cost: Free
Beats: basic header checks
Even with a good TLS fingerprint, a thin header set gives you away. Real browsers send a dozen headers in a consistent order; scripts often send three.
The fix is to send the full set a real Chrome sends, in the right order:
from curl_cffi import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/126.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Upgrade-Insecure-Requests": "1",
}
r = requests.get("https://example.com", headers=headers, impersonate="chrome")
The Sec-Fetch-* and Accept-* headers matter more than the User-Agent everyone obsesses over. Cloudflare cross-checks them against your TLS fingerprint.
One rule: keep every signal telling the same story. If you claim Chrome 126 in the User-Agent, impersonate Chrome in curl_cffi too. A mismatch is worse than a plain default.
Intermediate methods
Method 3: Rotate residential proxies
1020 error)Difficulty: Easy
Cost: Paid proxies
Beats: IP bans and reputation scoring
You can have a perfect fingerprint and still get blocked on IP alone. Cloudflare scores datacenter ranges harshly, and once an IP is flagged, nothing else you do matters.
Residential and mobile IPs solve this because they belong to real ISPs with real trust. Rotating them spreads your requests so no single IP looks like a bot.
Route your requests through a rotating proxy pool:
from curl_cffi import requests
# One endpoint that rotates the exit IP on every request
proxies = {
"http": "http://user:[email protected]:8000",
"https": "http://user:[email protected]:8000",
}
r = requests.get(
"https://example.com",
impersonate="chrome",
proxies=proxies, # every call exits from a fresh residential IP
timeout=20,
)
print(r.status_code)
A rotating gateway hands you a new IP per request, so you don't manage the pool yourself. If you'd rather build the rotation in code, the guide on proxy rotation in Python walks through it.
For heavy targets, residential proxies for web scraping hold up far better than datacenter IPs. This is the one spot where Roundproxies genuinely fits the job.
Watch out for one thing: rotating the IP on every single request breaks sessions. For flows that need a stable identity (login, cart, pagination), pin a "sticky" IP for the whole session instead of rotating mid-flow.
Method 4: Nodriver for JavaScript challenges
Difficulty: Medium
Cost: Free
Beats: JavaScript execution and fingerprint checks
When Cloudflare demands JavaScript, no HTTP library will save you. You need a real browser that doesn't announce itself as automated.
Nodriver is the 2026 default for this. It's the successor to undetected-chromedriver, from the same author, and it patches navigator.webdriver and CDP leaks at the driver level. The Nodriver project on GitHub is actively maintained against new detection.
It runs async and needs no separate WebDriver binary:
import nodriver as uc
async def main():
browser = await uc.start(headless=False) # headed mode passes more checks
page = await browser.get("https://example.com")
await page.sleep(5) # let the JS challenge resolve
html = await page.get_content()
print(html[:500])
browser.stop()
uc.loop().run_until_complete(main())
The sleep after get is deliberate. Cloudflare's JS challenge needs a couple of seconds to run and set the cf_clearance cookie; grab the HTML too early and you scrape the challenge page instead of your data.
Run it headed (headless=False) whenever you can. Headless Chrome leaks signals that headed Chrome doesn't, and on Linux servers you can fake a display with xvfb to stay headed without a monitor.
Method 5: SeleniumBase UC Mode for Turnstile
Difficulty: Medium
Cost: Free
Beats: Cloudflare Turnstile
Turnstile is Cloudflare's CAPTCHA replacement. It usually runs invisibly, but when it shows a checkbox, a plain stealth browser stalls.
SeleniumBase UC Mode handles it. It launches Chrome before attaching the driver, renames the console variables anti-bots scan for, and disconnects the driver during sensitive actions like page loads and clicks.
Its uc_gui_click_captcha() helper clicks the Turnstile checkbox for you:
from seleniumbase import SB
with SB(uc=True, headless=False) as sb:
# reconnect_time lets the challenge load before the driver reattaches
sb.uc_open_with_reconnect("https://example.com", reconnect_time=4)
sb.uc_gui_click_captcha() # clicks the Turnstile widget if present
html = sb.get_page_source()
print(html[:500])
uc_open_with_reconnect is what makes this work: it detaches the driver while the page loads so Cloudflare sees a clean browser, then reconnects to let you read the DOM.
SeleniumBase trades a little stealth for a lot of stability, which is why I reach for it in production over raw Nodriver.
If you already scrape with Selenium, the deeper walkthrough on bypassing Cloudflare with Selenium covers the same UC Mode setup end to end.
Advanced methods
Method 6: Camoufox for the hardest fingerprinting
Difficulty: Hard
Cost: Free
Beats: deep browser fingerprint checks
Sometimes every Chrome-based tool gets flagged because the target fingerprints Chromium itself. Switching engines changes the whole picture.
Camoufox is a hardened Firefox build made for stealth. It spoofs fingerprints at the browser level rather than patching them from the outside, so the leaks that betray automated Chrome simply aren't there.
It uses a Playwright-style API:
from camoufox.sync_api import Camoufox
# os= and locale= are randomized to a coherent, real-looking profile
with Camoufox(headless=False, os="windows", locale="en-US") as browser:
page = browser.new_page()
page.goto("https://example.com")
page.wait_for_timeout(5000) # allow challenge + render
print(page.content()[:500])
The os and locale arguments keep the fingerprint internally consistent — a Windows profile that also reports a US locale and a matching timezone, not a random mix that stands out.
Camoufox is heavier and slower to set up than Nodriver, so it's a fallback, not a default. Reach for it only when Chrome-based methods keep failing on a specific target.
Method 7: Reuse sessions and pace your requests
Difficulty: Medium
Cost: Free
Beats: rate limits and repeat challenges
Passing the challenge once is easy. Staying past it at scale is where scrapers quietly die.
Cloudflare hands you a cf_clearance cookie after you clear a challenge. It's valid from about 15 minutes to a few hours, tied to your IP and fingerprint. Solve the challenge once with a browser, then reuse that cookie with fast HTTP requests.
Pull the cookie from a Nodriver session and hand it to curl_cffi:
from curl_cffi import requests
# cf_clearance + user_agent captured from a solved browser session
cookies = {"cf_clearance": "PASTE_TOKEN_FROM_BROWSER"}
headers = {"User-Agent": "PASTE_THE_SAME_UA_THE_BROWSER_USED"}
r = requests.get(
"https://example.com/page-2",
impersonate="chrome",
cookies=cookies,
headers=headers,
timeout=15,
)
print(r.status_code) # 200 if the clearance cookie is still valid
The cookie only works from the same IP and User-Agent that earned it. Change either one and Cloudflare voids it, so keep the browser and the follow-up requests on the same sticky proxy.
Then pace yourself. Add randomized delays of a few seconds between requests, and don't hammer resources a human would never load in that order. Good timing prevents challenges better than any tool clears them.
After running rotation against tens of thousands of pages, the thing nobody tells you: 80% of blocks come from pacing and IP reputation, not fingerprinting. Fix those two first and half your "detection" problems vanish.
Which method should you use?
Don't reach for a browser when a header fix will do. Match the method to what the target actually checks.
| Your situation | Start with |
|---|---|
| Page loads fine in view-source, no JS needed | Method 1 + 2 |
Blocked on IP even with good fingerprint (1020) |
Method 3 |
| "Checking your browser" / JS challenge | Method 4 |
| Turnstile checkbox appears | Method 5 |
| Everything Chrome-based gets flagged | Method 6 |
| Thousands of pages, getting re-challenged | Method 7 |
A simple decision path:
Does the data appear without JavaScript?
├── Yes → curl_cffi + full headers (Method 1–2)
│ └── Still blocked? → add residential proxies (Method 3)
└── No → stealth browser (Method 4)
├── Turnstile appears? → SeleniumBase UC Mode (Method 5)
└── Still fingerprinted? → Camoufox (Method 6)
└── Scaling up? → reuse sessions (Method 7)
Two tools you'll see recommended elsewhere are dead weight in 2026. puppeteer-extra-stealth was deprecated and Cloudflare detects it on sight. FlareSolverr has stalled on maintenance and gets flagged too. Skip both and use the maintained tools above.
Cloudflare error codes and how to fix them
Cloudflare tells you exactly why it blocked you if you read the code. Most people don't, and they fix the wrong layer.
Here's the translation, matched to the method that actually solves each one. The full list lives in Cloudflare's official 1xxx error docs.
| Code | What it means | The real fix |
|---|---|---|
1020 |
Access denied by a firewall rule, usually IP reputation | Residential proxies (Method 3) |
1010 |
Your browser signature was flagged as automation | Stealth browser (Method 4) |
1015 |
You're rate limited | Slow down, add backoff (Method 7) |
1012 / 1009 |
IP or country banned by the site | Rotate to a clean residential IP |
403 (no code) |
Failed the composite bot score | Fix TLS + headers first (Method 1–2) |
"403 Forbidden on every request"
Your TLS fingerprint or headers are the tell. Switch to curl_cffi with impersonate and send a full header set before touching anything heavier.
"Endless 'Checking your browser' loop"
Your client can't run the challenge JavaScript, or it's grabbing the page before the challenge resolves. Move to Nodriver and add a sleep after the page load.
"It worked, then started failing after a while"
Your cf_clearance cookie expired, or you rotated to a new IP and voided it. Re-solve with the browser and keep follow-up requests on the same sticky IP.
"Turnstile checkbox won't clear"
You're running headless. Turnstile watches for a real render — switch to headed mode (use xvfb on a server) and call uc_gui_click_captcha().
Frequently asked questions
Can I bypass Cloudflare with headers and a User-Agent only?
Usually not. Modern Bot Management correlates your TLS fingerprint, IP reputation, and JavaScript behavior with your headers. A Chrome User-Agent sitting on a Python TLS stack gets flagged immediately, so headers alone rarely clear anything past the lightest protection.
Does Cloudflare detect headless browsers like Playwright?
Yes. Vanilla Playwright and Selenium leak automation signals that Cloudflare fingerprints in seconds. You need a patched build — Nodriver or SeleniumBase UC Mode — plus residential proxies and headed mode to bypass Cloudflare on protected sites.
How long does the cf_clearance cookie last?
Typically 15 minutes to a few hours, depending on the site's config and how human your behavior looks. It's tied to your IP and User-Agent, so reusing it only works while both stay identical. Re-solve the challenge when it expires.
Do residential proxies really help against Cloudflare?
Significantly. Cloudflare gives datacenter IP ranges much lower trust and targets them for extra scrutiny. Residential and mobile IPs carry ISP-level trust, which clears most 1020 reputation blocks that fingerprint fixes can't touch.
Is it legal to bypass Cloudflare?
It depends on what you scrape and where. Accessing public data is generally allowed in many places, but violating a site's Terms of Service, the CFAA in the US, or GDPR in the EU can carry real consequences. Get legal advice before anything commercial.
A note on responsible use
Bypassing a bot wall is a technical exercise, but it isn't consequence-free. A few ground rules keep you on the right side of it.
Scrape public data for legitimate reasons. Respect robots.txt and rate limits even when you can blow past them, hammering a site is how you get IPs banned and lawyers involved.
Terms of Service, the CFAA in the US, and GDPR in the EU can all apply depending on what you collect and where.
If a site offers an API, use it; it's faster and it's allowed. For anything commercial, talk to a lawyer, not a blog.
Where these methods genuinely don't help: they won't get you into anything behind a login you're not authorized for, and they won't make illegal data collection legal. The tools are neutral; how you point them isn't.
Wrapping up
The one idea to keep: Cloudflare scores every layer at once, so bypassing it means matching every layer at once. One clean signal on top of five dirty ones still fails.
Start cheap and escalate only when forced. curl_cffi with good headers clears a surprising number of sites for almost no cost. Add residential proxies for IP blocks, and reach for a stealth browser only when JavaScript stands in the way.
Go build the smallest version first: two residential IPs, curl_cffi with impersonate="chrome", and a retry loop. That handles most targets. Save Nodriver and Camoufox for the ones that fight back.
Quick reference: TLS-only site → Method 1. IP block → Method 3. JS challenge → Method 4. Turnstile → Method 5. Scaling → Method 7.