Getting blocked by websites while trying to scrape data or automate tasks? You're not alone. Anti-bot systems have become ruthless in 2026, using multi-layered defenses that analyze everything from your TLS handshake to your mouse movements.
This guide shows you 8 proven methods to bypass bot detection using free, open-source tools. No expensive APIs. No black-box solutions. Just working code you can implement today.
Whether you're building scrapers for price monitoring, lead generation, or research, these techniques will help you bypass bot detection systems that block standard automation tools.
What is Bot Detection and How Does It Work?
Bot detection identifies and blocks automated traffic from accessing websites. Modern anti-bot systems like Cloudflare, DataDome, Akamai, and PerimeterX use sophisticated techniques that go far beyond simple IP bans.
Here's what they analyze:
- TLS/JA3 fingerprinting: Your SSL handshake reveals whether you're using a real browser or a Python script
- Browser fingerprinting: Canvas, WebGL, audio context, fonts, and screen properties create unique identifiers
- Behavioral analysis: Mouse movements, typing patterns, scroll behavior, and timing between actions
- HTTP header validation: Missing or incorrectly ordered headers expose automation
- JavaScript challenges: Scripts that test browser APIs and look for automation markers
The good news? Each detection method has a corresponding bypass technique. Learning to bypass bot detection effectively requires understanding what triggers these systems in the first place.
Method 1: Use TLS Fingerprint Spoofing with curl_cffi
TLS fingerprinting is the first barrier you'll encounter when trying to bypass bot detection. The handshake happens before your request reaches the server, making it an early filter against automated traffic. Standard Python libraries like requests and httpx expose non-browser TLS fingerprints that trigger instant blocks.
curl_cffi solves this by impersonating real browser TLS signatures at the network level.
Installation
pip install curl_cffi
Basic Usage
from curl_cffi import requests
# Make a request that mimics Chrome's TLS fingerprint
response = requests.get(
"https://www.walmart.com/search?q=laptop",
impersonate="chrome"
)
print(response.status_code) # 200
print(response.text[:500])
The impersonate parameter accepts browser labels like chrome, chrome131, safari, and safari_ios. Each option configures the correct cipher suites, TLS extensions, and HTTP/2 settings.
Why This Works
When you send an HTTPS request, the client and server perform a TLS handshake. This handshake includes details about supported encryption algorithms and TLS extensions.
Different clients have unique signatures. Websites use algorithms like JA3 to hash these details into fingerprints. A fingerprint from requests looks nothing like Chrome—instant red flag.
curl_cffi uses a modified version of cURL that matches real browser signatures. The server sees a legitimate Chrome fingerprint and allows the request through.
Advanced: Session Management with Proxies
from curl_cffi import requests
session = requests.Session()
# Configure proxy rotation
proxies = {
"http": "http://user:pass@proxy1.example.com:8080",
"https": "http://user:pass@proxy1.example.com:8080"
}
response = session.get(
"https://target-site.com/data",
impersonate="chrome131",
proxies=proxies
)
Sessions maintain cookies and connection state across multiple requests. This mimics how real browsers behave when navigating between pages.
Method 2: Deploy Nodriver for CDP-Free Automation
Traditional browser automation tools like Selenium and Puppeteer use the Chrome DevTools Protocol (CDP). Anti-bot systems detect CDP artifacts—specific JavaScript properties and behaviors that only exist in automated browsers.
Nodriver eliminates this problem entirely. It communicates directly with Chrome without using WebDriver or CDP, leaving no automation markers behind.
Installation
pip install nodriver
Basic Scraping Example
import nodriver as nd
async def main():
# Launch browser - no chromedriver needed
browser = await nd.start()
# Navigate to target page
page = await browser.get("https://nowsecure.nl")
# Wait for content to load
await page.sleep(2)
# Extract page content
content = await page.get_content()
print(content[:1000])
# Clean up
await browser.stop()
if __name__ == "__main__":
nd.loop().run_until_complete(main())
Handling Dynamic Content
Nodriver's async architecture makes it perfect for JavaScript-heavy sites. You can wait for specific elements instead of using arbitrary sleep timers.
import nodriver as nd
async def scrape_dynamic_page():
browser = await nd.start()
page = await browser.get("https://example.com/products")
# Wait for specific element to appear
product_list = await page.select(".product-grid", timeout=10)
if product_list:
# Extract all product links
products = await page.select_all(".product-card a")
for product in products:
href = await product.get_attribute("href")
print(href)
await browser.stop()
Key Advantages Over Selenium
Nodriver strips away detectable components that Selenium leaves behind:
- No
navigator.webdriverflag set totrue - No CDP domain artifacts
- No automation-specific JavaScript properties
- Fresh browser profile for each session
The creator of Nodriver is the same developer behind Undetected ChromeDriver. Nodriver represents the next evolution—built from scratch without legacy detection vectors.
Method 3: Implement Camoufox for Firefox-Based Stealth
Most anti-bot systems focus detection efforts on Chromium because it dominates browser market share. Camoufox exploits this blind spot to bypass bot detection by using Firefox as its base, then injecting fingerprint spoofing at the C++ level—not through JavaScript patches that detection services easily identify.
Installation
pip install camoufox[geoip]
python -m camoufox fetch
The second command downloads the modified Firefox binary.
Basic Usage
from camoufox.sync_api import Camoufox
with Camoufox() as browser:
page = browser.new_page()
page.goto("https://www.crunchbase.com/organization/openai")
# Page content is accessible
content = page.content()
print(content[:2000])
Humanized Mouse Movements
Camoufox includes built-in human-like cursor simulation. This defeats behavioral analysis that looks for linear or absent mouse movements.
from camoufox.sync_api import Camoufox
# Enable humanized behavior with 2-second max cursor movement time
with Camoufox(humanize=2.0) as browser:
page = browser.new_page()
page.goto("https://protected-site.com")
# Click will use bezier curve mouse trajectory
button = page.locator("button.submit")
button.click()
The humanize parameter generates distance-aware trajectories with randomized acceleration curves. Detection systems that monitor cursor behavior see patterns indistinguishable from real users.
Advanced Fingerprint Configuration
For maximum control, you can specify exact fingerprint properties:
from camoufox.sync_api import Camoufox
config = {
'window.outerHeight': 1056,
'window.outerWidth': 1920,
'window.innerHeight': 1008,
'window.innerWidth': 1920,
'navigator.userAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0',
'navigator.platform': 'Win32',
'navigator.hardwareConcurrency': 12,
}
with Camoufox(config=config, headless=False) as browser:
page = browser.new_page()
page.goto("https://target-site.com")
Camoufox passes detection tests on CreepJS, BrowserLeaks, and other fingerprinting analysis tools. It remains undetected against Cloudflare Turnstile, DataDome, and Imperva.
Method 4: Rotate Proxies Intelligently
Even perfect fingerprinting fails if your IP address has bad reputation. Datacenter IPs are flagged almost instantly. Effective proxy rotation requires residential or mobile IPs combined with smart rotation logic.
Basic Rotation with Requests
import random
import requests
proxies_list = [
"http://user:pass@residential1.example.com:8080",
"http://user:pass@residential2.example.com:8080",
"http://user:pass@residential3.example.com:8080",
]
def get_with_rotation(url):
proxy = random.choice(proxies_list)
proxies = {"http": proxy, "https": proxy}
response = requests.get(url, proxies=proxies, timeout=30)
return response
This approach works for simple cases but lacks sophistication.
Session-Aware Rotation
Better proxy rotation maintains geographic and session consistency:
from curl_cffi import requests as cffi_requests
import random
class SmartProxyRotator:
def __init__(self, proxy_pool):
self.proxy_pool = proxy_pool
self.session_proxy_map = {}
def get_proxy_for_session(self, session_id, target_region=None):
# Return existing proxy for active sessions
if session_id in self.session_proxy_map:
return self.session_proxy_map[session_id]
# Filter by region if specified
if target_region:
regional_proxies = [
p for p in self.proxy_pool
if target_region in p
]
proxy = random.choice(regional_proxies)
else:
proxy = random.choice(self.proxy_pool)
self.session_proxy_map[session_id] = proxy
return proxy
def make_request(self, url, session_id):
proxy = self.get_proxy_for_session(session_id)
proxies = {"http": proxy, "https": proxy}
return cffi_requests.get(
url,
proxies=proxies,
impersonate="chrome"
)
This pattern keeps the same IP throughout a browsing session—just like real users behave.
Proxy Types Compared
Datacenter proxies are fast and cheap but easily detected. Use them only for non-protected sites.
Residential proxies route through real home internet connections. They blend in with legitimate traffic and work well against most protections. If you need reliable residential proxies, Roundproxies.com offers pools across multiple regions with high success rates.
ISP proxies combine datacenter speed with residential trust scores. They're assigned to real ISPs but hosted in datacenters.
Mobile proxies use 4G/5G connections and have the highest trust scores. Best for the most aggressive anti-bot systems.
Method 5: Spoof Headers Like a Real Browser
HTTP headers reveal more about your client than you might expect. Anti-bot systems check for:
- Missing standard headers
- Headers in wrong order
- Inconsistent User-Agent and Sec-CH-UA values
- Automation-specific headers
Complete Header Set for Chrome
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Sec-CH-UA": '"Google Chrome";v="131", "Chromium";v="131", "Not_A Brand";v="24"',
"Sec-CH-UA-Mobile": "?0",
"Sec-CH-UA-Platform": '"Windows"',
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"Connection": "keep-alive",
"DNT": "1",
}
response = requests.get("https://example.com", headers=headers)
Rotating User-Agents
Don't use the same User-Agent for every request. Rotate through realistic combinations:
import random
user_agents = [
# Windows Chrome
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
# Mac Safari
"Mozilla/5.0 (Macintosh; Intel Mac OS X 14_2_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15",
# Windows Edge
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0",
# Mac Chrome
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
]
def get_random_headers():
ua = random.choice(user_agents)
headers = {
"User-Agent": ua,
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.5",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
}
return headers
Keep your User-Agent consistent with Sec-CH-UA headers. A Chrome User-Agent with mismatched client hints triggers detection.
Method 6: Simulate Human Behavior Patterns
Behavioral analysis catches scrapers that move too fast, too predictably, or not at all. Real users pause, scroll erratically, and take time between actions.
Adding Random Delays
import random
import time
def human_delay(min_seconds=1, max_seconds=5):
"""Add realistic random delay between actions"""
delay = random.uniform(min_seconds, max_seconds)
time.sleep(delay)
def scrape_with_delays(urls):
for url in urls:
response = requests.get(url, headers=get_random_headers())
process_response(response)
# Wait 2-7 seconds between requests
human_delay(2, 7)
Simulating Scroll Behavior
For browser-based scraping, add scroll actions:
import nodriver as nd
import random
import asyncio
async def human_scroll(page):
"""Simulate natural scrolling patterns"""
viewport_height = await page.evaluate("window.innerHeight")
total_height = await page.evaluate("document.body.scrollHeight")
current_position = 0
while current_position < total_height:
# Random scroll distance
scroll_amount = random.randint(100, viewport_height)
current_position += scroll_amount
await page.evaluate(f"window.scrollTo(0, {current_position})")
# Random pause between scrolls
await asyncio.sleep(random.uniform(0.3, 1.5))
Typing Simulation
When filling forms, type character by character:
import asyncio
import random
async def human_type(element, text):
"""Type text with human-like delays between keystrokes"""
for char in text:
await element.send_keys(char)
# Variable delay between keystrokes (50-200ms)
delay = random.uniform(0.05, 0.2)
await asyncio.sleep(delay)
These patterns make your automation indistinguishable from real user behavior.
Method 7: Handle CAPTCHAs When They Appear
Despite best efforts, some sites will present CAPTCHA challenges. You have two options: solve them or avoid triggering them.
Prevention First
The best CAPTCHA is one you never see. Following all previous methods significantly reduces CAPTCHA frequency. Camoufox in particular avoids triggering CAPTCHAs in most cases through its comprehensive stealth features.
Manual Solving for Low Volume
For occasional CAPTCHAs during development or testing, manual solving works:
from camoufox.sync_api import Camoufox
with Camoufox(headless=False) as browser:
page = browser.new_page()
page.goto("https://site-with-captcha.com")
# Check if CAPTCHA appeared
captcha = page.locator(".captcha-container")
if captcha.is_visible():
print("CAPTCHA detected - solve manually")
input("Press Enter after solving...")
# Continue with scraping
data = page.locator(".target-data").text_content()
Automated Solving with Services
For production systems, CAPTCHA solving services handle challenges automatically. These services use human workers or AI to solve challenges and return tokens.
The integration pattern follows a request-response model: send the challenge details, poll for the solution, inject the token, and proceed.
Note that CAPTCHA solving adds latency and cost to your pipeline. Prevention through proper stealth remains the preferred approach.
Method 8: Use SeleniumBase UC Mode for Legacy Compatibility
If you have existing Selenium scripts, SeleniumBase with Undetected Chrome (UC) mode provides an upgrade path without major rewrites.
Installation
pip install seleniumbase
Basic UC Mode Usage
from seleniumbase import Driver
# Launch in undetected mode
driver = Driver(uc=True)
driver.get("https://cloudflare-protected-site.com")
# Wait for page to fully load
driver.sleep(3)
# Extract content
content = driver.page_source
print(content[:2000])
driver.quit()
UC Mode with Proxy
from seleniumbase import Driver
driver = Driver(
uc=True,
proxy="user:pass@proxy.example.com:8080"
)
driver.get("https://target-site.com")
When to Use SeleniumBase vs. Other Tools
SeleniumBase UC mode works well for:
- Migrating existing Selenium codebases
- Sites with moderate anti-bot protection
- Quick prototyping and testing
For maximum stealth against aggressive protections, Camoufox or Nodriver deliver better results. SeleniumBase remains a solid middle-ground option that balances familiarity with improved detection evasion.
Common Mistakes That Get You Blocked
Even with proper tools, certain mistakes guarantee detection:
Running from datacenter IPs without residential proxies. Anti-bot systems maintain IP reputation databases. Cloud provider ranges are heavily scrutinized.
Using headless mode on strict sites. Some protections specifically detect headless operation. Run in headful mode with a virtual display for maximum stealth.
Ignoring request patterns. Hitting 50 pages in 10 seconds screams automation. Real users browse at 3-7 second intervals.
Reusing the same fingerprint across sessions. Rotate fingerprints between scraping runs. The same browser fingerprint appearing from different IPs raises flags.
Neglecting mobile user-agents. Many sites have stricter protection for desktop traffic. Mobile fingerprints sometimes pass more easily.
Skipping error handling. When requests fail, don't immediately retry. Implement exponential backoff and proxy rotation on failures.
Putting It All Together
Here's a complete example combining multiple bypass techniques:
from curl_cffi import requests
import random
import time
class StealthScraper:
def __init__(self, proxy_list):
self.session = requests.Session()
self.proxy_list = proxy_list
def get_random_proxy(self):
proxy = random.choice(self.proxy_list)
return {"http": proxy, "https": proxy}
def human_delay(self):
time.sleep(random.uniform(2, 6))
def scrape(self, url):
response = self.session.get(
url,
impersonate="chrome131",
proxies=self.get_random_proxy(),
timeout=30
)
self.human_delay()
return response
# Usage
proxies = [
"http://user:pass@proxy1.example.com:8080",
"http://user:pass@proxy2.example.com:8080",
]
scraper = StealthScraper(proxies)
response = scraper.scrape("https://target-site.com/data")
print(response.text)
For browser-based scraping, Camoufox provides the most comprehensive solution:
from camoufox.sync_api import Camoufox
import random
import time
with Camoufox(humanize=2.0, headless=True) as browser:
page = browser.new_page()
urls = ["https://site.com/page1", "https://site.com/page2"]
for url in urls:
page.goto(url)
# Wait for content
page.wait_for_load_state("networkidle")
# Extract data
content = page.content()
process(content)
# Human-like delay
time.sleep(random.uniform(3, 8))
Final Thoughts
Knowing how to bypass bot detection in 2026 requires layered defenses—because anti-bot systems use layered attacks. No single technique works alone. Success comes from combining TLS fingerprint spoofing, browser stealth, intelligent proxy rotation, realistic headers, and human-like behavior patterns.
Start with curl_cffi for simple requests where you need TLS fingerprint matching. Move to Nodriver or Camoufox when you need full browser automation with JavaScript rendering. Add residential proxies from providers like Roundproxies when IP reputation becomes an issue.
The techniques to bypass bot detection covered in this guide are actively maintained and updated against evolving anti-bot systems. What works today might need adjustment tomorrow—stay current with project updates and be ready to adapt your approach.
Now go build something. The data is waiting.
Quick Comparison: Which Method Should You Use?
Choosing the right approach to bypass bot detection depends on your target site's protection level and your technical requirements.
| Method | Best For | Difficulty | Stealth Level |
|---|---|---|---|
| curl_cffi | API requests, static pages | Easy | Medium |
| Nodriver | Dynamic sites, moderate protection | Medium | High |
| Camoufox | Heavily protected sites, Cloudflare | Medium | Very High |
| Proxy Rotation | All scenarios at scale | Easy | Depends on proxy type |
| Header Spoofing | Basic protections | Easy | Low-Medium |
| Behavior Simulation | Behavioral analysis systems | Medium | High |
| CAPTCHA Handling | When prevention fails | Medium | N/A |
| SeleniumBase UC | Selenium migration, moderate protection | Easy | Medium-High |
Decision Framework
Start with curl_cffi if:
- You only need HTTP requests (no JavaScript rendering)
- Target sites use TLS fingerprinting but not browser fingerprinting
- Speed matters more than stealth
Choose Nodriver if:
- You need JavaScript rendering
- Target uses Cloudflare, Akamai, or similar WAFs
- You want async/await architecture
Go with Camoufox if:
- Maximum stealth is required
- Target uses DataDome, PerimeterX, or aggressive behavioral analysis
- You need Playwright API compatibility
- Firefox fingerprints might have higher success than Chrome
Troubleshooting Common Issues
Still Getting Blocked Despite Using These Methods
Check these common causes:
- Bad proxy quality: Free proxies are flagged immediately. Invest in quality residential proxies from providers like Roundproxies.com for reliable results.
- Outdated browser versions in fingerprints: Anti-bot systems know which browser versions are current. Using Chrome 99 fingerprints in 2026 stands out.
- Inconsistent fingerprint elements: Your User-Agent says Windows but your timezone says Australia—that's suspicious.
- Too fast request rates: Even with perfect stealth, 100 requests per minute from one session gets flagged.
Error: RecursionError with Nodriver Headless
Nodriver has stability issues in pure headless mode. Use virtual display instead:
# Install Xvfb
sudo apt-get install xvfb
# Run with virtual display
xvfb-run python your_script.py
Or use Camoufox's virtual headless mode which handles this automatically.
Cloudflare Turnstile Won't Pass
Turnstile is Cloudflare's newest CAPTCHA and the most challenging. Steps to maximize success:
- Use residential proxies exclusively—datacenter IPs fail almost always
- Enable humanized mouse movements in Camoufox
- Add random delays of 3-10 seconds after page load before interacting
- Maintain session cookies across requests to build trust score
If you still get blocked, the site may have custom rules or ML models that require more sophisticated approaches.
VPS/Docker Detection
Running from cloud infrastructure is harder than local machines. Detection systems fingerprint:
- Timezone vs IP location mismatches
- Missing hardware features (GPU, audio devices)
- Virtual environment artifacts
Mitigation strategies:
from camoufox.sync_api import Camoufox
# Match fingerprint to VPS location
config = {
'timezone': 'America/New_York', # Match your VPS region
'navigator.language': 'en-US',
'navigator.languages': ['en-US', 'en'],
}
with Camoufox(config=config) as browser:
page = browser.new_page()
page.goto("https://target.com")
What's Coming in 2026 and Beyond
The bot detection landscape evolves constantly. Here's what to expect:
AI-powered behavioral analysis will become standard. Systems will use machine learning to detect patterns invisible to rule-based systems. Mitigation: More sophisticated human behavior simulation.
Zero-trust browser verification will check hardware attestation. Proving you're running on real hardware becomes mandatory for some sites. Mitigation: Hardware-backed browser environments.
Collaborative fingerprint databases will share blocked fingerprints across sites. Getting blocked on one property leads to blocks everywhere. Mitigation: Never reuse fingerprints and rotate frequently.
Client-side ML models running in the browser will analyze behavior in real-time. Mitigation: Tools like Camoufox that operate at the C++ level rather than through JavaScript.
Stay updated by following the GitHub repositories for these tools. The maintainers actively adapt to new detection techniques and push updates regularly.
FAQ
What is the best tool to bypass bot detection in 2026?
Camoufox is currently the most effective open-source solution for bypassing bot detection. It implements fingerprint spoofing at the C++ level rather than through detectable JavaScript patches. For simpler HTTP requests without browser automation, curl_cffi provides TLS fingerprint matching that defeats many protections.
Can I bypass Cloudflare for free?
Yes, you can bypass Cloudflare using free tools like Camoufox, Nodriver, or SeleniumBase UC mode. These open-source libraries handle JavaScript challenges and fingerprint masking without requiring paid services. Residential proxies may be needed for consistent results on heavily protected sites.
Do I need proxies to bypass bot detection?
Proxies are often necessary but not always required. If your home IP has good reputation and you're scraping at low volume, you might succeed without proxies. For scale or when targeting sites with aggressive IP reputation checks, residential proxies become essential for sustained success.
Is web scraping legal in 2026?
Web scraping legality depends on what you scrape and how you use the data. Scraping publicly available data is generally legal in many jurisdictions. However, always respect robots.txt, avoid scraping personal data without consent, and check terms of service for specific sites. Consult legal counsel for commercial applications.