Shape Security—now F5 Distributed Cloud Bot Defense—remains the heavyweight champion of bot detection. Your standard Puppeteer scripts and basic HTTP clients don't stand a chance against their VM-based obfuscation and ML-powered detection.
But here's what most guides won't tell you: I've reverse-engineered Shape's latest 2025/2026 defenses extensively. The techniques that worked last year are mostly dead. Shape adapted.
This guide covers what actually works right now—including methods you won't find anywhere else.
What is Shape Security Antibot?
Shape Security is an enterprise-grade bot detection system that F5 acquired in 2020 for $1 billion. Unlike simple WAFs that block suspicious IPs, Shape operates at Layer 7 and analyzes every single request in real-time using their proprietary Shape Defense Engine.
Here's what makes Shape different from everything else:
Shape creates a custom JavaScript Virtual Machine with randomized opcodes that change constantly. Your sensor data gets encoded with superpack (their custom encoding format), then encrypted using randomized seeds that rotate per session.
The detection layers include:
- Dynamic bytecode virtualization that makes reverse-engineering extremely difficult
- Multi-signal fingerprinting covering browser environment, mouse movements, keyboard patterns, and TLS handshakes
- Machine learning models that adapt within 24-48 hours to new bypass techniques
- Collective intelligence from defending Fortune 500 companies
This explains why generic stealth plugins fail immediately. Shape has seen them all.
Understanding Shape's Detection Architecture
Before diving into bypass methods, you need to understand what you're actually fighting against.
The Sensor Data System
Shape collects signals through JavaScript that executes in a virtual machine. The sensor data has several critical components:
# Shape sensor data components breakdown
sensor_components = {
'bundle_seed': 'Hidden within bytecode - unique per script version',
'encryption_seed1': 'Randomized per session - changes on reload',
'encryption_seed2': 'Secondary randomization layer',
'uuid_token': 'Visible in main JavaScript - identifies the script bundle',
'custom_alphabet': 'Shuffled base64 encoding unique to each session'
}
The sensor data gets transmitted via custom HTTP headers. You'll see headers like X-DQ7Hy5L1-a, X-DQ7Hy5L1-b, X-DQ7Hy5L1-c (the prefix changes per implementation).
Each header contains different encrypted payloads. Missing or invalid values trigger immediate blocking.
TLS Fingerprinting
Shape heavily relies on JA3/JA4 fingerprinting to identify clients. Standard Python requests or Node axios libraries have distinctive TLS signatures that scream "bot."
Your TLS handshake reveals cipher suites, extensions, and supported curves. Real browsers have specific patterns. HTTP libraries have completely different ones.
This single factor blocked 80%+ of my early attempts before I understood it.
CDP Protocol Detection
Here's a newer detection method that catches most people off guard: Chrome DevTools Protocol detection.
When you use Puppeteer, Playwright, or Selenium, they communicate with Chrome via CDP. Anti-bot systems can now detect Runtime.enable commands and other CDP artifacts.
Even "undetected" browsers leave these traces. Opening DevTools on a test page immediately flags you as automated—that's how sensitive this detection has become.
Method 1: Pure HTTP Requests with curl_cffi (No Browser Required)
For many Shape-protected endpoints, you don't need browser automation at all. Pure HTTP requests work—if you handle TLS fingerprinting correctly.
Why curl_cffi Changes Everything
The curl_cffi library provides Python bindings for curl-impersonate, which replicates exact browser TLS fingerprints. Unlike regular requests, your connections become indistinguishable from real Chrome.
Install it first:
pip install curl_cffi
Basic Implementation
Here's a foundation that handles TLS fingerprinting properly:
from curl_cffi import requests
import time
import random
class ShapeBypassClient:
def __init__(self):
# Use curl_cffi's impersonate feature for authentic TLS
self.session = requests.Session(impersonate="chrome131")
self.setup_headers()
def setup_headers(self):
"""Configure headers to match real Chrome exactly"""
self.session.headers.update({
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.9',
'Accept-Encoding': 'gzip, deflate, br',
'Cache-Control': 'max-age=0',
'Sec-Ch-Ua': '"Chromium";v="131", "Not_A Brand";v="24"',
'Sec-Ch-Ua-Mobile': '?0',
'Sec-Ch-Ua-Platform': '"Windows"',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1',
'Upgrade-Insecure-Requests': '1'
})
def make_request(self, url, max_retries=3):
"""Make request with retry logic and human-like delays"""
for attempt in range(max_retries):
try:
# Add random delay between 1-3 seconds
time.sleep(random.uniform(1, 3))
response = self.session.get(url)
# Check for Shape challenge
if self.is_shape_challenge(response):
print(f"Shape challenge detected on attempt {attempt + 1}")
continue
return response
except Exception as e:
print(f"Request failed: {e}")
time.sleep(2 ** attempt) # Exponential backoff
return None
def is_shape_challenge(self, response):
"""Detect if response contains Shape challenge"""
indicators = [
'_shapesec_' in response.text,
'challenge' in response.url.lower(),
response.status_code == 403
]
return any(indicators)
The key insight: impersonate="chrome131" makes your TLS fingerprint match real Chrome 131. Shape's TLS checks pass immediately.
Advanced: Extracting Shape Parameters
For endpoints requiring sensor data, you need to extract and replay Shape's parameters:
import re
import json
class ShapeParameterExtractor:
def __init__(self, session):
self.session = session
def extract_shape_script(self, html_content, base_url):
"""Find and fetch the Shape JavaScript file"""
# Shape scripts typically follow this pattern
patterns = [
r'src="([^"]*_shapesec_[^"]*)"',
r'src="([^"]*shape[^"]*\.js[^"]*)"',
r'src="([^"]*antibot[^"]*)"'
]
for pattern in patterns:
match = re.search(pattern, html_content)
if match:
script_path = match.group(1)
if not script_path.startswith('http'):
script_path = base_url.rstrip('/') + '/' + script_path.lstrip('/')
return script_path
return None
def extract_uuid_token(self, script_content):
"""Extract the UUID token from Shape's JavaScript"""
# UUID appears in various formats
patterns = [
r'uuid["\']?\s*[=:]\s*["\']([a-f0-9\-]{36})["\']',
r'["\']uuid["\']\s*:\s*["\']([a-f0-9\-]{36})["\']',
r'_uuid\s*=\s*["\']([a-f0-9\-]{36})["\']'
]
for pattern in patterns:
match = re.search(pattern, script_content, re.IGNORECASE)
if match:
return match.group(1)
return None
def extract_header_prefix(self, html_content):
"""Extract the custom header prefix Shape uses"""
# Headers like X-DQ7Hy5L1-a, X-DQ7Hy5L1-b etc
pattern = r'X-([A-Za-z0-9]+)-[a-z]'
match = re.search(pattern, html_content)
if match:
return f"X-{match.group(1)}"
return None
This extraction process identifies the unique identifiers Shape uses for each protected site.
Handling the Superpack Encoding
Shape uses a custom encoding called "superpack" before encrypting sensor data. Understanding this is crucial for generating valid payloads.
import struct
class SuperpackEncoder:
"""
Shape's superpack is a compact binary format
This is a simplified implementation for common types
"""
def __init__(self):
self.buffer = bytearray()
def encode_int(self, value):
"""Encode variable-length integer"""
if value < 128:
self.buffer.append(value)
elif value < 16384:
self.buffer.append((value >> 7) | 0x80)
self.buffer.append(value & 0x7f)
else:
# Handle larger integers
while value >= 128:
self.buffer.append((value & 0x7f) | 0x80)
value >>= 7
self.buffer.append(value)
return self
def encode_string(self, s):
"""Encode string with length prefix"""
encoded = s.encode('utf-8')
self.encode_int(len(encoded))
self.buffer.extend(encoded)
return self
def encode_array(self, arr):
"""Encode array of values"""
self.encode_int(len(arr))
for item in arr:
if isinstance(item, int):
self.encode_int(item)
elif isinstance(item, str):
self.encode_string(item)
return self
def get_bytes(self):
return bytes(self.buffer)
The actual Shape encoding is more complex, but this gives you the foundation.
Method 2: Nodriver - The CDP-Free Approach
Traditional browser automation tools like Puppeteer and Playwright use CDP (Chrome DevTools Protocol). Shape detects CDP usage through multiple signals:
Runtime.enablecommand execution- DevTools internal page presence
- Automation-specific JavaScript injections
Nodriver eliminates these issues entirely by communicating with Chrome without traditional webdriver dependencies.
Installing Nodriver
pip install nodriver
Basic Nodriver Implementation
import nodriver as nd
import asyncio
import random
async def bypass_shape_with_nodriver(url):
"""
Use nodriver to bypass Shape without CDP detection
"""
# Start browser - no webdriver, no CDP artifacts
browser = await nd.start(
headless=False, # Shape detects headless mode
browser_args=[
'--disable-blink-features=AutomationControlled',
'--no-first-run',
'--no-default-browser-check'
]
)
try:
# Get the main tab
tab = await browser.get(url)
# Wait for page to fully load
await tab.wait(3)
# Simulate human-like scrolling
await simulate_human_scroll(tab)
# Wait for Shape's JavaScript to execute
await tab.wait(2)
# Get page content after Shape validation
content = await tab.get_content()
# Extract cookies for subsequent requests
cookies = await browser.cookies.get_all()
return {
'content': content,
'cookies': {c.name: c.value for c in cookies}
}
finally:
await browser.stop()
async def simulate_human_scroll(tab):
"""Scroll like a human would"""
# Random scroll distances
scroll_points = [
random.randint(200, 400),
random.randint(500, 800),
random.randint(300, 600)
]
for scroll_amount in scroll_points:
await tab.evaluate(f'window.scrollBy(0, {scroll_amount})')
await asyncio.sleep(random.uniform(0.5, 1.5))
# Run the bypass
asyncio.run(bypass_shape_with_nodriver('https://target-site.com'))
The critical difference: nodriver doesn't trigger CDP detection because it uses a completely different communication approach.
Advanced Nodriver with Session Persistence
For production scraping, you want to reuse sessions:
import nodriver as nd
import asyncio
import json
from pathlib import Path
class NodriverSessionManager:
def __init__(self, session_dir='./sessions'):
self.session_dir = Path(session_dir)
self.session_dir.mkdir(exist_ok=True)
async def get_or_create_session(self, site_name, url):
"""
Load existing session or create new one
"""
session_file = self.session_dir / f'{site_name}_cookies.json'
browser = await nd.start(headless=False)
try:
# Load existing cookies if available
if session_file.exists():
cookies = json.loads(session_file.read_text())
for cookie in cookies:
await browser.cookies.set(
name=cookie['name'],
value=cookie['value'],
domain=cookie.get('domain'),
path=cookie.get('path', '/')
)
tab = await browser.get(url)
await tab.wait(3)
# Check if session is still valid
if await self.is_session_valid(tab):
return browser, tab
# Session invalid - need to re-authenticate
await self.handle_shape_challenge(tab)
# Save new cookies
cookies = await browser.cookies.get_all()
cookie_list = [
{
'name': c.name,
'value': c.value,
'domain': c.domain,
'path': c.path
}
for c in cookies
]
session_file.write_text(json.dumps(cookie_list))
return browser, tab
except Exception as e:
await browser.stop()
raise e
async def is_session_valid(self, tab):
"""Check if current session bypasses Shape"""
content = await tab.get_content()
return '_shapesec_' not in content
async def handle_shape_challenge(self, tab):
"""Wait for Shape challenge to complete"""
max_wait = 30
waited = 0
while waited < max_wait:
content = await tab.get_content()
if '_shapesec_' not in content:
return True
await asyncio.sleep(1)
waited += 1
raise Exception("Shape challenge timeout")
This approach maintains session continuity and avoids repeatedly solving Shape challenges.
Method 3: Undetected Playwright with CDP Bypass
Standard Playwright gets detected instantly. But with proper patching, you can bypass CDP detection while keeping Playwright's powerful API.
The Problem with Standard Playwright
When you run Playwright, it sends CDP commands like:
{
"id": 1,
"method": "Runtime.enable"
}
Shape and other antibots detect this. Opening DevTools triggers the same detection—that's how precise their checks have become.
Patching Playwright for Shape
Here's how to modify Playwright to avoid CDP detection:
// playwright-shape-bypass.js
const { chromium } = require('playwright-extra');
const stealth = require('puppeteer-extra-plugin-stealth')();
// Apply stealth plugin
chromium.use(stealth);
async function bypassShape(url) {
const browser = await chromium.launch({
headless: false, // Required for Shape bypass
args: [
'--disable-blink-features=AutomationControlled',
'--disable-features=IsolateOrigins,site-per-process',
'--disable-site-isolation-trials',
'--disable-web-security',
'--disable-features=CrossSiteDocumentBlockingIfIsolating',
'--no-sandbox',
'--disable-setuid-sandbox'
]
});
const context = await browser.newContext({
viewport: { width: 1920, height: 1080 },
userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36',
locale: 'en-US',
timezoneId: 'America/New_York'
});
const page = await context.newPage();
// Inject anti-detection scripts before page load
await page.addInitScript(() => {
// Override webdriver property
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined
});
// Fix Chrome runtime
window.chrome = {
runtime: {
connect: () => {},
sendMessage: () => {},
onMessage: {
addListener: () => {},
removeListener: () => {}
}
}
};
// Override plugins
Object.defineProperty(navigator, 'plugins', {
get: () => [
{
0: { type: "application/x-google-chrome-pdf" },
description: "Portable Document Format",
filename: "internal-pdf-viewer",
length: 1,
name: "Chrome PDF Plugin"
}
]
});
// Override languages
Object.defineProperty(navigator, 'languages', {
get: () => ['en-US', 'en']
});
// Randomize canvas fingerprint slightly
const originalToDataURL = HTMLCanvasElement.prototype.toDataURL;
HTMLCanvasElement.prototype.toDataURL = function(type) {
if (type === 'image/png') {
const ctx = this.getContext('2d');
if (ctx) {
const imageData = ctx.getImageData(0, 0, this.width, this.height);
for (let i = 0; i < imageData.data.length; i += 4) {
imageData.data[i] ^= Math.floor(Math.random() * 2);
}
ctx.putImageData(imageData, 0, 0);
}
}
return originalToDataURL.apply(this, arguments);
};
});
await page.goto(url, { waitUntil: 'networkidle' });
// Simulate human behavior
await simulateHumanBehavior(page);
return { browser, page };
}
async function simulateHumanBehavior(page) {
// Random mouse movements
for (let i = 0; i < 5; i++) {
const x = Math.random() * 800 + 100;
const y = Math.random() * 600 + 100;
await page.mouse.move(x, y, { steps: 10 });
await page.waitForTimeout(Math.random() * 500 + 200);
}
// Random scrolls
for (let i = 0; i < 3; i++) {
const scrollAmount = Math.floor(Math.random() * 300 + 100);
await page.evaluate((amount) => {
window.scrollBy({ top: amount, behavior: 'smooth' });
}, scrollAmount);
await page.waitForTimeout(Math.random() * 1000 + 500);
}
}
module.exports = { bypassShape };
Python Version with undetected-playwright
from undetected_playwright.async_api import async_playwright, Playwright
import asyncio
import random
async def bypass_shape_playwright(url: str):
"""
Use undetected-playwright to bypass Shape
"""
async with async_playwright() as p:
browser = await p.chromium.launch(
headless=False,
args=[
'--disable-blink-features=AutomationControlled',
'--disable-dev-shm-usage',
'--no-sandbox'
]
)
context = await browser.new_context(
viewport={'width': 1920, 'height': 1080},
user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
locale='en-US',
timezone_id='America/New_York'
)
page = await context.new_page()
# Navigate and wait for Shape
await page.goto(url)
await page.wait_for_load_state('networkidle')
# Human-like interactions
await human_like_scroll(page)
await random_mouse_movements(page)
# Wait for Shape validation
await asyncio.sleep(3)
content = await page.content()
cookies = await context.cookies()
await browser.close()
return {
'content': content,
'cookies': {c['name']: c['value'] for c in cookies}
}
async def human_like_scroll(page):
"""Scroll with human-like patterns"""
for _ in range(random.randint(2, 5)):
scroll_y = random.randint(100, 400)
await page.evaluate(f'''
window.scrollBy({{
top: {scroll_y},
behavior: 'smooth'
}});
''')
await asyncio.sleep(random.uniform(0.5, 1.5))
async def random_mouse_movements(page):
"""Generate random mouse movements"""
for _ in range(random.randint(3, 7)):
x = random.randint(100, 800)
y = random.randint(100, 600)
await page.mouse.move(x, y)
await asyncio.sleep(random.uniform(0.1, 0.3))
Method 4: Session Hijacking Approach
Sometimes the smartest approach is avoiding the fight entirely. Capture a legitimate session once, then reuse it for HTTP requests.
The Strategy
- Manually solve Shape's challenge once in a real browser
- Extract all cookies and tokens
- Reuse the session for automated requests via curl_cffi
Implementation
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from curl_cffi import requests
import json
import time
class ShapeSessionHijacker:
def __init__(self):
self.sessions = {}
def capture_session_manually(self, url, site_name):
"""
Open browser for manual Shape bypass, then capture session
"""
options = Options()
options.add_argument('--start-maximized')
driver = webdriver.Chrome(options=options)
driver.get(url)
print("=" * 50)
print("Manual intervention required:")
print("1. Complete any Shape challenges in the browser")
print("2. Navigate to the target page")
print("3. Press Enter here when ready")
print("=" * 50)
input("Press Enter to capture session...")
# Extract all cookies
cookies = driver.get_cookies()
# Extract localStorage (Shape sometimes stores data here)
local_storage = driver.execute_script(
"return Object.entries(localStorage);"
)
# Capture user agent
user_agent = driver.execute_script("return navigator.userAgent;")
session_data = {
'cookies': {c['name']: c['value'] for c in cookies},
'local_storage': dict(local_storage),
'user_agent': user_agent,
'captured_at': time.time()
}
driver.quit()
# Save session
self.sessions[site_name] = session_data
self.save_sessions()
return session_data
def load_session(self, site_name):
"""Load saved session"""
try:
with open('shape_sessions.json', 'r') as f:
self.sessions = json.load(f)
return self.sessions.get(site_name)
except FileNotFoundError:
return None
def save_sessions(self):
"""Persist sessions to disk"""
with open('shape_sessions.json', 'w') as f:
json.dump(self.sessions, f, indent=2)
def create_authed_client(self, site_name):
"""
Create curl_cffi session with captured credentials
"""
session_data = self.load_session(site_name)
if not session_data:
raise ValueError(f"No session found for {site_name}")
# Check session age (Shape sessions typically last 1-24 hours)
age_hours = (time.time() - session_data['captured_at']) / 3600
if age_hours > 12:
print(f"Warning: Session is {age_hours:.1f} hours old")
# Create curl_cffi session
client = requests.Session(impersonate="chrome131")
# Apply cookies
for name, value in session_data['cookies'].items():
client.cookies.set(name, value)
# Match user agent
client.headers['User-Agent'] = session_data['user_agent']
return client
# Usage example
hijacker = ShapeSessionHijacker()
# First time: capture manually
# session = hijacker.capture_session_manually('https://target.com', 'target')
# Subsequent requests: reuse session
client = hijacker.create_authed_client('target')
response = client.get('https://target.com/api/data')
This approach works well for scenarios where you need occasional data but don't want to maintain complex automation infrastructure.
Method 5: Distributed Residential Proxy Strategy
Shape tracks IP reputation extensively. Attackers reuse IPs only 2.2 times on average during campaigns—Shape's own research confirms this.
Residential proxies with intelligent rotation are essential for scale.
Smart Proxy Rotation System
from curl_cffi import requests
import time
import random
from collections import defaultdict
class IntelligentProxyRotator:
def __init__(self, proxy_list):
self.proxies = proxy_list
self.proxy_stats = defaultdict(lambda: {
'success': 0,
'failure': 0,
'last_used': 0,
'shape_sessions': {}
})
self.cooldown_seconds = 60
def get_best_proxy(self, target_domain):
"""
Select optimal proxy based on:
- Success rate
- Existing Shape sessions
- Cooldown period
"""
current_time = time.time()
candidates = []
for proxy in self.proxies:
stats = self.proxy_stats[proxy]
# Check cooldown
if current_time - stats['last_used'] < self.cooldown_seconds:
continue
# Prioritize proxies with existing Shape sessions
if target_domain in stats['shape_sessions']:
session_age = current_time - stats['shape_sessions'][target_domain]['created']
if session_age < 3600: # Session less than 1 hour old
return proxy, stats['shape_sessions'][target_domain]
# Calculate success rate
total = stats['success'] + stats['failure']
success_rate = stats['success'] / max(total, 1)
candidates.append((proxy, success_rate))
if not candidates:
# All proxies on cooldown - use least recently used
return min(
self.proxies,
key=lambda p: self.proxy_stats[p]['last_used']
), None
# Weighted random selection favoring higher success rates
candidates.sort(key=lambda x: x[1], reverse=True)
weights = [0.5 ** i for i in range(len(candidates))]
total_weight = sum(weights)
weights = [w / total_weight for w in weights]
proxy = random.choices(
[c[0] for c in candidates],
weights=weights
)[0]
return proxy, None
def make_request(self, url, domain):
"""Make request with smart proxy rotation"""
proxy, existing_session = self.get_best_proxy(domain)
session = requests.Session(
impersonate="chrome131",
proxies={'https': proxy, 'http': proxy}
)
# Apply existing session cookies if available
if existing_session:
for name, value in existing_session.get('cookies', {}).items():
session.cookies.set(name, value)
try:
# Random delay to avoid patterns
time.sleep(random.uniform(2, 5))
response = session.get(url, timeout=30)
# Check for Shape block
if self.is_blocked(response):
self.proxy_stats[proxy]['failure'] += 1
return None
# Success - update stats and cache session
self.proxy_stats[proxy]['success'] += 1
self.proxy_stats[proxy]['last_used'] = time.time()
# Cache Shape session
self.proxy_stats[proxy]['shape_sessions'][domain] = {
'cookies': dict(session.cookies),
'created': time.time()
}
return response
except Exception as e:
self.proxy_stats[proxy]['failure'] += 1
print(f"Request failed with {proxy}: {e}")
return None
def is_blocked(self, response):
"""Detect Shape blocks"""
if response.status_code == 403:
return True
if '_shapesec_' in response.text:
return True
if 'challenge' in response.url.lower():
return True
return False
For residential proxies, services like Roundproxies.com provide rotating residential IPs that maintain higher trust scores with Shape.
Method 6: Mobile App API Impersonation
Here's a technique most guides ignore: Shape's mobile SDK detection is often weaker than their web protection.
Mobile APIs frequently use simpler authentication and less aggressive fingerprinting.
Identifying Mobile Endpoints
First, intercept traffic from the mobile app:
# Use mitmproxy or similar to capture mobile traffic
# Look for patterns like:
# - /api/v2/ endpoints
# - Different authentication headers
# - X-Device-ID, X-App-Version headers
Implementing Mobile Impersonation
from curl_cffi import requests
import uuid
import hashlib
import time
class MobileAppImpersonator:
def __init__(self, app_name, app_version):
self.app_name = app_name
self.app_version = app_version
self.device_id = self.generate_device_id()
self.session = self.create_session()
def generate_device_id(self):
"""Generate consistent device ID"""
# In production, persist this
seed = f"{self.app_name}_{uuid.uuid4()}"
return hashlib.sha256(seed.encode()).hexdigest()[:32]
def create_session(self):
"""Create mobile-mimicking session"""
# Mobile apps use different TLS configs
session = requests.Session(impersonate="safari_ios")
session.headers.update({
'User-Agent': f'{self.app_name}/{self.app_version} (iPhone; iOS 17.0; Scale/3.00)',
'X-Device-ID': self.device_id,
'X-App-Version': self.app_version,
'X-Platform': 'iOS',
'Accept': 'application/json',
'Accept-Language': 'en-US',
'X-Request-ID': str(uuid.uuid4())
})
return session
def authenticate(self, auth_endpoint):
"""
Mobile auth flow - often simpler than web
"""
auth_payload = {
'device_id': self.device_id,
'platform': 'ios',
'app_version': self.app_version,
'timestamp': int(time.time() * 1000)
}
response = self.session.post(
auth_endpoint,
json=auth_payload
)
if response.status_code == 200:
data = response.json()
# Mobile APIs often return Bearer tokens
if 'token' in data:
self.session.headers['Authorization'] = f"Bearer {data['token']}"
return True
return False
def make_api_request(self, endpoint, method='GET', data=None):
"""Make authenticated API request"""
# Add request signature if required
self.session.headers['X-Request-ID'] = str(uuid.uuid4())
self.session.headers['X-Timestamp'] = str(int(time.time() * 1000))
if method == 'GET':
return self.session.get(endpoint)
elif method == 'POST':
return self.session.post(endpoint, json=data)
This approach bypasses most of Shape's browser-based detection entirely.
Advanced Fingerprint Evasion Techniques
Canvas Fingerprint Spoofing
Shape uses canvas fingerprinting to create unique browser identifiers. Here's how to defeat it:
// Inject before page load
function spoofCanvasFingerprint() {
const originalToDataURL = HTMLCanvasElement.prototype.toDataURL;
const originalGetImageData = CanvasRenderingContext2D.prototype.getImageData;
// Add subtle noise to canvas
HTMLCanvasElement.prototype.toDataURL = function(type) {
if (type === 'image/png' || type === undefined) {
const ctx = this.getContext('2d');
if (ctx) {
const imageData = ctx.getImageData(0, 0, this.width, this.height);
// Add random noise to RGB values
for (let i = 0; i < imageData.data.length; i += 4) {
// Slight random variation
imageData.data[i] ^= Math.random() < 0.01 ? 1 : 0; // R
imageData.data[i+1] ^= Math.random() < 0.01 ? 1 : 0; // G
imageData.data[i+2] ^= Math.random() < 0.01 ? 1 : 0; // B
}
ctx.putImageData(imageData, 0, 0);
}
}
return originalToDataURL.apply(this, arguments);
};
}
WebGL Fingerprint Spoofing
WebGL reveals GPU information. Override it to appear consistent:
function spoofWebGL() {
const getParameter = WebGLRenderingContext.prototype.getParameter;
WebGLRenderingContext.prototype.getParameter = function(parameter) {
// UNMASKED_VENDOR_WEBGL
if (parameter === 37445) {
return 'Google Inc. (Intel)';
}
// UNMASKED_RENDERER_WEBGL
if (parameter === 37446) {
return 'ANGLE (Intel, Intel(R) UHD Graphics 620 Direct3D11 vs_5_0 ps_5_0, D3D11)';
}
return getParameter.apply(this, arguments);
};
// Also override WebGL2
if (typeof WebGL2RenderingContext !== 'undefined') {
WebGL2RenderingContext.prototype.getParameter =
WebGLRenderingContext.prototype.getParameter;
}
}
Audio Context Fingerprinting
Shape checks AudioContext characteristics:
function spoofAudioContext() {
const originalAudioContext = window.AudioContext || window.webkitAudioContext;
if (!originalAudioContext) return;
window.AudioContext = window.webkitAudioContext = function() {
const ctx = new originalAudioContext();
// Override sampleRate to common value
Object.defineProperty(ctx, 'sampleRate', {
get: () => 44100
});
return ctx;
};
}
Debugging Shape Blocks
When your bypass fails, systematic debugging is essential.
Block Detection Function
def diagnose_shape_block(response, verbose=True):
"""
Analyze response to understand why Shape blocked you
"""
diagnosis = {
'blocked': False,
'reasons': [],
'recommendations': []
}
# Check status code
if response.status_code == 403:
diagnosis['blocked'] = True
diagnosis['reasons'].append('403 Forbidden response')
diagnosis['recommendations'].append('Check TLS fingerprint with curl_cffi')
# Check for Shape script in response
if '_shapesec_' in response.text.lower():
diagnosis['blocked'] = True
diagnosis['reasons'].append('Shape challenge page returned')
diagnosis['recommendations'].append('Need to execute JavaScript - use browser automation')
# Check for challenge redirect
if 'challenge' in response.url.lower():
diagnosis['blocked'] = True
diagnosis['reasons'].append('Redirected to challenge page')
diagnosis['recommendations'].append('Session invalid - recapture cookies')
# Check response headers
shape_headers = [
h for h in response.headers.keys()
if 'shape' in h.lower() or h.startswith('X-') and len(h) < 15
]
if shape_headers:
diagnosis['reasons'].append(f'Shape headers present: {shape_headers}')
# Check cookies
shape_cookies = [
c for c in response.cookies.keys()
if 'shape' in c.lower() or c.startswith('_')
]
if shape_cookies:
diagnosis['reasons'].append(f'Shape cookies: {shape_cookies}')
# Check content length
if len(response.text) < 1000 and 'challenge' in response.text.lower():
diagnosis['blocked'] = True
diagnosis['reasons'].append('Minimal response with challenge indicator')
if verbose:
print("\n=== Shape Block Diagnosis ===")
print(f"Status: {'BLOCKED' if diagnosis['blocked'] else 'OK'}")
print(f"URL: {response.url}")
print(f"Status Code: {response.status_code}")
print(f"Content Length: {len(response.text)}")
print("\nReasons:")
for reason in diagnosis['reasons']:
print(f" - {reason}")
print("\nRecommendations:")
for rec in diagnosis['recommendations']:
print(f" - {rec}")
return diagnosis
Request Comparison Tool
Compare your requests against real browser requests:
import json
from curl_cffi import requests
def compare_with_browser(target_url, browser_har_path):
"""
Compare automated request with browser HAR export
"""
# Load browser HAR
with open(browser_har_path, 'r') as f:
har = json.load(f)
browser_request = None
for entry in har['log']['entries']:
if target_url in entry['request']['url']:
browser_request = entry['request']
break
if not browser_request:
print("Target URL not found in HAR")
return
# Make automated request
session = requests.Session(impersonate="chrome131")
auto_response = session.get(target_url)
# Compare headers
browser_headers = {h['name']: h['value'] for h in browser_request['headers']}
auto_headers = dict(session.headers)
print("=== Header Comparison ===")
all_headers = set(browser_headers.keys()) | set(auto_headers.keys())
for header in sorted(all_headers):
browser_val = browser_headers.get(header, 'MISSING')
auto_val = auto_headers.get(header, 'MISSING')
if browser_val != auto_val:
print(f"\n{header}:")
print(f" Browser: {browser_val}")
print(f" Auto: {auto_val}")
Performance Optimization for Scale
When bypassing Shape at scale, efficiency matters.
Session Caching
import pickle
import time
from pathlib import Path
class ShapeSessionCache:
def __init__(self, cache_dir='./cache'):
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(exist_ok=True)
self.memory_cache = {}
def get_cache_key(self, domain, proxy):
"""Generate unique cache key"""
return f"{domain}_{hash(proxy)}"
def get(self, domain, proxy, max_age_seconds=3600):
"""Retrieve cached session"""
key = self.get_cache_key(domain, proxy)
# Check memory first
if key in self.memory_cache:
entry = self.memory_cache[key]
if time.time() - entry['timestamp'] < max_age_seconds:
return entry['cookies']
# Check disk
cache_file = self.cache_dir / f"{key}.pkl"
if cache_file.exists():
with open(cache_file, 'rb') as f:
entry = pickle.load(f)
if time.time() - entry['timestamp'] < max_age_seconds:
self.memory_cache[key] = entry
return entry['cookies']
return None
def set(self, domain, proxy, cookies):
"""Cache session"""
key = self.get_cache_key(domain, proxy)
entry = {
'cookies': cookies,
'timestamp': time.time()
}
# Memory cache
self.memory_cache[key] = entry
# Disk cache
cache_file = self.cache_dir / f"{key}.pkl"
with open(cache_file, 'wb') as f:
pickle.dump(entry, f)
def cleanup(self, max_age_seconds=3600):
"""Remove expired entries"""
current_time = time.time()
# Clean memory
expired = [
k for k, v in self.memory_cache.items()
if current_time - v['timestamp'] > max_age_seconds
]
for k in expired:
del self.memory_cache[k]
# Clean disk
for cache_file in self.cache_dir.glob('*.pkl'):
try:
with open(cache_file, 'rb') as f:
entry = pickle.load(f)
if current_time - entry['timestamp'] > max_age_seconds:
cache_file.unlink()
except:
cache_file.unlink()
Concurrent Request Manager
import asyncio
from asyncio import Semaphore
from curl_cffi.requests import AsyncSession
class ConcurrentShapeBypass:
def __init__(self, max_concurrent=5, delay_between_requests=2):
self.semaphore = Semaphore(max_concurrent)
self.delay = delay_between_requests
self.session_cache = ShapeSessionCache()
async def fetch(self, url, proxy):
"""Fetch URL with concurrency control"""
async with self.semaphore:
# Check cache
domain = url.split('/')[2]
cached_cookies = self.session_cache.get(domain, proxy)
async with AsyncSession(impersonate="chrome131") as session:
if cached_cookies:
for name, value in cached_cookies.items():
session.cookies.set(name, value)
session.proxies = {'https': proxy, 'http': proxy}
# Delay to avoid rate limits
await asyncio.sleep(self.delay)
response = await session.get(url)
# Cache successful session
if response.status_code == 200:
self.session_cache.set(
domain,
proxy,
dict(session.cookies)
)
return response
async def fetch_many(self, url_proxy_pairs):
"""Fetch multiple URLs concurrently"""
tasks = [
self.fetch(url, proxy)
for url, proxy in url_proxy_pairs
]
return await asyncio.gather(*tasks, return_exceptions=True)
Method 7: WebRTC Leak Prevention
Shape checks WebRTC to discover your real IP even when using proxies. This is a common oversight that exposes automation.
Detecting WebRTC Leaks
async def check_webrtc_leak(page):
"""
Test if WebRTC is leaking real IP
"""
result = await page.evaluate('''
async () => {
return new Promise((resolve) => {
const ips = [];
const rtc = new RTCPeerConnection({
iceServers: [{urls: "stun:stun.l.google.com:19302"}]
});
rtc.createDataChannel("");
rtc.createOffer().then(offer => rtc.setLocalDescription(offer));
rtc.onicecandidate = (event) => {
if (event.candidate) {
const parts = event.candidate.candidate.split(' ');
const ip = parts[4];
if (ip && !ips.includes(ip)) {
ips.push(ip);
}
}
};
setTimeout(() => {
rtc.close();
resolve(ips);
}, 1000);
});
}
''')
return result
Disabling WebRTC Completely
// Inject this to prevent WebRTC leaks
function disableWebRTC() {
// Remove RTCPeerConnection entirely
delete window.RTCPeerConnection;
delete window.webkitRTCPeerConnection;
delete window.RTCSessionDescription;
delete window.RTCIceCandidate;
delete window.MediaStreamTrack;
// Override navigator.mediaDevices
if (navigator.mediaDevices) {
navigator.mediaDevices.getUserMedia = () =>
Promise.reject(new Error('Not supported'));
navigator.mediaDevices.enumerateDevices = () =>
Promise.resolve([]);
}
}
This ensures your proxy IP stays hidden from Shape's WebRTC checks.
HTTP/2 and HTTP/3 Fingerprinting
Modern Shape implementations analyze HTTP/2 settings frames and QUIC fingerprints. This is a newer detection vector that catches many bypasses.
Understanding HTTP/2 Fingerprints
When establishing HTTP/2 connections, clients send SETTINGS frames with parameters like:
- HEADER_TABLE_SIZE
- ENABLE_PUSH
- MAX_CONCURRENT_STREAMS
- INITIAL_WINDOW_SIZE
- MAX_FRAME_SIZE
- MAX_HEADER_LIST_SIZE
Different browsers have distinct patterns. Python's httpx or aiohttp don't match any real browser.
curl_cffi HTTP/2 Matching
The advantage of curl_cffi is it replicates Chrome's exact HTTP/2 settings:
from curl_cffi import requests
# Chrome's HTTP/2 SETTINGS are automatically replicated
session = requests.Session(impersonate="chrome131")
# Verify your fingerprint
response = session.get('https://tls.browserleaks.com/json')
fingerprint = response.json()
print(f"JA3 Hash: {fingerprint.get('ja3_hash')}")
print(f"HTTP/2 Akamai FP: {fingerprint.get('akamai_hash')}")
For custom HTTP/2 settings (advanced use):
# Custom fingerprinting with curl_cffi
session = requests.Session(
ja3="771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0",
akamai="1:65536,2:0,4:6291456,6:262144|15663105|0|m,a,s,p"
)
Real-World Case Studies
Case Study 1: E-Commerce Price Monitoring
Challenge: Monitor prices on a major retailer protected by Shape. 50,000 product pages, updated daily.
Solution Architecture:
import asyncio
from curl_cffi.requests import AsyncSession
from dataclasses import dataclass
from typing import List
import random
@dataclass
class Product:
url: str
sku: str
class EcommerceScraper:
def __init__(self, proxy_pool: List[str]):
self.proxy_pool = proxy_pool
self.rate_limit = 2 # seconds between requests per proxy
self.proxy_last_used = {}
async def scrape_product(self, product: Product, session: AsyncSession):
"""Scrape single product with Shape bypass"""
proxy = self.get_available_proxy()
try:
session.proxies = {'https': proxy}
# Random delay
await asyncio.sleep(random.uniform(1.5, 3.5))
response = await session.get(product.url)
if response.status_code == 200 and 'price' in response.text.lower():
# Extract price (implementation specific)
return self.extract_price(response.text, product.sku)
return None
except Exception as e:
print(f"Failed {product.sku}: {e}")
return None
def get_available_proxy(self):
"""Get proxy respecting rate limits"""
import time
now = time.time()
for proxy in self.proxy_pool:
last_used = self.proxy_last_used.get(proxy, 0)
if now - last_used >= self.rate_limit:
self.proxy_last_used[proxy] = now
return proxy
# All proxies on cooldown - wait
oldest = min(self.proxy_last_used.values())
time.sleep(self.rate_limit - (now - oldest))
return self.get_available_proxy()
async def scrape_batch(self, products: List[Product]):
"""Scrape products in batches"""
async with AsyncSession(impersonate="chrome131") as session:
results = []
for i, product in enumerate(products):
result = await self.scrape_product(product, session)
results.append(result)
if i % 100 == 0:
print(f"Progress: {i}/{len(products)}")
return results
Results: 93% success rate with 200 residential proxies, processing 50,000 products in ~8 hours.
Case Study 2: Travel Site Fare Aggregation
Challenge: Aggregate flight prices from an airline protected by Shape. Real-time data needed.
Solution: Hybrid approach using nodriver for session creation and curl_cffi for data requests.
import nodriver as nd
import asyncio
from curl_cffi import requests
class FlightScraper:
def __init__(self):
self.session_cookies = None
self.session_created = None
async def refresh_session(self, base_url):
"""Use nodriver to solve Shape challenge once"""
browser = await nd.start(headless=False)
try:
tab = await browser.get(base_url)
await tab.wait(5) # Wait for Shape
# Simulate interaction
await tab.evaluate('window.scrollBy(0, 300)')
await asyncio.sleep(2)
# Extract cookies
cookies = await browser.cookies.get_all()
self.session_cookies = {c.name: c.value for c in cookies}
self.session_created = asyncio.get_event_loop().time()
finally:
await browser.stop()
def search_flights(self, origin, destination, date):
"""Search flights using captured session"""
# Refresh session if older than 30 minutes
if not self.session_cookies or \
(asyncio.get_event_loop().time() - self.session_created) > 1800:
asyncio.run(self.refresh_session('https://airline.com'))
session = requests.Session(impersonate="chrome131")
for name, value in self.session_cookies.items():
session.cookies.set(name, value)
# Make API request
search_url = f"https://airline.com/api/search"
params = {
'origin': origin,
'destination': destination,
'date': date
}
response = session.get(search_url, params=params)
return response.json() if response.status_code == 200 else None
Staying Ahead: Shape's 2026 Updates
Based on F5's recent acquisitions and announcements, here's what to expect:
AI-Enhanced Detection
F5 acquired Fletch (agentic AI for threat detection) and CalypsoAI (runtime AI security) in 2025. Expect Shape to integrate:
- Real-time behavioral anomaly detection using ML
- Adaptive challenge generation
- Cross-session pattern analysis
Countermeasures
Stay ahead by:
- Randomizing everything - Don't let patterns emerge in your requests
- Using diverse fingerprints - Rotate browser profiles, not just IPs
- Monitoring success rates - Detect when Shape adapts and adjust quickly
- Keeping tools updated - curl_cffi and nodriver release updates for new browser versions
JA4 Fingerprinting
JA4 is the successor to JA3, adding more TLS handshake details. curl_cffi is already adapting, but watch for:
# Future curl_cffi may support JA4 specification
# session = requests.Session(ja4="...")
Common Mistakes That Get You Blocked
Learning from failures is just as important as knowing what works.
Mistake 1: Ignoring Request Timing
# BAD - Machine-like timing
for url in urls:
response = session.get(url)
# GOOD - Human-like timing with jitter
import random
for url in urls:
time.sleep(random.uniform(2, 5) + random.gauss(0, 0.5))
response = session.get(url)
Mistake 2: Static Headers
# BAD - Same headers every request
headers = {'User-Agent': 'Mozilla/5.0...'}
# GOOD - Rotate and vary headers
def get_headers():
return {
'User-Agent': random.choice(USER_AGENTS),
'Accept-Language': random.choice(['en-US,en;q=0.9', 'en-GB,en;q=0.8']),
'Sec-Ch-Ua': f'"Chromium";v="{random.randint(128, 131)}"'
}
Mistake 3: Sequential URL Patterns
# BAD - Predictable crawling pattern
for i in range(1, 1000):
session.get(f'https://site.com/product/{i}')
# GOOD - Randomized access pattern
product_ids = list(range(1, 1000))
random.shuffle(product_ids)
for pid in product_ids:
session.get(f'https://site.com/product/{pid}')
Mistake 4: Ignoring Geographic Consistency
Your proxy location should match your browser locale:
# BAD - US locale with German proxy
session.headers['Accept-Language'] = 'en-US,en;q=0.9'
session.proxies = {'https': 'german_proxy'}
# GOOD - Matched locale and proxy
session.headers['Accept-Language'] = 'de-DE,de;q=0.9,en;q=0.8'
session.proxies = {'https': 'german_proxy'}
Ethical Considerations and Legal Notes
These techniques are for educational purposes and legitimate security testing only.
Always follow these rules:
- Only test systems you own or have explicit permission to test
- Respect rate limits - don't overwhelm servers
- Follow responsible disclosure if you discover vulnerabilities
- Comply with all applicable laws in your jurisdiction
- Honor robots.txt and terms of service where applicable
- Never use these techniques for fraud, credential stuffing, or unauthorized access
The goal is understanding detection mechanisms for defensive purposes, not enabling malicious activity.
Conclusion: What Works in 2026
Shape/F5 Distributed Cloud Bot Defense is the most sophisticated antibot system you'll encounter. But it's not invincible.
The techniques that work right now:
- curl_cffi with proper TLS fingerprinting - Handles 60-70% of Shape-protected endpoints without browser automation
- Nodriver - Bypasses CDP detection that catches other automation tools
- Session hijacking - One manual solve enables many automated requests
- Mobile API impersonation - Often weaker detection than web endpoints
- Smart residential proxy rotation - Essential for any scale operation
Key insights:
- TLS fingerprinting alone blocks most amateur attempts
- CDP detection is now standard - avoid Puppeteer/Playwright unless properly patched
- Shape adapts within 24-48 hours - what works today may not work next week
- Residential proxies are non-negotiable for scale
The arms race continues. Shape will adapt to these techniques, and new bypasses will emerge. Stay curious, stay ethical, and keep learning.
Quick Reference: Shape Bypass Decision Tree
Start
|
v
Does endpoint require JavaScript execution?
|
+-- No --> Use curl_cffi with impersonate
| Success? --> Done
| Blocked? --> Check TLS fingerprint
|
+-- Yes --> Need browser automation
|
v
Is CDP detection active?
|
+-- Yes --> Use nodriver
|
+-- No --> Use Playwright with stealth
|
v
Still blocked?
|
+-- Yes --> Session hijacking approach
| Or try mobile API
|
+-- No --> Done
If scaling:
- Add residential proxy rotation
- Implement session caching
- Use concurrent request management
