Proxies in Rust let you route HTTP requests through intermediate servers, dodge rate limits, and scale your data extraction operations.
This guide shows you how to implement proxy support using reqwest, from basic setup to advanced rotation strategies that actually work in production.
Whether you're scraping at scale or just need to bypass geo-restrictions, you'll learn the exact patterns that work—and the gotchas that'll waste your time if you don't know about them.
Step 1: Set up basic proxy routing
Start with the simplest case—routing all requests through a single proxy.
use reqwest::{Client, Proxy};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Basic HTTP proxy
let proxy = Proxy::all("http://proxy.example.com:8080")?;
let client = Client::builder()
.proxy(proxy)
.build()?;
let response = client
.get("https://httpbin.org/ip")
.send()
.await?;
println!("Response: {}", response.text().await?);
Ok(())
}
But here's where most tutorials stop. What they don't tell you is that Proxy::all() only handles HTTP and HTTPS—it ignores WebSocket connections and other protocols. If you need protocol-specific routing:
// Different proxies for different protocols
let client = Client::builder()
.proxy(Proxy::http("http://http-proxy.com:8080")?)
.proxy(Proxy::https("http://https-proxy.com:8080")?)
.build()?;
SOCKS5 Support (The Part Nobody Mentions)
Enable the socks feature in your Cargo.toml:
[dependencies]
reqwest = { version = "0.12", features = ["socks"] }
Then use it like any other proxy:
let proxy = Proxy::all("socks5://127.0.0.1:1080")?;
SOCKS5 proxies are slower but harder to detect. They operate at a lower network level, making them ideal for bypassing application-layer filters.
Step 2: Add authentication (without leaking credentials)
Hard-coding credentials is amateur hour. Here's how to do it right:
use std::env;
fn create_authenticated_proxy() -> Result<Proxy, Box<dyn std::error::Error>> {
let proxy_url = env::var("PROXY_URL")?;
let username = env::var("PROXY_USER")?;
let password = env::var("PROXY_PASS")?;
let proxy = Proxy::all(proxy_url)?
.basic_auth(&username, &password);
Ok(proxy)
}
Custom Authentication Headers
Some proxy providers use non-standard auth. Here's how to handle them:
use reqwest::header::HeaderValue;
let proxy = Proxy::all("http://proxy.example.com:8080")?
.custom_http_auth(HeaderValue::from_static("Bearer your-api-key"));
Conditional Proxying (The Smart Way)
Don't proxy local traffic or trusted domains—it's slow and pointless:
let proxy = Proxy::custom(move |url| {
// Skip proxy for internal APIs
if url.host_str() == Some("api.internal.com") {
return None;
}
// Skip for localhost
if url.host_str().map_or(false, |h| h.starts_with("localhost")) {
return None;
}
// Proxy everything else
Some("http://proxy.example.com:8080".parse().unwrap())
});
Step 3: Build a rotation pool that doesn't suck
Static proxies get banned. Here's a rotation system that actually works:
use std::sync::{Arc, Mutex};
use rand::seq::SliceRandom;
#[derive(Clone)]
struct ProxyPool {
proxies: Arc<Mutex<Vec<String>>>,
failures: Arc<Mutex<std::collections::HashMap<String, u32>>>,
}
impl ProxyPool {
fn new(proxies: Vec<String>) -> Self {
Self {
proxies: Arc::new(Mutex::new(proxies)),
failures: Arc::new(Mutex::new(std::collections::HashMap::new())),
}
}
fn get_proxy(&self) -> Option<String> {
let mut proxies = self.proxies.lock().unwrap();
let failures = self.failures.lock().unwrap();
// Filter out proxies with too many failures
let healthy_proxies: Vec<_> = proxies
.iter()
.filter(|p| failures.get(*p).unwrap_or(&0) < &3)
.cloned()
.collect();
healthy_proxies.choose(&mut rand::thread_rng()).cloned()
}
fn mark_failure(&self, proxy: &str) {
let mut failures = self.failures.lock().unwrap();
*failures.entry(proxy.to_string()).or_insert(0) += 1;
}
fn mark_success(&self, proxy: &str) {
let mut failures = self.failures.lock().unwrap();
failures.remove(proxy);
}
}
Dynamic Client Creation (The Performance Killer)
Most rotation examples recreate the client for each request. Don't do that—it's slow as hell. Instead, use a custom proxy function:
use std::sync::atomic::{AtomicUsize, Ordering};
fn rotating_proxy() -> impl Fn(&reqwest::Url) -> Option<reqwest::Url> {
let proxies = vec![
"http://proxy1.com:8080",
"http://proxy2.com:8080",
"http://proxy3.com:8080",
];
let counter = AtomicUsize::new(0);
move |_url| {
let index = counter.fetch_add(1, Ordering::Relaxed) % proxies.len();
proxies[index].parse().ok()
}
}
// Use it
let client = Client::builder()
.proxy(Proxy::custom(rotating_proxy()))
.build()?;
Step 4: Handle failures like they'll actually happen
Proxies fail. Networks drop. Here's how to handle it without losing data:
use std::time::Duration;
use tokio::time::sleep;
async fn request_with_retry(
client: &Client,
url: &str,
max_retries: u32,
) -> Result<String, Box<dyn std::error::Error>> {
let mut retries = 0;
let mut backoff = Duration::from_millis(100);
loop {
match client.get(url).timeout(Duration::from_secs(10)).send().await {
Ok(response) if response.status().is_success() => {
return Ok(response.text().await?);
}
Ok(response) if response.status() == 407 => {
// Proxy auth failed - don't retry
return Err("Proxy authentication failed".into());
}
Ok(response) if response.status() == 429 => {
// Rate limited - exponential backoff
if retries >= max_retries {
return Err("Max retries exceeded".into());
}
sleep(backoff).await;
backoff *= 2;
retries += 1;
}
Err(e) if e.is_timeout() => {
// Timeout - try different proxy
if retries >= max_retries {
return Err("Request timeout".into());
}
retries += 1;
}
Err(e) if e.is_connect() => {
// Connection failed - proxy might be dead
return Err(format!("Proxy connection failed: {}", e).into());
}
_ => {
if retries >= max_retries {
return Err("Unknown error".into());
}
retries += 1;
}
}
}
}
Circuit Breaker Pattern
Stop hammering dead proxies:
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
use std::time::{SystemTime, UNIX_EPOCH};
struct CircuitBreaker {
failure_count: AtomicU32,
last_failure: AtomicU64,
threshold: u32,
timeout_secs: u64,
}
impl CircuitBreaker {
fn new(threshold: u32, timeout_secs: u64) -> Self {
Self {
failure_count: AtomicU32::new(0),
last_failure: AtomicU64::new(0),
threshold,
timeout_secs,
}
}
fn is_open(&self) -> bool {
let failures = self.failure_count.load(Ordering::Relaxed);
if failures < self.threshold {
return false;
}
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_secs();
let last = self.last_failure.load(Ordering::Relaxed);
if now - last > self.timeout_secs {
// Reset after timeout
self.failure_count.store(0, Ordering::Relaxed);
false
} else {
true
}
}
fn record_failure(&self) {
self.failure_count.fetch_add(1, Ordering::Relaxed);
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_secs();
self.last_failure.store(now, Ordering::Relaxed);
}
fn record_success(&self) {
self.failure_count.store(0, Ordering::Relaxed);
}
}
Step 5: Bypass detection with request fingerprinting
Here's the stuff that actually gets you past anti-bot systems:
Randomize TLS Fingerprints
Standard reqwest uses the same TLS configuration every time. Dead giveaway. Fix it:
use reqwest::tls;
let client = Client::builder()
.danger_accept_invalid_certs(false)
.min_tls_version(tls::Version::TLS_1_2)
.use_rustls_tls() // Different TLS implementation
.proxy(proxy)
.build()?;
Realistic Headers
Stop sending default reqwest headers:
use reqwest::header::{HeaderMap, HeaderValue, USER_AGENT, ACCEPT, ACCEPT_LANGUAGE, ACCEPT_ENCODING};
fn realistic_headers() -> HeaderMap {
let mut headers = HeaderMap::new();
// Rotate user agents
let user_agents = vec![
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15",
];
let ua = user_agents.choose(&mut rand::thread_rng()).unwrap();
headers.insert(USER_AGENT, HeaderValue::from_static(ua));
headers.insert(ACCEPT, HeaderValue::from_static("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"));
headers.insert(ACCEPT_LANGUAGE, HeaderValue::from_static("en-US,en;q=0.5"));
headers.insert(ACCEPT_ENCODING, HeaderValue::from_static("gzip, deflate, br"));
headers.insert("DNT", HeaderValue::from_static("1"));
headers.insert("Connection", HeaderValue::from_static("keep-alive"));
headers.insert("Upgrade-Insecure-Requests", HeaderValue::from_static("1"));
headers
}
let client = Client::builder()
.default_headers(realistic_headers())
.proxy(proxy)
.build()?;
Request Timing Patterns
Don't hammer endpoints like a bot:
use rand::Rng;
async fn human_like_delay() {
let mut rng = rand::thread_rng();
let delay = rng.gen_range(500..3000);
tokio::time::sleep(Duration::from_millis(delay)).await;
}
// Between requests
human_like_delay().await;
The Nuclear Option: Raw TCP Sockets
When reqwest isn't enough, go lower level:
use tokio::net::TcpStream;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
async fn raw_proxy_request(
proxy_addr: &str,
target_host: &str,
target_port: u16,
) -> Result<String, Box<dyn std::error::Error>> {
let mut stream = TcpStream::connect(proxy_addr).await?;
// Send CONNECT for HTTPS
let connect_request = format!(
"CONNECT {}:{} HTTP/1.1\r\nHost: {}:{}\r\n\r\n",
target_host, target_port, target_host, target_port
);
stream.write_all(connect_request.as_bytes()).await?;
let mut buffer = vec![0; 1024];
let n = stream.read(&mut buffer).await?;
let response = String::from_utf8_lossy(&buffer[..n]);
if !response.contains("200 Connection Established") {
return Err("Proxy connection failed".into());
}
// Now you have a raw tunnel - do whatever
// TLS handshake, custom protocols, etc.
Ok(response.to_string())
}
Performance Tricks Nobody Talks About
Connection Pooling
Reuse connections across proxy requests:
let client = Client::builder()
.pool_max_idle_per_host(10)
.pool_idle_timeout(Duration::from_secs(90))
.proxy(proxy)
.build()?;
DNS Caching
Avoid DNS lookups for every request:
use trust_dns_resolver::AsyncResolver;
use std::net::IpAddr;
async fn resolve_once(domain: &str) -> Result<IpAddr, Box<dyn std::error::Error>> {
let resolver = AsyncResolver::tokio_from_system_conf()?;
let response = resolver.lookup_ip(domain).await?;
response.iter().next()
.ok_or_else(|| "No IP found".into())
}
// Cache the result and reuse
HTTP/2 Multiplexing
Send multiple requests over one connection:
let client = Client::builder()
.http2_prior_knowledge() // Force HTTP/2
.proxy(proxy)
.build()?;
Next Steps
You now have production-grade proxy handling in Rust. But here's what separates the pros from the script kiddies:
- Monitor proxy health: Build a background task that continuously tests proxy availability
- Geographic distribution: Route requests through proxies in the target's region
- Session persistence: Some sites track sessions—use the same proxy for related requests
- Protocol detection: Automatically detect whether a site needs HTTP, HTTPS, or SOCKS5
The real power isn't in using proxies—it's in using them intelligently. Start with the basics, then layer on sophistication as you hit real-world limits.
