> ## Content Index
> Fetch the complete content index at: https://roundproxies.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# How to Set Up Airtop for AI Web Automation
- URL: https://roundproxies.com/blog/airtop/
- Published: 2025-08-10T00:37:34.000Z
- Updated: 2026-09-23T13:04:24.000Z
- Description: Learn to set up Airtop for AI web automation with proxies. Handle OAuth, 2FA, bypass anti-bot detection. Code examples included.
- Author: Marius Bernard
- Tags: #dated-689615fc4fa498a2c6ee0c51

Airtop is a cloud browser automation platform that lets you control browsers through natural language commands and handle complex authentication like OAuth, 2FA, and CAPTCHAs automatically.

This guide covers setting up Airtop from scratch, including proxy configuration for bypassing anti-bot detection and handling authenticated sessions.

## What You'll Learn

By the end, you'll be able to:

- Set up Airtop with proper API authentication
- Configure residential proxies for stealth scraping
- Handle complex authentication flows (OAuth, 2FA)
- Build automated scrapers with natural language commands
- Bypass anti-bot measures effectively

## Step 1: Get Your API Key and Initial Setup

You need an Airtop account and API key to get started.

### Setting Up Your Account

1. Head to [portal.airtop.ai](https://portal.airtop.ai/) and create a free account
2. Navigate to the API Keys section in your dashboard
3. Click "+ Create new key" and give it a descriptive name
4. Copy the generated key immediately (you won't see it again)

### Installing the SDK

Airtop provides SDKs for both TypeScript/Node.js and Python. Pick your weapon:

**For Node.js/TypeScript:**

```bash
npm install @airtop/sdk
# or with yarn
yarn add @airtop/sdk

```

**For Python:**

```bash
pip install airtop

```

### Environment Configuration

Create a `.env` file in your project root:

```bash
AIRTOP_API_KEY=your_api_key_here

```

## Step 2: Initialize Your First Browser Session

Let's create a basic browser session to test everything's working:

**TypeScript Example:**

```typescript
import { AirtopClient } from "@airtop/sdk";
import * as dotenv from 'dotenv';

dotenv.config();

const client = new AirtopClient({
    apiKey: process.env.AIRTOP_API_KEY
});

async function createSession() {
    try {
        // Create a browser session
        const session = await client.sessions.create({
            configuration: {
                // Session lasts 10 minutes by default
                timeoutMinutes: 10
            }
        });
        
        console.log(`Session created: ${session.id}`);
        
        // Create a window and navigate
        const window = await client.windows.create(session.id, {
            url: "https://example.com"
        });
        
        console.log(`Window created: ${window.id}`);
        
        return { session, window };
    } catch (error) {
        console.error("Failed to create session:", error);
    }
}

createSession();

```

**Python Example:**

```python
import os
from airtop import Airtop
from dotenv import load_dotenv

load_dotenv()

client = Airtop(
    api_key=os.getenv("AIRTOP_API_KEY")
)

async def create_session():
    try:
        # Create a browser session
        session = await client.sessions.create(
            configuration={
                "timeoutMinutes": 10
            }
        )
        
        print(f"Session created: {session.id}")
        
        # Create a window
        window = await client.windows.create(
            session_id=session.id,
            url="https://example.com"
        )
        
        print(f"Window created: {window.id}")
        
        return session, window
        
    except Exception as e:
        print(f"Failed to create session: {e}")

```

## Step 3: Configure Residential Proxies

Here's where things get interesting. Airtop has built-in residential proxy support with over 100 million IPs from 100+ countries. This matters for bypassing anti-bot detection.

### Using Airtop's Integrated Proxy

```typescript
const sessionWithProxy = await client.sessions.create({
    configuration: {
        proxy: {
            type: "residential",
            country: "US",  // ISO 3166-1 format
            sticky: true     // Keep same IP for 30 minutes
        }
    }
});

```

### Using Your Own Proxy

If you already have a proxy provider (like Bright Data, Oxylabs, or SmartProxy), you can bring your own:

```typescript
const sessionWithCustomProxy = await client.sessions.create({
    configuration: {
        proxy: {
            server: "http://proxy.example.com:8080",
            username: "your_username",
            password: "your_password"
        }
    }
});

```

### Domain-Specific Proxy Routing

This is a neat trick: you can route only specific domains through the proxy while letting others connect directly:

```typescript
const sessionWithSelectiveProxy = await client.sessions.create({
    configuration: {
        proxy: [
            {
                domainPattern: "*.wikipedia.org",
                relay: {
                    type: "residential",
                    country: "UK"
                }
            },
            {
                domainPattern: "*",  // All other domains
                relay: null          // No proxy
            }
        ]
    }
});

```

## Step 4: Handle Authentication Like a Pro

One of Airtop's killer features is handling complex authentication flows automatically. Here's how to scrape data behind login walls:

```typescript
async function scrapeWithAuth() {
    const session = await client.sessions.create();
    
    // Create a window
    const window = await client.windows.create(session.id, {
        url: "https://linkedin.com"
    });
    
    // Create a live view for manual login
    const liveView = await client.windows.createLiveView(
        session.id, 
        window.id
    );
    
    console.log(`Login here: ${liveView.url}`);
    console.log("Complete the login process (including 2FA if needed)");
    
    // Wait for user to complete login
    await new Promise(resolve => {
        setTimeout(resolve, 60000); // Wait 60 seconds
    });
    
    // Save the authenticated profile
    const profile = await client.profiles.create({
        sessionId: session.id,
        name: "linkedin-authenticated"
    });
    
    // Now you can reuse this profile for future sessions
    const newSession = await client.sessions.create({
        profileId: profile.id
    });
    
    // Extract data using natural language
    const data = await client.windows.pageQuery(
        newSession.id,
        window.id,
        {
            prompt: "Extract all job postings with company names, titles, and locations",
            configuration: {
                outputSchema: {
                    type: "array",
                    items: {
                        type: "object",
                        properties: {
                            company: { type: "string" },
                            title: { type: "string" },
                            location: { type: "string" }
                        }
                    }
                }
            }
        }
    );
    
    return data;
}

```

## Step 5: Use Natural Language Commands

Instead of writing complex selectors, use Airtop's AI to interact with pages naturally:

```typescript
async function naturalLanguageAutomation() {
    const session = await client.sessions.create({
        configuration: {
            proxy: { type: "residential", country: "US" }
        }
    });
    
    const window = await client.windows.create(session.id, {
        url: "https://producthunt.com"
    });
    
    // Extract structured data with a simple prompt
    const products = await client.windows.pageQuery(
        session.id,
        window.id,
        {
            prompt: `Find all new product launches from today. 
                     For each product, extract:
                     - Product name
                     - Description
                     - Vote count
                     - Maker name
                     Ignore sponsored listings`,
            configuration: {
                followPagination: true,  // Automatically handle pagination
                maxPages: 5
            }
        }
    );
    
    // Interact with the page
    await client.windows.act(session.id, window.id, {
        action: "Click on the first product that has more than 100 votes"
    });
    
    // Take a screenshot
    const screenshot = await client.windows.screenshot(
        session.id, 
        window.id
    );
    
    return products;
}

```

## Step 6: Build a Production-Ready Scraper

Let's put it all together with a real-world example: monitoring competitor pricing:

```typescript
import { AirtopClient } from "@airtop/sdk";
import * as fs from 'fs';

class CompetitorPriceMonitor {
    private client: AirtopClient;
    private profileId?: string;
    
    constructor(apiKey: string) {
        this.client = new AirtopClient({ apiKey });
    }
    
    async initialize() {
        // Check if we have a saved profile
        const profilePath = './competitor-profile.json';
        if (fs.existsSync(profilePath)) {
            const profile = JSON.parse(fs.readFileSync(profilePath, 'utf-8'));
            this.profileId = profile.id;
        }
    }
    
    async monitorPricing(competitorUrl: string) {
        try {
            // Create session with proxy rotation
            const session = await this.client.sessions.create({
                profileId: this.profileId,
                configuration: {
                    proxy: {
                        type: "residential",
                        country: "US",
                        sticky: false  // Rotate IP for each request
                    }
                }
            });
            
            const window = await this.client.windows.create(session.id, {
                url: competitorUrl
            });
            
            // Wait for page to load
            await this.client.windows.waitForLoad(session.id, window.id);
            
            // Extract pricing data
            const pricingData = await this.client.windows.pageQuery(
                session.id,
                window.id,
                {
                    prompt: `Extract all pricing plans with:
                            - Plan name
                            - Monthly price
                            - Annual price
                            - Top 3 features
                            - Any discounts or promotions`,
                    configuration: {
                        outputSchema: {
                            type: "object",
                            properties: {
                                plans: {
                                    type: "array",
                                    items: {
                                        type: "object",
                                        properties: {
                                            name: { type: "string" },
                                            monthlyPrice: { type: "number" },
                                            annualPrice: { type: "number" },
                                            features: {
                                                type: "array",
                                                items: { type: "string" }
                                            },
                                            discount: { type: "string" }
                                        }
                                    }
                                },
                                lastUpdated: { type: "string" }
                            }
                        }
                    }
                }
            );
            
            // Compare with previous data
            const previousData = this.loadPreviousData(competitorUrl);
            const changes = this.detectChanges(previousData, pricingData);
            
            if (changes.length > 0) {
                await this.notifyChanges(competitorUrl, changes);
            }
            
            // Save current data
            this.savePricingData(competitorUrl, pricingData);
            
            // Clean up
            await this.client.sessions.terminate(session.id);
            
            return pricingData;
            
        } catch (error) {
            console.error(`Error monitoring ${competitorUrl}:`, error);
            throw error;
        }
    }
    
    private detectChanges(previous: any, current: any): any[] {
        // Implementation for change detection
        const changes = [];
        // Compare pricing, features, etc.
        return changes;
    }
    
    private async notifyChanges(url: string, changes: any[]) {
        // Send notifications (email, Slack, etc.)
        console.log(`Price changes detected for ${url}:`, changes);
    }
    
    private loadPreviousData(url: string): any {
        // Load from database or file
        return null;
    }
    
    private savePricingData(url: string, data: any) {
        // Save to database or file
        fs.writeFileSync(
            `./pricing-data/${url.replace(/[^a-z0-9]/gi, '_')}.json`,
            JSON.stringify(data, null, 2)
        );
    }
}

// Usage
async function main() {
    const monitor = new CompetitorPriceMonitor(process.env.AIRTOP_API_KEY!);
    await monitor.initialize();
    
    const competitors = [
        "https://competitor1.com/pricing",
        "https://competitor2.com/plans",
        "https://competitor3.com/pricing"
    ];
    
    for (const url of competitors) {
        await monitor.monitorPricing(url);
        // Add delay to avoid rate limiting
        await new Promise(resolve => setTimeout(resolve, 5000));
    }
}

main().catch(console.error);

```

## Advanced Tips and Tricks

### 1\. Bypass Cloudflare and Other Anti-Bot Systems

```typescript
const stealthSession = await client.sessions.create({
    configuration: {
        proxy: {
            type: "residential",
            country: "US"
        },
        viewport: {
            width: 1920,
            height: 1080
        },
        userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
        // Airtop automatically handles fingerprinting
        stealth: true
    }
});

```

### 2\. Handle Dynamic Content and Infinite Scroll

```typescript
const scrollAndExtract = async (sessionId: string, windowId: string) => {
    // Scroll to load dynamic content
    await client.windows.act(sessionId, windowId, {
        action: "Scroll to the bottom of the page slowly over 5 seconds"
    });
    
    // Extract data after scrolling
    const data = await client.windows.pageQuery(sessionId, windowId, {
        prompt: "Extract all product cards that are now visible",
        configuration: {
            waitForStable: true  // Wait for content to stop changing
        }
    });
    
    return data;
};

```

### 3\. Smart CAPTCHA Handling

While Airtop handles many CAPTCHAs automatically, for complex ones you can use the live view:

```typescript
async function handleCaptcha(session: any, window: any) {
    const captchaDetected = await client.windows.pageQuery(
        session.id,
        window.id,
        {
            prompt: "Is there a CAPTCHA on this page? Return true or false"
        }
    );
    
    if (captchaDetected) {
        const liveView = await client.windows.createLiveView(
            session.id,
            window.id
        );
        
        console.log(`Manual intervention needed: ${liveView.url}`);
        // Wait for human to solve CAPTCHA
        await new Promise(resolve => setTimeout(resolve, 30000));
    }
}

```

## Pricing and Alternatives

### Airtop Pricing Tiers

- **Free Plan**: 5,000 credits, 1 simultaneous session
- **Starter**: $29/month, 3 simultaneous sessions, integrated proxy
- **Professional**: $89/month, 30 simultaneous sessions, custom proxy support
- **Enterprise**: $380+/month, 100+ sessions, SOC 2 compliance, dedicated support

Concurrency is usually what forces the upgrade before credits do. On the free plan every job runs strictly in series, which is fine while you are building and testing a script, and painful the moment you want to crawl fifty profiles in parallel. Estimate the number of sessions you need to have open at once, then pick the tier above it.

The proxy line matters just as much. Starter gives you Airtop's integrated proxy, which is the shortest path to residential-looking traffic. Professional lets you point sessions at your own provider, so if you already pay for proxy pools you keep that contract and that IP reputation. Enterprise adds SOC 2 and dedicated support, which tends to be the deciding factor when the automation logs into company accounts with real credentials.

### When to Use Airtop vs Alternatives

Airtop and the open-source browser drivers compose rather than compete. Each session exposes a `cdpWsUrl`, so an existing Puppeteer, Playwright, or Selenium script can connect to the hosted cloud browser over CDP and run unchanged, while the natural-language APIs (`pageQuery`, `paginatedExtraction`, `formFilling`, `monitor`) cover the extraction and form work that keeps breaking on selector changes. That makes migration incremental: keep your navigation code, hand off the brittle parts.

**Choose Airtop when:**

- You need to handle complex authentication (OAuth, 2FA) and reuse the resulting session across runs
- Natural language commands appeal to you, especially for paginated lists and form filling
- You're scraping JavaScript-heavy sites with anti-bot measures
- You need reliable proxy rotation without operating a browser fleet

**Consider Playwright/Puppeteer when:**

- You need fine-grained control over browser behavior
- You're building complex test suites
- Budget is extremely tight (both are free)
- You have existing infrastructure for proxy management

**Consider Selenium when:**

- You need support for legacy browsers
- Your team uses languages like Java or C#
- You have existing Selenium infrastructure

There are also two paths that skip the SDK entirely. Airtop ships as a native n8n node with an API-key credential and covers the full operation set (session, window, interaction, extraction including Smart Scrape, file, and agent run), which is the fastest way to hang browser steps off an existing workflow. Above that sits Agent Builder and the deployable agent, Mark, which runs compiled, deterministic jobs and gained a self-healing capability in August 2026 that recovers when a target site changes its layout, the exact failure mode that breaks hand-written selectors.

## Common Pitfalls to Avoid

1. **Not rotating proxies enough**: Some sites track IP patterns. Use `sticky: false` for aggressive sites.
2. **Not saving authenticated profiles**: Authentication is expensive. Always save profiles for reuse.
3. **Over-relying on natural language**: Sometimes explicit selectors are more reliable for production systems.

**Ignoring rate limits**: Even with proxies, respect rate limits. Add delays between requests:

```typescript
await new Promise(resolve => setTimeout(resolve, Math.random() * 3000 + 2000));

```

## Next Steps

Now that you have Airtop configured, consider these advanced implementations:

- Build a distributed scraping system with queue management
- Integrate with data pipelines using Apache Airflow
- Create a monitoring dashboard with real-time alerts
- Implement automatic proxy health checking and rotation

Remember, Airtop shines when you need to interact with complex, modern web applications that would typically require human intervention. The combination of cloud browsers, natural language processing, and integrated proxy support makes it a powerful tool for scenarios where traditional scraping fails.