> ## Content Index > Fetch the complete content index at: https://roundproxies.com/blog/llms.txt > Use this file to discover other available public pages before exploring further. # 7 ways to scrape Twitch in 2026 - URL: https://roundproxies.com/blog/scrape-twitch/ - Published: 2026-07-22T22:33:21.000Z - Updated: 2026-09-23T13:04:13.000Z - Description: Scrape Twitch in 2026 with 7 self-built methods. Working Python and Node code, no paid APIs. - Author: Marius Bernard - Tags: Web Scraping, #dated-690d0b93358d710426c14a8a You want viewer counts, stream titles, or chat logs off Twitch. You write a scraper, point it at twitch.tv, and get back a near-empty HTML shell. That's because Twitch renders almost everything client-side and serves its real data through APIs. Scrape the wrong layer and you get nothing. This guide covers seven ways to scrape Twitch, from the official API to the private GraphQL endpoint the site itself uses. Every method is something you build and run yourself. No paid scraping service. ## How do you scrape Twitch? Scraping Twitch means pulling public data (stream titles, viewer counts, chat, clips, VODs) from Twitch's servers with code instead of by hand. The fastest routes are Twitch's own Helix API and its private GraphQL endpoint, both returning clean JSON. For live chat, connect to Twitch IRC over WebSocket. Pick a method by how much data you need and how often. ## Why the twitch.tv HTML is useless to a scraper Open a channel page, view source, and search for the viewer count. It isn't there. Twitch ships a React app. The page arrives nearly empty, then JavaScript fires off API calls to fill it in. So the data you want lives in those API responses, not the HTML. Once you know that, scraping Twitch stops being about parsing pages and starts being about hitting the right endpoint. There are three data layers worth knowing: - **Helix** is the official, documented REST API. Clean, stable, rate-limited. - **GraphQL** (`gql.twitch.tv`) is the private API the website runs on. Richer, undocumented, no signup. - **IRC** is the chat backend. Real-time messages over WebSocket, readable anonymously. Every method below taps one of these. ## What data can you pull from Twitch? Before picking a method, know what's actually reachable. Not every field lives in every layer. Stream and channel data is the easy stuff. Titles, viewer counts, categories, language, uptime, follower totals, and profile details all come back as clean JSON from Helix or GraphQL. Video data covers VODs and clips: durations, view counts, creation dates, thumbnails, and the game each belongs to. GraphQL exposes more of this than Helix does. Chat is its own world. Over IRC you get every message live, plus the sender's badges, color, subscriber status, and user ID if you request tags. Category and directory data lets you rank games by concurrent viewers or pull the top live channels in a section. That powers most "what's trending" analytics. What you can't get cleanly: private messages, email addresses, watch history, or anything gated behind a user's own login. If a method promises those, it's crossing a line you shouldn't. ## The 7 methods at a glance Here's the full lineup before we get into code. Start at the top and work down as your needs grow. | # | Method | Difficulty | Auth | Best for | Block risk | | - | ------------------------- | ---------- | ---------------- | ---------------------------- | ---------- | | 1 | Official Helix API | Easy | App token | Clean metadata, ToS-safe | Low | | 2 | Private GraphQL API | Easy | Public Client-ID | Richer front-end data | Medium | | 3 | Anonymous IRC chat | Medium | None | Live chat capture | Low | | 4 | Headless browser | Medium | None | JS-only pages, players | Medium | | 5 | Persisted GraphQL queries | Hard | Public Client-ID | Scale + undocumented ops | Medium | | 6 | Proxy rotation | Hard | Depends | Polling live data at scale | Low | | 7 | EventSub push | Hard | App token | Real-time events, no polling | Very low | Quick take: for a weekend project, method 1 or 2 is all you need. For anything real-time or high-volume, jump to 3, 6, or 7. ## Method 1: Official Helix API Twitch gives you a free, documented API for its own data. It's the boring, correct starting point. ****Best for:** stream, user, clip, and game metadata ****Difficulty:** easy ****Auth:** app access token (free) ****Block risk:** low Register an app in the [Twitch developer console](https://dev.twitch.tv/docs/api/) to get a client ID and secret. Then grab an app token with the client-credentials flow. ```python import requests # Trade your client ID + secret for an app access token auth = requests.post("https://id.twitch.tv/oauth2/token", data={ "client_id": CLIENT_ID, "client_secret": CLIENT_SECRET, "grant_type": "client_credentials", }) token = auth.json()["access_token"] # valid ~60 days, cache it ``` That token has no user scopes, which is fine for reading public data. Cache it; don't request a fresh one every run. Now hit the streams endpoint. Both the `Client-Id` header and the `Bearer` token are required. ```python headers = { "Client-Id": CLIENT_ID, "Authorization": f"Bearer {token}", } # Top live streams for a game, 100 per page (the max) r = requests.get( "https://api.twitch.tv/helix/streams", headers=headers, params={"game_id": "509658", "first": 100}, # 509658 = Just Chatting ) print(r.json()["data"][0]["viewer_count"]) ``` Response is clean JSON: title, viewer count, started-at, language, tags. Paginate with the `cursor` value in `pagination` to walk past the first 100. The tradeoff is honesty about limits. Helix caps you around 800 points per minute per app, and it deliberately won't expose everything the website shows, like exact follower timestamps or full chat. Use Helix when you want stability and you're okay staying inside what Twitch chooses to document. ## Method 2: Private GraphQL API This is the endpoint Twitch's own website runs on, and it returns far more than Helix. No signup, no app registration. ****Best for:** follower counts, VOD metadata, channel panels, anything the site shows ****Difficulty:** easy ****Auth:** a public Client-ID baked into the site ****Block risk:** medium Every request to `https://gql.twitch.tv/gql` needs a `Client-ID` header. Twitch hardcodes a public one into its own HTML: `kimne78kx3ncx6brgo4mv6wki5h1ko`. ```python import requests CLIENT_ID = "kimne78kx3ncx6brgo4mv6wki5h1ko" # public web client, baked into twitch.tv query = 'query { user(login: "pokimane") { followers { totalCount } } }' r = requests.post( "https://gql.twitch.tv/gql", json={"query": query}, headers={"Client-ID": CLIENT_ID}, ) print(r.json()["data"]["user"]["followers"]["totalCount"]) ``` That returns a live follower count, no OAuth involved. The same query shape fetches stream status, VOD lists, clip data, and panel content. If you'd rather not hardcode the ID in case it ever rotates, scrape it from the homepage: ```python import re, requests html = requests.get("https://www.twitch.tv").text # Twitch embeds the client id in its bootstrap JSON client_id = re.search(r'"Client-ID":"(\w+)"', html).group(1) ``` Two things worth knowing. This API is undocumented, so a field can change without warning. And Twitch has hinted that heavy use tied to a logged-in account may draw consequences, so keep your queries anonymous with the public ID. For a broader primer on this pattern, see our guide on scraping GraphQL APIs. The schema itself is mirrored in the community [twitch-graphql-api repo](https://github.com/SuperSonicHub1/twitch-graphql-api). ## Method 3: Anonymous IRC chat capture Chat is a different backend entirely. You read it in real time over WebSocket, with zero authentication. ****Best for:** live chat logs, sentiment, emote frequency ****Difficulty:** medium ****Auth:** none ****Block risk:** low Twitch lets you log in anonymously with any nickname starting `justinfan` followed by numbers. No password, no token. Here's a minimal reader in Node using the `ws` library. ```javascript import WebSocket from "ws"; const ws = new WebSocket("wss://irc-ws.chat.twitch.tv:443"); ws.on("open", () => { ws.send("NICK justinfan" + Math.floor(Math.random() * 100000)); // anon login ws.send("JOIN #xqc"); // channel name, lowercase, with the # prefix }); ``` The random `justinfan` nick is what makes this read-only and account-free. You're now joined to the channel. Chat lines arrive as raw IRC. You parse the sender and message out of each `PRIVMSG`, and answer Twitch's keepalive so it doesn't drop you. ```javascript ws.on("message", (data) => { const line = data.toString(); if (line.startsWith("PING")) return ws.send("PONG :tmi.twitch.tv"); // keepalive const m = line.match(/:(\w+)!.* PRIVMSG #\w+ :(.*)/); if (m) console.log(`${m[1]}: ${m[2].trim()}`); // user: message }); ``` Miss the `PING`/`PONG` step and Twitch closes your socket after a few minutes. That single line is the most common reason chat scrapers die. Send `CAP REQ :twitch.tv/tags` right after connecting if you want badges, colors, and user IDs attached to each message. ## Method 4: Headless browser for what only renders in JS Some data never appears in an API call you can easily replay, like the state of the video player or a lazily hydrated category grid. For those, drive a real browser. ****Best for:** player state, embeds, pages that resist direct API calls ****Difficulty:** medium ****Auth:** none ****Block risk:** medium Playwright loads the page, runs the JavaScript, and hands you the rendered DOM. Slower and heavier than an API call, but it sees exactly what a viewer sees. ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.launch(headless=True) page = browser.new_page() page.goto("https://www.twitch.tv/directory/game/VALORANT") page.wait_for_selector('[data-a-target="preview-card-title-link"]') # wait for hydration titles = page.locator('[data-a-target="preview-card-title-link"]').all_inner_texts() print(titles) browser.close() ``` The `wait_for_selector` call matters. Grab the DOM too early and the grid is still empty. A sharper trick: skip the DOM and intercept the API calls the page makes. You get structured JSON instead of scraping rendered text. ```python page.on("response", lambda res: print(res.url) if "gql.twitch.tv" in res.url else None ) # log every GraphQL call the page fires, then replay it directly ``` This is how you reverse-engineer new endpoints. Watch what the real client requests, then reproduce it with method 2 or 5 and drop the browser. Playwright costs real CPU and RAM, so reserve it for pages you genuinely can't reach otherwise. Our Playwright scraping guide covers stealth and stability in more depth. The official [Playwright docs](https://playwright.dev/) are the reference. ## Method 5: Persisted GraphQL queries at scale When I first ran raw GraphQL queries across a few thousand channels, they worked. When I pushed past that, some operations started returning errors that the real website never hits. The reason is persisted queries. ****Best for:** high-volume pulls, undocumented operations, resilience ****Difficulty:** hard ****Auth:** public Client-ID ****Block risk:** medium Twitch's frontend uses Automatic Persisted Queries. Instead of sending the full query text, the client sends a stored operation name plus a SHA-256 hash. This saves bandwidth and mirrors real traffic. You send the same shape. ```python import requests payload = [{ "operationName": "StreamMetadata", "variables": {"channelLogin": "shroud"}, "extensions": {"persistedQuery": { "version": 1, "sha256Hash": "252a46e3f5b1ddc431b396e688331d8d020daec27079893ac7d4e6db759a7402", }}, }] # note the outer list: GraphQL batching in one POST r = requests.post("https://gql.twitch.tv/gql", json=payload, headers={"Client-ID": CLIENT_ID}) ``` The outer list is the batching trick. Twitch accepts an array of operations in one request, so you can fetch metadata for many channels per round trip instead of one at a time. You find the hashes by watching the network tab (method 4) and copying the `sha256Hash` for each operation you need. Save them; they only change when Twitch ships a new build. One catch worth flagging: Twitch sometimes attaches a `Client-Integrity` header to sensitive operations. Public read queries like stream metadata generally don't need it, but if you hit `integrity` errors, that operation is gated and not worth forcing. Batching plus persisted hashes is how you scrape tens of thousands of channels without hammering the endpoint with one request each. ## Method 6: Proxy rotation for live polling Live data changes by the second, so you end up polling. Poll hard enough from one IP and Twitch will throttle or block it. ****Best for:** frequent polling, large channel sets, geo-varied data ****Difficulty:** hard ****Auth:** depends on method ****Block risk:** low once rotating The fix is spreading requests across many IPs so no single address looks abusive. Rotate a pool and assign a different exit per request. ```python import requests, random proxies_pool = [ "http://user:pass@gw.example-proxy.net:7001", "http://user:pass@gw.example-proxy.net:7002", ] # residential IPs rotate best against Twitch's per-IP limits def get(url, **kw): proxy = random.choice(proxies_pool) return requests.get(url, proxies={"http": proxy, "https": proxy}, **kw) ``` Residential IPs work better than datacenter ranges here, because Twitch treats them as ordinary viewer traffic. This is the one place a proxy provider earns its keep, and it's what [Roundproxies](https://roundproxies.com/residential-proxies-pool/) residential pools are built for. Pair rotation with backoff. If a response comes back `429`, sleep before retrying rather than slamming the next IP. ```python import time def get_with_backoff(url, headers, tries=4): for i in range(tries): r = get(url, headers=headers) if r.status_code != 429: return r time.sleep(2 ** i) # 1s, 2s, 4s, 8s return r ``` Rotation without backoff just burns your pool. Together they keep you polling for a long time. For the wider pattern, see our notes on rotating proxies and handling rate limits. ## Method 7: EventSub instead of polling Polling asks "did anything change?" over and over. EventSub flips it: Twitch pushes you the event when it happens. ****Best for:** going live, follows, subs, raids, in real time ****Difficulty:** hard ****Auth:** app token ****Block risk:** very low You subscribe to event types like `stream.online`, and Twitch delivers them to a WebSocket connection or a webhook you host. For scraping, WebSocket is simpler because you don't need a public URL. ```python import websocket, json, requests def on_message(ws, msg): data = json.loads(msg) if data["metadata"]["message_type"] == "session_welcome": session_id = data["payload"]["session"]["id"] subscribe(session_id) # register your events using this id ws = websocket.WebSocketApp( "wss://eventsub.wss.twitch.tv/ws", on_message=on_message) ws.run_forever() ``` The first message Twitch sends is a `session_welcome` carrying a session ID. You use that ID to register subscriptions over Helix. ```python def subscribe(session_id): requests.post("https://api.twitch.tv/helix/eventsub/subscriptions", headers={"Client-Id": CLIENT_ID, "Authorization": f"Bearer {token}", "Content-Type": "application/json"}, json={"type": "stream.online", "version": "1", "condition": {"broadcaster_user_id": "12826"}, # channel's numeric id "transport": {"method": "websocket", "session_id": session_id}}) ``` Now you get a push the instant that channel goes live, no polling loop, almost no wasted requests. EventSub is more setup than a plain GET, but for tracking many channels' live status it's the cheapest method by far. Twitch's EventSub reference lists every event type. ## Which method should you use? Match your goal to the row. | If you need... | Use | Why | | ------------------------------------ | ----------------------------------- | --------------------- | | Clean stream/user metadata, ToS-safe | Helix API (1) | Documented and stable | | Data Helix hides (followers, VODs) | GraphQL (2) | Richer, no signup | | Live chat messages | Anonymous IRC (3) | Real-time, no auth | | Something only the player renders | Headless browser (4) | Sees the real DOM | | Tens of thousands of channels | Persisted queries (5) + proxies (6) | Batch and spread load | | Instant "went live" alerts | EventSub (7) | Push beats polling | Most projects end up combining two or three. A typical stack is GraphQL for metadata, IRC for chat, and proxies underneath once volume climbs. ## Common errors and how to fix them These are the failures you'll actually hit, with the exact messages. `**{"error":"Bad Request","status":400,"message":"The \"Client-ID\" header is missing from the request."}**` You called `gql.twitch.tv` without the `Client-ID` header. Add the public one from method 2. **`401 Unauthorized` on Helix** Your app token expired or you forgot the `Client-Id` header. Regenerate the token and send both headers. Chat socket closes after a few minutes You didn't answer `PING` with `PONG`. Add the keepalive from method 3. **`Login unsuccessful` on IRC** You tried an app token as an IRC password. IRC wants a user token or the anonymous `justinfan` nick, not an app token. **`429 Too Many Requests`** You're over the rate limit from one IP. Add backoff and rotate proxies (method 6). ## A note on responsible use Everything here reads public data, the same data any logged-out visitor sees. Keep it that way. Respect Twitch's terms of service and don't collect private or personal information. Throttle your requests so you're not degrading anyone's service. The GraphQL and IRC endpoints are undocumented and can change, so build in error handling and expect the occasional breakage. Don't tie high-volume private-API use to your personal account. ## Frequently asked questions **Is scraping Twitch legal?** Reading public data is generally fine, but Twitch's terms restrict automated collection. Stay on public data, throttle your requests, and don't republish anything private. When in doubt, use the official Helix API. **Do I need a Twitch account to scrape it?** Not for methods 2, 3, and 4\. The private GraphQL API uses a public Client-ID, IRC reads anonymously, and a headless browser needs no login. Only Helix and EventSub want an app registration. **How much data can I pull before getting blocked?** Helix allows roughly 800 points per minute per app. The private endpoints have looser but real limits. One IP polling aggressively gets throttled fast, which is why method 6 exists. **Can I scrape Twitch chat history?** Live chat, yes, via IRC. Past chat isn't served by any public endpoint, so you have to capture it in real time and store it yourself as it streams in. **Which method is fastest to set up?** The private GraphQL API (method 2). It's a single POST with one header and no registration, and it returns more than Helix. ## Wrapping up Scraping Twitch is really about picking the right layer. The HTML is a dead end; the APIs behind it are where the data lives. Start with Helix (method 1) or the private GraphQL endpoint (method 2). Add IRC for chat, a headless browser for the rare JS-only page, and proxies plus EventSub once you're operating at scale. Build these yourself and you own the whole pipeline, no vendor, no per-request bill, no black box. Next, try wiring GraphQL metadata to IRC chat capture for a single channel, then scale the pattern out.