Web scraping with Python, without a stealth stack

If you collect with Python, start from the documented door, not a random Chrome disguise. This overview is official libraries and APIs. It is not a Beautiful Soup plus stealth Chromium recipe. The 2023 gists that skip permission are the pages we are not refreshing.

What web scraping is is the definition. This URL is the language note for people who already chose Python.

The order in a repo

1. Find the official SDK or HTTP API. Use requests or httpx against that contract. Identify the app. Store tokens in the repo's env, not in a browser cookie store.
2. If there is no API and the page is public and permitted, a slow HTTP client with an honest UA may be enough. User-agent honesty in research. Public pages, slowly. robots.txt and terms come first.
3. If you own the host and you need a real browser, Playwright lives in this repo, pointed at your hostname. Playwright for pages you own. MaskWright does not ship it.
4. If none of those doors open, stop. Do not add undetected-chromedriver. We will not hide a scraper.

I will not paste a selector tutorial. Selectors are how you read a tree you were already invited to read. They are not the interesting part.

Libraries I will name as doors, not as disguises

Official platform SDKs when the job is Reddit, X, ads reporting, or a shop catalog. Those are Python because the vendor shipped Python. Official APIs versus headed collection. Rate limits are not a puzzle. Honor the headers. Sleep. Exit.

httpx or requests for a documented JSON URL. Parse with the stdlib or a JSON library. Write a CSV you can provenance.

A parser on HTML you were allowed to fetch. The parser is not the ethics. The fetch is.

I will not name stealth wrappers. I will not name solver bindings. I will not name a "real Chrome" launcher as the default next step after a 403.

What Python is not for here

LinkedIn inboxes. We will not scrape LinkedIn inboxes. Instagram harvest scripts. Instagram public research versus a scrape. Email lists from storefronts. Personal data and collection. Those jobs stay refused even when the language is Python.

A scraping-browser control plane. What a scraping browser is. If your requirements.txt is a headed fleet plus a proxy rotator, you skipped the overview.

An exit in HTTP_PROXY is fine for an allowed client that must leave through a line you brought. It is not a residential disguise.

What the file tree looks like

`` E:\work\research-client\ .venv\ .env # tokens, not cookies src\client.py # official HTTP, honest UA out\2026-08-16.csv # leftover with a provenance line ``

No user-data-dir copied from MaskWright. No undetected extra. No solver package. If the tree grows a stealth folder, the overview failed.

Local Windows notes

The virtualenv lives next to the repo, not under maskwright-data. Work profiles stay closed while the client runs. Do not import maskwright-cookies.json into a session. That file is plaintext and it is a login.

Write source, time, client, ua in a sidecar next to the leftover. If you are choosing a hosted API instead of a script, judge it on permission and pace, not on stealth demos. How to judge a web scraping tool. Python is not mandatory. Permission is.

A 403 with a challenge is a stop. Your next file is not a stealth extra. It is the vendor's API docs, or an empty folder.

MaskWright 0.1 will not grow a Python runner. Bulk start will not invoke your script. If you need both a profile and a client, they still do not share a folder.

A first script that only hits a documented /v1/reports URL and writes ten rows is a finished overview. A first script that launches Chrome is a skipped overview. Keep the boring file.

If pip freeze grows a browser extra you cannot explain as "our staging host," remove it. The overview is finished when the client has a name, a limit handler that sleeps or exits, and a leftover with a provenance line.

The Scraping hub will keep this overview boring on purpose. Official client, honest name, wait on the limit, store a leftover you can defend.

FAQ

Should I start with Beautiful Soup?

Start with the API docs. A parser is how you read a tree you were already invited to fetch. The fetch is the ethics.

Can MaskWright run my Python client?

No. Keep the venv next to the repo. Keep work profiles closed while it runs.