.env.example

# quora-scraper — copy to .env next to the scripts and fill in.
#
#   cp .env.example .env
#   python3 env_config.py        # prints what was picked up, WITHOUT secrets
#
# Credentials belong here and never on a command line: a secret in argv is
# readable by anything that can run `ps` and lands in shell history.
# `.env` is in .gitignore; keep it that way.
#
# Precedence, highest first:
#   an explicit flag  ->  an exported environment variable  ->  .env  ->  default
#
# Every value below is a PLACEHOLDER. Anything still carrying `{...}` braces
# is treated as unset, so a copied example is never sent to an API as if it
# were a key — the alternative is a 401 a long way from its cause.

# ---------------------------------------------------------------------------
# 2Captcha — one key, four separately billed products
# ---------------------------------------------------------------------------
# Captcha solving, the Scraping Browser API, proxies and fingerprints all sit
# behind this one key: https://2captcha.com
#
# Worth knowing before you buy any of it: on this site NONE of it is
# required to get data. Playwright's own bundled Chromium, from an ordinary
# DATACENTRE address with no key and no proxy, was served HTTP 200 and the
# full feed on eight of the first fourteen fetches on 2026-09-15 — a stronger
# result than a home connection would have been, since datacentre ranges are
# what a bot manager scores first. The six refusals cleared on the next
# attempt about a minute later.
#
# What the paid products buy is the thing that actually limits a run here:
# RATE. Everything after those first fourteen fetches from that one address
# was refused, across three hours and a 45-minute rest. Many addresses is how
# you fetch a lot of Quora; one address and a --delay is how you fetch a
# little of it for free.
TWOCAPTCHA_KEY=your_2captcha_api_key_here

# ---------------------------------------------------------------------------
# The Scraping Browser API — a remote browser over CDP
# ---------------------------------------------------------------------------
#   country-{cc}   picks the exit country. Quora serves the same markup
#                  everywhere; what the country changes is which language
#                  host a bare quora.com lands on, so prefer naming the
#                  language host in the URL and treat this as a rate lever
#   pid-{id}       a profile with persistent cookies, ONE live connection
#                  each — reuse pids rather than minting one per run
#
# --concurrency above 1 is refused on this site anyway (a Quora feed has no
# per-batch address), so one run per pid is the shape here regardless.
#
# NOTE: selenium_scraper.py cannot use this. chromedriver's `debuggerAddress`
# takes a bare host:port with nowhere to put a password; Playwright and
# pyppeteer authenticate on the WebSocket upgrade and work fine.
QUORA_CDP_ENDPOINT=ws://{login}-zone-scraping_browser-country-us-pid-{profileId}:{password}@cb.2captcha.com:9222

# ---------------------------------------------------------------------------
# A proxy exit
# ---------------------------------------------------------------------------
# Read as a single exit, or use --proxy-file for a pool.
#
# What a proxy is and is NOT for on this site:
#
#  * It is NOT needed to get in. An ordinary datacentre address with no proxy
#    at all was served the full feed — see the note above.
#
#  * It DOES spread the rate, which is the thing that actually stops a long
#    run here. Cloudflare's challenge on this site tracks the ADDRESS's
#    recent request rate. A rested address recovers on the next attempt; a
#    busy one stops recovering altogether. --delay is the cheaper lever; a
#    pool is the one that scales.
#
# Note Selenium cannot authenticate a proxy at all (--proxy-server takes an
# address only). 2Captcha's IP-whitelist mode is the way around that:
# whitelist your address and ask /proxy/generate_white_list_connections for
# connections, which give you one `host:port` per exit.
#
# Measured caveat (2026-09-15): a set of those still required a username and
# password over SOCKS5 — see TROUBLESHOOTING.md. Check yours before assuming
# Selenium can use them.
QUORA_PROXY=http://{user}:{password}@us.proxy.2captcha.com:2334

# Through a proxy, images, video/audio and fonts are not downloaded: the
# proxy is billed by traffic and nothing this scraper reads is in them.
# Set to 1 to load them anyway (same as --load-media). Left empty = block.
QUORA_LOAD_MEDIA=

# Proxy traffic cap per run, in MB (10^6 bytes), same as --proxy-budget-mb.
# Empty = 10. 0 = no cap. Once a run has moved more than this through a
# local proxy it stops, keeps the rows it has and exits 6 (5 with none).
QUORA_PROXY_BUDGET_MB=

# ---------------------------------------------------------------------------
# A default URL, so a run needs no --url
# ---------------------------------------------------------------------------
# Any of the three feeds, on any of the twenty-four language hosts:
#
#   topic     https://www.quora.com/topic/Machine-Learning
#             https://es.quora.com/topic/Aprendizaje-autom%C3%A1tico
#   question  https://www.quora.com/What-is-machine-learning-4
#   profile   https://www.quora.com/profile/Yann-LeCun
#
# Prefer a question or profile URL when you want upvotes, view counts,
# creation times or the full answer text: Quora sends a topic's answers over
# a later XHR rather than in the page, so those columns are null on topic
# rows. The `data_source` column on every row says which view built it.
#
# Deliberately NOT here: an output prefix. `--out` already carries a
# non-empty default, so the loader would never see it as unset and the
# variable would be silently inert — a setting that looks configurable and is
# not.
QUORA_URL=https://www.quora.com/topic/Machine-Learning