web-research-cascade.

#What it is

A skill that changes what my agent does when a fetch gets blocked. It enforces two habits, in order: first decide which sources would genuinely best answer the question (the primary document, the official docs, the actual announcement), then actually get those sources. When a chosen URL returns a 403, the same URL is worked through stronger fetch methods until the real page comes through. Two bundled scripts, standard-library Python only, MIT licensed.

#The itch

I asked my agent to read a vendor's official announcement and check it against their API docs. The answer came back fluent and confident. Only in the transcript did I see what had happened: the announcement URL had returned a 403, and the agent had quietly summarized a third-party blog that quoted the announcement instead. Nothing in the output said the evidence had gotten weaker. A 403 is a transport problem, and the agent was treating it as a reason to drop the source. This skill turns that silent fallback into a visible escalation.

#How it works

Per URL, the cascade escalates only when the current stage is blocked: a 403, 401, or 429, an empty or truncated body, or a CAPTCHA page where content should be.

Stage 2 does most of the work: prepend https://r.jina.ai/ to the full URL and fetch that instead. On top of the ladder sit shortcuts for sources that block the generic path entirely: Reddit through its .json route in your logged-in browser, single X tweets through the bundled x_tweet.py, Zendesk help centers through their article JSON API.

One design decision needs explaining: Firecrawl ships as a script, deliberately without the vendor's MCP server. That server injects an instruction that makes its metered search the primary search tool, which overrides the cascade and burns credits on every search. The script has no instructions to inject, costs no standing context, and stays scrape-only.

#See it run

Four scenarios, one ending per stage. Pick one and watch how far the same URL has to climb. Each run is a scripted replay; this page makes no network requests.

#In real use

In my sessions the skill triggers reliably on research tasks and on blocked fetches. It is instructions the model follows, so there is no hard stop if the model ever ignores it; I cannot promise it always fires. The other limits, stated in the repo:

• Stage 4 needs a browser MCP attached to your real session. Without one, login-walled sources stay unreachable, and stages 1 to 3 cannot substitute for it.
• Paywalls you have not paid for stay closed. The cascade gets you to pages you could read in your own browser anyway, and every site's terms of service still apply to you.
• The unofficial routes are borrowed time. The X syndication endpoint can stop working any day, and the docs say so plainly.

Version 1.0.0 shipped on 2026-07-11 with the four-stage cascade and both scripts.

#Install

$ git clone --depth=1 https://github.com/belschak/web-research-cascade.git \
  ~/.claude/skills/web-research-cascade

That is the whole install. Stages 1 and 2 need no keys or accounts. Stage 3 is optional: get a free Firecrawl key, cp .env.example .env, paste it in.

#Links

---