Add skill: firecrawl

[leonardogrig]

Add Firecrawl skill: web scraping, search, and site mapping for Refly agents

What this adds

A new firecrawl skill (skills/firecrawl/) that gives Refly agents full web scraping, search, and crawling capabilities through the Firecrawl CLI.

Why Firecrawl belongs in the Refly skill registry

Refly already has great search skills, but they're focused on search - finding links and snippets. Firecrawl fills a different gap: it's the web data extraction layer that agents actually need once they know what to look at.

Here's what Firecrawl adds on top of the existing search skills:

• Full page scraping to clean Markdown - not just search snippets, but the actual page content optimized for LLM context windows
• JavaScript-rendered pages - handles SPAs and dynamic content with the --wait-for flag
• Site-wide URL discovery - the map command discovers all URLs on a domain, useful for understanding docs structure
• Structured data extraction - extract data into JSON schemas
• Parallel batch scraping - scrape multiple pages concurrently up to your rate limit
• Image and news search - --sources images,news for multi-source research
• File-based output - -o flag writes directly to file, avoids flooding agent context

The key difference: when an agent uses a search skill to find "React Server Components docs," it gets search results. When it uses Firecrawl, it can scrape those pages, get clean Markdown, and actually work with the content - all in one step with firecrawl search "query" --scrape.

What's in the skill

• README.md - Standard skill documentation with features, triggers, installation, and tags
• SKILL.md - Full skill definition with:
  • Three core commands: search, scrape, map
  • Agent-friendly conventions: file-based output to .firecrawl/, incremental reading patterns, parallel execution examples
  • Built-in auth flow with browser-based login and fallback to manual API key
  • Detailed option references for every command

How agents use it

# Search the web and scrape results in one shot
firecrawl search "React 19 migration guide" --scrape -o .firecrawl/search-react19.json

# Scrape a specific docs page to clean Markdown
firecrawl scrape https://docs.example.com/api -o .firecrawl/api-docs.md

# Discover all URLs on a documentation site
firecrawl map https://docs.example.com --search "authentication" -o .firecrawl/auth-urls.txt

# Batch scrape multiple pages in parallel
firecrawl scrape https://site1.com -o .firecrawl/1.md &
firecrawl scrape https://site2.com -o .firecrawl/2.md &
wait

Setup

npm install -g firecrawl-cli
firecrawl login --browser

The skill handles auth errors gracefully - if a command fails after login, it prompts the user to re-authenticate via browser or manual API key.

Follows repo conventions

• Matches the skills/<name>/README.md + SKILL.md structure used by all other skills
• Uses the search-data category alongside the existing search skills
• Standard frontmatter with name, version, category, description

---

I work on Firecrawl's developer relations team. Happy to iterate on anything here or adjust the skill to better fit Refly's conventions.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds two documentation files for the Firecrawl CLI skill: a concise README and an expanded SKILL.md documenting installation, authentication, commands (Search, Scrape, Map), options, examples, parallel execution, output handling, and troubleshooting.

Changes

Cohort / File(s)	Summary
Firecrawl CLI docs skills/firecrawl/README.md, skills/firecrawl/SKILL.md	Added comprehensive documentation for the Firecrawl CLI: purpose, installation, authentication (browser & API key), command reference (Search, Scrape, Map) with flags and examples, output formats and processing tips (jq), parallel execution guidance, handling large outputs, and troubleshooting notes.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 I hopped through docs with a twitch and a grin,
Wrote search, scrape, map — let the crawling begin,
Flags lined like carrots in neat little rows,
Parallel hops where the data wind blows,
I nibble the README — off to new shows!

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Add skill: firecrawl' directly and clearly summarizes the main change—adding a new firecrawl skill with documentation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 3

🤖 Fix all issues with AI agents

In `@skills/firecrawl/README.md`:
- Around line 22-23: In the README feature list update the "Scrape" bullet so
the output formats use the proper noun "Markdown" (capitalize the 'M') instead
of "markdown"; locate the line containing the "**Scrape** - Single page content
extraction with multiple output formats (markdown, HTML, links, screenshots)"
string and change "markdown" to "Markdown" while leaving the rest of the text
unchanged.

In `@skills/firecrawl/SKILL.md`:
- Around line 26-32: The fenced code block in SKILL.md is missing a language
identifier which triggers markdownlint MD040; update the block delimiter for the
snippet in skills/firecrawl/SKILL.md to include a language (e.g., "text") so it
becomes ```text and the closing ``` remains, ensuring the block around the "🔥
firecrawl cli v1.0.2" status lines is annotated; no other content changes
needed.
- Around line 206-207: Update the format behavior text under the "**Single
format**" bullet so that "markdown" is capitalized as "Markdown" (locate the
bullet starting with "**Single format**: Outputs raw content (markdown text,
HTML, etc.)" in SKILL.md and change "markdown" → "Markdown"); leave other text
unchanged.

[coderabbitai]

Actionable comments posted: 3

🤖 Fix all issues with AI agents

In `@skills/firecrawl/README.md`:
- Around line 3-5: Update the README and any related docs to use "Markdown"
(capital M) consistently as a proper noun wherever the term appears (e.g., in
the opening description lines shown), replacing lowercase "markdown"
occurrences; search for and apply the same capitalization across the Firecrawl
skill docs to keep terminology consistent (ensure phrases like "clean Markdown
content", "optimized for LLM context windows", and other occurrences use
"Markdown").

In `@skills/firecrawl/SKILL.md`:
- Line 119: Update the category list in SKILL.md where the option `--categories
<categories>` is documented: change the lowercase "github" to the proper noun
"GitHub" so the line reads `--categories <categories>` - Comma-separated:
GitHub, research, pdf; ensure only the capitalization is changed and no other
text is altered.
- Line 4: The example output for the command string "firecrawl --status" in
SKILL.md currently shows "firecrawl cli v1.0.2" which is outdated; update that
example output line to "firecrawl cli v1.1.1" (leave the frontmatter "version:
1.0.0" unchanged) so the sample output matches the current Firecrawl CLI
version.