快速判断
- 01这个 Skill 是干嘛的
- 结合网页抓取和 Agent 流程完成站点、竞品与市场信息分析。
- 02它能解决什么问题
- 解决研究资料分散在大量网页、动态内容难提取和人工汇总耗时的问题。
- 03它适合谁来用
- 适合需要批量网页研究、竞品监测和市场情报整理的分析人员。
web-agent
用网页抓取与 Agent 流程完成竞品、市场和站点信息分析。
主要能力
Compare two or more companies, products, or platforms across pricing, features, positioning, and docs. Use this skill whenever the user says "compare X vs Y", "how does X stack up against Y", "alternatives to X", "competitive landscape of …", "X vs Y vs Z", or asks for a competitor matrix. Uses search to discover competitors when the user only names a category, then scrape for each competitor's homepage, pricing page, and features/docs. Returns a normalized comparison matrix as JSON.
使用方式
仓库级安装会保留完整目录;本次核验的入口是 agent-core/src/skills/definitions/competitor-analysis/SKILL.md。使用前先阅读英文 README 与原始 SKILL.md,并按当前 Agent 的目录规范安装。
适用边界
抓取必须遵守 robots、站点条款与频率限制,结果还需核对来源时间。
许可与来源
来源:GitHub 公开仓库。核验许可:MIT。本页是贴近原仓库结构的中文导读,具体参数、依赖和更新记录以英文 README 为准。
Firecrawl Web Agent
<img src=".internal/agent.jpg" alt="Firecrawl Agent" />
<img src="https://media1.giphy.com/media/v1.Y2lkPTc5MGI3NjExcWhub2Jmd3NvejdhaTFsb3RvZWtpb2Q3cDVpN2pzYjVqeTgxdDEwbiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/CVyWVobjHwYGJiRz6r/giphy.gif" alt="Firecrawl Agent Demo" width="100%" />
Firecrawl runs a research-grade autonomous agent at firecrawl.dev/app/agent, powered by Spark 1 models optimized for structured web research. This repo gives you the open-source foundation to build your own — fork it, swap models, add skills, and deploy however you want.
Get started
# 1. Install the Firecrawl CLI and authenticate
npx -y firecrawl-cli@latest init -y --browser
# 2. Scaffold an agent project
firecrawl create agent -t next
Open Source
Each layer builds on the one below it. Start at the top for a ready-to-use app, or go lower in the stack for finer control over the primitives.
| Layer | Description | Get started |
|---|---|---|
| Next.js Template | Chat UI, streaming, Skills, Subagents, structured output | firecrawl create agent -t next |
| Express Template | API server with Skills, Subagents, structured output | firecrawl create agent -t express |
| ↑ | ||
| Agent Core | Orchestrator built on Deep Agents (LangChain). Skills, Subagents, structured output | firecrawl create agent -t library |
| ↑ | ||
| Firecrawl AI SDK | Search, Scrape, Interact as Vercel AI SDK tools | npm i firecrawl-aisdk |
| ↑ | ||
| Firecrawl SDK | Core API client for Scrape, Search, Crawl, Extract | npm i @mendable/firecrawl-js |
| ↑ | ||
| API Reference | REST API, use from any language | docs.firecrawl.dev |
Examples
| Level | Examples |
|---|---|
| Next.js | Full template |
| Express | API server |
| Agent Core | Basic · Structured output · Parallel Subagents · With Skills · Streaming |
| Firecrawl AI SDK | npmjs.com/package/firecrawl-aisdk |
How it works
The agent combines web tools with an AI model in a loop — it plans, acts, observes, and repeats until the task is done. The harness is Deep Agents (from LangChain), which gives us the plan-act loop, parallel task sub-agent spawning, and on-demand SKILL.md loading out of the box. Our agent-core wires Firecrawl's tools into that runtime and layers on structured output and streaming.
- Harness — Deep Agents. Provides the agent loop, sub-agent spawning, skills loading, and context management.
- Tools — Search, Scrape, Interact (browser automation), bash. Powered by firecrawl-aisdk.
- Skills — reusable SKILL.md playbooks. Auto-discovered from
agent-core/src/skills/definitions/, loaded on demand via Deep Agents' skills middleware. - Subagents — parallel workers for independent tasks, spawned via Deep Agents'
tasktool. Each has its own tool set and session state (e.g. an isolated interact browser session). - Output — structured results via
formatOutput(JSON) and data processing viabashExec, a set of bash tools powered by just-bash.
Project structure
| Directory | What's inside |
|---|---|
agent-core/ | Core agent logic, orchestrator, Skills, tools |
agent-templates/ | Deployment templates - Next.js, Express, Library |
License
MIT