February 25, 2026

Apify MCP Server: Web Scraping for AI Agents

Not everything lives behind an API. Competitor pricing pages, job boards, product catalogs, real estate listings, review aggregations — the data your agent needs is often sitting on a website with no structured way to access it.

That's the gap the apify mcp server fills. Apify has been the go-to web scraping platform for years, and their MCP server brings that entire infrastructure — 2,000+ pre-built scrapers, headless browser support, proxy rotation — into any MCP-compatible agent framework with a single config entry.

This guide covers setup, the actors worth knowing about, and how to combine Apify with other tools for workflows that go beyond simple extraction.

What Is the Apify MCP Server

The apify mcp server exposes Apify's web scraping and automation platform as tools that AI agents can call via the Model Context Protocol. When connected, your agent gets access to:

Actor execution — Run any of Apify's 2,000+ pre-built scrapers (called "actors") or custom-built ones. Each actor is a specialized scraper designed for a specific site or data type.
Dataset access — Read results from scraping runs. Actors output structured data (JSON, CSV, Excel) that agents can process directly.
Crawling — Navigate multi-page sites, follow pagination, and extract data across hundreds or thousands of pages in a single run.
Key-value store — Store and retrieve intermediate data between scraping runs.

The important distinction from raw HTTP requests: Apify handles the hard parts of scraping that break naive approaches. JavaScript rendering, anti-bot detection, proxy rotation, rate limiting, retries. Your agent says "get the pricing data from this page" and Apify handles the infrastructure that makes that actually work at scale.

The server runs locally via stdio, like most MCP servers. It communicates with the Apify cloud platform where actors execute, so your local machine doesn't need to run headless browsers.

How to Install and Configure Apify MCP

Get your API token

Create an Apify account and grab your API token from the dashboard at console.apify.com. The free tier includes $5/month in platform credits — enough for hundreds of simple scraping runs during development.

Claude Code setup

Add the server to your Claude Code configuration:

{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": ["-y", "@apify/mcp-server"],
      "env": {
        "APIFY_TOKEN": "apify_api_your-token-here"
      }
    }
  }
}

The apify claude code integration is that simple. On next session start, Claude Code discovers the Apify tools and can use them for any task that requires web data extraction. For a deeper dive into actor selection, search-to-scrape pipelines, and cost management, see our full Apify MCP tutorial.

OpenClaw setup

mcp_servers:
  apify:
    command: npx
    args: ["-y", "@apify/mcp-server"]
    env:
      APIFY_TOKEN: "${APIFY_TOKEN}"

Verify the connection

Ask your agent to scrape something simple:

Get the title and first paragraph from https://example.com

If the connection works, you'll see a tool call to the Apify MCP server, followed by structured content from the page. If it fails, verify your API token is valid and that npx can access the package.

Best Apify Actors for AI Agent Workflows

Apify's actor library has thousands of scrapers, but a handful consistently prove useful in apify ai agent workflows:

Web Scraper (Generic)

The general-purpose actor that handles most websites. Give it a URL and CSS selectors (or let the agent figure them out), and it returns structured data. Handles JavaScript rendering, pagination, and multi-page crawling.

Best for: sites you haven't scraped before, one-off extraction tasks, pages that don't have a dedicated actor.

Google Search Results Scraper

Extracts search results from Google with full metadata — titles, URLs, snippets, People Also Ask, related searches. Useful when you want raw SERP data rather than Tavily's pre-processed results.

Best for: SEO analysis, SERP monitoring, keyword research workflows.

Amazon Product Scraper

Pulls product data including prices, ratings, reviews, seller information, and availability. A single run can extract hundreds of products from a search query or category page.

Best for: competitive pricing analysis, market research, product catalog monitoring.

LinkedIn Profile Scraper

Extracts public profile data — job titles, company history, skills, education. Essential for sales prospecting workflows where you need enrichment beyond what contact databases provide.

Best for: lead enrichment, recruiting pipelines, prospect research.

Website Content Crawler

Deep-crawls an entire site and extracts all text content. Unlike the page-level scrapers, this one follows internal links and builds a comprehensive content map.

Best for: competitive content analysis, knowledge base extraction, site migration audits.

How to reference actors

Your agent calls actors by name or ID. In practice, you'd prompt something like:

Scrape the pricing page at https://competitor.com/pricing and extract 
the plan names, prices, and features for each tier.

The agent selects the appropriate actor (usually the generic Web Scraper for unknown sites), configures the run parameters, executes it, and returns the structured results. The Apify platform handles rendering, retries, and anti-bot measures transparently.

Combining Apify with Other MCP Servers

Apify becomes significantly more powerful when paired with other tools. Three patterns I use regularly:

Search → Scrape

Tavily finds relevant pages, Apify extracts the full data. This is the most common pattern — use search for discovery, scraping for depth.

Example workflow: "Research competitor pricing in the project management space." Tavily finds the pricing pages, Apify extracts the detailed plan comparisons and feature matrices that search snippets only summarize.

See our Tavily MCP server guide for setup details on the search side.

Scrape → Analyze with trends

Apify pulls product data or content from websites, Google Trends adds search interest context. "Scrape the top 10 products in this category and cross-reference with search trend data to identify which ones are gaining momentum."

Scrape → Enrich → Outreach

Apify extracts prospect information from directories or company pages, Apollo enriches with contact data, Reoon verifies emails, Instantly sends outreach. This is the full sales pipeline pattern — Apify fills the gap when prospects aren't in traditional contact databases.

For more on composing multi-tool workflows, see our agent automation guide.

Apify vs Browser Use vs Custom Scrapers

Three approaches to giving agents web scraping capabilities, each with different tradeoffs:

Feature	Apify MCP	Browser Use	Custom Scraper
Setup time	Minutes (config entry)	Minutes (config entry)	Hours to days
Pre-built scrapers	2,000+ actors	None	None
JavaScript rendering	Yes (cloud)	Yes (local browser)	You build it
Anti-bot handling	Proxy rotation, fingerprinting	Basic	You build it
Interactive pages	Via actors	Full browser control	You build it
Runs where	Apify cloud	Local machine	Your infrastructure
Cost	$5/mo free credits + usage	Free (open source)	Infrastructure costs
Maintenance	Apify maintains actors	You maintain	You maintain

When to use Apify

Structured data extraction at scale. When you need pricing tables, product catalogs, search results, or any data that follows a predictable pattern across pages. The pre-built actors save days of development for common sites, and Apify's infrastructure handles the anti-bot arms race so you don't have to.

When to use Browser Use

Interactive web tasks that go beyond extraction. Filling forms, navigating multi-step flows, clicking through dynamic interfaces, interacting with pages that require login. Browser Use gives the agent full browser control — it's doing what a human would do, not just extracting data.

When to write a custom scraper

Almost never, in 2026. Between Apify's 2,000+ actors for structured extraction and Browser Use for interactive tasks, the cases where you need custom scraping code have shrunk dramatically. The exception: highly specialized internal tools or data sources with unique formats that no existing actor covers and that don't require browser interaction.

Frequently Asked Questions

How do I set up the Apify MCP server?

Create an Apify account, get your API token from the console dashboard, and add a configuration entry to your agent framework. For Claude Code, add a JSON block to your settings file with npx @apify/mcp-server as the command and your token as an environment variable. For OpenClaw, add the equivalent YAML. The server runs locally via stdio and connects to Apify's cloud for actor execution. The free tier includes $5/month in credits — enough for development and testing. Full details on the Apify tool page.

What can the Apify MCP server scrape?

Virtually any public website. Apify has 2,000+ pre-built actors for common sites (Google, Amazon, LinkedIn, social media platforms, real estate sites, job boards) plus a generic Web Scraper that handles arbitrary pages. The platform handles JavaScript rendering, anti-bot measures, proxy rotation, and rate limiting. Your agent specifies what data to extract, and Apify handles the infrastructure that makes extraction reliable at scale. Results come back as structured JSON that agents can process directly.

Does Apify work with Claude Code?

Yes. Add the Apify MCP server to your Claude Code settings file and the scraping tools become available automatically. Claude Code's parallel task execution works well with Apify — the agent can kick off multiple scraping runs simultaneously for comparative analysis. For example, asking Claude Code to scrape pricing pages from five competitors in parallel completes in roughly the same time as scraping one, since Apify handles each run independently in the cloud.

How much does the Apify MCP server cost?

The MCP server package is free and open-source. Apify's platform pricing is usage-based: the free tier includes $5/month in platform credits, which covers hundreds of simple actor runs. Paid plans start at $49/month for higher compute and storage. Costs vary by actor complexity — simple page extraction uses minimal credits, while crawling thousands of pages with JavaScript rendering costs more. Most development and light production workflows fit comfortably within the free tier.

Should I use Apify or Browser Use for AI agent scraping?

Use Apify for structured data extraction at scale — pricing tables, product listings, search results, anything with a predictable pattern across many pages. Apify's pre-built actors and cloud infrastructure handle the heavy lifting. Use Browser Use for interactive web tasks — logging into sites, filling forms, navigating multi-step flows, or interacting with dynamic JavaScript interfaces. They complement each other: Apify for bulk structured extraction, Browser Use for complex interactive scenarios. Many production agents connect both.