Is crawlee-python still maintained?

Yes, active. Last change today. Latest release v1.11.2, 4 days ago.

How widely used is crawlee-python?

Widely used. 9.7k stars, 2 years old, by an organization. As with any download, get it from the official repo.

Assembly instructions

apify/crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.

Python#apify#automation#beautifulsoup#crawler

Before you start

Is it free?: YesOpen source (Apache-2.0).
Still maintained?: Yes, activeLast change today. Latest release v1.11.2, 4 days ago.
How widely used?: Widely used9.7k stars, 2 years old, by an organization. As with any download, get it from the official repo.

Difficulty: Moderate
Time: about 15 min
Stars: ★ 9,668
Activity: Updated this month

Saved setup from 7 h ago

Will it work on macOS?

Yes, it installs on your macOS

The README gives a pipx / uv install for macOS.

Your system

macOS Windows Linux

Start here

Website ↗crawlee.devThe project's own site.

Heads-upBuilding it yourself? It also needs PostgreSQL and Redis.

Install with a package manager

pipx / uvNeeds pipx or uv

uvx 'crawlee[cli]' create my-crawler

From the README, exactly as the maintainers wrote it.

Did you get it running?

No reports for macOS yet.

Still stuck? Get a written guide

GitManual AI Agent reads the 9 key files and writes a step-by-step guide for macOS. Once written, it's saved for everyone.

Parts list

1×Python >=3.10runtime
1×uv package manager
1×PostgreSQL service
1×Redis service

Share this manual

Is this your repo?

Add a badge to your README so newcomers find these steps.

[![Setup manual](https://gitmanual.com/apify/crawlee-python/badge.svg)](https://gitmanual.com/apify/crawlee-python)

Good to know

Has AGENTS.md — instructions for coding agents. You can open Claude Code (or another agent) in the cloned folder and ask it to do the setup for you.
Nested projects: docs/pyproject.toml, src/crawlee/project_template/{{cookiecutter.project_name}}/pyproject.toml, src/crawlee/project_template/{{cookiecutter.project_name}}/requirements.txt, website/package.json and 1 more

Details

License: Apache-2.0
Open issues: 81
Branch: master

Files we read (9)

.github/workflows/_checks.yaml

.github/workflows/factory_review.yaml

.github/workflows/manual_release_beta.yaml

.github/workflows/manual_release_docs.yaml

.github/workflows/manual_release_stable.yaml

.github/workflows/on_schedule_tests.yaml

CONTRIBUTING.md

README.md

pyproject.toml

1,369 files in the repo

Contact

apify/crawlee-python

Before you start

Your setup manual for crawlee-python

What's inside

Handy for

Yes, it installs on your macOS

Start here

Install with a package manager

Did you get it running?

Still stuck? Get a written guide

Yes, it installs on your macOS

Start here

Install with a package manager

Did you get it running?

Still stuck? Get a written guide

More like crawlee-python that run on macOS