Crawl API

Automate content extraction from any domain. Simply define the root URL and retrieve the full website content as Markdown, Text, HTML, or JSON files.

Кредитная карта не требуется
  • Map entire site structures in one request
  • Capture both static and dynamic web content
  • Flexible for SEO, AI, and compliance needs
  • Integrates with popular dev frameworks and no-code
ДОВЕРЯЮТ 20,000+ КЛИЕНТОВ ПО ВСЕМУ МИРУ

                              const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer ', 'Content-Type': 'application/json'},
  body: '[{"url":"https://il.linkedin.com/company/bright-data"}]'
};

fetch('https://api.brightdata.com/datasets/v3/trigger', options)
  .then(response => response.json())
  .then(response => console.log(response))
  .catch(err => console.error(err));






                              
                            
                              import requests

url = "https://api.brightdata.com/datasets/v3/trigger"

payload = [{"url": "https://il.linkedin.com/company/bright-data"}]
headers = {
    "Authorization": "Bearer ",
    "Content-Type": "application/json"
}

response = requests.request("POST", url, json=payload, headers=headers)

print(response.text)
                              
                            

Easy to start, easier to scale

  1. Choose target domain
    Define target URL and connect to the API with a single line of code
  2. Send request
    Edit crawl parameters and insert your custom logic using Python or JavaScript
  3. Get your data
    Retrieve website data as Markdown, Text, HTML, or JSON files
Read documentation

Developer-first experience

Quick Start

Connect to the Crawl API with a single line of code, or use the control panel to get results directly through the Control Panel.

Custom Collection

Use request parameters to customize collection and delivery, including pagination, scheduling and log collection.

Data Parsing

Efficiently converts raw HTML into structured data files, delivered as Markdown, Text, HTML, or JSON, directly to your database.

Crawl API pricing

pay as you go plan icon
Оплата по мере потребления
$1.5 /1K ЗАПИСЕЙ
Никаких обязательств
Start free trial

Оплата по мере использования без ежемесячных обязательств
25% OFF
2nd plan icon
Базовый тариф
$1.27
$0.95 /1K ЗАПИСЕЙ
$499 Ежемесячная
Start free trial
Use this coupon code: APIS25

Создано для команд, стремящихся масштабировать свои операции
25% OFF
3rd plan icon
бизнес
$1.12
$0.84 /1K ЗАПИСЕЙ
$999 Ежемесячная
Start free trial
Use this coupon code: APIS25

Предназначен для больших команд с обширными операционными потребностями
25% OFF
4th plan icon
ПРЕМИУМ
$1.05
$0.79 /1K ЗАПИСЕЙ
$1999 Ежемесячная
Start free trial
Use this coupon code: APIS25

Расширенная поддержка и функции для критических операций
Компания
Элитные услуги по обработке данных для бизнеса высшего уровня.
СВЯЖИТЕСЬ С НАМИ
  • Аккаунт-менеджер
  • Пакеты по заказу
  • Премиальный SLA
  • Приоритетная поддержка
  • Индивидуальное обучение
  • SSO
  • Настройки
  • Журналы аудита
Compliant proxies

Leading the way in ethical web data collection

Bright Data sets the gold standard in compliance, effectively self-regulating the industry. With transparent operations validated by top security firms, clear peer consent, and pioneering compliance units, we ensure legitimate and safe data collection. Upholding international privacy laws and utilizing tools like BrightBot, we minimize your legal exposure, making partnership with us a strategic move to curtail legal risks and associated costs.

Start free trial

Every 15 minutes, our customers scrape enough data to train ChatGPT from scratch.

API for Seamless Crawl Data Access

Comprehensive, Scalable, and Compliant Crawl Data Extraction

FLEXIBLE

Tailored to your workflow

Get structured data in JSON, NDJSON, or CSV files through Webhook or API delivery.

SCALABLE

Built-in infrastructure and unblocking

Get maximum control and flexibility without maintaining proxy and unblocking infrastructure. Easily scrape data from any geo-location while avoiding CAPTCHAs and blocks.

STABLE

Battle-proven infrastructure

Bright Data’s platform powers over 20,000+ companies worldwide, offering peace of mind with 99.99% uptime, access to 150M+ real user IPs covering 195 countries.

COMPLIANT

Industry leading compliance

Our privacy practices comply with data protection laws, including the EU data protection regulatory framework, GDPR, and CCPA – respecting requests to exercise privacy rights and more.

Хотите узнать больше?

Поговорите с экспертом, чтобы обсудить ваши потребности в скрапинге