HermesBELARABI
العربيةSearch/CheckingSign in
AcademyDirectorySolutionsStudioProjectsCompareSearch Releases Pricing Account Sign in
Directory → SKILL
SKILLoptionalHermes Optional Skills

Scrapling

Scrape sites with stealth browsing and Cloudflare bypass

Web ScrapingBrowserCloudflareStealthCrawlingSpiderOptionalHermes skill
Last registry verification2026-08-18v1.0.0FEUAZUR
Open original source ↗اقرأ بالعربية
Plain meaning

What does it add to Hermes?

Scrape sites with stealth browsing and Cloudflare bypass

Scrapling is a skill related to the web and browser automation. It can let the agent open pages, read websites, or perform browser steps, depending on the tools the resource exposes.

This plain-language explanation is based on the publisher description. The original text remains visible for verification.

Use it when

Use it when your goal in the web and browser automation is clear and you can limit it to the data and actions it actually needs.

Skip it when

Do not add it merely to experiment when Hermes already has a simpler path, or when you cannot review its source and permissions.

Who is it for?

Best for users who want a repeatable way of working inside Hermes.

Safe first test

Start on a public site without signing in, and extract one heading before allowing clicks or typing.

Original publisher description

Scrape sites with stealth browsing and Cloudflare bypass

✓
Data source

This entry was indexed from Hermes Optional Skills. Our explanation interprets the type and domain without inventing a capability not present upstream.

!
Security review

The source is official or editorially reviewed, but you still need to review permissions and version compatibility.

Safe setup path

Inspect, install, then test.

  1. 01
    Open the source

    Match the publisher, license, and description to your need. Check the real update history.

  2. 02
    Review permissions and secrets

    Never paste a secret value into this site. Use environment-variable names and grant the smallest scope.

  3. 03
    Copy setup only after review

    The controls below copy text. They do not execute commands on your device.

  4. 04
    Test with a non-sensitive task

    Inspect the visible tools, then exclude write or delete tools you do not need.

Install command

Review the command, then copy it.

hermes skills install scrapling

Hermes Belarabi does not execute this command. Installation happens on your device and remains subject to Hermes scanning and your review.

The full skill definition

Exactly what Hermes loads when this skill runs.

Reproduced from the official documentation. Read it before enabling the skill: this text becomes the agent's instructions.

Scrape sites with stealth browsing and Cloudflare bypass.

Skill metadata

A lookup table. Do not read it all; find the row that applies to you.

SourceOptional — install with hermes skills install official/research/scrapling
Pathoptional-skills/research/scrapling
Version1.0.0
AuthorFEUAZUR
LicenseMIT
Platformslinux, macos, windows
TagsWeb Scraping, Browser, Cloudflare, Stealth, Crawling, Spider
Related skillsduckduckgo-search, domain-intel

Reference: full SKILL.md

Explains the idea itself. Read it slowly; the later sections build on it.

Scrapling ↗ is a web scraping framework with anti-bot bypass, stealth browser automation, and a spider framework. It provides three fetching strategies (HTTP, dynamic JS, stealth/Cloudflare) and a full CLI.

This skill is for educational and research purposes only. Users must comply with local/international data scraping laws and respect website Terms of Service.

When to Use

Explains the idea itself. Read it slowly; the later sections build on it.

  • Scraping static HTML pages (faster than browser tools)
  • Scraping JS-rendered pages that need a real browser
  • Bypassing Cloudflare Turnstile or bot detection
  • Crawling multiple pages with a spider
  • When the built-in web_extract tool does not return the data you need

Installation

Ordered, practical steps. Run one and confirm it worked before moving on.

Shell2 lines
pip install "scrapling[all]"
scrapling install

Minimal install (HTTP only, no browser):

Shell1 line
pip install scrapling

With browser automation only:

Shell2 lines
pip install "scrapling[fetchers]"
scrapling install

Quick Reference

A lookup table. Do not read it all; find the row that applies to you.

ApproachClassUse When
HTTPFetcher / FetcherSessionStatic pages, APIs, fast bulk requests
DynamicDynamicFetcher / DynamicSessionJS-rendered content, SPAs
StealthStealthyFetcher / StealthySessionCloudflare, anti-bot protected sites
SpiderSpiderMulti-page crawling with link following

CLI Usage

Explains the idea itself. Read it slowly; the later sections build on it.

Extract Static Page

Shell1 line
scrapling extract get 'https://example.com' output.md

With CSS selector and browser impersonation:

Shell3 lines
scrapling extract get 'https://example.com' output.md \
  --css-selector '.content' \
  --impersonate 'chrome'

Extract JS-Rendered Page

Shell4 lines
scrapling extract fetch 'https://example.com' output.md \
  --css-selector '.dynamic-content' \
  --disable-resources \
  --network-idle

Extract Cloudflare-Protected Page

Shell4 lines
scrapling extract stealthy-fetch 'https://protected-site.com' output.html \
  --solve-cloudflare \
  --block-webrtc \
  --hide-canvas

POST Request

Shell2 lines
scrapling extract post 'https://example.com/api' output.json \
  --json '{"query": "search term"}'

Output Formats

The output format is determined by the file extension:

  • .html -- raw HTML
  • .md -- converted to Markdown
  • .txt -- plain text
  • .json / .jsonl -- JSON

Python: HTTP Scraping

Explains the idea itself. Read it slowly; the later sections build on it.

Single Request

Python6 lines
from scrapling.fetchers import Fetcher

page = Fetcher.get('https://quotes.toscrape.com/')
quotes = page.css('.quote .text::text').getall()
for q in quotes:
    print(q)

Session (Persistent Cookies)

Python8 lines
from scrapling.fetchers import FetcherSession

with FetcherSession(impersonate='chrome') as session:
    page = session.get('https://example.com/', stealthy_headers=True)
    links = page.css('a::attr(href)').getall()
    for link in links[:5]:
        sub = session.get(link)
        print(sub.css('h1::text').get())

POST / PUT / DELETE

Python3 lines
page = Fetcher.post('https://api.example.com/data', json={"key": "value"})
page = Fetcher.put('https://api.example.com/item/1', data={"name": "updated"})
page = Fetcher.delete('https://api.example.com/item/1')

With Proxy

Python1 line
page = Fetcher.get('https://example.com', proxy='http://user:pass@proxy:8080')

Python: Dynamic Pages (JS-Rendered)

Explains the idea itself. Read it slowly; the later sections build on it.

For pages that require JavaScript execution (SPAs, lazy-loaded content):

Python4 lines
from scrapling.fetchers import DynamicFetcher

page = DynamicFetcher.fetch('https://example.com', headless=True)
data = page.css('.js-loaded-content::text').getall()

Wait for Specific Element

Python5 lines
page = DynamicFetcher.fetch(
    'https://example.com',
    wait_selector=('.results', 'visible'),
    network_idle=True,
)

Disable Resources for Speed

Blocks fonts, images, media, stylesheets (~25% faster):

Python5 lines
from scrapling.fetchers import DynamicSession

with DynamicSession(headless=True, disable_resources=True, network_idle=True) as session:
    page = session.fetch('https://example.com')
    items = page.css('.item::text').getall()

Custom Page Automation

Python11 lines
from playwright.sync_api import Page
from scrapling.fetchers import DynamicFetcher

def scroll_and_click(page: Page):
    page.mouse.wheel(0, 3000)
    page.wait_for_timeout(1000)
    page.click('button.load-more')
    page.wait_for_selector('.extra-results')

page = DynamicFetcher.fetch('https://example.com', page_action=scroll_and_click)
results = page.css('.extra-results .item::text').getall()

Python: Stealth Mode (Anti-Bot Bypass)

Explains the idea itself. Read it slowly; the later sections build on it.

For Cloudflare-protected or heavily fingerprinted sites:

Python10 lines
from scrapling.fetchers import StealthyFetcher

page = StealthyFetcher.fetch(
    'https://protected-site.com',
    headless=True,
    solve_cloudflare=True,
    block_webrtc=True,
    hide_canvas=True,
)
content = page.css('.protected-content::text').getall()

Stealth Session

Python5 lines
from scrapling.fetchers import StealthySession

with StealthySession(headless=True, solve_cloudflare=True) as session:
    page1 = session.fetch('https://protected-site.com/page1')
    page2 = session.fetch('https://protected-site.com/page2')

Element Selection

Explains the idea itself. Read it slowly; the later sections build on it.

All fetchers return a Selector object with these methods:

CSS Selectors

Python3 lines
page.css('h1::text').get()              # First h1 text
page.css('a::attr(href)').getall()      # All link hrefs
page.css('.quote .text::text').getall() # Nested selection

XPath

Python2 lines
page.xpath('//div[@class="content"]/text()').getall()
page.xpath('//a/@href').getall()

Find Methods

Python3 lines
page.find_all('div', class_='quote')       # By tag + attribute
page.find_by_text('Read more', tag='a')    # By text content
page.find_by_regex(r'\$\d+\.\d{2}')       # By regex pattern

Similar Elements

Find elements with similar structure (useful for product listings, etc.):

Python2 lines
first_product = page.css('.product')[0]
all_similar = first_product.find_similar()
Python5 lines
el = page.css('.target')[0]
el.parent                # Parent element
el.children              # Child elements
el.next_sibling          # Next sibling
el.prev_sibling          # Previous sibling

Python: Spider Framework

Explains the idea itself. Read it slowly; the later sections build on it.

For multi-page crawling with link following:

Python23 lines
from scrapling.spiders import Spider, Request, Response

class QuotesSpider(Spider):
    name = "quotes"
    start_urls = ["https://quotes.toscrape.com/"]
    concurrent_requests = 10
    download_delay = 1

    async def parse(self, response: Response):
        for quote in response.css('.quote'):
            yield {
                "text": quote.css('.text::text').get(),
                "author": quote.css('.author::text').get(),
                "tags": quote.css('.tag::text').getall(),
            }

        next_page = response.css('.next a::attr(href)').get()
        if next_page:
            yield response.follow(next_page)

result = QuotesSpider().start()
print(f"Scraped {len(result.items)} quotes")
result.items.to_json("quotes.json")

Multi-Session Spider

Route requests to different fetcher types:

Python16 lines
from scrapling.fetchers import FetcherSession, AsyncStealthySession

class SmartSpider(Spider):
    name = "smart"
    start_urls = ["https://example.com/"]

    def configure_sessions(self, manager):
        manager.add("fast", FetcherSession(impersonate="chrome"))
        manager.add("stealth", AsyncStealthySession(headless=True), lazy=True)

    async def parse(self, response: Response):
        for link in response.css('a::attr(href)').getall():
            if "protected" in link:
                yield Request(link, sid="stealth")
            else:
                yield Request(link, sid="fast", callback=self.parse)

Pause/Resume Crawling

Python2 lines
spider = QuotesSpider(crawldir="./crawl_checkpoint")
spider.start()  # Ctrl+C to pause, re-run to resume from checkpoint

Pitfalls

Explains the idea itself. Read it slowly; the later sections build on it.

  • Browser install required: run scrapling install after pip install -- without it, DynamicFetcher and StealthyFetcher will fail
  • Timeouts: DynamicFetcher/StealthyFetcher timeout is in milliseconds (default 30000), Fetcher timeout is in seconds
  • Cloudflare bypass: solve_cloudflare=True adds 5-15 seconds to fetch time -- only enable when needed
  • Resource usage: StealthyFetcher runs a real browser -- limit concurrent usage
  • Legal: always check robots.txt and website ToS before scraping. This library is for educational and research purposes
  • Python version: requires Python 3.10+