Spider Web Scraper/Crawler
Scrape & Crawl the web with Spider - the fastest open source web scraper & crawler.
Spider is the fastest open source web scraper & crawler that returns LLM-ready data. To get started using this node you need an API key from Spider.cloud.
Get Started
Go to the Spider.cloud website and sign up for a free account.
Then go to the API Keys and create a new API key.
Copy the API key and paste it into the "Credential" field in the Spider node.
Scrape & Crawl
Choose either "Scrape" or "Crawl" in the mode dropdown.
Enter the URL you want to scrape or crawl in the "Web Page URL" field.
If you chose "Crawl", enter the maximum amount of pages you want to crawl in the "Limit" field. If no value or 0 is entered, the crawler will crawl all pages.
Example
Last updated