Crawling API
Easy to use API to crawl and scrape websites
Crawler
Move your crawling queues to the cloud
Proxy Backconnect
Advanced host:port rotating proxy to use in apps
Scraper API
Structured data API automatically scrape websites
Leads API
Get access to various company emails
Crawling Storage
Store your crawled or scraped data on the cloud
General
Manage your account and other generic questions
Top frequently asked questions
Crawler
How many times are requests re-tried?
By default a request is retried 110 times for a period of 48 hours from the first time it starts to be processed.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Proxy
Can the proxy be used with https?
Yes, the proxy can be used for https websites, but the connection to the proxy must be done with http. So when connecting to the proxy use http://proxy.proxycrawl.com and not https://proxy.proxycrawl.com All https traffic that you send to any website using the proxy, will continue to be securely transfered and we won't be able to access it in any way.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Crawling api
Do I need to know how to code to use the Crawling API?
At the moment the only way to use ProxyCrawl is by integrating it to your product or service, therefore you will need to know development or know/pay someone which can help you. Please note that we have some very easy to integrate libraries.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Crawling api
Can I use the free requests for javascript?
Yes you can use the free requests for javascript or normal requests.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Crawling api
Do you sell a list of proxies?
We want to offer more than just a list or lists of proxies, therefore our API and Crawler products are built on top of thousands of residential and data center worldwide proxies combined with artificial intelligence plus our engineering team which helps delivering the best data results possible. If you are looking just for proxies and you don't want to get any of the other benefits, we recommend that you use the Proxy Backconnect
Was this helpful?![Thumbs up]()
![Thumbs down]()
Leads api
Are the leads you provide updated?
All the leads you get from the Leads API are up to date. We don't work with outdated sources.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Leads api
How much will I be charged when I upgrade?
We prorate the charges, so only the the amount according to the remaining days of the month period will be charged.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Scraper api
How can I request a specific Scraper API scraper?
If the Scraper API is not giving you the data you need, please let us know more details on the pages you are crawling and we will build a classifier for it.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Scraper api
How can I cancel my subscription?
Visit the subscription plan page and cancel your subscription plan there.
Was this helpful?![Thumbs up]()
![Thumbs down]()
Scraper api
What is the difference between Crawling API and Scraper API?
The Crawling API is a generic API that can be used to crawl any website and get the raw source page. It also gives the option to use our classified scrapers. The Scraper API is an API that focuses on auto scraping and parsing HTML pages into structured data. One of their differences as well, the Crawling API is tiered pricing and payments are made at the end of every month based on the request consumption while the Scraper API is subscription-based.
Was this helpful?![Thumbs up]()
![Thumbs down]()