WebScrapingAPI is one of the numerous scraping APIs. And like others, you can’t be so sure of the claims they make on their website. How then do you know if it is right for you or not? It is for this reason that we wrote this WebScrapingAPI review.
Quick Preview show
In this WebScrapingAPI review, we’ll reveal to you, all you need to know about this scraping API. This includes the API latency (speed) test, website compatibility test, geotargeting support, the proxy option provided, data parsing, pricing, ease of usage, and customer support, among others.
It is important you know that we didn’t consider WebScrapingAPI in isolation. At the time of carrying out the performance test, we tested other similar proxy APIs for web scraping such as ScraperAPI, ScrapingBee, Zenscrape, and Scrapestack, among others. This is to ensure you get a feel of how it is related to its competitors.
WebScrapingAPI Review Summary Table
| Feature | WebScrapingAPI |
|---|---|
| Price | Starts at $29/month |
| Features | Supports a wide range of websites, including dynamic and JavaScript-heavy websites. Offers a variety of scraping methods, including headless browser, proxy, and API. |
| Ease of use | Easy to use and set up. Offers a user-friendly interface and detailed documentation. |
| Support | Offers 24/7 support via email and live chat. |
| Reputation | Has a good reputation among users. |
What is WebScrapingAPI?
WebScrapingAPI is a scraping API for web data collection. It is a ready-use API that collects web data either in raw HTML or structured JSON. This web scraping tool takes away the headaches associated with proxies. It handles headless browsers for JavaScript rendering, manages and rotates proxies, and also solves captchas automatically. WebScrapingAPI is also equipped with an advanced anti-bot bypass system, which makes your web scraping task more undetectable.
Aside from the regular ScraperAPI which it offers, it comes with two more specialized scraper APIs. These include Google Search Results Scraper and Amazon Product Scraper. With WebScrapingAPI, you do not need to develop your own web scraper or even manage one. And to make things even better for you, you only get to pay for successful requests. But don’t rush to pay for a plan yet, there’s still more food and a few not-so-good things I will reveal in this WebScrapingAPI review.
WebScrapingAPI Latency Speed Test: How Fast is the Scraping API?
The latency of an API is the time taken for an API to return a response after getting a request. For web scraping performance, this determines how many pages you can scrape within a period of time. API latency is time measurement. In our own case, we use seconds to measure it. To find out how fast the WebScrapingAPI is, we use the Postman software, an API performance testing tool.
As stated in the introduction, we didn’t measure the API latency speed in isolation, we measured it alongside that of competitors. To make the data easy for you to grasp, we present the API speed performance for 2 competitors of WebScrapingAPI. This includes ScraperAPI and ScrapingBee. Below is the speed measured from the performance test.
| Web Services | WebScrapingAPI (secs) | ScraperAPI (secs) | ScrapingBee (secs) |
|---|---|---|---|
| 5.08 | 7.46 | 4.71 | |
| Bing | 1.2 | 1.92 | 4.28 |
| Yahoo | 2.28 | 1.17 | 4.43 |
| 1.37 | X | 4.47 | |
| 1.2 | X | 4.78 | |
| Amazon | 7.53 | 1.32 | 5.04 |
| AliExpress | 0.98 | 2.92 | 6.02 |
| Craigslist | 3.14 | 3.55 | 4.53 |
| Walmart | 1.2 | 2.31 | 13.65 |
| Booking | 2.73 | 2.34 | 7.08 |
As you can see from the table above, WebScrapingAPI is the fastest scraping API when compared with ScrapingBee and ScraperAPI. Speed ranges from 0.98 seconds for AliExpress to 7.53 seconds for Amazon. On average, the response time is 3.9 seconds. This makes WebScrapingAPI faster than both ScraperAPI and ScrapingBee. In fact, for websites such as Bing, Walmart, Facebook, and Instagram, the response time is a little over 1 second — below 1 second for AliExpress.
WebScrapingAPI Site Compatibility Test
One of the major issues with web scraping is blocks. Websites do not like being scraped and as such, integrate some form of anti-bot and anti-scraping systems to block scrapers. Scraping APIs are tools for avoiding blocks when scraping. But how good is WebScrapingAPI for this?
To test how effective WebScrapingAPI’s anti-bot bypass system is, we tested it against known targets that are difficult to scrape because of their anti-bot systems. Booking is one of the most difficult to scrape websites as it is protected by PerimeterX, a strong anti-bot system. The other targets we tested WebScrapingAPI API on include FastPeopleSearch which is protected by multiple anti-bot systems including Cloudflare and captchas. Sephora.com is protected by Cloudflare, and Datadome is protected by its own anti-bot system (Datadome).
WebScrapingAPI was able to access all of these targets without getting blocked. Not many scraping APIs could access FastPeopleSearch because of its multiple anti-bot systems. For Sephora, normal bots couldn’t load them. But WebScrapingAPI did. It is safe to say WebScrapingAPI is at the same level of effectiveness as ScraperAPI.
The targets mentioned are for testing anti-bot systems. As you can see from the speed testing, they are also perfect for your regular websites. These Google, Bing, Yahoo, Facebook, Instagram, Amazon, AliExpress, Walmart, Craigslist, and Booking, among others. However, for most of these popular websites or any other websites that are difficult to scrape, you’ll need to use their premium proxy option which will consume more credits.
Geolocation Support: Which Location IPs Does WebScrapingAPI Offers?
Do you want to scrape localized data? If the data of interest has geotargeting dimension, the. The location of the IP address is an important factor. If you need to scrape Google Germany, you’ll need a German IP address. Does WebScrapingAPI allow you to choose the location of IPs? Yes, they do. This makes them perfect for scraping localized content. But what are the locations supported? For datacenter proxies, this scraping API supports the below 10 countries.
If your tasks require residential proxies, you’re also covered. This is because WebScrapingAPI supports 12 countries in their residential proxy pool. The list of countries is provided below.
For enterprise customers, there are about 195 countries available on request for their residential proxy network. One unique thing that you’ll come to appreciate about WebScrapingAPI is its support for geotargeting for datacenter proxies. Most scraping APIs most notably ScrapingBee only support geotargeting for residential/mobile proxies.
WebScrapingAPI Proxy: What Proxy Options are Available?
At the core of web scraping APIs are proxies without which rate-limiting websites will block them. With WebScrapingAPI, you do not have to worry about the request limits of websites. This is because WebScrapingAPI has a massive proxy network with over 100 million IPs in its pool that you can utilize. The proxies offered by this scraping API are rotating proxies. With this, you can’t get blocked because of too many requests.
Currently, you can only get datacenter proxies and residential proxies from WebScrapingAPI — mobile proxies are not supported. The datacenter proxies offered are private proxies with less likelihood of already being blacklisted. These private proxies are the default proxies used by WebScrapingAPI. The cost fewer credits to use but are susceptible to blocks by popular websites with strong anti-bot systems. For difficulty to scrape websites, I recommend you make use of their residential proxies. However, this will cost you 10 credits as opposed to 1 credit for datacenter proxies. With JavaScript rendering, residential proxies will cost you 25 credits per request.
WebScrapingAPI JavaScript Rendering: How Well Does It Renders and Execute JavaScript?
One of the core tasks of WebScrapingAPI is to handle JavaScript rendering using a headless browser (Chrome). But how well does it do that? By default, this scraping API does not render JavaScript. However, if you want to render JavaScript, you can pass the value (1) to the render_js parameter. With this, web pages will be accessed using headless Chrome, and as such, JavaScript will be rendered. But does a simple cURL command or API call be able to execute complex JavaScript events? Unlike in the case of ScraperAPI, WebScrapingAPI does have advanced support for JavaScript rendering.
You can set some parameters so the browser will wait for specific page elements to load before scraping. These parameters are known as the await parameters, and they include the wait_until, wait_for, and wait_for_css parameters. This scraping API also has support for JavaScript instruction which is a feature that helps you automate certain tasks such as clicking buttons, scrolling down, and many more. This gives it better handling and control over JavaScript rendering than others that just support basic rendering. To use this feature, you will have to stringify the instructions. It is also easier with the WebScraperAPI SDK/ libraries.
Language Support: What Programming Language Does WebScrapingAPI Provide Support?
All programmers can make use of WebScrapingAPI. This is because it is not language-dependent. It is made available as a REST API platform. This means that all you need to interact with it is a simple API call. This makes WebScrapingAPI integrate with any language including Python, Java, NodeJS, PHP, C#, Go, and many others. You can even use cURL to interact with it.
However, there is also something interesting about it. This scraping API does offer an SDK for popular programming languages. Currently, you can get a WebScrapingAPI SDK for Python, NodeJS, Rust, Java, PHP, and Scrapy Python framework for web scraping and crawling. With the SDK provided, using the API and accessing some of its features like using extraction and automation support becomes easier.
Data Parsing Support: Does WebScrapingAPI offer Automatic Parsing Support?
The work of many scraping APIs stops the moment they deliver the raw HTML of a web page. Parsing data is left to you. Is WebScrapingAPI one of such? It might interest you to know that this scraping API stands in also as a parser. In fact, for Google Search and Amazon, it provides specialized scraping APIs that will return structured JSON of important data points. But what about other web pages? This API also does have support for parsing data from them, but you’ll need to put in a little work.
With the extraction rule feature of WebScrapingAPI, you’re able to scrape specific data from web pages. And yes, you do not need an SDK to do that. The SDK makes it easier. But even without it, all you have to do is stringify the extraction rules and pass it as a parameter. This makes this scraping API a full-fledged solution for web scraping for developers. It manages proxies, bypasses captchas, avoids blocks, and parses data.
WebScrapingAPI Pricing: Is It worth its price?
The WebScrapingAPI REST API is a paid tool. As a new user, you are given 5000 API credits to try out the service. That is where the free things end. And this is what makes the pricing important. The WebScrapingAPI spending is based on successful requests. However, the packages and plans are not packages based on the number of requests but on API credits. This makes it a little confusing. Let starts with the pricing based on API credit and see how it can be affected by your requests.
Pricing for WebScrapingAPI starts from $49 for their starter plan. This gives you 100K credits. Other plans include the Grow plan (1 million credits) sold for $149, the Business plan (3 million credits), sold for $299, and the Pro plan (10 million) sold for $799. There is also an enterprise plan available. These plans vary not just in the number of API credits, but also in the number of concurrent requests allowed, and geo-targeting options. Compared to its competitors, WebScrapingAPI pricing is like that of ScraperAPI and ScrapingBee but shared more resemblance with ScraperAPI.
Charge per Request consideration and Reality
The API credits you purchase will be used to access the API and one request does not equal one credit. When scraping APIs were first introduced, their pricing was based on requests. However, the different features have led to the current pricing system. The credit cost per request is based on your combination of proxy and JavaScript preference. It varies between 1 – 25 credits per request. This is the same as what is obtainable from its competitors such as ScrapingBee and ScraperAPI.
As you can see from above, if you send a web request via their private proxies with Javascript rendering turned off, the cost is 1 API credit. However, using their residential proxies instead of the private (datacenter) proxies, the cost per request is 10 API credits. Residential proxies with JavaScript rendering will cost you 25 API credits. Unlike in the case of ScraperAPI and ScrapingBee, WebScrapingAPI does not have a 3rd proxy option that cost more credits. Considering what it will cost you to manage a web scraper and all the headaches associated with that, the WebScrapingAPI pricing and API request cost is worth it.
Ease of Usage: How Easy is WebScrapingAPI for Use?
The WebScrapingAPI can be said to be an easy-to-use web data extraction solution. One good thing about it is that it provides two methods through which it can be used — REST API and SDK. The first method makes it possible to retrieve data from web pages by sending a simple API request. https://api.webscrapingapi.com/v1?api_key=<YOUR_API_KEY>&url=<TARGETED_URL>. This is the format, replace the placeholders with the appropriate details.
The format above is the most basic request. You can add parameters to give custom commands and choose other specific features as in the case of using residential proxies and JavaScript rendering. Even though this method is easy, you’ll appreciate the WebScrapingAPI SDK even more. The SDK is available for multiple programming languages including NodeJS, Python, Java, Rust, and PHP. They also have an SDK for Scrapy, the popular scraping framework for Python.
WebScrapingAPI Customer Support: How Helpful is the Customer Agent?
Customer service is a key indicator of how serious a business is. No matter the level of self-service options you provide, you still need the service of dedicated humans to provide support. How does WebScrapingAPI stand in this regard? The service has a team of customer service agents to respond to customers’ queries and help them out.
However, as with most other scraping APIs out there, it still does not have live chat support. Customer support is provided via the contact form and email and responses are usually within a few hours.
WebScrapingAPI Review Verdict: Is This Scraping API Recommended?
From the above, you can see how WebScrapingAPI performs in terms of speed and evading anti-bot systems. You can also see how well it renders JavaScript and its other core features. No doubt, this web scraping API has proven to be a force to be reckoned with. Even though it has its cons as with most other REST APIs for web scraping, we highly recommend it for its speed, reliability, and its advanced JavaScript execution support, among others.
WebScrapingAPI Alternatives
WebScrapingAPI is one of the fastest web scraping APIs and it works. However, if for any reason, you’ll want to make use of an alternative, then below are the top 3 alternatives to WebScrapingAPI.
ScraperAPI — Best WebScrapingAPI Alternative
ScraperAPI is arguably the best proxy API for web scraping. Its strength lies in how effective it is against anti-bot systems. It is also a fast option and has a pricing tier similar to that of WebScrapingAPI. However, unlike WebScrapingAPI, you do not get extraction rule support and as such, parsing is left to you.
ScrapingBee — Best for JavaScript Rendering Support
ScrapingBee is a lot like WebScrapingAPI in terms of its JavaScript rendering support. If you need an alternative that still provides the same level of JavaScript rendering support, extraction rules for data parsing, and JavaScript instruction support then ScrapingBee is the option. However, ScraperAPI is faster than ScrapingBee.
Proxycrawl — Best for Automatic Data Parsing
Proxycrawl Scraper API is one of the few scraping APIs that provide support for automatic data parsing for many popular web services. With this service, you can get structured JSON with important data points for popular social media, e-commerce, and booking web services.
