Crawl Browser is a tool to help you extract data from web pages more easily through automation tools or techniques.
With the credentials we provide, you can choose to use the Crawl Browser locally or on our We will open a fingerprinting browser for you with a proxy network to handle automation scripts as well as data collection for you through browser automation and unlockers.
Application Scenarios:
Widely used in data analysis, market monitoring and other fields. Compatible with mainstream framework technology language (such as Puppeteer, Selenium, playwright), suitable for e-commerce price monitoring, public opinion analysis, academic research and other scenarios.
Quick Start
Get Account Password
Visit , register an account, go to the Personal Centre, and get your account password through in Scraping Browser.
How to use
Request samples
Below, you'll find sample Python requests. For examples in other programming languages, please refer to the relevant sections:
import asyncio
from playwright.async_api import async_playwright
const AUTH = 'PROXY-FULL-ACCOUNT:PASSWORD';
const SBR_WS_SERVER = `wss://${AUTH}@upg-scbr.abcproxy.com`;
async def run(pw):
print('Connecting to Scraping Browser...')
browser = await pw.chromium.connect_over_cdp(SBR_WS_SERVER)
try:
print('Connected! Navigating to Target...')
page = await browser.new_page()
await page.goto('https://example.com', timeout= 2 * 60 * 1000)
# Screenshot
print('To Screenshot from page')
await page.screenshot(path='./remote_screenshot_page.png')
# html content
print('Scraping page content...')
html = await page.content()
print(html)
finally:
# In order to better use the Scraping browser, be sure to close the browser
await browser.close()
async def main():
async with async_playwright() as playwright:
await run(playwright)
if _name_ == '_main_':
asyncio.run(main())
from selenium.webdriver import Remote, ChromeOptions
from selenium.webdriver.chromium.remote_connection import ChromiumRemoteConnection
from selenium.webdriver.common.by import By
# Enter your credentials - the zone name and password
AUTH = 'PROXY-FULL-ACCOUNT:PASSWORD'
REMOTE_WEBDRIVER = f'https://{AUTH}@hs-scbr.abcproxy.com'
def main():
print('Connecting to Scraping Browser...')
sbr_connection = ChromiumRemoteConnection(REMOTE_WEBDRIVER, 'goog', 'chrome')
with Remote(sbr_connection, options=ChromeOptions()) as driver:
# get target URL
print('Connected! Navigating to target ...')
driver.get('https://example.com')
# screenshot
print('screenshot to png')
driver.get_screenshot_as_file('./remote_page.png')
# html content
print('Get page content...')
html = driver.page_source
print(html)
if __name__ == '__main__':
main()
const puppeteer = require('puppeteer-core');
const AUTH = 'PROXY-FULL-ACCOUNT:PASSWORD';
const WS_ENDPOINT = `wss://${AUTH}@upg-scbr.abcproxy.com`;
(async () => {
console.log('Connecting to Scraping Browser...');
const browser = await puppeteer.connect({
browserWSEndpoint: SBR_WS_ENDPOINT,
defaultViewport: {width: 1920, height: 1080}
});
try {
console.log('Connected! Navigating to Target URL');
const page = await browser.newPage();
await page.goto('https://example.com', { timeout: 2 * 60 * 1000 });
//1.Screenshot
console.log('Screenshot to page.png');
await page.screenshot({ path: 'remote_screenshot.png' });
console.log('Screenshot be saved');
//2.Get content
console.log('Get page content...');
const html = await page.content();
console.log("source Htmml: ", html)
} finally {
// In order to better use the Scraping browser, be sure to close the browser after the script is executed
await browser.close();
}
})();
const pw = require('playwright');
const AUTH = 'PROXY-FULL-ACCOUNT:PASSWORD';
const SBR_CDP = `wss://${AUTH}@upg-scbr.abcproxy.com`;
async function main() {
console.log('Connecting to Scraping Browser...');
const browser = await pw.chromium.connectOverCDP(SBR_CDP);
try {
console.log('Connected! Navigating to target...');
const page = await browser.newPage();
// Target URL
await page.goto('https://www.windows.com', { timeout: 2 * 60 * 1000 });
// Screenshot
console.log('To Screenshot from page');
await page.screenshot({ path: './remote_screenshot_page.png'});
// html content
console.log('Scraping page content...');
const html = await page.content();
console.log(html);
} finally {
// In order to better use the Scraping browser, be sure to close the browser after the script is executed
await browser.close();
}
}
if (require.main === module) {
main().catch(err => {
console.error(err.stack || err);
process.exit(1);
});
}
Step 1：Selection of use cases for obtaining relevant scripts.
Step 2：Specify region (not required), select account number.
Step 3：
Run the script in the REQUEST area and wait for the processing to return the result in RESPONSE and CONSOLE to view the preview result and result log.The right side of the RESPONSE area can be switched to html and download the html file.
All information herein is provided on an “as is” basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on this page. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website’s terms of service or receive a scraping license.
