Undetected ChromeDriver is an optimized version of Selenium’s ChromeDriver and is designed to bypass anti-bot services used for automated browser bot detection. This tool is particularly effective at bypassing the detection mechanisms of services such as Imperva, DataDome and Distil Networks. It can also help bypass certain Cloudflare protections.
What is an Undetected ChromeDriver?
Undetectable, Undetected or by its official name Undetected ChromeDriver is a Python library that provides an optimized version of Selenium’s ChromeDriver. This has been patched to limit detection by anti-bot services such as
- Imperva
- DataDome
- Distil Networks
- Cloudflare
How Does Undetected ChromeDriver Work?
Undetected ChromeDriver reduces detection using the following techniques:
- Renaming Selenium Variables: Replaces the variables used by Selenium with the names used by real browsers.
- Realistic User-Agent Usage: Prevents detection by using real-world User-Agent strings.
- Simulation of Natural Human Interaction: Mimics the user’s natural behavior, making detection more difficult.
- Cookie and Session Management: Manages cookies and sessions correctly when browsing websites.
- Proxy Support: Allows the use of proxies to bypass IP blocking and avoid speed limitations.
Which of our main proxy server products can you use in Undetected ChromeDriver?
Use of Undetected ChromeDriver for Web Scraping
Using Undetected ChromeDriver for web scraping in Python is quite simple. First, install the required package with the following command:
pip install undetected-chromedriver
You can then create a Python script like the one below:
import undetected_chromedriver as uc
# Chrome tarayıcıyı başlat
driver = uc.Chrome()
# Hedef sayfaya bağlan
driver.get("https://www.example.com")
# İstediğiniz veriyi çekme işlemleri...
# Tarayıcıyı kapat
driver.quit()
This script connects to the specified website and allows you to pull the data you want. You can also customize browser options using uc.ChromeOptions(). For example, to run the browser in headless mode:
options = uc.ChromeOptions()
options.headless = True
driver = uc.Chrome(options=options)
Proxy Integration
You can avoid IP blocking and speed limitations by using a proxy. You can add the proxy as follows:
options = uc.ChromeOptions()
options.add_argument("--proxy-server=http://proxy_ip:port")
driver = uc.Chrome(options=options)
If the proxy requires user:pass authentication, you can use the seleniumwire library:
from seleniumwire import undetected_chromedriver as uc
proxy_options = {
'proxy': {
'http': 'http://kullanici_adi:sifre@proxy_ip:port',
'https': 'https://kullanici_adi:sifre@proxy_ip:port',
'no_proxy': 'localhost,127.0.0.1'
}
}
driver = uc.Chrome(seleniumwire_options=proxy_options)
Limitation and Alternatives
While Undetected ChromeDriver is effective at bypassing many anti-bot systems, it may fall short on websites with more advanced detection mechanisms. Also, it does not support features like proxy rotation on its own. In such cases, you may need to use more advanced solutions or additional tools.
- The success rate is quite high on low-security sites. On websites with standard bot detection mechanisms, you can operate as if you were logging in manually.
- At medium levels of protection, bot detection by services such as Cloudflare may come into play. In this case, additional solutions (e.g. browser fingerprinting or cookie storage) may be required.
- Highly secure sites are likely to fail due to JavaScript-based behavior analysis and advanced bot detection systems. In such cases, it may be more effective to use advanced solutions or consider manual data extraction methods.
Some users report that the browser takes a little longer to launch than the standard ChromeDriver. This is due to additional modifications to circumvent bot detection systems. However, this difference is usually negligible in long-term runs.
Usage Style and Best Practices
To use Undetected ChromeDriver efficiently, it is important to consider the following best practices:
Perform IP Address Rotation for Frequent Use:
Continuous requests from the same IP address can increase the risk of being detected. You can change your IP address regularly by using proxy services or a VPN. You can use an auto-rotating proxy with the Rotating Residential Proxy service you purchased from our site, or if you have a list of proxies you have purchased, you can make your own rotation with the article “How to Rotate Proxies in Python?” which we explained in another article.
Hide Browser Fingerprinting:
The fingerprint of the scanner can give away whether you are a bot or not. selenium-stealth or similar libraries to make your fingerprint similar to real users.
Store and Use Cookies:
Websites store users’ past session information in cookies. In web scraping operations, you can provide a more natural use by saving cookies of your previous sessions.
Use Headless Mode with Care:
Although headless mode is useful for web scraping, some sites may consider it as bot activity. If you are having trouble being detected, you can try disabling headless mode.
Imitate Human Behavior:
Making random clicks while browsing websites, adding random delays between pages and simulating mouse movements can reduce the likelihood of detection.
Conclusion
While Undetected ChromeDriver is a powerful tool to bypass bot detection systems, it may not work perfectly in all cases. Depending on your usage scenario, integrating additional methods such as proxy support, human behavior emulation, browser fingerprint management, etc. can significantly reduce the likelihood of detection.
For more sophisticated systems, alternative solutions such as Playwright, Puppeteer or API-based data extraction methods can be considered. Remember that scraping operations must be carried out in an ethical and legal framework!
