How Can I Prevent Selenium Webdriver from Being Detected by Websites?

Selenium Webdriver: Preventing Detection by Modifying the navigator.webdriver Flag

Introduction

Selenium, a popular web automation framework, faces challenges when websites implement detection mechanisms that flag Selenium-driven browsers. This article aims to address such detection issues by providing effective methods to modify the navigator.webdriver flag, thereby preventing Selenium's detection.

Approach

1. Add Chrome Arguments:

   • --disable-blink-features=AutomationControlled disables the automation-detection feature.

   options = webdriver.ChromeOptions()
   options.add_argument('--disable-blink-features=AutomationControlled')
   driver = webdriver.Chrome(options=options)

2. Execute CDP Command:

   • execute_cdp_cmd allows direct manipulation of the browser's Chrome DevTools Protocol (CDP).
   • Modify the user-agent to further mimic real user behavior.

   driver.execute_cdp_cmd('Network.setUserAgentOverride', {'userAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})

3. Modify navigator.webdriver Property:

   • Change the property value of navigator.webdriver to undefined.

   driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")

4. Exclude Automation Switches:

   • Add excludeSwitches to prevent the collection of automation-related switches.

   options.add_experimental_option("excludeSwitches", ["enable-automation"])

5. Disable Automation Extension:

   • Turn off the useAutomationExtension option to further reduce detection.

   options.add_experimental_option('useAutomationExtension', False)

Sample Code

Combining the above methods into a single code block:

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options)
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {'userAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
driver.get('https://www.website.com/')

Historical Context

The navigator.webdriver flag is defined by the W3C as a way to indicate that a user agent is under remote control.

Caution

Tweaking these parameters may interfere with browser navigation and make the WebDriver instance detectable.

Updates

• Selenium Python clients now support the execute_cdp_cmd method.
• Additional parameters have been added to further improve detection prevention.

Conclusion

By following these methods, you can effectively modify the navigator.webdriver flag and prevent websites from detecting your Selenium-driven browser. However, it's important to handle these changes with caution to avoid detection.

The above is the detailed content of How Can I Prevent Selenium Webdriver from Being Detected by Websites?. For more information, please follow other related articles on the PHP Chinese website!