Cloudflare Turnstile not always visible and checked on this particolar site

[thomvas]

Hi, i'm testing SelenuimBase 4.40.6 on keepa.com, where there is Cloudflare Turnstile.

I need a solution with driver and not sb. Also, the solution must works on Windows, MacOS, Linux, and Linux without GUI, without showing the browser.

I created a loop to check the bypass code repetitively.

My script open the page https://tinyurl.com/pm5xvzw8, it is a shorter link because the link is too long.

After that, sometimes the site shows Turnstile, sometimes not, so i add cdp.gui_click_element.

I use driver.cdp.gui_click_element("#turnstile div") because the Turnstile is in a popup in shodow-root (check attached images).

The site loads content only if i put the mouse on an element, so i use ActionChains for ".fa.fa-search" element.

At this point, sometimes the site shows Turnstile again, sometimes not, so i add cdp.gui_click_element again.

After that, i extract the ASINs from the elements.

Watch the attached video to see the code running.

My code is:

from seleniumbase import Driver
from selenium.webdriver import ActionChains

url = "https://tinyurl.com/pm5xvzw8"

i = 0

while True:

    print(f"\nCheck n. {i}")

    driver= Driver(uc=True)

    driver.uc_activate_cdp_mode(url)
    
    try:
        driver.cdp.gui_click_element("#turnstile div")
    except Exception as e:
        print(f"First no captcha: {e}")

    driver.reconnect()

    try:
        driver.wait_for_element_visible(".fa.fa-search", timeout=15)
        search_icon = driver.find_element("css selector", ".fa.fa-search")
        ActionChains(driver)\
            .move_to_element(search_icon)\
            .perform()
    except Exception as e:
        print(f"Error .fa.fa-search: {e}")
        driver.save_screenshot("error_fafa-search.png")

    driver.disconnect()
    
    try:
        driver.cdp.gui_click_element("#turnstile div")
    except Exception as e:
        print(f"Second no captcha: {e}")

    driver.reconnect()

    try:
        driver.wait_for_element_visible(".productContainer", timeout=15)
        product = driver.find_elements(".productContainer")

        for prod in product[0:5]:
            asin = prod.find_element("css selector", "a[href*='#!product/']")
            asin = asin.get_attribute("href")
            asin = asin.split("-")[-1]
            print(f"ASIN: {asin}")
        
    except Exception as e:
        print(f"Error .productContainer: {e}")
        driver.save_screenshot("error_productContainer.png")
    
    driver.quit()
    
    i += 1

The HTML code of the Turnstile is:

<div id="popup3"><div id="overlayShadowTop3"></div><div class="relativeAlignCenter flex"><div class="hidden generalOverlay" style="max-height: 80%; overflow-y: auto; opacity: 1; display: block; transform-origin: 50% 50% 0px;"><div class="text_long mdc-elevation--z10" style="margin:0px;text-align:justify; color: #444; max-width:95%;position: relative;"><div id="popupTitle" style="font-size: 18px; margin: 0 40px 15px 0;display: inline-block; color: #111">Anti-bot check</div><br><span id="popupMessage"><div id="turnstile"><div><input type="hidden" name="cf-turnstile-response" id="cf-chl-widget-2gdn3_response"></div></div></span></div></div></div></div>

Problems:

1. Turnstile sometimes resolves itself automatically, without use cdp.gui_click_element("#turnstile div"), so i can't test if my code is correct for resolve Turnstile. How can I make it not be automatically resolved, so I can use cdp.gui_click_element("#turnstile div") and see if it works?

2. When Turnstile appears, I see the mouse moving towards it, but I'm not sure if it works properly. Also, sometimes Turnstile seems to only load partially, see the image attached "POPUP Turnstile Incomplete"

3. What do I need to change or add, to make my code suitable for running on Windows, Linux, MacOS and Linux without GUI? I need a solution without showing the browser, such as window_position="-2400,-2400".

Thanks :)

VIDEO
https://github.com/user-attachments/assets/01c26777-3bea-49fb-93a5-df5bb8f0970a

POPUP Turnstile

POPUP Turnstile Incomplete

SHADOW-ROOT

[mdmintz]

Since you can't see into closed Shadow-root elements, methods like sb.cdp.gui_click_element(selector) and sb.uc_gui_click_captcha() do their best to click on the CAPTCHA through the Shadow-root, but they can't see inside to know if the CAPTCHA has already been solved or is even there anymore. To force the likelihood of the CAPTCHA not getting solved automatically, you can change your user agent to something fake, eg: agent="cool". That'll likely force the CAPTCHA to not get solved automatically.

If the CAPTCHA is only partially loaded, add more wait time before calling one of the clicking methods. As mentioned above, there's no way to see if the CAPTCHA has finished loading since it's inside closed Shadow-root.

You can set the window_position to "x,y" coordinates to start the window off-screen, but note that PyAutoGUI can only click on visible browser windows, which means that if you have to click the CAPTCHA with one of the special methods, the window will need to be on screen and visible. On Linux, you'll need the SB() manager for the special virtual display that allows you to run a headed browser in a GUI-less environment. There are lots of examples of bypassing Cloudflare in GitHub Actions here: https://github.com/mdmintz/undetected-testing/actions/runs/16293201161/job/46007958250

[thomvas]

thank you! You are the best!