![]() I would conclude that there is something on selenium's end that is causing me to be identified as a bot, when I'm using the browser as a regular user. There's nothing in the request headers that would suggest it is any different from a regular browser. My user agent is unspoofed and it's no different from my normal browsing experience. I have all my regular extensions that I have installed on my profile. I have used a custom profile, with custom cache path. Is there something specific about selenium that's making google identify me as a bot automatically? Then I logged out, returned to the login page, entered my credentials, pressed the reCAPTCHA box.and it asked me to solve the image selection problem again!Īt this point I'm thinking, I just solved the captcha successfully half a minute ago, exhibited a bunch of manual human actions, but I'm still being identified as a bot. So I solve the captcha and successfully log in. Now I'm thinking, OK, so if google thinks I'm a bot, how about I solve the captcha in the selenium-launched browser once, let them know I'm good, and then it won't happen again? Maybe it identifies the browser as a new client, and just needs to know that this new client is not a bot. Maybe I just need to solve the captcha once?.So I decided to use my own profile's cache tPreference("_directory", PATH_TO_MY_PROFILE_CACHE) Īnd verified that all of my cached resources are there. In firefox you can see this by going to about:cache and it will say something like anonymous6337741624277931373webdriver-profile\cache2, and there isn't much there. ![]() I verified that all of the websites that I have saved my credentials were there in the selenium-launched browser, but when I confronted the reCAPTCHA, it determined I was a bot and asked for image selectionīy default, selenium uses a custom cache path that is cleaned up after the session is over. I can specify a custom profile to use, so I simply passed in my own firefox profile stored in APPDATA/roaming/mozilla/profiles. Selenium by default creates a new profile, so it has no cookies or browsing history. Maybe I don't have any cookies or browsing history?.So the browser starts up, I type in the URL, I click a few other links, I come back to the login page, type in username password, then proceed to click on the captcha box.and I'm a bot. There are many theories that talk about things like mouse movement, keyboard strokes, etc. So I tried a few different things to try to look more human: Which of course means a lot of stuff is going on under the hood, but perhaps the selenium instance of firefox is not "human" enough? So I'm doing nothing much but starting firefox, using selenium. I decided to launch firefox using this piece of java code WebDriver driver = new FirefoxDriver(new FirefoxProfile()) I then became curious what was the difference between me launching firefox through the executable, and me launching firefox through selenium. I started up a regular instance of firefox (that is, without selenium), went to the website, clicked the checkbox, and it determined that I was a human and let me go. Recently the website changed their login system by adding google's reCAPTCHA, and everytime I try to click the checkbox, google determines that I am a bot and asks me to select a bunch of images. I use selenium to start up firefox and log onto a website to scrape some data a few times a day. Now, the problem came back and this code is not working anymore to bypass the protection.Preface: my goal is not to solve captcha using automation tools, but to attempt to understand why a browser that is being launched by selenium is being identified as a bot in the first place, and how selenium contributes to this. Options.add_experimental_option('useAutomationExtension', False)ĭriver = webdriver.Chrome(options=options, executable_path=r'PATH_TO\chromedriver.exe')ĭriver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", ) Options.add_experimental_option("excludeSwitches", ) I had this problem before and corrected it by using some options as below (python): from selenium import webdriver I can access the site with my ordinary chrome but when I try with chrome driver, it doesn't work. I'm having some trouble trying to access a web site () with a chrome driver and selenium (I'm quite being "blocked").
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |