I am writing a python script which includes opening up an URL and do some activity on it. I am facing an issue when i execute below code then Firefox browser starts but it is not able to browse URL. What could be wrong here..?
I also tried to add proxy exception but that isn't solve issue.
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('WEBSITE_URL')
So, Pls suggest what is wrong here.
Related
I am using selenium for a crawling project, but I struggle with a specific webpage (both chrome and firefox).
I found 2 workarounds that work to an extend but I want to know why this issue happens and how to avoid it.
1) Opening chrome manually and then opening selenium with my user profile.
If i manually start chrome and then run:
from selenium import webdriver
options.add_argument(r"user-data-dir=C:\Users\User\AppData\Local\Google\Chrome\User Data")
driver = webdriver.Chrome(options=options)
the page loads as intended
2) Passing a variable in the request
by appending /?anything to the url the page loads as intended in selenium
For some reason the webpage has a function in the header despite not loading... I suspect this could be a clue but I do not know enough to determine the cause.
I'm learning how selenium crawls data, but I find that when a website opens through selenium, it's different from what I used to get when I used other normal browsers. Even I add headers. And I'm very confused.
I really want to upload two contrast pictures, but I can't upload them in stackoverflow at present. I even tried to open the chrome driver and enter the web address manually, but the result is still different.
I use Python 3.6, selenium and chrome 75.0.3770.80
from selenium import webdriver
driver = webdriver.Chrome() #创建driver实例
url = 'https://www.free-ss.ooo'
driver.get(url)
At present, I can't post pictures on stack overflow, but I just want to figure out how I can use selenium to get normal web pages.
Aha,I found the problem, really because the target site detected selenium, the solution is to add options
Chrome_options. add_experiment_option ('excludeSwitches', ['enable-automation'])
Faced same issue and was able to resolve it by removing or fixing appropriate user-agent argument and it worked fine in both headless and non-headless mode.
The resolution was inspired by PDHide post
I am trying to get the actual user agent that I am using in Selenium, at the moment with the chromedriver.
I found a Java version of this problem:
How to get userAgent information in Selenium Web driver.
Does someone know how to do that in Python?
The same manner as inside your link:
user_agent = driver.execute_script("return navigator.userAgent;")
PS: Using execute_script method you can run JS inside your driver.
Hope it helps you!
from selenium import webdriver
import time
import datetime
site = 'URL'
browser = webdriver.Ie()
browser.implicitly_wait(1)
browser.get(site)
#time.sleep(1)
browser.get_screenshot_as_file('screen_shot_test'
+str(datetime.datetime.now())+'.png')
browser.close()
I am trying to save a screenshot of my webpage but when I try to screenshot it I get the error saying "Unable to get Browser" even though the webpage I am trying to reach is open and ready to be screenshotted. I am new to selenium and very confused, all help is appreciated, Thanks.
I ran your code on my machine using the Chrome driver(don't have IE) and it works perfectly. Although this isn't a solution to your code(was going to just comment but don't have enough reps) but try changing the web browser. It may be a problem with the current version of IE on your machine(try updating?)
I keep getting this error:
https://sites.google.com/a/chromium.org/chromedriver/help/chromedriver-crashes
I get it when running the command:
python Web.py
However when I go into the file and run the lines 1 by 1, I don't get the error. However I always get the error when the Web.py file has finished. When I run the lines 1 by 1, it's very basic things but i feel like I"m not ending my script correctly.
import selenium
from selenium.webdriver.common.keys import Keys
import time
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('espn.com')
I want the window with espn.com to stay on the screen, not quit when the script has finished.
I'm running this on Python. I can share my setup, maybe that's something I did incorreclty but any help would be appeciated.
You're passing an invalid url.
You need to pass the url like this:
driver.get("http://www.espn.com")
This might work in your browser, but it won't with selenium. Type in "espn.com" in your browser and then copy / paste the url and you'll see that it's actually the above url.
You should also specify the "chromedriver.exe" path.
You are getting this error because you had not installed the chrome driver for selenium on your Machine. Selenium by default provides the driver for Firefox so when you use the webdriver for Firefox, it won't rise any error. To resolve this issue with Chrome you can download the Chrome webdriver from here.
and you can specify the driver as
from selenium import webdriver
d = webdriver.Chrome(executable_path='<your Chrome driver path>')
Adding to what #Pythonista said , it's better if you keep the URL as a raw string than a normal string
driver.get(r'http://www.espn.com')
so that it won't take the slash as an escape sequence in few cases.
Hope it helps.
Try to update chrome and get updated/latest chrome driver, recently chrome made several updates in its driver you can download the last one from the link below:
https://chromedriver.storage.googleapis.com/2.27/chromedriver_win32.zip