I have been trying to send a URL link to a text area. I am using python3 and Chrome Version 109.0.5414.119. It worked fine on my local OSX machine but when I tried to automate it on a Linux Machine. It started behaving weirdly. So I have this feeling that It can be a Chrome Driver-related issue.
When I send a URL like this:
l.send_keys("https://google.com")
But becomes:
/google.comhttps:
Then I tried to debug this behavior by sending the following:
>>> l.send_keys("/") # /|
>>> l.send_keys("/") # |/
So the position of the cursor is going ahead of the line for the second /. I was not expecting this. I wonder if you can shed light on how to solve this?
Possibly you are trying to invoke send_keys() too early within the <input> field even before the element have rendered properly.
Solution
Ideally to send a character sequence to the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "element_css_locator"))).send_keys("https://google.com")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
Using selenium and python, I'm opening a document by making click on a link which opens in the same tab. When I make a screenshot without using '--headless' it takes the document screenshot, but when I activate '--headless' the screenshot is from the previous page (the one where I make click to the document to be opened)
I have tried different browsers and ways (opening in new tab and switching but not working when using '--headless') but not working...
Any idea why '--headless' is behaving like this?
While taking a screenshot ideally you need to induce WebDriverWait for the visibility_of_element_located() of some static and visible element e.g. some <h1> / <h2> element on the desired page and then take the screenshot as follows:
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "css_visible_element")))
driver.save_screenshot("Federico Albrieu.png")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I solved the problem.
I was using the action
ActionChains(driver).move_by_offset('xcoordinate', 'ycoordinate).click().perform()
to open the desired document. Apparently, while using headless, selenium can't make click in the document coordinate (?) but it can when not using headless.
I tried instead making click with element_to_be_clickable((By.XPATH, 'XPATH'))).click() and it works when using headless.
Thanks!
I'm having a really hard time locating elements on this website. My ultimate aim here is scrape the score card data for each country. The way I've envisioned doing that is to have selenium click list view (as opposed to the default country view) and loop going in and out of all the country profiles, scraping the data for each country along the way. But methods to locate elements on the page that worked for me in the past, have been fruitless with this site.
Here's some sample code outlining my issue.
from selenium import webdriver
driver = webdriver.Chrome(executable_path= "C:/work/chromedriver.exe")
driver.get('https://www.weforum.org/reports/global-gender-gap-report-2021/in-full/economy-profiles#economy-profiles')
# click the `list` view option
driver.find_element_by_xpath('//*[#id="root"]/div/div[1]/div[2]/div[2]/svg[2]')
As you can see, I've only gotten as far as step 1 of my plan. I've tried what other questions have suggested as far as adding waits, but to no avail. I see the site fully loaded in my DOM, but no xpaths are working for any element on there I can find. I apologize if this question is posted too often, but do know that any and all help is immensely appreciated. Thank you!
The element is inside an iframe you need to switch it to access the element.
Use WebDriverWait() wait for frame_to_be_available_and_switch_to_it()
driver.get("https://www.weforum.org/reports/global-gender-gap-report-2021/in-full/economy-profiles#economy-profiles")
wait=WebDriverWait(driver,10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "iFrameResizer0")))
wait.until(EC.element_to_be_clickable((By.XPATH,"//*[name()='svg' and #class='sc-gxMtzJ bRxjeC']"))).click()
You need following imports.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
Another update:
driver.get("https://www.weforum.org/reports/global-gender-gap-report-2021/in-full/economy-profiles#economy-profiles")
wait=WebDriverWait(driver,10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "iFrameResizer0")))
driver.find_element_by_xpath("//*[name()='svg' and #class='sc-gxMtzJ bRxjeC']").click()
You incorrectly click List view. Your locator has to be stable. I checked it did not work.
So, in order to click the icon, use WebDriverWait:
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
wait = WebDriverWait(driver, timeout=30)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".sc-gzOgki.ftxBlu>.background")))
list_view = driver.find_element_by_css_selector(".sc-gzOgki.ftxBlu>.background")
list_view.click()
Next, to get the unique row locator, use the following css selector:
.sc-jbKcbu.kynSUT
It will give use the list of all countries.
I'm newish to StackOverflow, and programming in general.
I am automating my company's billing system (which only works on IE) using Selenium, and I need to press a Filter button in order to display on the page what our database says. I have tried many variations of xpath in order to find the element, but nothing seems to have worked. Here is the HTML line for the Filter button:
<input style="background:green;color:white;" type=button class=bttn onclick="form1.formfilter.value='1';form1.submit();" value="FILTER">
Is there a way to find this element by value="FILTER"?
I have read the docs and looked for this answer, but I haven't found it, or I'm just not searching the right keywords.
To click on FILTER button using the attribute value="FILTER" you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.bttn[value='FILTER']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='bttn' and #value='FILTER']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
First of all, I'm completely new in web scraping, html and selenium, so my question might not seem meaningful to you.
What I'm trying to do: automated click on an image on a webpage. Manual click on this image allows to display new information on a part of the webpage.
How I tried to do it: I found something specific to this image, for example:
<img src="images/btn_qmj.gif" border="0">
So I just entered in python:
xpath = '//img[#src="images/btn_qmj.gif"]'
driver.find_elements_by_xpath(xpath)
Problem: this returns me an empty list. I partially understood why, but I don't have any solution.
image of the html code inspector
I included here an image of the html tree I obtain on the web inspector. As one can see, the tree to reach my line of interest gives "/html/frameset/frame1/#document/html/body/center/table/tbody/tr[2]/td/a/img".
The issue is that I cannot access -by using an xpath- anything that comes after the #document. If I try to copy the xpath of my line of interest, it tracks it back only up to the 2nd "/html". It is like the page is subdivided, compartmentalized, with inner parts I cannot access. I suppose there is a way to access the #document content, maybe it refers to another webpage, but I'm stuck at that point.
I would be very grateful if anyone could help me on this.
Have you tried switching to the frame first?
driver.switch_to.frame('gauche')
You need to switch control to frame first to handle image on it. Please see below code for your reference
driver.switch_to_frame(frame)
WebDriverWait(driver, 20).until(
EC.visibility_of_element_located((By.XPATH, "'//img[#src="images/btn_qmj.gif"]'"))).click()
driver.switch_to_default_content()
Note :: You need to add below imports
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
The element is present inside an iframe.In order to access the element you need to switch it first.
Induce WebDriverWait() and frame_to_be_available_and_switch_to_it()
Induce WebDriverWait() and visibility_of_element_located()
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"gauche")))
WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//img[#src='images/btn_qmj.gif']"))).click()
You need to import following libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
I'm trying to code nicely against a web site with AJAX like functionality, and using pysaunter (http://element34.ca/products/saunter/pysaunter).
When I use the available synchronization method wait_for_available, perhaps improperly, my code does more or less what I want, but the Selenium server node throws asserts like following while the class is not yet present:
org.openqa.selenium.remote.ErrorHandler$UnknownServerException: Unable to locate element: {"method":"css selector","selector":".ng-scope.ready.idle"}
I'd like to use WebDriverWait, I think like this:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(self.driver, 30).until(EC.presence_of_element_located((By.CLASS_NAME,'idle')))
But when I try that, I still get the above exception from a Firefox remote webdriver, and the following from a chrome remote webdriver:
13:09:22.525 WARN - Exception: no such element
(Session info: chrome=29.0.1547.76)
(Driver info: chromedriver=2.0,platform=Mac OS X 10.8.5 x86_64) (WARNING: The server did not provide any stacktrace information)
Is it possible to avoid exceptions from Selenium Server when looking for an element that will likely not be present right away, when running remote webdriver using Python?
Can anyone point me to an example of the proper way to use WebDriverWait from pysaunter? I'm starting from here:
http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp
and
(see also Element 34 blog posting "WebDriverWait and Python" from July 9th, 2012)
TIA
If you look into the WebDriverWait code you will see that you can give the constructor a list of exceptions to ignore. One such list is pre-defined, 'IGNORED_EXCEPTIONS', that is set to [NoSuchElementException]. So you can just add 'ignored_exceptions=IGNORED_EXCEPTIONS' to the WebDriverWait constructor, i.e.:
WebDriverWait(self.driver, 30, ignored_exceptions=IGNORED_EXCEPTIONS).until(...)
Then those exceptions will be ignored and it will continue to try until it succeeds or times out.