New to Selenium. I'm trying to get it so that I can enter a name into a search box and then click on the correct name.
I have managed to write some code to go to a website and then enter what I want and press the search button. The problem is that it then shows a list of items so I'd have to click again.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from webdriver_manager.chrome import ChromeDriverManager
from time import sleep
index = 'AMZN'
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://www.marketscreener.com')
sleep(6)
text_box = driver.find_element_by_css_selector('#autocomplete')
text_box.send_keys(index)
#select.select_by_index(1)
driver.find_element_by_xpath("//*[#id='recherche_menu']/table/tbody/tr[1]/td/button/img").click()
When I enter the name and don't click It presents a dropdown list so I am trying to select the first item from the dropdown as that is usually what I will want.
I attempted this by using the commented out select.select_by_index code line. But it doesn't quite work.
I just tried also using text_box.send_keys(Keys.DOWN, Keys.RETURN) to move down into the dropdown field, but this doesn't work and just returns the same as what I currently get from clicking.
To be clear what I mean is that currently the code will return this:
But I want it to go straight to the Amazon page so it will return this:
Any help appreciated.
Thanks
This page use many JavaScript to listen the action of the mouse and the key.If you didn't focus on the searchbox,the dropdown will be disappeared.(and you couldn't find it in the source code).
Try to use ActionChains,this works fine on my PC:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.action_chains import ActionChains
from time import sleep
index = 'AMZN'
driver = webdriver.Chrome()
driver.get('https://www.marketscreener.com')
action = ActionChains(driver)
sleep(4)
text_box = driver.find_element_by_css_selector('#autocomplete')
action.move_to_element(text_box)
action.click(text_box)
action.send_keys(index)
action.perform()
sleep(1)
driver.find_element_by_xpath('//*[#id="AC_tRes"]/li[1]').click()
If the result couldn't show you,maybe you need to increase the time of sleep().
Related
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import Select
from selenium.webdriver.chrome import options
import unittest
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
#link to website
website = 'https://www.ncei.noaa.gov/access/monitoring/climate-at-a-glance'
path = ('../chromedriver') #Folder location where is the chromedriver
driver = webdriver.Chrome(path)
driver.get(website)
driver.maximize_window()
#Selection of the sections where the information I am looking for is located
state_click = driver.find_element(by=By.XPATH, value='//*[#id="show-statewide"]/a').click()
time_series_click = driver.find_element(by=By.XPATH, value='.//*[#id="time-series"]/div[3]/button').click()
#selection of the years (for all files the same range of 1950 - 2021)
star_year_dropdown =Select(driver.find_element(by=By.ID, value='begyear'))
star_year_dropdown.select_by_visible_text('1950')
end_year_dropdown = Select(driver.find_element(by=By.ID, value='endyear'))
end_year_dropdown.select_by_visible_text('2021')
#selection of the parameter to download: Average temperature
parameter_dropdown = Select(driver.find_element(by=By.ID, value='parameter'))
parameter_dropdown.select_by_visible_text('Average Temperature')
#Creating a loop to loop through all the states and all the months:
#state selection
select_state = driver.find_element(by=By.XPATH, value='.//*[#id="state"]')
opcion_state = select_state.find_elements(by=By.TAG_NAME, value='option')
#month selection
select_month = driver.find_element(by=By.XPATH, value = '//*[#id="month"]')
opcion_month = select_month.find_elements(by = By.TAG_NAME, value='option')
for option in opcion_month:
option.click()
for option in opcion_state:
option.click()
time.sleep(3)
plot = driver.find_element(by=By.XPATH, value='.//input[#id="submit"]').click()
dowload = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="csv-download"]'))).click()
time.sleep(3)
The code works fine, but in the plot and download functions created (at the end of the whole) when trying to click it gives an error and cannot be solved. I think it is because the web page at the time of executing that command does not display the button in the screen to plot the graph and download the csv. I have tried modifying the waiting time, but not work. Let's see if someone can help me.
Thanks in advance!!
So the problem that you are having with downloading the CSV file is not that the command is wrong (it does work), however, the issue is that the download CSV button is not visible on the page, which prevents you from clicking it.
A way to get around having to visibly see the element on the page and still click it you can do the following:
driver.execute_script("arguments[0].click();", driver.find_element(By.XPATH, '//*[#id="csv-download"]'))
This would be the preferred method, otherwise you would have to scroll to where the button is visible on the page and then click the button. The page has a scrolling effect if trying to click on a button which you can do the following (but the previous method is preferred as it is cleaner and does not take any additional time to do):
from selenium.common.exceptions import ElementClickInterceptedException
download_btn = driver.find_element(By.XPATH, '//*[#id="csv-download"]')
try:
download_btn.click() # the first time you try to click it will throw an error, while navigating the button into the page view
except ElementClickInterceptedException: # catch the exception and proceed as if clicking again
download_btn.click()
Other comments on your code
you have option.click() twice in your code, and it is unclear if you want to click the month or state option - there may be a confusion about which options are clicked. I would suggest renaming your iterator variables appropriately so that you know which buttons are being clicked
One of my dad's friends wants me to make a bot that automates buying gun primer because they are always sold out. I have a script with some ammunition as a test (because its actually in stock), but as I try and figure out how to click the dropdown, the terminal always spits out this error
selenium.common.exceptions.InvalidArgumentException: Message: invalid type: null, expected a string at line 1 column 12
For reference, I am using Python 3.9 with Selenium 3.141.0 and this is my code:
#imports funcs from selenium
from selenium import webdriver
#chooses your browser
driver = webdriver.Firefox()
#gets your store
url = driver.get("https://www.midwayusa.com/product/2090655809")
#opens the website
driver.get(url)
#makes variables for html elements
button = driver.find_element_by_xpath('//*[#id="productSelectorContainer"]/div[1]/button')
#le button click has arrived
button.click()
I think I see the issue. You have these two lines
#gets your store
url = driver.get("https://www.midwayusa.com/product/2090655809")
#opens the website
driver.get(url)
driver.get() doesn't return anything so null gets stored into url. Then on the next line you are navigating to null. These lines are actually redundant and looks like you misunderstood how to use them. You only need the first line, with some corrections.
I removed the redundant line, corrected the other, and made some adjustments to some of your comments to make them more accurate.
#imports funcs from selenium
from selenium import webdriver
#launches the browser
driver = webdriver.Firefox()
#navigates to your store website
driver.get("https://www.midwayusa.com/product/2090655809")
#stores the button in a variable
button = driver.find_element_by_xpath('//*[#id="productSelectorContainer"]/div[1]/button')
#le button click has arrived
button.click()
Faults:
url = driver.get("https://www.midwayusa.com/product/2090655809")
is not a valid just use driver.get('url')
You need to click the consent otherwise it overlap all the clicks otherwise.
Here's an example of clicking the 50 quantity of the item at hand and then proceeding.
wait = WebDriverWait(driver, 5)
driver.get("https://www.midwayusa.com/product/2090655809")
driver.maximize_window()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#cookie-consent-btn"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#productSelectorContainer > div.product-filter.original-selector > button"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#Quantity > ol > li:nth-child(1) > button"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"button.big-primary-button.add-to-cart-button"))).click()
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Outputs
I Got this code:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
myurl = "https://foobar.pl"
driver = webdriver.Chrome()
driver.get(myurl)
select = Select(driver.find_element_by_xpath('/html/body/div/select'))
select.select_by_visible_text('foobar')
time.sleep(5)
after selecting "foobar" an input field appears.
but after I try:
driver.find_element_by_xpath('/html/body/div/div[2]/input').click()
I get
ElementNotVisibleException: element not visible
How can I update driver, so it would see input, without refreshing page (I would loose my selection)?
Can you please try the code snippet below. You might be needing to wait between clicking and getting the field that you would like to retrieve.
select = Select(driver.find_element_by_xpath('/html/body/div/select'))
select.select_by_visible_text('foobar')
time.sleep(5)
driver.find_element_by_xpath('/html/body/div/div[2]/input').click()
You first need to click to the field, then wait, then click again.
This commonly occurs when JavaScript modifies the page after you interact with it. The solution is to use WebDriverWait:
from selenium.webdriver.support import ExpectedConditions as EC
wait = WebDriverWait(driver, 5)
input = wait.until(EC.element_to_be_clickable(By.xpath('/html/body/div/div[2]/input'))
input.click()
I am accessing this page and I have to pick one of Combo value that shows existing resume. I am getting error:
selenium.common.exceptions.ElementNotVisibleException: Message:
Element is not currently visible and so may not be interacted with
Code I am trying is:
from selenium import webdriver
from selenium.webdriver.support.select import Select
from time import sleep
from bs4 import BeautifulSoup
driver = webdriver.Firefox()
driver.get('https://www.glassdoor.com/job-listing/developer-one-north-interactive-JV_IC1128808_KO0,9_KE10,31.htm?jl=1584069572')
select_box = driver.find_element_by_id('ExistingResume')
select_box.click();
select_box = Select(select_box)
sleep(5)
select_box.select_by_value(RESUME_TEXT_VALUE)
Updated The required element is right there
Update #2: Checked that element is not visible in static html. Guess via JS being loaded.
Update #3: OK I made following changes which prints tag name that is select:
select_box = driver.find_element_by_id('ExistingResume')
print(select_box.tag_name)
Now mission is to select the value from that combo
You can use Keys.ARROW_DOWN to get the option and Keys.RETURN to select. See below:
>>> from selenium.webdriver.common.keys import Keys
>>> driver.find_element_by_id("ExistingResumeSelectBoxIt").click()
>>> d = driver.find_element_by_id("ExistingResumeSelectBoxIt")
>>> d.send_keys(Keys.ARROW_DOWN)
>>> d.send_keys(Keys.RETURN)
>>> driver.find_element_by_id("ExistingResumeSelectBoxIt").text
u'mesut gunes resume eng.pdf'
You must be logged in and have a resume too.
The error message is clean enough. The problems is that there is no element with id "ExistingResume" on the website.
If you open the console in your browser and type $("#ExistingResume"); jQuery cannot find that element on the website neither.
Make sure you didn't make a typo in the id or you have all the necessary permissions to access that element. (Maybe it's visible only for logged in users.)
First of all, you have to be logged in and have at least one existing uploaded resume.
You cannot directly control the select element with the resumes - it is invisible. Control the wrapping combo element that is visible.
Complete working code (including logging in):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get('https://www.glassdoor.com/job-listing/developer-one-north-interactive-JV_IC1128808_KO0,9_KE10,31.htm?jl=1584069572')
# log in
driver.find_element_by_css_selector("div.actions span.signin").click()
driver.find_element_by_css_selector("form.signInForm input.signin-email").send_keys("login")
driver.find_element_by_css_selector("form.signInForm input.signin-password").send_keys("password")
driver.find_element_by_css_selector("form.signInForm button.loginDlgSignInBtn").click()
# wait for the combo to appear
resume_name = "myResume.pdf"
# open up combo
combo = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "ExistingResumeSelectBoxIt")))
combo.click()
# select resume
resume_item = combo.find_element_by_xpath("//li[#data-val = '%s']" % resume_name)
resume_item.click()
I'm new to selenium and trying to automate the download of some government data. When using the code below. I manage to navigate to the right page and enter the right parmeter in the form, but then can't find a way to click the 'submit' button. I've tried find_element_by_partial_link_text("Subm").click() and I've tried find_element_by_class_name on a number of class names. Nothing works. Any ideas?
import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
main_url="http://data.stats.gov.cn/english/easyquery.htm?cn=E0101"
driver = webdriver.Firefox()
driver.get(main_url)
time.sleep(8)
driver.find_element_by_partial_link_text("Industry").click()
time.sleep(8)
driver.find_element_by_partial_link_text("Main Economic Indicat").click()
time.sleep(8)
driver.find_element_by_id("mySelect_sj").click()
time.sleep(3)
driver.find_element_by_class_name("dtText").send_keys("last72")
time.sleep(4)
try:
driver.find_element_by_class_name("dtFoot").click()
except:
driver.find_element_by_class_name("dtFoot").submit()
Solved my own problem, the key was using
driver.find_element_by_class_name(`dtTextBtn`)
instead of
driver.find_element_by_class_name(`dtTextBtn f10`)
The latter was what I saw in the source code, but the f10 blocked selenium.