I am trying to extract all available elements for the Xpath, and I did try element with 's' and without and cant seem to make it work. being 'Element' its alright but only returns me the first result and with 'Elements' it gives me an error : "AttributeError: 'list' object has no attribute 'find_elements'"
My code :
from selenium.webdriver.common.by import By
from selenium import webdriver
url = 'https://automira.ro/dealeri-autorizati/lista'
PATH = 'C:\\Users\\czoca\\AppData\\Roaming\\Microsoft\\Windows\\Start Menu\\Programs\\Python 3.6\\chromedriver.exe'
driver = webdriver.Chrome(PATH)
driver.get(url)
driver.maximize_window()# For maximizing window
driver.implicitly_wait(100)# gives an implicit wait for 20 seconds
dealers = driver.find_elements(By.XPATH, '/html/body/div[4]/div/div[3]/div/div[1]')
for dealer in dealers:
name = dealer.find_elements(By.XPATH, "/html/body/div[4]/div/div[3]/div/div[1]/div/div/h4/a").text
email = dealer.find_elements(By.XPATH, '/html/body/div[4]/div/div[3]/div/div[2]/div/div/div[3]/a').text
phone = dealer.find_elements(By.XPATH, '/html/body/div[4]/div/div[3]/div/div[2]/div/div/div[2]/a').text
print(name,email,phone)
Any ideias?
Thanks!
find_elements method returns a list object. you can iterate on list object to get all the elements you need.
for n in name:
print(n.text)
In your code, dealers returns a list of WebElements, so you can use find_elements. But in for loop - dealer returns only one WebElement per iteration, so you have to use find_element
dealers = driver.find_elements(By.XPATH, '/html/body/div[4]/div/div[3]/div/div[1]')
for dealer in dealers:
# you should use 'find_element' for name, email and phone
name = dealer.find_element(By.XPATH, "/html/body/div[4]/div/div[3]/div/div[1]/div/div/h4/a").text
email = dealer.find_element(By.XPATH, '/html/body/div[4]/div/div[3]/div/div[2]/div/div/div[3]/a').text
phone = dealer.find_element(By.XPATH, '/html/body/div[4]/div/div[3]/div/div[2]/div/div/div[2]/a').text
print(name,email,phone)
Related
I trying to get the specific element (Minimum Amount) with selenium but its returns empty
options = Options()
options.headless = True
browser = webdriver.Firefox(options=options)
browser.get('https://www.huobi.com/en-us/trade-rules/exchange')
time.sleep(5)
name = browser.find_element(by=By.CSS_SELECTOR, value='.dt-wrap+ .exchange-item span:nth-child(4)')
print(name.text) # Return Empty
how can do it with selenium or beautifulsoap?
Data is also populating from external source via API. So you can easily pull all the required data whatever you need using only requests module.
Example:
import requests
api_url = 'https://www.huobi.com/-/x/pro/v2/beta/common/symbols?r=mhzzzd&x-b3-traceid=6c1acdfbf0a19d63cc05c62de990a55c'
req = requests.get(api_url).json()
for item in req['data']:
print(item['display_name'])
Output:
REN/HUSD
NKN/HT
BRWL/USDT
NSURE/BTC
ITC/USDT
SPA/USDT
CTC/USDT
EVX/BTC
EUL/USDT
USTC/USDT
SUKU/USDT
KAN/BTC
NFT/USDC
LOOKS/USDT
IOI/USDT
DORA/USDT
BAT/USDT
QSP/ETH
WXT/USDT
RING/ETH
NEAR/ETH
SWFTC/BTC
LINK/HUSD
RUFF/USDT
EFI/USDT
DIO/USDT
AVAX/USDC
GSC/ETH
RAD/BTC
INSUR/USDT
NODL/USDT
H2O/USDT
BTC/HUSD
FIRO/ETH
KCASH/BTC
XPNT/USDT
STPT/BTC
XCN/USDT
ETC/BTC
OCN/ETH
BTC/EUR
MAN/BTC
OP/USDC
OXT/BTC
DASH/USDT
KSM/USDT
SD/USDT
YGG/BTC
... so on
I think your CSS_SELECTOR is wrong. Maybe do it with a list and take the element you want?
Something like:
exchange_items: list = driver.find_elements(By.XPATH, '//div[#class="exchange-item"]')
target = exchange_items[3]
print(target.text)
Here we take all items and choose the 4.
In that specific case use .get_attribute('innerText') or .get_attribute('innerHTML') to get your goal with selenium:
browser.find_element(By.CSS_SELECTOR, '.exchange-item span:nth-of-type(4)').get_attribute('innerText')
I want to get all the links in the class tag like the image below.
enter image description here
driver.find_elements_by_xpath('/html/body/div[2]/div[2]/div[2]/div/div[2]/div[2]/div/div[1]/div[1]/div/div'):
url_video = a.get_property('href')
print(url_video)
i get the result is none
I use the 'a' tag to get all the links. I just want to get the links in the specified class. Please help me
This my code:
from selenium import webdriver
import time
browser=webdriver.Chrome()
time.sleep(6)
elements = browser.find_elements_by_xpath('/html/body/div[2]/div[2]/div[2]/div/div[2]/div[2]/div/div[1]/div[1]/div/div')
for element in elements:
videoUrl = element.get_attribute('href')
print(videoUrl)
----> The result is none
.find_elements_by_xpath() return a list, you should use element['href'] on each element of the list.
elements = driver.find_elements_by_xpath('...')
for element in elements:
videoUrl = element['href']
print(videoUrl)
What I mean is that the website I'm using has 2 dropmenus named province with the exact same id, so how do I tell python which dropmenu in particular I wanna select. Of course this is assuming that the issue is that python always picks the first id it sees
from selenium import webdriver
from selenium.webdriver.support.ui import Select
# There are two dropmenu with the same xpath. first time it works fine
# 2nd time it throws an error about element not interactable
Prov = Select(web.find_element_by_xpath('//*[#id="province"]'))
Prov.select_by_index(2)
def Start():
# once opened it will fill in the confirm your age
Day = Select(web.find_element_by_xpath('//*[#id="bday_day"]'))
Day.select_by_index(2)
Month = Select(web.find_element_by_xpath('//*[#id="bday_month"]'))
Month.select_by_index(4)
Month = Select(web.find_element_by_xpath('//*[#id="bday_year"]'))
Month.select_by_index(24)
Prov = Select(web.find_element_by_xpath('//*[#id="province"]'))
Prov.select_by_index(5)
Button = web.find_element_by_xpath('//*[#id="popup-subscribe"]/button')
Button.click()
# have to go through select your birthday
Start()
# 2 seconds is enough for the website to load
time.sleep(2)
# this throws and error.
Prov = Select(web.find_element_by_xpath('//*[#id="province"]'))
Prov.select_by_index(5)
Selenium has functions
find_element_by_... without s in word element to get only first element
find_elements_by_... with s in word elements to get all elements
Selenium doc: 4. Locating Elements¶
So you can get all elements as list (even if there is only one element in HTML)
(if there is no elements then you get empty list)
elements = web.find_elements_by_xpath('//*[#id="province"]')
and later slice it
first = elements[0]
second = elements[1]
last = elements[-1]
list_first_and_second = elements[:1]
EDIT:
You can also try to slice directly in xpath like
(it starts counting at one, not zero)
'//*[#id="province"][2]'
or maybe
'(//*[#id="province"])[2]'
but I never used it to confirm if it will work.
BTW:
All ID should have unique names - you shouldn't duplicate IDs.
If you check documentation 4. Locating Elements¶ then you see that there is find_element_by_id without char s in word element - to get first and the only element with some ID - but there is no find_elements_by_id with char s in word elements - to get more then one element with some ID.
EDIT:
Minimal working code with example HTML in code
from selenium import webdriver
from selenium.webdriver.support.ui import Select
html = '''
<select id="province">
<option value="value_A">A</options>
<option value="value_B">B</options>
</select>
<select id="province">
<option value="value_1">1</options>
<option value="value_2">2</options>
</select>
'''
driver = webdriver.Firefox()
driver.get("data:text/html;charset=utf-8," + html)
all_elements = driver.find_elements_by_xpath('//*[#id="province"]')
first = all_elements[0]
second = all_elements[1]
prov1 = Select(first)
prov2 = Select(second)
print('--- first ---')
for item in prov1.options:
print('option:', item.text, item.get_attribute('value'))
for item in prov1.all_selected_options:
print('selected:', item.text, item.get_attribute('value'))
print('--- second ---')
for item in prov2.options:
print('option:', item.text, item.get_attribute('value'))
for item in prov2.all_selected_options:
print('selected:', item.text, item.get_attribute('value'))
EDIT:
There are two province.
When you use find_element in Start then you get first province in popup - and you can fill it. When you click button then it closes this popup but it doesn't remove first province from HTML - it only hide it.
Later when you use find_element you get again first province in hidden popup - and this time it is not visible and it can't use it - and this gives error. You have to use second province like in this example.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time
def Start():
# once opened it will fill in the confirm your age
Day = Select(web.find_element_by_xpath('//*[#id="bday_day"]'))
Day.select_by_index(2)
Month = Select(web.find_element_by_xpath('//*[#id="bday_month"]'))
Month.select_by_index(4)
Month = Select(web.find_element_by_xpath('//*[#id="bday_year"]'))
Month.select_by_index(24)
# it uses first `province`
Prov = Select(web.find_element_by_xpath('//*[#id="province"]'))
Prov.select_by_index(5)
Button = web.find_element_by_xpath('//*[#id="popup-subscribe"]/button')
Button.click()
web = webdriver.Firefox()
web.get('https://www.tastyrewards.com/en-ca/contest/fritolaycontest/participate')
# have to go through select your birthday
Start()
# 2 seconds is enough for the website to load
time.sleep(2)
# `find_elements` with `s` - to get second `province`
all_province = web.find_elements_by_xpath('//*[#id="province"]')
second_province = all_province[1]
Prov = Select(second_province)
Prov.select_by_index(5)
I use Selenium + Python and I work on a Page that data is filled from JS and is Dynamic every 10 seconds but it's not important because I will run it once a week, I want to wait as long as the td with id='e5' get its value and be filled Or rather until the site is fully loaded, The site address is as follows :
Site Address
But i dont find Suitable Expected Conditions for this job :
driver = webdriver.Firefox()
driver.get('http://www.tsetmc.com/Loader.aspx?ParTree=151311&i=2400322364771558')
WebDriverWait(driver, 10).until(EC.staleness_of((By.ID, 'e5')))
print(driver.find_element_by_id('e5').text)
driver.close()
I speak about this tag if you cant find it :
There is an Expected Condition for that:
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, 'e5')))
You don't need to wait for the element to be stale, you need it to be visible on the DOM.
Try this code to constantly check elements' text value
import time
pause = 1 # set interval between value checks in seconds
field = driver.find_element_by_id('e5')
value = field.text
while True:
if field.text != value:
value = field.text
print(value)
time.sleep(pause)
If you want to use WebdriverWait try
field = driver.find_element_by_id('e5')
value = field.text
while True:
WebDriverWait(driver, float('inf')).until(lambda driver: field.text != value)
value = field.text
print(value)
I'm trying to get a value from a class name but the onlything a could get thill now is a [ ] output.
so, what I'm supposed to do in the following code?
from selenium import webdriver
import time
options = webdriver.ChromeOptions()
options.add_argument('lang=pt-br')
driver = webdriver.Chrome(
executable_path=r'./chromedriver.exe', options=options)
driver.get('https://economia.uol.com.br/cotacoes/cambio/')
time.sleep(5)
dolar = driver.find_elements_by_class_name('currency2')
time.sleep(5)
print(dolar)
You don't need selenium to get that information, try:
import requests
u = "https://api.cotacoes.uol.com/currency/intraday/list/?format=JSON&fields=bidvalue,askvalue,maxbid,minbid,variationbid,variationpercentbid,openbidvalue,date¤cy=1"
j = requests.get(u).json()
dolar = j['docs'][0]['bidvalue']
# 5.5916
Demo
Notes:
If you need other info, like daily variation (variationpercentbid) search on json the object:
Change the currency value at the end of the url for a different currency, for example, currency=5, will give you the EUR values.
You can use get_attribute to retrieve value of web element, Also in your code you there is no element with class name currency2.
Please find below example :
<input class="field normal" name="currency2" value="5,59" data-audience-click="{"reference":"ativar-campo-texto","component":"currency-converter"}" xpath="1">
Code to retrieve value:
driver.get('https://economia.uol.com.br/cotacoes/cambio/')
currency = driver.find_element_by_name('currency2')
print currency.get_attribute("value")
Output::
5,59