I am using the below code to get data from http://www.bddk.org.tr/BultenHaftalik. Two table elements have the same class name. How can I get just one of them?
from selenium import webdriver
import time
driver_path = "C:\\Users\\Bacanli\\Desktop\\chromedriver.exe"
browser = webdriver.Chrome(driver_path)
browser.get("http://www.bddk.org.tr/BultenHaftalik")
time.sleep(3)
Krediler = browser.find_element_by_xpath("//*[#id='tabloListesiItem-253']/span")
Krediler.click()
elements = browser.find_elements_by_css_selector("td.ortala")
for element in elements:
print(element.text)
browser.close()
If you want to select all rows for one column only that match a specific css selection, then you can use :nth-child() selector.
Simply, the code will be like this:
elements = browser.find_elements_by_css_selector("td.ortala:nth-child(2)")
In this way, you will get the "Krediler" column rows only. You can also select the first child if you want to by applying the same idea.
I guess what you want to do is to extract the text and not the numbers, try this:
elements = []
for i in range(1,21):
css_selector = f'#Tablo > tbody:nth-child(2) > tr:nth-child({i}) > td:nth-child(2)'
element=browser.find_element_by_css_selector(css_selector)
elements.append(element)
for element in elements:
print(element.text)
browser.close()
Related
im am trying to get the values i added to a list with selenium and print them out. But i am only getting this: <generator object at 0x000001B924EC7990>. How can i print the values in the list.
I also tried to shorten the xpath with "//tr[#class= 'text3'][11]/td" but it didnt work.
Like you can see i tried to loop through the list and convert it in text, but it also didnt work.
Would this work range(driver.find_elements(By.XPATH,"//table[2]/tbody/tr/td[2]/table[1]/tbody/tr/td[3]/table/tbody/tr[2]/td[position() >= last()]"))?
Can you guys help me out?
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.implicitly_wait(10)
website = "https://langerball.de/"
driver.get(website)
for i in range(7):
xpath_test = "//table[2]/tbody/tr/td[2]/table[1]/tbody/tr/td[3]/table/tbody/tr[2]/td[position() >= last()]"
a = driver.find_elements(By.XPATH, xpath_test)
test_li = []
test_li.append(a)
print(b.text for b in test_li)
driver.find_elements method returns a list of web elements while you are looking for their text values. Web element text value can be received by applying the .text method on a web element.
So, you should iterate over the received list of web elements and extract text from each web element in the list.
Also test_li = [] should be defined out of the loop.
So your code could be something like this:
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.implicitly_wait(10)
website = "https://langerball.de/"
driver.get(website)
test_li = []
for i in range(7):
xpath_test = "//table[2]/tbody/tr/td[2]/table[1]/tbody/tr/td[3]/table/tbody/tr[2]/td[position() >= last()]"
a_list = driver.find_elements(By.XPATH, xpath_test)
for a in a_list:
test_li.append(a.text)
print(b.text for b in test_li)
P.S.
I'm not sure about the rest of your code: the for i in range(7) loop and the xpath_test XPath expression
This worked
test_li = []
xpath_test = "//table[2]/tbody/tr/td[2]/table[1]/tbody/tr/td[3]/table/tbody/tr[2]/td[position() <= last()]"
a_list = driver.find_elements(By.XPATH, xpath_test)
for a in a_list:
test_li.append(a.text)
print(test_li)
how would I get the text "Premier League (ENG 1)" extracted from this HTML tree? (marked part)
I treid ti get the text with xpath, css selector, class... but I seem to cant get this text extracted.
Basically I want to create a list and go over all "class=with icon" elements that include a text (League) and append the text to that list.
This was my last attempt:
def scrape_test():
alleligen = []
#click the dropdown menue to open the folder with all the leagues
league_dropdown_menue = driver.find_element_by_xpath('/html/body/main/section/section/div[2]/div/div[2]/div/div[1]/div[1]/div[7]/div')
liga_dropdown_menue.click()
time.sleep(1)
#get text form all elements that conain a league as text
leagues = driver.find_elements_by_css_selector('body > main > section > section > div.ut-navigation-container-view--content > div > div.ut-pinned-list-container.ut-content-container > div > div.ut-pinned-list > div.ut-item-search-view > div.inline-list-select.ut-search-filter-control.has-default.has-image.is-open.active > div > ul > li:nth-child(3)')
#append to list
alleligen.append(leagues)
print(alleligen)
But I dont get any output.
What am I missing here?
(I am new to coding)
try this
path = "//ul[#class='inline-list']//li[first()+1]"
element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, path))).text
print(element)
the path specifies the element you want to target. the first // in the path means that the element you want to find is not the first element in the page and exists somewhere in the page. li[first()+1] states that you are interested in the li tag after the first li.
The WebDriverWait waits for the webpage to load completely for a specified number of seconds (in this case, 5). You might want to put the WebdriverWait inside a try block.
The .text in the end parses the text from the tag. In this case it is the text you want Premier League (ENG 1)
Can you try :
leagues = driver.find_elements_by_xpath(“//li[#class=‘with-icon’ and contains(text(), ‘League’)]”)
For league in leagues:
alleligen.append(league.text)
print(alleligen)
If you know that your locator will remain on the same position in that list tree, you can use the following where the li element is taken based on its index:
locator= "//ul[#class='inline-list']//li[2]"
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, locator))).text
I want to extract all h2 elements of the div element. The code that I've used is this:
browser = webdriver.Chrome()
browser.get("https://www.mmorpg.com/play-now")
time.sleep(2)
item_list_new=[]
link = browser.find_element_by_xpath("//div[#class='freegamelist']")
names = link.find_element_by_tag_name('h2')
x = names.text
item_list_new.append(x)
print(item_list_new)
But when I run this, I only get the first 'h2' element of the div element.
Can somebody tell me what am I doing wrong and also please guide me with the correct way of doing it?
Thanks in advance.
you need to write names = link.find_elements_by_tag_name('h2')
Your code should be
browser = webdriver.Chrome()
browser.get("https://www.mmorpg.com/play-now")
time.sleep(2)
item_list_new=[]
link = browser.find_element_by_xpath("//div[#class='freegamelist']")
names = link.find_elements_by_tag_name('h2')
x = names.text
item_list_new.append(x)
print(item_list_new)
find_element_by_tag_name gives the first element and find_elements_by_tag_name gives all the matching elements
Try to get all header values as below:
link = browser.find_element_by_xpath("//div[#class='freegamelist']")
names = link.find_elements_by_tag_name('h2')
item_list_new = [x.text for x in names]
print(item_list_new)
or you can simplify
names = browser.find_elements_by_xpath("//div[#class='freegamelist']//h2")
item_list_new = [x.text for x in names]
print(item_list_new)
You actually want to use the function find_elements_by_tag_name that sounds almost similar, as pointed out here.
I want to select first 100 elements in a list so I user action chain method.
Please suggest any other method, if any. With the code I used below, I can select elements in the list but I am not able to click any element:
for r in range(1, 100):
r = str(r)
print r
row = GlobalVar.Driver.find_element_by_xpath("/html/body/table/tbody/tr[2]/td/table/tbody/tr[2]/td/div/div[3]/table/tbody/tr[2]/td/table/tbody/tr/td[1]/table/tbody/tr[2]/td/select/option["+r+"]")
action_chains = ActionChains(GlobalVar.Driver)
action_chains.key_down(Keys.CONTROL).key_down(Keys.SHIFT).click(row).key_up(Keys.SHIFT).key_up(Keys.CONTROL).perform()
It is a select tag, you can go with select class in python.
ele = GlobalVar.Driver.find_element_by_xpath("/html/body/table/tbody/tr[2]/td/table/tbody/tr[2]/td/div/div[3]/table/tbody/tr[2]/td/table/tbody/tr/td[1]/table/tbody/tr[2]/td/select")
select = Select(ele)
for index in range(1, 100):
select.select_by_index(index)
It is not advisable to use absolute xpath. please try to use relative path if possible or other locator types.
On a typical eBay search query where more than 50 listings are returned, such as this, eBay displays in the a grid format (whether you have it set up to display as grid or a list).
I'm using class name to pull out the prices using WebDriver:
prices = webdriver.find_all_elements_by_class_name("bidsold")
The challenge: although all prices on the page look identical in structure, the ones that are crossed out (where Buy It Now is not available and it's Best offer accepted) are actually contained within a child span of the above span:
I could pull these out separately by repeating the find_all_elements_by_class_name method with class sboffer, but (i) I will lose track of the order, and more importantly (ii) it will roughly double the time it takes to extract the prices.
The CSS selector for both types of prices also differ, as do the XPaths.
How do we catch all prices in one go?
Try this:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.ebay.com/sch/i.html?rt=nc&LH_Complete=1&_nkw=Columbia+Hiking+Pants&LH_Sold=1&_sacat=0&LH_BIN=1&_from=R40&_sop=3&LH_ItemCondition=1000&_pgn=2')
prices_list = driver.find_elements_by_css_selector('span.amt')
prices_on_page = []
for span in prices_list:
unsold_item = span.find_elements_by_css_selector('span.bidsold.bold')
sold_item = span.find_elements_by_css_selector('span.sboffer')
if len(sold_item):
prices_on_page.append(sold_item[0].text)
elif len(unsold_item):
prices_on_page.append(unsold_item[0].text)
elif span.text:
prices_on_page.append(span.text)
print prices_on_page
driver.quit()
In this case, you will have track of the order and you will only query the specific span element instead of the entire page. This should improve performance.
I would go for xpath- below code worked for me. It grabbed 50 prices!
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.ebay.com/sch/i.html?rt=nc&LH_Complete=1&_nkw=Columbia+Hiking+Pants&LH_Sold=1&_sacat=0&LH_BIN=1&_from=R40&_sop=3&LH_ItemCondition=1000&_pgn=2')
my_prices = []
itms = driver.find_elements_by_xpath("//div[#class='bin']")
for i in itms:
prices = i.find_elements_by_xpath(".//span[contains(text(),'$')]")
val = ','.join(i.text for i in prices)
my_prices.append([val])
print my_prices
driver.quit()
Result is
[[u'$64.95'], [u'$59.99'], [u'$49.95'], [u'$46.89,$69.99'], [u'$44.98'], [u'$42.95'], [u'$39.99'], [u'$39.99'], [u'$37.95'], [u'$36.68'], [u'$35.96,$44.95'], [u'$34.99'], [u'$34.99'], [u'$34.95'], [u'$30.98'], [u'$29.99'], [u'$29.99'], [u'$29.65,$32.95'], [u'$29.00'], [u'$27.96,$34.95'], [u'$27.50'], [u'$27.50'], [u'$26.99,$29.99'], [u'$26.95'], [u'$26.55,$29.50'], [u'$24.99'], [u'$24.99'], [u'$24.99'], [u'$24.99'], [u'$24.98'], [u'$24.98'], [u'$24.98'], [u'$24.98'], [u'$24.98'], [u'$22.00'], [u'$22.00'], [u'$22.00'], [u'$22.00'], [u'$18.00'], [u'$18.00'], [u'$17.95'], [u'$11.99'], [u'$9.99'], [u'$6.00']]