Until now I used a for cycle to get all the elements on a page in a certain path with this script:
for username in range(range_for_like):
link_username_like = "//article/div[2]/div[2]/ul/div/li[" + str(num) + "]/div/div[1]/div/div[1]/a[contains(#class, 'FPmhX notranslate zsYNt ')]"
user = browser.find_element_by_xpath(link_username_like).get_attribute("title")
num += 1
sleep(0.3)
But sometimes my cpu will exceed 100%, which is not ideal.
My solution was to find all the elements in one line using find_elements_by_xpath but in doing so, I can't figure out how to get all the "title" attributes.
I know that the path changes for every title, //article/div[2]/div[2]/ul/div/li[" + str(num) + "]/div/div[1]/div/div[1]/a that's why I kept increasing the num variable, but how can I use this tecnique without a cycle for?
What's the most efficient way in term of performance to get all the attributes? I don't mind if it does take also 2 minutes or more
Here how you can get all the people that like your photo by xpath:
//div[text()='Likes']/..//a[#title]
Code below get first 12 liker:
likes = browser.find_elements_by_xpath("//div[text()='Likes']/..//a[#title]")
for like in likes:
user = like.get_attribute("title")
To get all likes you have to scroll, for that you can get total likes you have and then scroll until all likes will be loaded. To get total likes you can use //a[contains(.,'likes')]/span xpath and convert it to integer.
To scroll use javascript .scrollIntoView() to last like, final code would look like:
totalLikes = int(browser.find_element_by_xpath("//a[contains(.,'likes')]/span").text)
browser.find_element_by_xpath("//a[contains(.,'likes')]/span").click()
while true:
likes=browser.find_elements_by_xpath("//div[text()='Likes']/..//a[#title]")
likesLen = len(likes)
if (likesLen == totalLikes - 1)
break
browser.execute_script("arguments[0].scrollIntoView()", likes.get(likesLen-1))
for like in likes:
user = like.get_attribute("title")
How it works:
With //div[text()='Likes'] I found unique div with window contains likes. Then to get all likes that is li I go to parent div with /.. selector and then get all a with title attribute. Because all likes not loading immediately you have to scroll down. For that I get total likes amount before click to likes. Than I scroll to last like (a[#title]) to force instagram to load some data until total likes I got not equals to list of likes. When scroll completes I just iterate throw all likes in list I got inside while loop and get titles.
Related
I created a selenium script to check the number of cart is shown zero (0)> However, it's returned empty although this field is zero.
Scripts:
shopping_cart_qty = self.driver.find_element_by_xpath("//span[contains(#class,'topnav-cart-qty')]").text
self.assertEqual('0', shopping_cart_qty, "The shopping car is not empty")
Return:
The shopping car is not empty
!= 0
Expected :0
Actual :
Try with
shopping_cart_qty = self.driver.find_element_by_xpath("//span[contains(#class,'topnav-cart-qty')]").get_attribute("value")
shopping_cart_qty = self.driver.find_element_by_xpath("//span[contains(#class,'topnav-cart-qty')]").get_attribute("textContent")
or
shopping_cart_qty = self.driver.find_element_by_xpath("//span[contains(#class,'topnav-cart-qty')]")
driver.execute_script("arguments[0].scrollIntoView()",shopping_cart_qty )
print(shopping_cart_qty.text)
if element is not visible then text will return empty as it considers the visibility also, you can use textContent to retrieve the text irrespective of the visibility .
But the best way is to scrollintoview first and then get text
I'm trying to fetch the web-table data using for loop. And the table has pagination up-to 42. here my code:
driver.get()
#identification and Locators
stack = driver.find_elements_by_xpath("//*[#id='container']/div/div[4]/table/tbody/tr/td[10]/div/ul/li")
quant = driver.find_elements_by_xpath("//*[#class='admin__data-grid-wrap']/table/tbody/tr/td[7]/div")
link = driver.find_elements_by_xpath("//*[#class='admin__data-grid-wrap']/table/tbody/tr/td[15]/a")
#Start a procedure
for i in driver.find_elements_by_xpath("//*[#id='container']/div/div[2]/div[2]/div[2]/div/div[2]/div/div[2]/button[2]"):
for steck,quanty,links in zip(stack,quant,link):
stuck = steck.text
quantity = quanty.text
linkes = links.get_attribute("href")
if stuck != 'No manage stock':
word = "Default Stock: "
stock = stuck.replace(word, '')
stocks = int(stock)
quanties = int(float(quantity))
if stocks < 0:
print(stocks,quanties,linkes)
stacks = abs(stocks)
total = stacks+quanties+1
print(total)
i.click()
driver.implicitly_wait(10)
print("Next Page")
This code fetches data from the 1st page. after clicking the next page. the 2nd for-loop didn't fetch 2nd-page data from web-table.
Most likely your query driver.find_elements_by_xpath("//*[#id='container']/div/div[2]/div[2]/div[2]/div/div[2]/div/div[2]/button[2]") only returns one element (the actual button to go to the next page) so I guess you should read the number of page and use it for an outer loop (or at least, you might have to rebind the selection on the HTML element representing the clickable button because it might change when a new page of the table is loaded) :
driver.get()
# Read the number of page and store it as an integer
nb_pages = int(driver.find_element_by_id('someId').text)
# Repeat your code (and rebind your selections, notably the one
# on the button to go to the next page) on each page of the table
for page in nb_pages:
# lines below are adapted from your code, I notably removed you first loop
stack = driver.find_elements_by_xpath("//*[#id='container']/div/div[4]/table/tbody/tr/td[10]/div/ul/li")
quant = driver.find_elements_by_xpath("//*[#class='admin__data-grid-wrap']/table/tbody/tr/td[7]/div")
link = driver.find_elements_by_xpath("//*[#class='admin__data-grid-wrap']/table/tbody/tr/td[15]/a")
# loop removed here (i also splited the string for readability
# (but it don't change the actual string value)
i = driver.find_elements_by_xpath(
"//*[#id='container']/div/div[2]/div[2]/div[2]"
"/div/div[2]/div/div[2]/button[2]")[0]
for steck, quanty, links in zip(stack, quant, link):
# your logic ...
# ...
# Load the next page:
i.click()
If you can't read the number of page, you may also use a while loop and exit it when you can't find a button to load the next page whit something like:
while True:
i = driver.find_elements_by_xpath(
"//*[#id='container']/div/div[2]/div[2]/div[2]"
"/div/div[2]/div/div[2]/button[2]")
if not i:
break
i = i[0]
# the rest of your logic
# ...
i.click()
This is only a guess (as we don't have a sample HTML code of the page / table structure that you are trying to use).
I am trying to write code with selenium python.
I work on site like https://www.thewatchcartoononline.tv/anime/south-park-season-1. As you can see this page is the page for the series with links to all the episode of the series.
I want to get the link of a given episode (the user chooses which one).
Important to note that not every series page has the same naming format for the episodes, some series has only "Episode 1" in the link text, others may have "South park season 1 episode 1" in the link, so I cant count on the naming format of the link's text.
this is the code I used to get the link to the episode (episode_num is given by the user)
episode_num = 1
chrome_driver = Chrome()
chrome_driver.get("https://www.thewatchcartoononline.tv/anime/south-park-season-1")
# This xpath takes you to the div of the episode list and then it search for a link which has a certain text in it
links = chrome_driver.find_elements_by_xpath(
f"//*[#id='sidebar_right3']//"
f"a[contains(text(), 'Episode {episode_num}')]"
)
However when I check links I see that there are more than one link. I get both episode 1 and episode 10 (since both of them contain the string "Episode 1")
Is there a way to get only the link I want? (maybe to make selenium take the link that doesn't have any digit after the text I want)
EDIT:
Well, kind of ugly, but in Xpath 1.0 I think this is the best you can do.
links = chrome_driver.find_elements_by_xpath(f"//*[#id='sidebar_right3']//a[(contains(., 'Episode {episode_num} ')) or (substring(text(), string-length(text()) - string-length('Episode {episode_num}') +1) = 'Episode {episode_num}') or (contains(., 'Episode {episode_num}-')) ]")
Finds Episode 10-11 for episode_num = 10 but not for episode_num = 11.
Checks for:
is Episode x in the text()
text() ends with Episode x
is Episode x- in the text()
I was checking the urls of the episodes. Wouldn't be a better approach relying on the #href instead of the text()? This is a bit shorter:
links = chrome_driver.find_elements_by_xpath(f"//*[#id='sidebar_right3']//a[(contains(#href, 'episode-{episode_num}-')) or (substring(#href, string-length(#href) - string-length('episode-{episode_num}') +1) = 'episode-{episode_num}')]")
Checks for:
is episode-x- in the url
url ends with episode-x
Try following xpath.Use last() option this should give count 1.
links = chrome_driver.find_elements_by_xpath("(//*[#id='sidebar_right3']//a[contains(text(), 'Episode {episode_num}')])[last()]")
print(len(links))
I'm trying to iterate over a number of elements returned by matching class names, which I have stored in an array users. The print(len(users)) outputs as 12, which is accurately correct as to how many returned there should be. This is my code:
def follow():
time.sleep(2)
# iterate here
users = []
users = browser.find_elements_by_class_name('wo9IH')
print(len(users))
for user in users:
user_button = browser.find_element_by_css_selector('li.wo9IH div.Pkbci').click()
#user_button = browser.find_element_by_xpath('//div[#class="Pkbci"]/button').click()
However currently, only index [0] is being .click()'d and the program is terminating after this first click. What would be the problem as to why the index being iterated isn't incrementing?
resource: image - red shows what's being iterated through and blue is each button being .click()'d
try this,
You can directly make array of buttons rather than li array,
Go click all buttons contains text as Follow,
simple,
browser.maximize_window()
users = []
users = browser.find_elements_by_xpath('*//button[text()='Follow']')
print(len(users)) # check it must be 12
for user in users:
browser.execute_script("arguments[0].click()", user)
# user.click() Go click all buttons
Find all your css_selector elements as a list and then iterate that list to perform .click()
yourList = browser.find_elements_by_css_selector('w0o9IH div.Pkbci')
users = browser.find_elements_by_class_name('wo9IH') returns a list of selenium.webdriver.remote.webelement.WebElement instances that can also be transversed.
In your implementation of the iteration, the above fact about the items in the list is overlooked and the entire page is search by transversing the page source from the WebDriver instance (i.e. browser.find_element_by_css_selector).
Here is how to go about getting the button in the matched WebElements:
for user_web_element in users:
# The next line given that there is only a single <button>
# in the screenshot for the matched WebElements.
user_button = user_web_element.find_element_by_tag_name('button')
user_button.click()
There are multiple buttons on my page containing similar href. They only differ with id_invoices. I want to click one button on page using xpath and href which looks like:
href="/pl/payment/getinvoice/?id_invoices=461"
I can select all buttons using:
invoices = driver.find_elements_by_xpath("//a[contains(#href, '/payment/getinvoice/')]")
but I need to select only button with highest id_invoices. Can it be done? :)
What you can do is:
hrefList = driver.find_elements_by_xpath("//a[contains(#href, '/payment/getinvoice/')]/#href")
for i in hrefList:
hrefList[i]=hrefList[i].split("id_invoices=")[-1]
max = max(hrefList)
driver.find_elements_by_xpath("//a[contains(#href, '/payment/getinvoice/?id_invoices="+str(max))+"'"+"]").click()
I don't know much about python so giving you a direction/algorithm to achieve same
Using getAttribute('#href');
You will get strings of URLs
You need to split all element after getText() you get in invoice List.
Split by= and pick up the last array value.
Now you need to typecast string to int, as last value after = will be a number
Now you just need to pick the highest value.
Since you have an XPath that returns all the desired elements, you just need to grab the href attribute from each one, split the href by '=' to get the id (2nd part of string), find the largest id, and then use the id to find the element you want and click on it.
invoices = driver.find_elements_by_xpath("//a[contains(#href, '/payment/getinvoice/')]")
ids = []
for invoice in invoices
ids.append(invoice.get_attribute("href").split('=')[2])
results = list(map(int, ids)) // you can't do max on a list of string, you won't get the right answer
id = max(results)
driver.find_element_by_xpath("//a[#href='/pl/payment/getinvoice/?id_invoices=" + id + "']").click