Confirm whether to write comments with xpath - python

Could you make an example with the site? I want to ignore the posts with comments and just click the ones without comments.
from selenium import webdriver
import time
path = "C:\chromed\chromedriver.exe"
driver = webdriver.Chrome(path) #path
'''
'''
driver.get("http://cineaste.co.kr/") #url
time.sleep(0.5)
postclick = driver.find_element_by_xpath("//h3[.='영화이야기']/following::div[#class='widget-small-box'][1]//li[#class='ellipsis'][not(contains(.,'+'))]") #로그인창 활성화
postclick.click()
driver.close()
I tried this, but there was an error.
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//h3[.='영화이야기']/following::div[#class='widget-small-box'][1]//li[#class='ellipsis'][not(contains(.,'+'))]"}
(Session info: chrome=81.0.4044.138)
I want to click on a post that doesn't have any comments yet. And I want to skip posts with comments.
I'm still a beginner. Help me.

The Element you are looking for is not available.
You need to recreate the Xpath. The Xpath you created is incorrect. See following image :-

You'll need these imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Example with the first entry with no comments in the "movie-reviews" block :
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//h3[.='영화감상평']/following::div[#class='widget-small-box'][1]//li[#class='ellipsis'][not(contains(.,'+'))][1]"))).click()

Related

selenium find_element_by_css_selector for the long class name

I have tried multiple times with other instruction codes with space from the tutorial which worked fine. However, when I just changed the URL and the following class, it would give the error saying
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
(Session info: chrome=100.0.4896.88)
Everything worked when I used the tutorial code.
Here is my code (I have solved a few chrome driver problems from the internet)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options)
driver.get("https://raritysniper.com/nft-drops-calendar")
time.sleep(1)
link = driver.find_element_by_css_selector(".w-full.h-full.align-middle.object-cover.dark:brightness-80.dark:contrast-103.svelte-f3nlpp").get_attribute("alt")
print(link)
I am trying to get attributes of each projects and make them into csv.
(Please refer to the screenshot)
Screen shot of HTML that I am trying to extract
it would be wonderful if anyone could depict the problem I got with the code.
Thank you!
The CSS_SELECTOR that you are using
.w-full.h-full.align-middle.object-cover.dark:brightness-80.dark:contrast-103.svelte-f3nlpp
does not really match any element in the HTML.
Instead, you should use this CSS_SELECTOR:
div.w-full.h-full.align-middle img:not(.placeholder)
In code:
driver.maximize_window()
wait = WebDriverWait(driver, 30)
driver.get("https://raritysniper.com/nft-drops-calendar")
#time.sleep(1)
first_element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.w-full.h-full.align-middle img:not(.placeholder)")))
print(first_element.get_attribute('alt'))
print(first_element.get_attribute('src'))
Import:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Output:
NEON PLEXUS
https://media.raritysniper.com/featured/neon-plexus_1648840196045_3.webp
Process finished with exit code 0

Selenium no such element: Unable to locate element (trying to login)

I am trying to use Selenium to download a file from https://id.opswat.com, in order to access it I need to login, but I can't, because when I try to retrieve the email input, it gives me an error ... This is my code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://id.opswat.com/login')
driver.find_element_by_name('email').send_keys('email#email.es')
driver.find_element_by_class_name('form--button is-primary is-fullwidth button').click()
driver.find_element_by_name('password').send_keys('pass')
driver.find_element_by_class_name('form--button is-primary is-fullwidth button').click()
And this is the error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[name="email"]"}
The login on that page is a bit special, because first you have to enter the email, check that it is well formed, click to continue, and you will see the password entry and the button to log in ...
Maybe something very basic is missing me, but I can't find it ... (I've tried waits, but neither ...)
First, your line
driver.find_element_by_class_name('form--button is-primary is-fullwidth button').click()
will always fail because class_name function does not handle spaces in the class. I'd suggest changing that to
driver.find_element_by_css_selector('.form--button.is-primary.is-fullwidth.button').click()
Next, You need to wait for the page to load before you try to interact with each field. This works for me:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get('https://id.opswat.com/login')
WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.NAME, 'email'))).send_keys('email#email.es')
driver.find_element_by_css_selector('.form--button.is-primary.is-fullwidth.button').click()
WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.NAME, 'password'))).send_keys('pass')
driver.find_element_by_css_selector('.form--button.is-primary.is-fullwidth.button').click()
I can't see anything obviously wrong with it, have you checked if you can get the driver.find_element_by_class_name('emailInputBox')? Or any other element in the page?
Does it work when getting by XPath?
//*[#id="root"]/div/section/div/div/div[2]/form/div[2]/div/div/div/input
You could also try waiting for the element to be present like in this answer:https://stackoverflow.com/a/50324722/10832847

unable to select an element using xpath using selenium

I'm trying to automate a process using selenium and have been able to open a webpage and click on links, however I stumbled upon a table in which a link needs to be clicked but I'm unable to select that link and getting an error. need help to select that particular element
right now this is what I've done
elem2=browser.find_elements_by_xpath('/html/body/div[3]/table/tbody/tr[1]/td[2]/div[2]/table/tbody/tr[7]/td[3]/a::text')
elem2.click()
you can see in the picture that I'm trying to access the findhtml.org link.
the error that I get is
InvalidSelectorException: Message: invalid selector: Unable to locate an element with the xpath expression /html/body/div[3]/table/tbody/tr[1]/td[2]/div[2]/table/tbody/tr[7]/td[3]/a::text because of the following error:
SyntaxError: Failed to execute 'evaluate' on 'Document': The string '/html/body/div[3]/table/tbody/tr[1]/td[2]/div[2]/table/tbody/tr[7]/td[3]/a::text' is not a valid XPath expression.
(Session info: chrome=81.0.4044.113)
First you have to switch to the iframe
Example:
frame = browser.find_elements_by_xpath('//iframe[contains(#src, \'hbx.media.net\')]')
browser.switch_to.frame(frame)
Now you can click
link = browser.find_elements_by_xpath('//a[contains(#href, \'http://www.findhtml.org\')]')
link.click()
To click on the specific link try below code.
Induce WebDriverWait() and presence_of_element_located() and following xpath.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://publicrecords.netronline.com/state/IL/county/dupage")
element=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.XPATH,"//a[#href='http://www.dupageco.org/PropertyInfo/PropertyLookUp.aspx' and contains(.,'Go to Data')]")))
element.location_once_scrolled_into_view
element.click()
Please note the element is not inside any iframe
browser.get('https://publicrecords.netronline.com/state/IL/county/dupage')
wait = WebDriverWait(browser, 20)
wait.until(EC.element_to_be_clickable((By.XPATH, "//td[contains(text(),'DuPage Supervisor of Assessments')]//following-sibling::td[2]//a"))).click()
output:
Note : please add below imports to your solution
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

Can not crawl website with Selenium in python (no errors but didn't work)

I'm trying scrawl comments from the link: http://www.phimmoi.net/phim/the-gioi-phep-mau-i1-6113/xem-phim.html. This is the code, I used:
find_comment = browser.find_elements_by_css_selector("div[class='_3-8y _5nz1 clearfix']")
for i in find_comment:
element_comment = i.find_element_by_css_selector("span[class='_5mdd']")
print(element_comment.text)
But nothing happened: no errors, no exceptions and nothing was printed.
Has the website lock crawled? If yes, please help me how to know.
The comments on the website you posted are in an iframe, so you will need to switch to the iframe before you can locate the comments.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# first switch to the iframe
WebDriverWait(browser, 10).until(
EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[contains(#title, 'fb:comments')]")))
# then get comments
comments_list = browser.find_elements_by_xpath("//span[#class='_5mdd']/span")
# iterate the comments
for comment in comments_list:
# print the comment text element -- the span which contains the commment text
print(comment.text)
find_comment = browser.find_elements_by_css_selector("._3-8y _5nz1 clearfix")
for i in read_more:
element_comment = i.find_element_by_css_selector("._5mdd")
print(element_comment.text)
Reference:
https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.find_elements_by_css_selector

selenium in python. trying to select element

UPDATE: I found out what caused the issue in my original post, but don't know how to solve. Here is some more of my code. I first did a search, after which my page in firefox jumped to www.flickr.com/searc/?text=volleybal.
but apparently my browser object still sits on www.flickr.com. How can I update it?
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('https://www.flickr.com')
s = browser.find_element_by_id('search-field')
s.send_keys('volleyball')
s.submit()
s = browser.find_element_by_class_name("style-button")
#s = browser.find_element_by_xpath("//li[#data-style-value = 'minimalism']")
s.click()
original post:
I'm experimenting with selenium, and I want to select the following element:
<li class="style-button minimalist" data-style-value="minimalism" data-tooltip-title="Minimalist" tabindex="0" role="button" aria-label="Minimalist">
I have tried the following, but none of them work:
s = browser.find_element_by_class_name("style-button minimalist")
s = browser.find_element_by_class_name("style-button.minimalist")
s = browser.find_element_by_css_selector("li.style-button.minimalist")
s = browser.find_element_by_css_selector(".style-button.minimalist")
How can I select this element?
You could try an XPath here:
s = browser.find_element_by_xpath("//li[contains(#class, 'style-button')]")
Hope this helps a bit.
Solved. Added browser.refresh()after s.submit()
The best option is to use absolute xpath. It for that button will be:
/html[1]/body[1]/div[1]/div[1]/div[3]/div[1]/div[1]/div[1]/div[2]/ul[1]/li[3]
Moreover, I found that sometimes the page is not loaded and the selenium searches for the element, which results in error. You can wait for the element to appear. You can find more information here.
With the combination of these, your code will be:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Firefox()
browser.get('https://www.flickr.com/')
s = browser.find_element_by_id('search-field')
s.send_keys('volleyball')
s.submit()
s = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, "/html[1]/body[1]/div[1]/div[1]/div[3]/div[1]/div[1]/div[1]/div[2]/ul[1]/li[3]")))
s.click()

Categories