Can't access to HTML elements with Selenium (python) - python

I'm new in python scripting. I'm trying to develop a bot that can access websites and work on it with Selenium.
I can do basics things like enter text in a search bar but I am blocked with a popup.
I try to access the popup element but I can't. It seems the element I want to reach are on a "modal" popup.
I have tried many things, I can access to the "modal"popup:
popup = self.__driver.find_element_by_xpath("//div[#id='modal']")
but I can't access to any element right under the revious one:
element = self.__driver.find_element_by_xpath("//div[#id='region-modal-right-panel']")
Can somebody help me with this problem? Thank you in advance for your help!!
If my explanations wasn't clear, do not hesitate to ask for mor details.
EDIT: link to the popup: https://app.libertex.com/products/currency/EURUSD/#modal_newInvest_EURUSD

Related

Selenium: A specific page in website is coming blank / white screen

Using selenium I can browse through the entire website except for the login screen on a specific website. When I checked page source I see some js codes.
Using a normal chrome browser I can access the expected login screen. Can anyone help me to overcome this issue? Thanks in advance.
Chances are that the website is detecting that you are using a bot and is blocking you from accessing its login screen for that reason. I can't know for sure that this is the reason because I haven't seen your issue in person, but most good websites don't like having non-human users interact with their pages, and a login page is exactly the type of page that the website would block you from accessing.
You can try to change some things to make Selenium less detectable by the website, but it's quite difficult and inconsistent at best. You can find some more information about achieving this here, but I wouldn't expect too much.

Selenium button click with Python

I'm pretty new to programming and Зython so trying to learn more by projects. Currently, workıng on Selenium and trying to automate my Steam login. The code looks like this. parts about the username and password inputs seem to working. I tried several different find methods for my button click but not worked. Using Chrome as a browser. I need some help on how to make that button click work. Thank you.
def login():
browser.get("URL")
browser.find_element_by_id("input_username").send_keys("username")
browser.find_element_by_id("input_password").send_keys("pasword")
element = browser.find_elements_by_class_name("btnv6_blue_hoverfade btn_medium")
element.click()
login()
find_elements_by_class_name will return a list of elements and handle all elements within that list you need to iterate through your list. Currently, you are trying to handle a single web element so you can use find_element_by_class_name to retrieve your desired element, and then you can perform an action on it.
element = browser.find_element_by_class_name("btnv6_blue_hoverfade btn_medium")

Python Selenium -- Searching for a link but finding cards

Using Python + Selenium to create a web crawler/scraper to notify me when new homework is posted. Managed to log into the main website, but you need to click a link to select your course.
After searching through the HTML manually, I found this information about the link I usually click (The blue box is the link).
However, no button that seems clickable. So I searched the page for the link I knew it should redirect me to, and I found this:
It looks like a card, which is a new data structure/object for me. How can I use an automated web crawler to click this link?
Try the following:
ui.WebDriverWait(self.driver, timeout).until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".title.ellipsis")))
driver.find_element_by_css_selector(".title.ellipsis").click()
Hope it helps you!

Trouble retrieving this element in selenium

Sign in
Hey Guys,
Pretty simple question here but I'm having trouble with it for some odd reason. What is the best way to "grab" this button in selenium (using python but a generalized solution would work as well). Its a button, but its html is not a button which means this has not been working for me.
driver.find_element(By.XPATH, '//button[text()="Sign in"]')
Any help would be great, thanks!
You should be able to access it using the a tag instead of button like this
driver.find_element(By.XPATH, "//a[contains(text(), 'Sign in')]")
Selenium docs on locating elements by XPath
and
A good SO answer to help locate an element that contains given text

Download File from website without filename? Python 3.4

I dont know where to exactly start here, and I have to admit my knowledge of python and websites are limited. However In the past ive done some requests from an api and accessed a file or two from a website but I had some examples to build off of. In this case I have no written example to help me through the process so I dont really know where to start or if "requests" is even the way to go.
What I have is a distributor's website that has a file with product information.
If I were to download this file manually I would have to login, navigate to the download section of the website. At this point a popup appears where I select the brand I want to download, I have options to select from as far as data I would like to gather, a text box to name the file and a download button that has no url.
Im sure all this seems pretty vague since I dont know what info would be helpful at this point.
A nudge in the right direct would be great!!
Thanks
Screen shot of popup
It sounds like there may not be an API, in instances like this using a web automation solution such as selenium could get you the desired result.
For your case it sounds like you will need to find the button elements and then click them
From their basic example:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.send_keys("pycon")
enter code here`elem.send_keys(Keys.RETURN)
based on your example html code, after you load the page, you could use the following to find the button and click it
elem = driver.find_element_by_id("downloadBtn")
elem.click()
You can use a http library like Request to download this. but you may offer the username and password, you can study from its examples.
If the site you wish to download from has no JavaScript you will need to parse to navigate to the file you wish, consider using RoboBrowser. Selenium may be overkill for this.
Here is a basic example:
robo = RoboBrowser(history=True, parser="html.parser")
robo.open("http://www.python.org")
search = robo.get_form(action="/search/")
search["q"].value = "Really awesome search query"
robo.submit_form(search)

Categories