Dear Stack Overflowers,
I have a short Python script that automatically goes through the Texas Unemployment Website and fills out all of your information every 2 weeks. This year they started implementing a reCAPTCHA. This steps prevents the script from getting to the next step.
I've tried selecting the captcha by ID and Class but haven't had any luck. The class name changes when you hover over the box as well which makes things more difficult. The first picture below is when I'm not hovered over the box and the second picture is with the mouse over the box.
I've been using Selenium in Python to find elements on the webpage and make clicks. Here's my last attempt using class name:
driver.find_elements_by_class_name('recaptcha-checkbox goog-inline-block recaptcha-checkbox-unchecked rc-anchor-checkbox').click()
Anybody have any ideas for selecting the reCAPTCHA?
Related
I'm new to using Selenium, and I am having trouble figuring out how to click through all iterations of a specific element. To clarify, I can't even get it to click through one as it's a dropdown but is defined as an element.
I am trying to scrape fanduel; when clicking on a specific game you are presented with a bunch of main title bets and in order to get the information I need to click the dropdowns to get to that information. There is also another drop down that states, "See More" which is a similar problem, but assuming this gets fixed I'm assuming I will be able to figure that out.
So far, I have tried to use:
find_element_by_class_name()
find_element_by_css_selector()
I have also used them in the sense of elements, and tried to loop through and click on each index of the list, but that did not work.
If there are any ideas, they would be much appreciated.
FYI: I am using beautiful soup to scrape the website for the information, I figured Selenium would be helpful making the information that isn't currently accessible, accessible.
This image shows the dropdowns that I am trying to access, in this case the dropdown 'Win Margin'. The HTML code is shown to the left of it.
This also shows that there are multiple dropdowns, varying in amount based off the game.
You can also try using action chains from selenium
menu = driver.find_element_by_css_selector(".nav")
hidden_submenu = driver.find_element_by_css_selector(".nav # submenu1")
ActionChains(driver).move_to_element(menu).click(hidden_submenu).perform()
Source: here
I am writing a script in Python using selenium that automates a store checkout. It launches to the site, finds the first item that matches the criteria, hits checkout, then proceeds top purchase. I am trying to find out how to make it select a specific color. The shop I am buying from updates the store every week with new items, so I won't have the XPATH available to me.
There is a button to change the color, and I need help being able to select the button.
The store has multiple buttons with the product in different colors as a way to choose between them.
The button to change colors has ONE specific attribute titled "data-style-name"="(color)"
I will not have any other attribute, link, or ID known to me, thus searching for this attribute seems like the only way to find it. The picture above is a snippet from pre-existing items on the storepage as an example.
(Note that above it says class="selected" because the "black" jacket has been clicked on while taking this screenshot. Example of a jacket that has not been clicked on:
Anyone have any ideas on the way to go about this?
Inspired by this and this, and a bit of reading up on CSS selectors but something like:
elem = driver.find_element_by_css_selector('button[data-style-name]')
Might work for you
I am having trouble finding elements on a customer-facing webpage that I am scraping data from, using Robot Framework + Selenium. My trouble, I think, has to do with the desired data existing in a popup/frame. The data I seek is located on a customer's invoice, which pops up when I press a button ("View Current Invoice"). I've been successful with logging into the site and navigating around, and at one point I was successful pressing the View Current Invoice button to cause the invoice to pop up - but forgot to commit that code and lost it. :-(
In any case, eve if I manually enter the popped up invoice by pressing the button when my script is expecting it to be pressed, I can't seem to scrape the subsequent data. I have tried to identify elements on the invoice using locators (from Right-Click-Inspect capability built into Firefox and Chrome; Katalaon Recorder; Selenium IDE; etc.). I get what looks like a valid locator (almost always Xpath); yet when I run my Robot script, it fails to find the element in question. I have spent a lot of time poring over the page's source code, but since I am not as savvy with HTML/JS/CSS as I should be, I haven't been successful.
Here is a screenshot of the invoice button:
And here is what I see when the button is pressed. I want to scrape all the invoice data, like Amount Due, Invoice Number, Due Date, etc.
Does anyone have any idea what I am missing here? What would you do to get the data on the invoice if you were in my shoes? I know my question probably sounds vague and naiive, but I am at the end of my rope, so to speak. I am willing to share page source code, more screenshots, whatever is required.
EDIT I used Rahul Rai's method to inspect the popup while it was popped up; then searched for "iframe". There were 10 matches; #7, when clicked on, resulted in the invoice popup being highlighted in blue:
I assume this means this is the iframe referencing the popup? If so, I should be able to find information about the "handle" to the iframe in the inspection code, but I don't see anything there that matches the locators I am used to (e.g. name, id, xpath). I even tried "Select Frame 1599252503952", but that just resulted in a
"Element with locator '1599252503952' not found" error.
As per above screen you have shared I can see your Invoice details are inside iframe. So after clicking on View Current Invoice button you can use below code to navigate inside frame and then scrape required information.
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(#src,'invoice_detail_container']")))
#Code to scrape data
ele = driver.find_element_by_xpath('<xpath>')
print(ele.txt)
......
......
#After your work is done in this frame to navigate back to main window
driver.switch_to.default_content()
Note: I have assumed your main frame for invoice is not in side any other iframe ( Based on screen shared). Also before elements start there is no other nested frame. If there is any other nested frame you need to navigate first into that.
I was finally able to scrape data from the Invoice popup after inspecting the HTML source, and seeing this:
<iframe frameborder="0" src="/cmc/invoice_detail_container.pyt?direction=//my.hughesnet.com/cmc/invoice_detail.pyt%3Finvnumber%1234-567890&portletId=863" name="1599391562960" class="cboxIframe" allowfullscreen="true" webkitallowfullscreen="true" mozallowfullscreen="true"></iframe>
I was then able to use the Select Frame keyword in Robot Framework, specifying the iframe locator for the popup, using the 'class' strategy. I also had to explicitly select the main body frame first. In the end, the code that allowed me to enter the iframe and scrape was:
Select Frame body
Select Frame class:cboxIframe
Big thank you to Rahul Rai for pushing me closer to the solution; and thanks to the others who answered as well.
You need to switch your site to frame/popup, you can use like below example, may be it will help you.
IList<IWebElement> textfields = new List<IWebElement>();
textfields = driver.FindElements(By.TagName("iframe"));
driver.SwitchTo().Frame(textfields[count); // number of textfields list.
please try to implement as per your scenario, let me know if any question.
You can try to do :
driver.switch_to_active_element()
and then scrape the popup to close it. Then I think it will be okay...
I am currently using pyautogui to automate a web application. I am very new to the software so every single step has been a learning experiennce. I am currently trying to select a specific checkbox however since there are multiple check boxes I am not able to do so. I am currently using a screenshot of the checkbox icon in order to click on it so I figured this was my problem. Is there anyway in pyautogui to select a specific image even if there are duplicates present?
I've tried playing around with the coordinates, but since this application needs to be as user friendly as possible I thought setting specific coordinates could be tricky.
Checkcoords=pyautogui.locateOnScreen("checkbox.png"#Locate the checkbox, but since there are multiple checkboxes it just goes to the first one present and not the one that i need
pyatuogui.click(Checkcoords) # used to click the checkbox
Not getting any error messages, just goes to the wrong checkbox.
I have a script which gets me info from Polish Avon's website. So essentially every month they change prices, and to make my girl's life easier I just download the prices to have a look up table in excel.
Anyways, so I have the script which navigates to this website:
https://www.avon.pl/szukaj/po-kodzie-produktu/
Once the page is loaded it enters a number between 00000 and 99999 into the search box, which I find using xpath:
find_box_path = '//*[#id="ShopByProductNumber"]/div[2]/div[3]/div[1]/div/input'
Only some of the codes are valid, so if the search is successful, the script will click on the item, which opens in a new window and processes the information, if nothing is found it moves on to the next number. The script checks for the xpath to figure out if the code is valid or not. The following exctract would click on the element and open a new tab:
# ------------ click the product ------------
find_item_text_element = WebDriverWait(driver, 10).until(lambda driver: driver.find_element_by_xpath(find_text_path))
find_item_text_element.click()
time.sleep(0.4)
The find_text_path variable is declared earlier as:
find_text_path = '//*[#id="ShopByProductNumber"]/div[2]/div[3]/p/a'
The interesting bit, is that for more than a year, my script worked like a charm. Only 2 days ago the script was running, I got through maybe 25000 combinations until it stooped. From that point, when the script gets to the bit above, it shuts down and resets. I understand why it resets - that is intended - but I have no idea why it won't click on the element. The IDE doesn't show any error.
I use XPather to find the xpaths, and the one above is a valid xpath. And as I said it work fine until now. I understand that website itself could have done something to prevent automation, but I don't see the problem. Can anyone see/point out the problem? Maybe some workarounds?
Location of both elements in question
EDIT:
The issue was resolved. The zoom in my chrome profile I was using, was set to 105% instead of 100%. This cause the webdriver to click wrong spot on the page.
Can you show us the specific html code which is causing the issue? I'm not able to find it on the website. Maybe the element you're trying to reach is now included in a frame, but I can only suppose without the html.