I'm learning how to do RPA, and a lot of what I want to accomplish involves manipulating checkboxes and text fields on Chrome. I'm trying to find a way to either:
Return a list of all elements of a specific type in Chrome (For example, all the element IDs of checkboxes, buttons, or text fields).
or
Have a user click on an element or use a hot key while hovering over an element to get the element ID.
The idea is that if I can have the element to manipulate selected by the user while the program is running, I can automate tasks more efficiently without having to change the parameters by manually inspecting the element in Chrome and changing the code.
Basically I'm trying to create a crude element selector in Python, or to just display a list of elements so the user can choose which element to interact with.
Currently I'm attempting to find the syntax to return all elements in the active window using pywinauto, and am exploring the use of Beautiful Soup to parse the HTML. However I assume there must be a simple one or two line code to do this, so if possible I would like to learn how to do it correctly rather than hack together a crude function that accomplishes it ineffectively.
Related
I'm working on a project trying to autonomously monitor item prices on an Angular website.
Here's what a link to a particular item would look like:
https://www.<site-name>.com/categories/<sub-category>/products?prodNum=9999999
Using Selenium (in Python) on a page with product listings, I can get some useful information about the items, but what I really want is the prodNum parameter.
The onClick attribute for the items = clickOnItem(item, $index).
I do have some information for items including the presumable item and $index values which are visible within the html, but I'm doubtful there is a way of seeing what is actually happening in clickOnItem.
I've tried looking around using dev-tools to find where clickOnItem is defined, but I haven't been successful.
Considering that I don't see any way of getting prodNum without clicking, I'm wondering, is there's a way I could simulate a click to see where it would redirect to, but without actually loading the link- as this would take way too much time to do for each item?
Note: I want to get the specific prodNumber. I want to be able to hit the item page directly without first going though the main listing page.
I need a way to find a button with a specific value in its text,which is a sequence of numbers like 768.I have tried with wd.find_element_by_xpath('//*[contains(text()='768')]')and with the function wd.find_element_by_css_selector('//input[value*='768']')but nothing works.The original xpath i need to locate and interact with is this (//*[#id="a-autoid-2-offer-2"]/span/input).The are other words in the description of the element but i need to identify it by value alone,and then interact with the button.
Help please,I'm stuck on this for days now.
I'm new to using Selenium, and I am having trouble figuring out how to click through all iterations of a specific element. To clarify, I can't even get it to click through one as it's a dropdown but is defined as an element.
I am trying to scrape fanduel; when clicking on a specific game you are presented with a bunch of main title bets and in order to get the information I need to click the dropdowns to get to that information. There is also another drop down that states, "See More" which is a similar problem, but assuming this gets fixed I'm assuming I will be able to figure that out.
So far, I have tried to use:
find_element_by_class_name()
find_element_by_css_selector()
I have also used them in the sense of elements, and tried to loop through and click on each index of the list, but that did not work.
If there are any ideas, they would be much appreciated.
FYI: I am using beautiful soup to scrape the website for the information, I figured Selenium would be helpful making the information that isn't currently accessible, accessible.
This image shows the dropdowns that I am trying to access, in this case the dropdown 'Win Margin'. The HTML code is shown to the left of it.
This also shows that there are multiple dropdowns, varying in amount based off the game.
You can also try using action chains from selenium
menu = driver.find_element_by_css_selector(".nav")
hidden_submenu = driver.find_element_by_css_selector(".nav # submenu1")
ActionChains(driver).move_to_element(menu).click(hidden_submenu).perform()
Source: here
I am trying to create an Instagram scraping bot that collects a list of Followers and Following using Python + Selenium.
However, that list keeps on loading when the user scrolls until the list is exhausted. I am attaching a screenshot for reference (some content hidden due to privacy reasons):
Now, I believe I have two ways to achieve this:
Keep reading the usernames, and then keep on scrolling.
Keep scrolling till the end, and then read all usernames together from the source code.
I've been trying to figure this out using the second method. However, I am not able to figure out how to know when there is no more content to scroll. How can I achieve this (provided that I don't know anything about the length of this element)?
Reason for not using Method 1: When scrolling, the DOM keeps getting refreshed, so it is hard to keep track of which usernames have been read.
One way to do this is is to keep track of the amount of child elements the div has that contains the li elements for the followers. If it doesn't increase after a scroll event, you've reached the end of the list.
I've been trying to figure this out for a week now. I'm attempting to find a way to allow a user to select an element on a webpage and to return that element ID.
Either of these two processes would work.
The user could click an element to identify the element ID, like with an element selector.
The program could display a list of all elements of a certain type like a checkbox, and the user could select which checkbox/text field/button to interact with. I tried to do this with the below code using Selenium, but for some reason this code just returns [] no matter how many checkboxes are on the page.
checkboxes = driver.find_elements_by_css_selector("#data_configuration_datamaps_ct_fields_body input[type='checkbox']")
print(checkboxes)
Every time I find a way to do this, it's with another programming language. When I've tried to ask this in various discords and subreddits, I'm always told to post the specific element I'm trying to work with.
The problem is, I'm trying to do this to develop tools to help my own workflow when doing RPA. So I need a way to select an element at runtime and not to predefine it.
I'm able to do this with other programming languages but have not found a way to do it with Python despite everyone saying Python is ideal for automation.
Is there really no way to accomplish this?