I have created an automation via selenium in order to improve the time spent for downloading raw data per user:
What I found out now is that when a user has no data to download the "download" button is not visible and then the script stops running.
Sorry for uploading screenshots but it is impossible to share URL since you have to log in first.
Is there a way to avoid this?
Related
Newbie here. Trying to run a list of links (over 1000s) to files from 1GB to 50GB pr file.
The final link is in a button you click on.
So need help to setup selenium to parse a list of links and click on the download button to download the file in the link. I guess there needs to be some kind of pausing or timer or else the browser will crash with downloads. Maybe theres a smart way i'm just aiming for the stars.
the xcode of the button is: //*[#id="menu-75-option-3"]/div
thanks.
tried to get selenium to find all links on but failing
I have been working on a python code which shows the time spent on an applicaton based on keyboard and mouse movement.
I can set the start and stop time, the output of this code is the duration I spend on each application in the set duration.
I have also added "idle time" which happens when I am not working on the system.
My question is,
In case I am using a browser app like chrome or firefox or others, I need the name of the website.
This will help me track how much time I am spenting on those websites
You should 100% use Selenium. It's a browser automation tool that uses python to automate web browsers. I personally use it to automate YouTube uploads. You can do what your trying to do with a few commands, selenium is one of the best tools for this usecase!
Here's the github! - https://github.com/SeleniumHQ/selenium
I am currently trying to write a python script that will open my companies inventory system which is a link in google chrome, sign in, and then click the save as excel button that is posted on top of a specific page. This will hopefully automate the process of opening the link, navigating over to the tab, clicking export, then exporting this data daily.
Any idea of where to start? I was thinking maybe can get this done using web scraping but not sure with the log in details needed. Also, how can I export this file once in? Just need some ideas to start me on this journey. Any and all help is appreciated!
Simply start with selenium python automation
Divide you whole process in smaller tasks and write python code
for that each task:)
I have tens of thousands of URLs which I want to save their webpages to my computer.
I'm trying to open and save these webpages using Chrome automated by pywinauto.
I'm able to open the webpages using the following code:
from pywinauto.application import Application
import pyautogui
chrome_dir = 'C:\Program Files\Google\Chrome\Application\chrome.exe'
start_args = ' --force-renderer-accessibility --start-maximized https://pythonexamples.org/'
app = Application(backend="uia").start(chrome_dir+start_args)
I want to further send a shortcut to the webpage to save it as a mhtml. Ctrl+Shift+Y is the shortcut of a Chrome extension (called SingleFile) that saves a webpage as mhmtl. Then I want to close the tab by typing "Ctrl + F4", before I open another one and repeat the same process.
The keys are not successfully sent to Chrome.
# Sent shortcut (Ctrl+Shift+Y)
pyautogui.press(['ctrl', 'shift', 'y'])
# Close the current tab:
pyautogui.press(['ctrl', 'f4'])
I'm stuck at this step. What's the right way to do this? Thank you!
Tried other alternatives like Selenium, but it was blocked by the remote server.
Why are you using Chrome to get the website data? Generally, using an external application directly (ie. emulating a user) is a horrible and inefficient way to do anything. If your objective is to quickly get and store the data from a website, you should be talking directly to the website, using something like the requests module, which lets you quickly and easily send an HTTP request and get all of the website data. To get MHTML data, you can try something like this.
I am trying to automate the process of downloading webpages with technical documentation which I need to update every year or so.
Here is an example page: http://prod.adv-bio.com/ProductDetail.aspx?ProdNo=1197
From this page, the desired end result would be having all the html links saved as pdf's.
I am using wget to download the .pdf files
I can't use wget to download the html files, because the .html links on the page can only be accessed by clicking through from the previous page.
I tried using Selenium to open the links in Firefox and print them to pdf's, but the process is slow, frequently misses links, and my work proxy server forces me to re-authenticate every time I need to access a page for a different product.
I could open a chrome browser using chromedriver but could not handle the print dialog, even after trying pywinauto per an answer to a similar question here.
I tried taking screenshots of the html pages using Selenium, but could not find out how to get the whole webpage without capturing the entire screen.
I have been through a ton of links related to this topic but have yet to find a satisfying solution to this problem.
Is there a cleaner way to do this?