My scenario is: I had to write a script to download a huge amount of files by using Selenium and Python 3.6, and now, I have to download files using the same technologies.
The point is that this script won't be executed on my own computer.
Is it possible, using chrome webdriver, to get the default download folder of chrome?
As of now I have this code:
dlPth="C:\\Users\\genieelecpsim\\Downloads\\"
nwPth="C:\\Users\\genieelecpsim\\Downloads\\Exports"
for file in os.listdir(dlPth):
if file.startswith("export") and file.endswith(".csv"):
print(str(years[i])+"-"+str(months[j])+"-"+str(days[k]))
newfile=os.path.join(nwPth,str(years[i]) +"-" +(str(months[j]) if months[j]>=10 else "0"+str(months[j]))+"-" +(str(days[k]) if days[k]>=10 else "0"+str(days[k])) +".csv")
shutil.move(os.path.join(dlPth,file),newfile)
print (newfile)
break
What I want to do here is something like:
dlPth=# Chrome's default download directory
nwPth=dlPth+"\\Export"
Is it possible? Thanks for your response!
EDIT: First of all, thanks to all for your quick answers, and it seems that my topic's a dupplicate, but as I'm not using the same configuration than this one, I'm wondering if this method works with py3.6 and Selenium 3.0.2... I'm sorry I can't directly comment your answers as I'm new here, but thanks to everyone!
You can change download folder by using specific preferences when starting your driver. You should set this:
("download.default_directory", yourWantedPath)
Not sure how are you starting and configuring your driver so can't help you with more code, but this is the preference you are looking for.
You could find useful things here.
The answers did not really help me, but thanks anyway. I find another way to do so by using the OS library:
# dlPth will be the path to the download directory of the current user (on the system)
dlPth=os.path.join(os.getenv('USERPROFILE'), 'Downloads')
# destPth will just be a directory where I'll put all my (renamed) files in.
destPth=dlPth+"\\Exports\\"
Thanks for answering, I posted the answer here so anyone looking for some help on this topic will be able to see it
Related
I have a problem. Let's say I have a website (e.g. www.google.com). Is there any way to create a file with a .url extension linking to this website in python? (I am currently looking for a flat, and I am trying to save shortcuts on my hard drive only to apartment offers posted online matching my expectations ) I've tried to use the os and requests module to create such files, but with no success. I would really appreciate the help. (I am using python 3.9.6 on Windows 10)
This is pretty straightforward. I had no idea what .URL files were before seeing this post, so I decided to drag its URL to my desktop. It created a file with the following contents which I viewed in Notepad:
[InternetShortcut]
URL=https://stackoverflow.com/questions/68304057/internet-shortcut-in-python
So, you just need to write out the same thing via Python, except replace the URL with the one you want:
test_url = r'https://www.google.com/'
with open('Google.url','w') as f:
f.write(f"""[InternetShortcut]
URL={test_url}
""")
With regards to your current attempts:
I've tried to use os and requests module to create such file
It's not clear what you're using requests or os for, since you didn't provide a Minimal Reproduceable Example of what you'd tried so far; so, if there's a more complex element to this that you didn't specify, such as automatically generating the file while you're in your browser, or something like that, then you need to update your question to include all of your requirements.
Here's the issue i am facing now.
I could launch chrome driver. However my selenium code suddenly doesnt work and pops up above image.
Hope someone can shed light as i couldn't find a solution online. .
Generally BarthRid's answer is correct, providing full path to userdata folder works with Chrome 90+. But if you store userdata folder in the same directory as your script, you can use pathlib to manage things more easily.
import pathlib
script_directory = pathlib.Path().absolute()
options.add_argument(f"user-data-dir={script_directory}\\userdata")
This way allows you to store actual script's directory in a variable, and via formatted string put it in selenium's options argument. Useful while dealing with multiple instances.
I had a similar problem to yours and #scott degen.
options.add_argument line of your pasted screenshot
I changed mine from:
options.add_argument("user-data-dir=selenium")
to the full directory, and it seems to be working better now.
options.add_argument("user-data-dir=C:\environments\selenium")
I changed mine from:
options.add_argument("user-data-dir=selenium")
to the full directory, and it seems to be working better now.
dir_path = os.getcwd()
chrome_option.add_argument(f'user-data-dir={dir_path}/selenium')
It seems Chrome changes its Current Working Directory during startup and so the relative path won't work. You can try to convert the user-data-dir to absolute path then pass to chrome. See Bug: 1058347
I am learning how to do webscraping, crawlers etc. and I came across this repo. I understand how the code works, what the input and outputs should be, but how do I run it in a terminal on Windows? How do I call the respective .txt files and test the search engine?
I saw that someone else asked that and the creator showed them this link here. But it still doesn't explain how to actually apply it to files.
The author of logicx24 has hard coded the target text files in querytexts.py. See line 122 which reads:
q = Query(['pg135.txt', 'pg76.txt', 'pg5200.txt'])
The list input to Query are all references to files that exist in the corpus directory. Try changing that to include a different file in their corpus directory. Better yet, add a new target text file of your own and use that.
Good luck!
Why are you using text files? I don't get it. Either way, you could just use Python itself to do that. Use the selenium library for Python. There's a tutorial to installing this here. Once that's done, just use this code if you're using Google:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("https://www.google.com")
search = driver.find_element_by_css_selector(".gLFyf.gsfi")
time.sleep(5)
search.send_keys("Desired Input Text Goes Here")
search.send_keys(Keys.RETURN)
Don't worry if it takes a while to load. It usually does that. If you want to reduce the amount of time it takes, use a lower number for the parameter on line 8 (time.sleep(5)). Assuming you've gone ahead and learned a bit more about Selenium, there isn't really much else to talk about apart from one thing. That is, line 7 (search = driver.find_element_by_css_selector(".gLFyf.gsfi"). Assuming you've learned advanced CSS selectors already (if you have literally no experience in web development, specifically HTML and CSS, you can just copy-paste the code), the .gLFyf.gsfi is simply the CSS selector for the search bar in Google. You can find the selector for the search bar in any engine by just looking through the source code using Ctrl + Shift + I on Windows. You can use any other Selenium element selector for this as long as it works. Make sure to also change the URL on line 6 (driver.get("https://www.google.com")) to match that of your search engine if you're not using Google.
Sorry if this seemed a bit vague or strange. If you don't really care, feel free to download Selenium, copy-paste the code, and move on. Otherwise, I suggest also learning Selenium and HTML/CSS if you haven't already.
Settings for the Mozilla Firefox browser my be changed by entering about:config in the address bar, which will open a configuration file. I would like to be able to edit, delete and add parameters to this file in Python. How can I do this?
The configuration files used in firefox are actually a couple of javascript files that you can find under the profile directory: prefs.js and user.js.
You can use python to append lines to those files (user.js recommended) like:
user_pref(<preference>, <value>)
However, I don't really think is a good idea to use python for this.
You can check
Firefox Profiles
to get to the OS dependent profile,
also,
here is a duplicate question to point you in the right direction.
Also, maybe
this Javascript Python bridge will better assist you editing the files.
im working a project lately to get download progress info remotely when im away.
i googled a bit but i couldnt find any useful info. on this issue
im using Internet Download Manager.
please help :/ i appreciate any suggestions
im thinking about making software with python but Also C is Ok. if there is a solution
I dont know exactly how to do it but you can get IDM download progress( such as Transfer rate, TimeLeft... ) by reading the values of the ListView of Internet Download Manager windows (they call it "hook"? )
i just found idm has log file for every download in its temporary folder im trying to parse information from log file i dunno if it is possible sync.ly get info..but i'll try