I have some files that contain one URL per line, like
https://url/url/url.com/page-1.jpg
https://url/url/url.com/a.mp3
https://url/url/url.com/b.mp3
....
I try to code for:
import wget
with open ("5074_url.txt", encoding='utf-8', mode = 'r') as f:
for line in list(f): # OR f.readlines()
filename = wget.download(line)
print (filename)
but raise Error msg:
Traceback (most recent call last):
File ".\Geturl2.py\", line 33, in <module>
filename = wget.download(line)
File "C:\Program Files (x86)\Python\lib\site-packages\wget.py", line 506, in download
(fd, tmpfile) = tempfile.mkstemp(".tmp", prefix=prefix, dir=".")
File "C:\Program Files (x86)\Python\lib\tempfile.py", line 342, in mkstemp
return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
File "C:\Program Files (x86)\Python\lib\tempfile.py", line 260, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
OSError: [Errno 22] Invalid argument: '.\\page-1.jpg\ngjf6wrvy.tmp'
\ngjf6wrvy.tmp What's this ? There's no this in file, I'm sure that.
I'm using Python 3.6.5 on windows 10
I know it could use urllib.request.urlretrieve(url, filename) ,
but it has filename option. I don't wanna change filename.
So How don't change filename ?
[Solved]
import wget
with open ("5074_url.txt", encoding='utf-8', mode = 'r') as f:
for url in f.readlines():
filename = wget.download(url.strip())
print (filename)
Thank you so much for help !!!
I don't know what you're trying to achieve exactly, but here's an example with requests:
#!/usr/bin/env python
import os
import requests
with open('test.txt', 'r') as f:
for url in f.readlines():
r = requests.get(url.strip())
print(r)
This will "download" each URL contained in test.txt and store in memory. The variable r contains the Response object.
Related
I want to download some files from gdrive with a python script. Files contain spaces and nonEnglish characters and I want to save them exactly with the same names. I try to create a similar file with spaces and special characters, but python gives me always
OSError: [Errno 22] Invalid argument::
from urllib.request import urlopen
from urllib import request
from urllib.parse import unquote
import os
base_dir = os.getcwd()
print(base_dir)
url = 'https://drive.google.com/u/0/uc?id=1KD3dIMaJU6SY8NX38jOW3DWQYjPGN-9A&export=download'
resource = request.urlopen(request.Request(url))
content = resource.getheader('Content-Disposition')
filename = content.split('*=')[1]
filename = unquote(filename.split("''")[1])
filepath = os.path.join(base_dir, filename)
with urlopen(url) as file:
file_content = file.read()
# Save to file
with open(filepath, "w") as f:
f.write(file_content)
This is the full message:
Traceback (most recent call last):
File "C:\Users\user\Downloads\Telegram Desktop\ChatExport_2023-01-07\gdrive_links.py", line 26, in <module>
with open(filepath, "w") as f:
OSError: [Errno 22] Invalid argument:
'C:\\Users\\user\\Downloads\\Telegram Desktop\\ChatExport_2023-01-07\\DilKoçu1 (BEN KİMİM? SİSTEMİMİZ NASIL İŞLİYOR?).mp4'
I am using python 3.10.4 on Windows 11
I would like to decompress a bunch of .bz2 files contained in a folder (where there are also .zst files). What I am doing is the following:
destination_folder = "/destination_folder_path/"
compressed_files_path="/compressedfiles_folder_path/"
dirListing = os.listdir(compressed_files_path)
for file in dirListing:
if ".bz2" in file:
unpackedfile = bz2.BZ2File(file)
data = unpackedfile.read()
open(destination_folder, 'wb').write(data)
But I keep on getting the following error message:
Traceback (most recent call last):
File "mycode.py", line 34, in <module>
unpackedfile = bz2.BZ2File(file)
File ".../miniconda3/lib/python3.9/bz2.py", line 85, in __init__
self._fp = _builtin_open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'filename.bz2'
Why do I receive this error?
You must be sure that all the file paths you are using exist.
It is better to use the full path to the file being opened.
import os
import bz2
# this path must exist
destination_folder = "/full_path_to/folder/"
compressed_files_path = "/full_path_to_other/folder/"
# get list with filenames (strings)
dirListing = os.listdir(compressed_files_path)
for file in dirListing:
# ^ this is only filename.ext
if ".bz2" in file:
# concatenation of directory path and filename.bz2
existing_file_path = os.path.join(compressed_files_path, file)
# read the file as you want
unpackedfile = bz2.BZ2File(existing_file_path)
data = unpackedfile.read()
new_file_path = os.path.join(destination_folder, file)
with bz2.open(new_file_path, 'wb') as f:
f.write(data)
You can also use the shutil module to copy or move files.
os.path.exists
os.path.join
shutil
bz2 examples
Hello Python Developers,
I want to write a python script that will search a directory for any files with the extension ".err", then format the file (which is an XML format), then overwrite the file with the correct formatting. Here is the code I have so far:
import xml.dom.minidom
import glob
import os
path = "/qond/apps/tomcat_qx/webapps/qxiqonddb/qxtendQueues/qdocqueue/responses/"
os.chdir(path)
for file in glob.glob("*.err"):
with open(path + file) as f:
xml_data = f.read()
xml = xml.dom.minidom.parse(xml_data)
xml_formatted = dom.toprettyxml()
f.write(xml_formatted)
f.close()
Many Thanks in advance!
Edit:
The current issue I face with the above code is:
Traceback (most recent call last):
File "qxtend_formatter.py", line 12, in <module>
xml = xml.dom.minidom.parse(xml_data)
File "/usr/lib64/python2.6/xml/dom/minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "/usr/lib64/python2.6/xml/dom/expatbuilder.py", line 922, in parse
fp = open(file, 'rb')
IOError: [Errno 36] File name too long: '<?xml version="1.0" encoding="UTF-8"?>\n<soapenv:Envelope xmlns:soapenv="http://schemas.xm.....
It seems as though it tries to save the file name as the file contents, but I would like it to keep whatever filename is had.
I have resolved this issue by doing two things:
Ensuring that the OS user has full access to the file (chmod 777)
Creating an '.f' instance to read and a '.fl' instance to write the file
My code now looks like this:
from xml.dom import minidom
import glob
import os
path = "/qond/apps/tomcat_qx/webapps/qxiqonddb/qxtendQueues/qdocqueue/responses/"
os.chdir(path)
for file in glob.glob("*.err"):
with open(file, "r") as f:
xml_data = f.read()
doc = minidom.parseString(xml_data)
with open(file, "w") as fl:
fl.write(doc.toprettyxml(indent=" ", newl="\n"))
In a Folder called Assignment Parser, I've my parsing.py file along with a auth.txt file. Trying to open this auth.txt file. But getting an error that says :
(base) C:\Users\Ajay\Desktop\Python\Assignment Parser>python parsing.py
Traceback (most recent call last):
File "parsing.py", line 27, in <module>
main()
File "parsing.py", line 8, in main
file = open(file_path / "auth.txt","r")
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Ajay\\Desktop\\Python\\Assignment Parser\\auth.txt'
Code:
from pathlib import Path
import os
def main():
# read file
# C:\Users\Ajay\Desktop\Python\Assignment Parser\
file_path = Path("C:/Users/Ajay/Desktop/Python/Assignment Parser/")
file = open(file_path / "auth.txt","r")
# file = open("auth.txt", "r")
lines = file.readlines()
file.close()
Where is this going wrong? PFA for the screenprint.
Try this:
from pathlib import Path
import os
def main():
# read file
# C:\Users\Ajay\Desktop\Python\Assignment Parser\
file_path = Path("C:/Users/Ajay/Desktop/Python/Assignment Parser/")
file = open(os.path.join(file_path, "auth.txt"), "r")
# file = open("auth.txt", "r")
lines = file.readlines()
file.close()
I think the problem is in file extension, I see parsing has .py extension but auth is not
please try file = open(file_path / "auth", "r") again (just delete .txt extension)
As you have your python file in same folder as your text file. You can directly use below code.
def main():
file = open("./auth.txt")
lines = file.readlines()
file.close()
Also make sure, your syder working directory is set to this folder path "C:/Users/Ajay/Desktop/Python/Assignment Parser"
I've just rebuilt my Raspberry Pi and hence installed the latest version of the Dropbox API and now my program doesn't work. I think this is due to point 1 in these breaking changes: https://github.com/dropbox/dropbox-sdk-python/releases/tag/v7.1.0. I'm sure this question from SO (Dropbox API v2 - trying to upload file with files_upload() - throws TypeError) solves my problem... but as a newbie, I can't figure out how to actually implement it - and anyway, I'm already using f.read()... can anyone help?
This is my code:
def DropboxUpload(file):
sourcefile = "/home/pi/Documents/iot_pm2/dropbox_transfer/" + filename
targetfile = "/" + filename
dbx = dropbox.Dropbox(cfg.dropboxtoken)
f = open(sourcefile, "r")
filecontents = f.read()
try:
dbx.files_upload(filecontents, targetfile, mode=dropbox.files.WriteMode.overwrite)
except dropbox.exceptions.ApiError as err:
print(err)
f.close()
And this is the error:
Traceback (most recent call last):
File "/home/pi/Documents/iot_pm2/dropbox_uploader.py", line 20, in <module>
DropboxUpload(filename)
File "/home/pi/Documents/iot_pm2/dropbox_uploader.py", line 12, in DropboxUpload
dbx.files_upload(filecontents, targetfile, mode=dropbox.files.WriteMode.overwrite)
File "/usr/local/lib/python3.5/dist-packages/dropbox/base.py", line 2125, in files_upload
f,
File "/usr/local/lib/python3.5/dist-packages/dropbox/dropbox.py", line 272, in request
timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/dropbox/dropbox.py", line 363, in request_json_string_with_retry
timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/dropbox/dropbox.py", line 407, in request_json_string
type(request_binary))
TypeError: expected request_binary as binary type, got <class 'str'>
Thanks in advance.
You need to supply bytes, but you're supplying str.
You can get bytes by changing the file mode to binary. I.e., instead of:
f = open(sourcefile, "r")
do:
f = open(sourcefile, "rb")