I used requests module and extracted a zip from a server.
I need to extract it, and store it in a folder.
I create a sample directory and extract the file using :
prep_zip = zipfile.ZipFile(BytesIO(response.content))
prep_zip.extractall(final_directory)
print(response.status_code)
print('The library '+name+'has been extracted in '+final_directory)
## final_directory = C:\\Users\XXXX\PycharProjects\Project1\ctl
When doing the above, all files are cleanly extracted and I can see the content inside ctl.
However, I want to create a subdirectory within ctl. eg: ctlA.
So the folder structure is: C:\\Users\XXXX\PycharProjects\Project1\ctl\ctlA\
I want to extract all files to the new path
another_directory=os.path.join(final_directory, r'ctlA')
prep_zip = zipfile.ZipFile(BytesIO(response.content))
prep_zip.extractall(another_directory)
print(response.status_code)
print('The library '+name+'has been extracted in '+final_directory)
This gives rise to a failure:
Error message :
File "C:\XXXX\Python38\lib\zipfile.py", line 1647, in extractall
self._extract_member(zipinfo, path, pwd)
File "C:\XXXX\Python38\lib\zipfile.py", line 1701, in _extract_member
open(targetpath, "wb") as target:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\XXXX\\PycharmProjects\\Project1\\ctl\\ctlA\\resources\\project\\Root.type.Files\\lib.type.File\\use-Lib.type.File\\{762f94cf-ed27-43e9-a0eb-39bbe5712253}.type.File\\Code Cache.type.File\\js.type.File\\index-dir.type.File\\1.type.DIR_SIGNIFIER.xml'
How is adding another folder to the path, failing the extract method?
Related
Problem statement:
While automatically copying files between input directories, and output directories my program fails on a path that contains unicode (most likely Korean) characters.
The whole script is publicly available under: This Link
The file that causes the error is also publicly available: File That Causes the Error
The specific part of the code that fails seems to be:
for root, _, filenames in os.walk(maybe_dir):
for file in filenames:
# Prepare relative paths:
relative_dir = os.path.relpath(root, maybe_dir)
relative_file = os.path.join(relative_dir, file)
# Get unique filename:
unique_filename = uuid.uuid4().hex
unique_filename_with_ext = unique_filename + file_extension
new_path_and_filename = os.path.join(
full_output_path, unique_filename_with_ext
)
current_file = os.path.abspath(os.path.join(root, file))
# Copying files:
shutil.copy(current_file, new_path_and_filename)
The error:
Traceback (most recent call last):
File "F:\Projects\SC2DatasetPreparator\src\directory_flattener.py", line 96, in <module>
directory_flattener(
File "F:\Projects\SC2DatasetPreparator\src\directory_flattener.py", line 60, in directory_flattener
shutil.copy(current_file, new_path_and_filename)
File "D:\Programs\Python3_10\lib\shutil.py", line 417, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "D:\Programs\Python3_10\lib\shutil.py", line 254, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'F:\\Projects\\SC2DatasetPreparator\\processing\\directory_flattener\\input\\2017_IEM_XI_World_Championship_Katowice\\IEM XI - World Championship - StarCraft II Replays\\RO24\\Group A\\Solar Vs herO\\Ùë¦ý+ñÝü¼ ý×¼Û¦£Ù¦£ ýºÇÛÁ¼ - ÝåáÙäêÙ¿+Ýè© (Û¦ÁÝùêýØÿ ý£áýé¦) (2) Solar vs Hero game 1.SC2Replay'
The error itself is unexpected as I am using absolute paths and the script works for 6 other directories before it fails on that specific file.
The path clearly exists and can be accessed manually:
Steps to attempt to reproduce the error are as follows:
Clone the repository: Branch 1.1.0_testing
Place the File That Causes the Error in ./processing/directory_flattener/input/test_dir
Run the script
Closing Remarks:
It seems that the script worked before on Python 3.7 because I have verified the output that I have received before updating to Python 3.10 and within the directory mapping that is created the files with unicode characters in their path are present:
{
"ce2f4610891e472190a0852c617b35e8": "RO24\\Group A\\Solar Vs herO\\\u00d9\u00eb\u00a6\u00fd+\u00f1\u00dd\u00fc\u00bc \u00fd\u00d7\u00bc\u00db\u00a6\u00a3\u00d9\u00a6\u00a3 \u00fd\u00ba\u00c7\u00db\u00c1\u00bc - \u00dd\u00e5\u00e1\u00d9\u00e4\u00ea\u00d9\u00bf+\u00dd\u00e8\u00a9 (\u00db\u00a6\u00c1\u00dd\u00f9\u00ea\u00fd\u00d8\u00ff \u00fd\u00a3\u00e1\u00fd\u00e9\u00a6) (2) Solar vs Hero game 1.SC2Replay",
"dcc82d633910479c95d06ef418fcf2e0": "RO24\\Group A\\Solar Vs herO\\\u00fd\u00fb\u00a6\u00d9\u00a6\u00e4\u00fd\u00e4\u00f1 \u00d9\u00aa\u00bc\u00dd\u00f6\u00e4 - \u00d9\u00d7\u00ff\u00d9\u00ec\u00f6 Solar vs Hero game 2.SC2Replay",
}
While searching for an answer I have only stumbled upon similar problems in Python 2 where the .decode() method was suggested as a solution. Applying such measures did not help to solve the issue.
The cause for this error was the Maximum Path Length Limitation which limited the ability to use paths longer than 260 characters on Windows.
The error was fixed by adding a prefix of "\\?\" to the path that was used to access and copy the file.
This means that the following line of code was changed:
current_file = os.path.abspath(os.path.join(root, file))
Into this:
current_file = "\\\\?\\" + os.path.abspath(os.path.join(root, file))
I'm trying to import data from a csv file saved on my hard drive into a database that I created. However when I go to run the code to iterate through the file and add to the database, I get the following error:
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:/Users/ajb56/Documents/Programming/CGS_sales_database.csv'
The file will open using the designated path, but i'm unsure why python is saying it is invalid.
Here is the code snippet where I define the directory and attempt to use the file in the for loop. The error I get is on the on the line with the for loop.
localFilePath = "C:/Users/ajb56/Documents/Programming/CGS_sales_database.csv"
for file in os.listdir(localFilePath):
if file.endswith(".csv"):
insertRows(localFilePath+ "/" + file, conn)
You set localFilePath to a file name but then try to use it as a directory name. I don't know your intent, but you should either remove the file name to get all CSVs in the directory
import os
localFilePath = "C:/Users/ajb56/Documents/Programming"
for file in os.listdir(localFilePath):
if file.endswith(".csv"):
insertRows(os.path.join(localFilePath, file), conn)
or skip the loop completely
localFilePath = "C:/Users/ajb56/Documents/Programming/CGS_sales_database.csv"
insertRows(localFilePath conn)
How to upload csv file to Dropbox with Python
I tried all the examples in this post bellow, neither works
upload file to my dropbox from python script
I am getting error:
FileNotFoundError: [Errno 2] No such file or directory: 'User\pb\Automation\test.csv'
My username: pb
Folder name: Automation
file name: test.csv
import pathlib
import dropbox
import re
# the source file
folder = pathlib.Path("User/pb/Automation") # located in folder
filename = "test.csv" # file name
filepath = folder / filename # path object, defining the file
# target location in Dropbox
target = "Automation" # the target folder
targetfile = target + filename # the target path and file name
# Create a dropbox object using an API v2 key
token = ""
d = dropbox.Dropbox(token)
# open the file and upload it
with filepath.open("rb") as f:
# upload gives you metadata about the file
# we want to overwite any previous version of the file
meta = d.files_upload(f.read(), targetfile, mode=dropbox.files.WriteMode("overwrite"))
# create a shared link
link = d.sharing_create_shared_link(targetfile)
# url which can be shared
url = link.url
# link which directly downloads by replacing ?dl=0 with ?dl=1
dl_url = re.sub(r"\?dl\=0", "?dl=1", url)
print (dl_url)
FileNotFoundError: [Errno 2] No such file or directory: 'User\\pb\\Automation\\test.csv'
The error message is indicating that you're supplying a local path of 'User\pb\Automation\test.csv' but nothing was found at that path on your local filesystem.
Based on the path format, it looks like you're on macOS, but you have the wrong path for accessing your home folder. The path should start with "/", and the home folders are located under "Users" (not "User"), so your folder definition should probably be:
folder = pathlib.Path("/Users/pb/Automation")
Or, use pathlib.Path.home() to automatically expand the home folder for you:
pathlib.Path.home() / "Automation"
I'm slowly starting to study about python and wanted to write simple script which use tinify api, takes photos from unoptimalized directiory, compress it and put to optimalized directory. So far works partly, I mean weirdly I need to keep photos in main directory and unoptimalized one. If I dont have another copy in one of these directories, I have error. another thing is that after I launch script, only first photo is compressed and put inside optimalized directory, and then error appears.
So far I'm experimenting on two photos: lew.jpg and kot.jpg
My directory structure is like this:
Main root directory with script, and two other directories inside (optimalized and unoptimalized)
def optimalizeImages():
for fname in os.listdir('./unoptimalized'):
if fname.endswith(".jpg"):
print(fname)
source = tinify.from_file(fname)
print("processing", fname)
os.chdir("./optimalized")
source.to_file("optimalized-" + fname)
print("changed", fname)
optimalizeImages()
Error after processing first image:
Traceback (most recent call last):
File "python.py", line 20, in <module>
optimalizeImages()
File "python.py", line 11, in optimalizeImages
source = tinify.from_file(fname)
File "/home/grzymowiczma/.local/lib/python2.7/site-packages/tinify/__init__.py", line 79, in from_file
return Source.from_file(path)
File "/home/grzymowiczma/.local/lib/python2.7/site-packages/tinify/source.py", line 13, in from_file
with open(path, 'rb') as f:
IOError: [Errno 2] No such file or directory: 'kot.jpg'
and if i keep photos only in root directory, no error but also no any action, if i keep them only in unoptimalized i get same error:
Traceback (most recent call last):
File "python.py", line 20, in <module>
optimalizeImages()
File "python.py", line 11, in optimalizeImages
source = tinify.from_file(fname)
File "/home/grzymowiczma/.local/lib/python2.7/site-packages/tinify/__init__.py", line 79, in from_file
return Source.from_file(path)
File "/home/grzymowiczma/.local/lib/python2.7/site-packages/tinify/source.py", line 13, in from_file
with open(path, 'rb') as f:
IOError: [Errno 2] No such file or directory: 'lew.jpg'
There are two issues here, both because you are using relative filenames. os.listdir() gives you those relative filenames, without a path. Instead of ./unoptimalized/kot.jpg, you get just kot.jpg.
So when you try to load the image:
source = tinify.from_file(fname)
you tell the library to load image.jpg without any context other than the current working directory. And that's not the right directory, not if os.listdir('./unoptimalized') worked to list all image files; that indicates that the current working directory is in the parent directory of both unoptimalized and optimalized.
You solved that by putting an image file with the same name in the parent directory, but that's not quite the right way to solve this. More on this below.
The next issue is that you change the working directory:
os.chdir("./optimalized")
You do this for the first image, so now the current working directory has changed to optimalized. When you then loop back up for the next file, you are now in the wrong location altogether to read the next file. Now lew.jpg, which might exist in ./unoptimalized or the parent directory, can't be found because it is not there in ./optimalized.
The solution is to add on the directory name. You can do so with os.path.join(), and not changing directories:
def optimalizeImages():
input_dir = './unoptimalized'
output_dir = './optimalized'
for fname in os.listdir(input_dir):
if fname.endswith(".jpg"):
print(fname)
source = tinify.from_file(os.path.join(input_dir, fname))
print("processing", fname)
source.to_file(os.path.join(output_dir, "optimalized-" + fname))
print("changed", fname)
Note that this still depends heavily on the current working directory being correct, but at least it is now stable and stays at the parent directory of both optimalized and unoptimalized. You may want to change that to using absolute paths.
And a side note on language: In English, the correct spelling of the two terms is optimized and unoptimized. I didn't correct those in my answer to keep the focus on what is going wrong.
This is one of my first python scripts designed to go into a directory and rename all the files of a certain extension to the same thing. For instance, go into a music directory and change the name of all the album artwork files to something like folder.jpg so that they're easily recognized and displayed by music playing software like foobar2000.
I'm currently getting the error in the title. I've tried:
os.chmod(pathname, 0777)
and
os.chmod(pathname, stat.S_IWOTH)
to allow other entities to edit the directory I'm looking at, but the error remains. I'm not too familiar with Windows's permissions system or with Python so this is mystifying me. Any idea where I'm going wrong?
#Changes file names in every subdirectory of the target directory
import os
import sys
import shutil
#Get target path
pathname = raw_input('Enter path for music directory (ex. C:\\Music): ')
try:
os.chdir(pathname)
except:
print('Failed. Invalid directory?')
sys.exit()
#Get variables for renaming
fn = raw_input('Enter desired file name for all converted files: ')
ft = raw_input('Enter the file extension you want the program to look for (ex. .jpg): ')
outname = fn + ft
#Create path tree
for path, subdirs, files in os.walk (pathname):
#Search tree for files with defined extention
for name in files:
if name.lower().endswith(ft):
#Rename the files
src = os.path.join(path, name)
print (src)
dst = os.path.join(path, outname)
print dst
shutil.move(src, dst)
print('Complete')
A second more benign issue is that when I use a print statement to check on what files are being edited, it seems that before the first .jpg is attempted, the program processes and renamed a file called d:\test\folder.jpg which doesn't exist. This seems to succeed and the program fails in the last loop the second time through, when an existing file is processed. The program runs like this:
>>> ================================ RESTART ================================
>>>
Enter path for music directory (ex. C:\Music): d:\test
Enter desired file name for all converted files: folder
Enter the file extension you want the program to look for (ex. .jpg): .jpg
d:\test\folder.jpg
d:\test\folder.jpg
d:\test\Anathema\Alternative 4\AA.jpg
d:\test\Anathema\Alternative 4\folder.jpg
Traceback (most recent call last):
File "D:\Documents\Programming\Python scripts\Actually useful\bulk_filename_changer.py", line 29, in <module>
shutil.move(src, dst)
File "C:\Python27\lib\shutil.py", line 301, in move
copy2(src, real_dst)
File "C:\Python27\lib\shutil.py", line 130, in copy2
copyfile(src, dst)
File "C:\Python27\lib\shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: 'd:\\test\\Anathema\\Alternative 4\\folder.jpg'
>>>
The glitch seems to be benign because it doesn't cause an error, but it might be related to an error I have in the code that leads to the IOError I get when the first "real" file is processed. Beats me...
I am assuming that d: is not a CD drive or something not writable. If so then have you checked the properties of your files? Are any of them read only? There would also need to be more than one of the file type in the folder to cause this, I believe.