Currently, I have the following code...
file_name = content.split('=')[1].replace('"', '') #file, gotten previously
fileName = "/" + self.feed + "/" + self.address + "/" + file_name #add folders
output = open(file_name, 'wb')
output.write(url.read())
output.close()
My goal is to have python write the file (under file_name) to a file in the "address" folder in the "feed" folder in the current directory (IE, where the python script is saved)
I've looked into the os module, but I don't want to change my current directory and these directories do not already exist.
First, I'm not 100% confident I understand the question, so let me state my assumption:
1) You want to write to a file in a directory that doesn't exist yet.
2) The path is relative (to the current directory).
3) You don't want to change the current directory.
So, given that:
Check out these two functions: os.makedirs and os.path.join. Since you want to specify a relative path (with respect to the current directory) you don't want to add the initial "/".
dir_path = os.path.join(self.feed, self.address) # will return 'feed/address'
os.makedirs(dir_path) # create directory [current_path]/feed/address
output = open(os.path.join(dir_path, file_name), 'wb')
This will create the file feed/address/file.txt in the same directory as the current script:
import os
file_name = 'file.txt'
script_dir = os.path.dirname(os.path.abspath(__file__))
dest_dir = os.path.join(script_dir, 'feed', 'address')
try:
os.makedirs(dest_dir)
except OSError:
pass # already exists
path = os.path.join(dest_dir, file_name)
with open(path, 'wb') as stream:
stream.write('foo\n')
Commands like os.mkdir don't actually require that you make the folder in your current directory; you can put a relative or absolute path.
os.mkdir('../new_dir')
os.mkdir('/home/you/Desktop/stuff')
I don't know of a way to both recursively create the folders and open the file besides writing such a function yourself - here's approximately the code in-line. os.makedirs will get you most of the way there; using the same mysterious self object you haven't shown us:
dir = "/" + self.feed + "/" + self.address + "/"
os.makedirs(dir)
output = open(os.path.join(dir, file_name), 'wb')
Related
excerpt from my python script located at C:\Users\my_name\documents\python_projects\randomness\random.py :
some_number = 3452342
filename = str(some_number) + '.csv'
# file 3452342.csv is stored in C:\Users\my_name\documents\python_projects\randomness\history
# call a function that takes the filename as the parameter
my_func(r'history\filename')
It triggers the following error:
FileNotFoundError: [Errno 2] File b'history\filename' does not exist: b'history\filename'
what exactly is going wrong here?
How can I pass the filename to my_func when it is located in a sub-folder?
thanks in advance
The answer to your question what's going wrong:
Python tried to open the file with the name literally "filename" in the subdirectory named "history", which is doesn't exist. You should pass the filename variable's value instead as follows:
You should use os.path.join().
import os
some_number = 3452342
filename = str(some_number) + '.csv'
workdir = "C:\Users\my_name\documents\python_projects\randomness\history"
my_func(os.path.join(workdir, filename))
Or if the file 3452342.csv placed in a subfolder (called history) of the the main script's directory, then you can use:
import os
some_number = 3452342
filename = str(some_number) + '.csv'
my_func(os.path.join("history", filename))
Alternatively you can simply use string concatenation:
import os
some_number = 3452342
filename = str(some_number) + '.csv'
my_func("history/" + filename)
Another approach using Python's format():
import os
some_number = 3452342
filename = str(some_number) + '.csv'
my_func("history/{}".format(filename))
First try to get the current path then join the path you got with the name of your file.
import os
some_number = 3452342
filename = str(some_number) + '.csv'
path_file = os.path.join(os.getcwd(), filename)
my_func(path_file)
for more about how to work with path using python check out this.
Common pathname manipulations
First, to be platform independent you should use os.path.join to concatenate directories.
Second, like #k88 pointed out, you need to pass the variable filename into your path, not the string 'filename'
The clean way would be:
import os
some_number = 3452342
filename = str(some_number) + '.csv'
# file 3452342.csv is stored in C:\Users\my_name\documents\python_projects\randomness\history
# call a function that takes the filename as the parameter
my_func(os.path.join('history', filename))
Relative Or Absolute Path?
If your history subfolder is a fixed subfolder of your script's directory, you should even consider to determine your target filename as an absolute path like this (see also this answer with comments):
base_dir = os.path.dirname(os.path.abspath(__file__))
my_func(os.path.join(base_dir, 'history', filename))
I've tried to write some code which will rename some files in a folder - essentially, they're listed as xxx_(a).bmp whereas they need to be xxx_a.bmp, where a runs from 1 to 2000.
I've used the inbuilt os.rename function to essentially swap them inside of a loop to get the right numbers, but this gives me FileNotFoundError [WinError2] the system cannot find the file specified Z:/AAA/BBB/xxx_(1).bmp' -> 'Z:/AAA/BBB/xxx_1.bmp'.
I've included the code I've written below if anyone could point me in the right direction. I've checked that I'm working in the right directory and it gives me the directory I'm expecting so I'm not sure why it can't find the files.
import os
n = 2000
folder = r"Z:/AAA/BBB/"
os.chdir(folder)
saved_path = os.getcwd()
print("CWD is" + saved_path)
for i in range(1,n):
old_file = os.path.join(folder, "xxx_(" + str(i) + ").bmp")
new_file = os.path.join(folder, "xxx_" +str(i)+ ".bmp")
os.rename(old_file, new_file)
print('renamed files')
The problem is os.rename doesn't create a new directory if the new name is a filename in a directory that does not currently exist.
In order to create the directory first, you can do the following in Python3:
os.makedirs(dirname, exist_ok=True)
In this case dirname can contain created or not-yet-created subdirectories.
As an alternative, one may use os.renames, which handles new and intermediate directories.
Try iterating files inside the directory and processing the files that meet your criteria.
from pathlib import Path
import re
folder = Path("Z:/AAA/BBB/")
for f in folder.iterdir():
if '(' in f.name:
new_name = f.stem.replace('(', '').replace(')', '')
# using regex
# new_name = re.sub('\(([^)]+)\)', r'\1', f.stem)
extension = f.suffix
new_path = f.with_name(new_name + extension)
f.rename(new_path)
What I'm hoping to accomplish is to scan all the files in the specified directory and remove the numbers that is contained in the name. The problem is I can't change the directory with the code that is below.
I'm currently in the Udacity Full Stack Nanodegree Program so if anyone can help me out that is also in the program that would be a plus.
Here is the code:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
print("Our current working directory is ", saved_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, file_name.translate("0123457689"))
os.chdir(saved_path)
print("Our current working directory is ", saved_path)
rename_files()
Here is the output I get:
Our current working directory is /Users/bill/Documents/web/LocalServer
Our current working directory is /Users/bill/Documents/web/LocalServer
Our current working directory is /Users/bill/Documents/web/LocalServer
Update 1:
I've finally changed directories but I still can't rename files. (Ex: 68chicago.jpg to chicago.jpg)
Here is my current code:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
new_path = os.getcwd()
print("Our current working directory is ", new_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, file_name.translate("0123457689"))
print("Our current working directory is ", new_path)
rename_files()
Here is my current output:
Our current working directory is /Users/bill/Documents/web/LocalServer
Our current working directory is /Users/bill/Documents/web/LocalServer/prank
Our current working directory is /Users/bill/Documents/web/LocalServer/prank
Update 2:
I've finally solved the problem thanks to #Dan.
Here is his code:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
new_path = os.getcwd()
print("Our current working directory is ", new_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, ''.join([i for i in file_name if not i.isdigit()])) # This works on my machine
os.chdir(saved_path)
print("Our current working directory is ", saved_path)
rename_files()
It looks like you want to change to a dir and then change back to the first directory? If that's the case you want something like this:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
new_path = os.getcwd()
print("Our current working directory is ", new_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, ''.join([i for i in file_name if not i.isdigit()])) # This works on my machine
os.chdir(saved_path)
print("Our current working directory is ", saved_path)
rename_files()
Your problem is that you use .translate() incorrectly. It does not change the file names at all, so rename actually renames X into X:
'68chicago.jpg'.translate('0123457689')
# '68chicago.jpg'
Try using list comprehension:
def clean_name(name):
return ''.join(x for x in name if not x.isdigit())
# 'chicago.jpg'
Here's a version that doesn't require changing the current directory. Just specify the directory where files are to be renamed. If you specify the full path to the old and new name changing the directory isn't required.
This also shows how to use translate properly. maketrans takes one, two, or three parameters (see docs). The three-parameter version takes two strings of equal length for 1:1 translating, plus a third parameter of characters to delete. It returns a dictionary suitable to be used with translate.
import os
def rename_files(directory):
xlat = str.maketrans('','','0123457689')
file_list = os.listdir(directory)
for file_name in file_list:
old_name = os.path.join(directory,file_name)
new_name = os.path.join(directory,file_name.translate(xlat))
os.rename(old_name,new_name)
rename_files('/Users/bill/Documents/web/LocalServer/prank')
The primary problem is your code isn't using the str.translate() method in the proper manner—your need to pass it a translation table for it to work.
This following works and avoids renaming files that don't need to be (i.e. doesn't rename them to their current name). This takes a little extra processing, but is probably faster that performing a useless OS-level operation.
Note: A more robust solution would also need to handle any exceptions the os.rename() call might raise—which definitely could occur for a number of reasons.
For example:
Perhaps a file with the translated name already exists.
All the characters of the original file get removed, so the new file name is the empty string.
File permissions or attribute prevent renaming.
etc.
The code:
from contextlib import contextmanager
import string
import os
#contextmanager
def temp_cd(path):
"""Temporarily change the current directory to path, yield, then restore it."""
saved_path = os.getcwd()
yield
os.chdir(saved_path)
def rename_files(path):
TRANS_TABLE = str.maketrans(dict.fromkeys(string.digits))
with temp_cd(path):
for file_name in os.listdir(path):
if any(ch in file_name for ch in string.digits):
# os.rename(file_name, file_name.translate(TRANS_TABLE))
print(file_name, '->', file_name.translate(TRANS_TABLE))
rename_files("/Users/bill/Documents/web/LocalServer/prank")
I am trying to convert a url like "www.example.com/images/dog.png" into directories from the current directory.
So I get a folder named "www.example.com", inside that "images" and finally inside that the file saved as "dog.png"
I've tried using urllib.url2pathname(path) but it keeps appending P:\ to the start of it.
You can use os.makedirs() to create the directory tree, but that will fail if the final directory already exists. So you can test if it exists before attempting to create the directory tree, or use try: ... except OSError:. In Python 3 you can supply an exist_ok parameter to over-ride this behaviour, see the Python docs of os.makedirs for further info.
#!/usr/bin/env python
import os
cwd = os.getcwd()
url = "www.example.com/images/dog.png"
fullname = os.path.join(cwd, url)
path, basename = os.path.split(fullname)
if not os.path.exists(path):
os.makedirs(path)
with open(fullname, 'w') as f:
f.write('test\n')
If your system doesn't support directory names containing periods you can translate them to another character, eg _, like this:
fullname = fullname.replace('.', '_')
(just insert this after the fullname = os.path.join(cwd, url) line).
And as jwilner mentions in the comments, it's more efficient to use
path = os.path.dirname
than path, basename = os.path.split(fullname) if you don't need the base component of the file name (in this example "dog.png").
This is probably a simple question, but I'm brand new to python and programming in general.
I'm working on a simple program to copy/move .mp3 files from on location to another while mirroring the directory structure of the source location. What I have so far works, however it also creates new folders in the destination location even if the source folder contained no mp3 files. I only want to create the new directories if the source contains .mp3s, otherwise it could lead to a bunch of empty folders in the destination.
Here is what I have so far:
import os
import shutil #Used for copying files
##CONFIG
source_dir = "C:\Users\username\Desktop\iTunes\\" #set the root folder that you want to scan and move files from. This script will scan recursively.
destPath = "C:\Users\username\Desktop\converted From iTunes" #set the destination root that you want to move files to. Any non-existing sub directories will be created.
ext = ".mp3" #set the type of file you want to search for.
count = 0 #initialize counter variable to count number of files moved
##
##FIND FILES
for dirName, subdirList, fileList in os.walk(source_dir):
#set the path for the destination folder(s)
dest = destPath + dirName.replace(source_dir, '\\')
#if the source directory doesn't exist in the destination folder
#then create a new folder
if not os.path.isdir(dest):
os.mkdir(dest)
print('Directory created at: ' + dest)
for fname in fileList:
if fname.endswith(ext) :
#determine source & new file locations
oldLoc = dirName + '\\' + fname
newLoc = dest + '\\' + fname
if os.path.isfile(newLoc): # check to see if the file already exists. If it does print out a message saying so.
print ('file "' + newLoc + fname + '" already exists')
if not os.path.isfile(newLoc): #if the file doesnt exist then copy it and print out confirmation that is was copied/moved
try:
shutil.move(oldLoc, newLoc)
print('File ' + fname + ' copied.')
count = count + 1
except IOError:
print('There was an error copying the file: "' + fname + '"')
print 'error'
print "\n"
print str(count) + " files were moved."
print "\n"
so if the folder structure is something like:
root->
band 1->
album name->
song.m4a,
song2.m4a
right now it will create all those folders in the destination driectory, even though there are no .mp3s to copy.....
Any help is appreciated!
I think I would create my own wrapper around copy for this sort of thing:
def fcopy(src,dest):
"""
Copy file from source to dest. dest can include an absolute or relative path
If the path doesn't exist, it gets created
"""
dest_dir = os.path.dirname(dest)
try:
os.makedirs(dest_dir)
except os.error as e:
pass #Assume it exists. This could fail if you don't have permissions, etc...
shutil.copy(src,dest)
Now you can just walk the tree calling this function on any .mp3 file.
The simplest thing to do I can think of for your existing code would be to just make it skip over any folders that don't have any .mp3 files in them. This can easily be done by adding the following items and if statement to the top of your loop. The itertools.ifilter() and fnmatch.fnmatch() functions can be used together to simplify checking for files with the proper extension.
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
if not any(ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)):
print ' skipping "{}"'.format(dirName)
continue
...
You will also have to change the os.mkdir(dest) to os.makedirs(dest) in the code further down to ensure that any subdirectories skipped by earlier iterations get created when there's a need to copy files to a corresponding subbranch of the destination directory.
You could optimize things a bit by creating and saving a possibly empty iterator of matching files that have the extension, and then use it again later to to determine what files to copy:
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
# generate list of files in directory with desired extension
matches = ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)
# skip subdirectory if it does not contain any files of interest
if not matches:
continue
...
... create destination directory with os.makedirs()
...
# copy each file to destination directory
for fname in matches:
... copy file
Would shutils.copytree not do what you want in fewer lines?