We Keep Coding

Creating a directory using part of a variable name

New to python and Im trying to create a set of directories, and create a file in each one, the input name will be a string starting with a > sign but I don't want the directory to contain the >.
I've tried to do the following;
seq_id = ">seq"
dirname = seq_id[1:]
print(dirname)
if not os.path.isdir('./' + dirname + '/'):
os.mkdir('./' + dirname + '/')
print("directory made")
It will not make the directory when I used the seq_id[1:] bit but it will print it. So I don't really get why it won't create the directory.
I ultimately want to build a function that would take a list of seq_ids from a file, >seq1 >seq2 >seq3 etc and create a directory for each one.
(Working with python3.5)

You must know the directories will be created at your current directory position.
If you are in folder /tmp/foo/, your script in /tmp/bar/myscript.py and you execute it with python ../bar/myscript.py the directories will be created in /tmp/foo/, not /tmp/bar/.
More of that, because you just skip possible errors with if not os.path.isdir('./' + dirname + '/'): and never prints nothing, you will not know if directory already exists.
You could do something like:
import os
strings = ['>foo', '>bar', '>baz']
def make_directories(input_list):
for string in input_list:
dirpath = os.path.join('./', string[1:])
try:
os.mkdir(dirpath)
except FileExistsError:
print('Directory {} already exists'.format(dirpath))
else:
print('Directory {} created'.format(dirpath))
make_directories(strings)
It uses os.path.join instead of homemade concatenation. It's a best practice you should always follow.
It uses try / except instead of if, again it's a best practice you should follow (Reference)
It prints things, so you know what is going on.

Spaces in directory path python

I'm a noob at coding Python and I've run into something that no amount of Googling is helping me with.
I'm trying to write a simple Directory listing tool and I cannot seem to deal with Spaces in the directory name in OSX.
My code is as follows:
def listdir_nohidden(path):
import os
for f in os.listdir(path):
if not f.startswith('.'):
yield f
def MACListDirNoExt():
import os
MACu = PCu = os.environ['USER']
MACDIR = '/Users/'+MACu+'/Desktop//'
while True:
PATH = raw_input("What is the PATH you would like to list?")
if os.path.exists(PATH):
break
else:
print "That PATH cannot be found or does not exist."
NAME = raw_input ("What would you like to name your file?")
DIR = listdir_nohidden(PATH)
DIR = [os.path.splitext(x)[0] for x in DIR]
f = open(''+MACDIR+NAME+'.txt', "w")
for file in DIR:
f.write(str(file) + "\n")
f.close()
print "The file %s.txt has been written to your Desktop" % (NAME)
raw_input ("Press Enter to exit")
For ease of trouble shooting though I think this could essentially be boiled down to:
import os
PATH = raw_input("What is the PATH you would like to list")
os.listdir(PATH)
When supplying a directory path that contains spaces /Volumes/Disk/this is a folder it returns
"No such file or Directory: '/Volumes/Disk/this\\ is\\ a\\ folder/'
It looks like its escaping the escape...?

Check the value returned from raw_input() for occurences of '\\' and replace them with ''.
a = a.replace('\\', '')

I just ran into this, and I'm guessing that what I was hastily doing is also what you were trying. In a way, both #zwol and #trans1st0r are right.
Your boiled down program has nothing wrong with it. I believe that if you put in the input /Volumes/Disk/this is a folder, everything would work fine.
However, what you may have been doing (or at least, what I was doing) is dragging a folder from the Finder to the Terminal. When you drag to the Terminal, the OS automatically escapes spaces for you, so what ends up getting typed into the Terminal is /Volumes/Disk/this\ is\ a\ folder.
So either you can make sure that what you "type in" doesn't have those backslashes, or you can use #trans1st0r's suggestion as a way to support the dragging functionality, though the latter will cause issues in the edge case that your desired path actually has backslashes in it.

How to process files from one subfolder to another in each directory using Python?

I have a basic file/folder structure on the Desktop where the "Test" folder contains "Folder 1", which in turn contains 2 subfolders:
An "Original files" subfolder which contains shapefiles (.shp).
A "Processed files" subfolder which is empty.
I am attempting to write a script which looks into each parent folder (Folder 1, Folder 2 etc) and if it finds an Original Files subfolder, it will run a function and output the results into the Processed files subfolder.
I made a simple diagram to showcase this where if Folder 1 contains the relevant subfolders then the function will run; if Folder 2 does not contain the subfolders then it's simply ignored:
I looked into the following posts but having some trouble:
python glob issues with directory with [] in name
Getting a list of all subdirectories in the current directory
How to list all files of a directory?
The following is the script which seems to run happily, annoying thing is that it doesn't produce an error so this real noob can't see where the problem is:
import os, sys
from os.path import expanduser
home = expanduser("~")
for subFolders, files in os.walk(home + "\Test\\" + "\*Original\\"):
if filename.endswith('.shp'):
output = home + "\Test\\" + "\*Processed\\" + filename
# do_some_function, output

I guess you mixed something up in your os.walk()-loop.
I just created a simple structure as shown in your question and used this code to get what you're looking for:
root_dir = '/path/to/your/test_dir'
original_dir = 'Original files'
processed_dir = 'Processed files'
for path, subdirs, files in os.walk(root_dir):
if original_dir in path:
for file in files:
if file.endswith('shp'):
print('original dir: \t' + path)
print('original file: \t' + path + os.path.sep + file)
print('processed dir: \t' + os.path.sep.join(path.split(os.path.sep)[:-1]) + os.path.sep + processed_dir)
print('processed file: ' + os.path.sep.join(path.split(os.path.sep)[:-1]) + os.path.sep + processed_dir + os.path.sep + file)
print('')
I'd suggest to only use wildcards in a directory-crawling script if you are REALLY sure what your directory tree looks like. I'd rather use the full names of the folders to search for, as in my script.
Update: Paths
Whenever you use paths, take care of your path separators - the slashes.
On windows systems, the backslash is used for that:
C:\any\path\you\name
Most other systems use a normal, forward slash:
/the/path/you/want
In python, a forward slash could be used directly, without any problem:
path_var = '/the/path/you/want'
...as opposed to backslashes. A backslash is a special character in python strings. For example, it's used for the newline-command: \n
To clarify that you don't want to use it as a special character, but as a backslash itself, you either have to "escape" it, using another backslash: '\\'. That makes a windows path look like this:
path_var = 'C:\\any\\path\\you\\name'
...or you could mark the string as a "raw" string (or "literal string") with a proceeding r. Note that by doing that, you can't use special characters in that string anymore.
path_var = r'C:\any\path\you\name'
In your comment, you used the example root_dir = home + "\Test\\". The backslash in this string is used as a special character there, so python tries to make sense out of the backslash and the following character: \T. I'm not sure if that has any meaning in python, but \t would be converted to a tab-stop. Either way - that will not resolve to the path you want to use.
I'm wondering why your other example works. In "C:\Users\me\Test\\", the \U and \m should lead to similar errors. And you also mixed single and double backslashes.
That said...
When you take care of your OS path separators and trying around with new paths now, also note that python does a lot of path-concerning things for you. For example, if your script reads a directory, as os.walk() does, on my windows system the separators are already processed as double backslashes. There's no need for me to check that - it's usually just hardcoded strings, where you'll have to take care.
And finally: The Python os.path module provides a lot of methods to handle paths, seperators and so on. For example, os.path.sep (and os.sep, too) wil be converted in the correct seperator for the system python is running on. You can also build paths using os.path.join().
And finally: The home-directory
You use expanduser("~") to get the home-path of the current user. That should work fine, but if you're using an old python version, there could be a bug - see: expanduser("~") on Windows looks for HOME first
So check if that home-path is resolved correct, and then build your paths using the power of the os-module :-)
Hope that helps!

How to substitute spaces with underscore in a dir using ipython?

Sometimes it is useful to substitute spaces with underscore. On linux machine, this works fine for me:
find /tmp/ -depth -name "* *" -execdir rename 's/ /_/g' "{}" \;
On windows, I'd like to do it using ipython. I think many people might have met this problem, how do you implement it in ipython?
Thank you!
Edit:
My apologize for the misunderstanding. Here is my script:
import os
def rm_space():
for filename in os.listdir("."):
if filename.find(" ") > 0:
newfilename = filename.replace(" ", "_")
os.rename(filename, newfilename)
This piece of code does substitute the spaces with underscore; however, there's a problem: How to substitute recursively?
I think this is a very common problem, that there might be already an idiomatic way to solve it, (just like the shell script above).

Code copy from stackoverflow and edit, works on my Mac.
import os
import sys
directory = sys.argv[1] # parse through file list in the current directory
for filename in os.listdir(directory): # parse through file list in the current directory
if filename.find(" ") > 0: # if an space is found
newfilename = filename.replace(" ","_") # convert underscores to space's
old_file_path = os.path.join(directory, filename)
new_file_path = os.path.join(directory, newfilename)
os.rename(old_file_path, new_file_path) # rename the file, note that arg[] of os.rename is path_of_file, that explains 2 lines of code above

How to substitute recursively?
Use OS.walk() rather than listdir

Python: Dynamically add relative path, based on OS

I wrote my first program in Python for my dad to convert about 1000 old AppleWorks files that he has (the AppleWorks format, .cwk, is no longer supported) to .docx. To clarify, the program does not actually convert anything, all it does is copy/paste whatever text is in the documents you specify to another document of any file-type you want.
The program works fine on my Windows Laptop, however it encounters problems in my dad's Mac laptop.
File-path in Windows is indicated with a \ whereas in Mac it's /. So when the program reaches the copy and paste variables it stops working if the slash in the respective string is the wrong way around.
Is there a way to get Python to dynamically add on the Input and Output folders to my copy and paste variables depending on the OS without the use of strings?
If there are any other improvements you can see, feel free to say so, I am interested in releasing this as Freeware, possibly with the tKinter GUI and want to make as user friendly as possible.
As it stands the program does have some problems (converting apostrophe's into omega symbols and the like). Feel free to try the program and see if you can improve it.
import os, os.path
import csv
from os import listdir
import sys
import shutil
path, dirs, files = os.walk(os.getcwd() + '/Input').next()
file_count = len(files)
if file_count > 0:
print "There are " + str(file_count) + " files you have chosen to convert."
else:
print "Please put some files in the the folder labelled 'Input' to continue."
ext = raw_input("Please type the file extension you wish to convert to, making sure to preceed your selection with '.' eg. '.doc'")
convert = raw_input("You have chosen to convert " + str(file_count) + " files to the " + ext + " format. Hit 'Enter' to continue.")
if convert == "":
print "Converter is now performing selected tasks."
def main():
dirList = os.listdir(path)
for fname in dirList:
print fname
# opens files at the document_input directory.
copy = open(os.getcwd() + "\Input\\" + fname, "r")
# Make a file called test.docx and stick it in a variable called 'paste'
paste = open(os.getcwd() + "\Output\\" + fname + ext, "w+")
# Place the cursor at the beginning of 'copy'
copy.seek(0)
# Copy all the text from 'copy' to 'paste'
shutil.copyfileobj(copy,paste)
# Close both documents
copy.close()
paste.close()
if __name__=='__main__':
main()
else:
print "Huh?"
sys.exit(0)
Please let me know if I wasn't clear or left out some info...

Use os.path.join
For example, you could get the path of the Input subdirectory with
path = os.path.join(os.getcwd(), 'Input')

os.path.join is the platform-independent way to join paths:
>>> os.path.join(os.getcwd(), 'Input', fname)
'C:\\Users\\Blender\\Downloads\\Input\\foo.txt'
And on Linux:
>>> os.path.join(os.getcwd(), 'Input', fname)
'/home/blender/Downloads/Input/foo.txt'