We Keep Coding

For loop over multiple files in a folder

I'm attempting to extract a single value from each file contained in folder A. The code runs without throwing an error but returns an empty array for Mfinal. Does anyone see where things might be going wrong?
Mfinal=[]
path = r'C:Desktop/thesis/hrfiles/A'
all_files = glob.glob(path + '/*.csv')
for filename in all_files:
df=pd.dataframe(filename)
mass=df[9]
m=mass[-1]
Mfinal.append(m)

Even if m == None, Mfinal cannot be empty if for loops without any errors.
So, the reasonable suspicion here is that all_files is empty.
Thus, nothing is found by glob.glob.
If you are on Window, try
path = r'C:\Desktop\thesis\hrfiles\A'
all_files = glob.glob(path + '\*.csv')

I agree with ghchoi on the initial path here. Fairly sure there needs to be \ following C:.

Use Python to Create Folders Recursively with Folder Names Matching Filenames

Is there a Python method to create directories recursively? I have this path:
/home/data/ with files 'table1.csv', 'table2.csv', ... , 'table1000.csv' in it
I would like to create:
/home/data/table1 and move 'table1.csv' in it;
/home/data/table2 and move 'table2.csv' in it;
.
.
.
/home/data/table1000 and move 'table1000.csv' in it;
The folder names would have to match the csv file names.
How can I do it recursively? I know os.makedirs() should probably be used not sure how it works exactly.
Many thanks.
NOTE: 'table1' and 'table2' etc are just dummy example file names. The real filenames are a bit complex.

I would work the following way in Python:
1.Get all files in folder in a list
2.Loop Through the filenames of the list and:
Create Folder with the exact name (provided that you do not have duplicates)
Move file in the folder
Next File
A simple search in the net would get you ready examples on how to do each step of the above.
Edit: Below a simple example for csv files.
import glob, os
import shutil
dir="D:\Dropbox\MYDOCS\DEV\python\snippets"
os.chdir(dir)
for file in glob.glob("*.csv"):
dst=dir+"\\"+file.replace(" ","_").replace(".csv","")
os.mkdir(dst)
print(dst)
shutil.move(file,dst)
Used windows paths, since I use windows, you ll need to change that to linux paths.

Use mkdir to create each dir, from the os library.
https://docs.python.org/2/library/os.html
For each dir move the current file using shutil.move.
How to move a file in Python
Each iteration should look like this:
for i in range(1, 1001):
os.mkdir('/home/data/table' + str(i))
shutil.move('/home/data/table' + str(i) + '.csv', '/home/data/table' + str(i) + '/table' + str(i) + '.csv')

Going through all folders in Python

I want to go through all folders inside a directory:
directory\
folderA\
a.cpp
folderB\
b.cpp
folderC\
c.cpp
folderD\
d.cpp
The name of the folders are all known.
Specifically, I am trying to count the number of lines of code on each of the a.cpp, b.cpp, c.pp and d.cpp source files. So, go inside folderA and read a.cpp, count lines and then go back to directory, go inside folderB, read b.cpp, count lines etc.
This is what I have up until now,
dir = directory_path
for folder_name in folder_list():
dir = os.path.join(dir, folder_name)
with open(dir) as file:
source= file.read()
c = source.count_lines()
but I am new to Python and have no idea if my approach is appropriate and how to proceed. Any example code shown will be appreciated!
Also, does the with open handles the file opening/closing as it should for all those reads or more handling is required?

I would do it like this:
import glob
import os
path = 'C:/Users/me/Desktop/' # give the path where all the folders are located
list_of_folders = ['test1', 'test2'] # give the program a list with all the folders you need
names = {} # initialize a dict
for each_folder in list_of_folders: # go through each file from a folder
full_path = os.path.join(path, each_folder) # join the path
os.chdir(full_path) # change directory to the desired path
for each_file in glob.glob('*.cpp'): # self-explanatory
with open(each_file) as f: # opens a file - no need to close it
names[each_file] = sum(1 for line in f if line.strip())
print(names)
Output:
{'file1.cpp': 2, 'file3.cpp': 2, 'file2.cpp': 2}
{'file1.cpp': 2, 'file3.cpp': 2, 'file2.cpp': 2}
Regarding the with question, you don't need to close the file or make any other checks. You should be safe as it is now.
You may, however, check if the full_path exists as somebody (you) could mistakenly delete a folder from your PC (a folder from list_of_folders)
You can do this by os.path.isdir which returns True if the file exists:
os.path.isdir(full_path)
PS: I used Python 3.

Use Python 3's os.walk() to traverse all subdirectories and files of a given path, opening each file and do your logic. You can use a 'for' loop to walk it, simplifying your code greatly.
https://docs.python.org/2/library/os.html#os.walk

As manglano said, os.walk()
you can generate a list of folder.
[src for src,_,_ in os.walk(sourcedir)]
you can generate a list of file path.
[src+'/'+file for src,dir,files in os.walk(sourcedir) for file in files]

How does one rename multiple files using python?

I have lots of programming experience but this is my first python script. I am trying to add the prefix "00" to all the files in a specific folder. First I read the names of all the files and save them in an array. Then I sort through the array and add the prefix "00" then use the os.rename function but somewhere along the way I've messed up something.
import sys, os
file_list = []
for file in os.listdir(sys.argv[1]):
file_list.append(file)
for i in file_list:
file_list[i] = prevName
newName = '00' + file_list[i]
os.rename(prevName, newName)
I have a .py file in the folder with all the files I want to rename. The .py file contains the script above. When i double click the .py file a cmd window flashes and disappears and none of the file names have been changed. Any help would be appreciated, sorry if this is a very obvious mistake, my python level is quite n00b at the moment.

In addition to the answer by #Padraic, also make following changes to your code.
import sys, os
file_list = []
for f in os.listdir(sys.argv[1]):
file_list.append(f)
for i in range(len(file_list)):
prevName = file_list[i]
if prevName != 'stackoverflow.py': # Mention .py file so that it doesnt get renamed
newName = '00' + file_list[i]
os.rename(prevName, newName)

Check your indentation. The second for loop is not indented correctly.
for i in file_list:
file_list[i] = prevName
You are not iterating correctly. for loops in Python are like foreach loops you may know from other programming languages. i in for i in file_list actually gives you the list's elements, so you should be doing
for i in range(len(file_list)):
file_list[i] = ......
although it is not very pythonic nor generally a good idea to modify the collection that you're currently iterating over.

Your code errors because you provide no args so sys.argv[1] would give an IndexError, you would need to call the script with the dir name from a cmd prompt not double click it:
python your_script directory <- argv[1]
Or change the code and specify the path, you also need to join the path to the filename.
path = "full_path"
for f in os.listdir(path):
curr,new = os.path.join(path,f), os.path.join(path,"00{}".format(f))
os.rename(curr,new)
os.listdir returns a list so just iterate over that, you don't need to create a list and append to it.
for i in file_list: would also make each i a filename not an index so that would cause another error but as above you don't need to do it anyway.

Making a new file in subdirectories using python

I am trying to create a file in a subdirectory, both of which will not exist when the program is first run. When I do this:
newfile = open('abc.txt','w')
It will create abc.txt just fine but the following will cause an error, saying the file or directory does not exist
newfile = open('folder/abc.txt','w')
I tried using os.makedirs to create the directory first but that failed as well raising the same error. What is the best way to create both the folder and file?
Thanks

>>> import os
>>> os.makedirs('folder')
>>> newfile = open('folder' + os.sep + 'abc.txt', 'w')
>>> newfile.close()
>>> os.listdir('folder')
['abc.txt']
This works for me

A couple of things to check:
os.makedirs takes just the path, not the file. I assume you know this already, but you haven't shown your call to makedirs so I thought I'd mention it.
Consider passing an absolute, not a relative, path to makedirs. Alternatively, use os.chdir first, to change to a directory in which you know you have write permission.
Hope those help!