We Keep Coding

Python: read folders that are not ZIP folders

Currently I read folders individually like so:
input_location = r'path\202206'
I would like to read in all the folders at once + not read in the zipped files.
Essentially the logic id like to perform is input_location = r'path\202206' + r'path\202207' + r'path\202207' + r'path\202208'
I cant just do input_location = r'path\ as it may read in those zip files I do not want.
Is there a way to read in the folders without reading in the zip files? Or explicitly list the folder names in one variable (input_location)?

IIUC: Collecting all directories from a directory can be done using a simple list comprehension as follows.
The glob library is nice, as it returns the full filepath, whereas a function such as os.listdir() only returns the filenames.
import os
from glob import glob
dirs = [f for f in glob('/path/to/files/*') if os.path.isdir(f)]
Output:
['/path/to/files/202207',
'/path/to/files/202206',
'/path/to/files/202208',
'/path/to/files/202209']
Then, your script can iterate over the list of directories as required.
For completeness, the directory content is a follows:
202206
202206.zip
202207
202207.zip
202208
202208.zip
202209
202209.zip

If you use pathlib from Pythons standard library you can get all entries in the folder and check if the entry is a folder.
from pathlib import Path
for entry in Path('/path/to/folder').glob('*'):
if entry.is_dir():
print(entry)

A os base approach. Notice that os.listdir returns the content of the directory in a basename form and not as a path.
import os
def my_dirs(wd):
return list(filter(os.path.isdir, (os.path.join(wd, f) for f in os.listdir('.'))))
working_dir = # add path
print(*my_dirs(working_dir), sep='\n')
Remarks: to make your program platform independent you always stuffs like os.path.join or os.sep for path manipulation

How to print the names of files from a folder?

I'm trying to print the names of all the files from a folder directory. I have a folder called "a", and in that folder there are 3 NC files, lets call them "b","c","d", whose directory I want to print. How would I do this?
For example, given my path to the folder is
path=r"C:\\Users\\chz08006\\Documents\\Testing\\a"
I want to print the directories to all the files in the folder "a", so the result should print:
C:\\Users\\chz08006\\Documents\\Testing\\a\\b.nc
C:\\Users\\chz08006\\Documents\\Testing\\a\\c.nc
C:\\Users\\chz08006\\Documents\\Testing\\a\\d.nc
So far, I've tried
for a in path:
print(os.path.basename(path))
But that doesn't seem to be right.

I think you're looking for this:
import os
path = r"C:\\Users\\chz08006\\Documents\\Testing\\a"
for root, dirs, files in os.walk(path):
for file in files:
print("{root}\\{file}".format(root=root, file=file))

You can have a list of file names in a folder using listdir().
import os
path = "C:\\Users\\chz08006\\Documents\\Testing\\a"
l = os.listdir(path)
for a in l:
print(path + a)

You made a couple mistakes. You were using os.path.basename, that only returns the name of the file or folder represented at the end of a path after the last file separator.
Instead, use os.path.abspath to get the full path of any file.
The other mistake was one of using the wrong variable inside the loop (print(os.path.basename(path) instead of using the variable a)
Also, dont forget to use os.listdir to list the files inside the folder before looping.
import os
path = r"C:\Users\chz08006\Documents\Testing\a"
for file in os.listdir(path): #using a better name compared to a
print(os.path.abspath(file)) #you wrote path here, instead of a.
#variable names that do not have a meaning
#make these kinds of errors easier to make,
#and harder to spot

Renaming filenames in python without changing the actual number order?

I have files like:
00001.jpg
00002.jpg
.
.
.
01907.jpg
I want to add some files to this directory which are named the same. But their names should continue like
01908.jpg
01909.jpg
.
.
12906.jpg
I couldn't manage to do that. How can i make this happen?
Thanks a lot:)
I tried
import os
files=[]
files = sorted(os.listdir('directory'))
b=len(files)
for i in range(0,b):
a=files[i]
os.rename(a,(a+1))
print (files)

you have a source dir (which contains the badly/identical named files) and a target dir (which contains files that should not be overwritten).
I would:
list the target dir & sort like you did (the rest of your attempt is clearly off...)
get the last item and parse as integer (without extension): add 1 and that gives the next free index.
loop in the source dir
generate a new name for the current file using the new computed index
use shutil.move or shutil.copy to move/copy the new files with the new name
like this:
import os,shutil
s = "source_directory"
d = "target_directory"
files = sorted(os.listdir(d))
highest_index = int(os.path.splitext(files[-1])[0])+1
for i,f in enumerate(sorted(os.listdir(s)),highest_index):
new_name = "{:05}.png".format(i)
shutil.copy(os.path.join(s,f),os.path.join(d,new_name))

You can do this:
import os
directory1 = 'path to the directory you want to move the files to'
directory2 = 'path to the directory you want to move the files to'
for file in ordered(os.listdir(directory2)):
counter = len(os.listdir(directory1))
file_number = int(file.split('.')[0]) #Get the file number
os.rename(os.path.join(directory2, file), os.path.join(directory1 + str(file_number + counter)))
What I have done:
Looped over the files that I wanted to rename and move.
Found the number of files, which I assumed that it is going to be the same as the name of the last file in this directory, in the main directory which the files are going to be moved to and made sure it will keep updating itself so that no overwrites happen.
Then I got the number of the current file in the loop.
Finally, I used os.rename to rename and move the file from the 1st directory to the 2nd.

Using os.system() in a specific directory only

I have a directory containing mutliple files with similar names and subdirectories named after these so that files with like-names are located in that subdirectory. I'm trying to concatenate all the .sdf files in a given subdirectory to a single .sdf file.
import os
from os import system
for ele in os.listdir(Path):
if ele.endswith('.sdf'):
chdir(Path + '/' + ele[0:5])
system('cat' + ' ' + '*.sdf' + '>' + ele[0:5] + '.sdf')
However when I run this, the concatenated file includes every .sdf file from the original directory rather than just the .sdf files from the desired one. How do I alter my script to concatenate the files in the subdirectory only?

this is a very clumsy way of doing it. Using chdir is not recommended, and system either (deprecated, and overkill to call cat)
Let me propose a pure python implementation using glob.glob to filter the .sdf files, and read each file one by one and write to the big file opened before the loop:
import glob,os
big_sdf_file = "all_data.sdf" # I'll let you compute the name/directory you want
with open(big_sdf_file,"wb") as fw:
for sdf_file in glob.glob(os.path.join(Path,"*.sdf")):
with open(sdf_file,"rb") as fr:
fw.write(fr.read())
I left big_sdf_file not computed, I would not recommend to put it in the same directory as the other files, since running the script twice would result in taking the output as input as well.
Note that the drawback of this approach is that if the files are big, they're read fully into memory, which can cause problems. In that case, replace
fw.write(fr.read())
by:
shutil.copyfileobj(fr,fw)
(importing shutil is necessary in that case). That allows packet copy instead of full-file read/write.
I'll add that it's probably not the full solution you're expecting, since there seem to be something about scanning the sub-directories of Path to create 1 big .sdf file per sub-directory, but with the provided code which doesn't use any system command or chdir, it should be easier to adapt to your needs.

Rename and move file with Python

I have a Python script that compares existing file names in a folder to a reference table and then determines if it needs to be renamed or not.
As it loops through each filename:
'oldname' = the current file name
'newname' = what it needs to be renamed to
I want rename the file and move it to a new folder "..\renamedfiles"
Can I do the rename and the move at the same time as it iterates through the loop?

Yes you can do this. In Python you can use the move function in shutil library to achieve this.
Let's say on Linux, you have a file in /home/user/Downloads folder named "test.txt" and you want to move it to /home/user/Documents and also change the name to "useful_name.txt". You can do both things in the same line of code:
import shutil
shutil.move('/home/user/Downloads/test.txt', '/home/user/Documents/useful_name.txt')
In your case you can do this:
import shutil
shutil.move('oldname', 'renamedfiles/newname')

os.rename (and os.replace) won't work if the source and target locations are on different partitions/drives/devices. If that's the case, you need to use shutil.move, which will use atomic renaming if possible, and fallback to copy-then-delete if the destination is not on the same file system. It's perfectly happy to both move and rename in the same operation; the operation is the same regardless.

To do both of the operations, you can use the os.rename(src, dest) function.
You should have the wanted directory to save the file in, and the new file name. You can do this for every file you run across in your loop.
For example:
# In Windows
dest_dir = "tmp\\2"
new_name = "bar.txt"
current_file_name = "tmp\\1\\foo.txt"
os.rename(current_file_name, os.path.join(dest_dir, new_name))
The rename function allows you to change the name of the file and it's folder at the same time.
To prevent any errors in renaming and moving of the file, use shutil.move.

Since Python 3.4, working with paths is done easily with pathlib. Moving/renaming a file is done with rename or replace (will unconditionally do the replace). So combining with the parent attribute and the concat operator, you can do:
from pathlib import Path
source = Path("path/to/file/oldname")
target = source.replace(source.parent / "renames" / "newname")

Create a Python file in your desired directory and write something like that :
import os
for filename in os.listdir("."):
if(filename ...):
newFilename = ...
os.rename(filename, newFilename)