I cannot figure out the best way to find a specific folder and send files to another specific folder, especially if the users directory is slightly different than what I have coded.
I'm working on a program that has a folder of content to grab from and the user basically picks items and when they're done, it creates a folder full of things including the images they chose. I've gotten it to work (and creating all necessary folders in the user's directory works fine but once it becomes more complex, it doesn't work some of the time) but I would like it to work every time, regardless of the user and where they've placed my program on their computer.
an example of relevant code I currently have which I'm sure is redundant compared to what I could be using instead:
init python:
import os
import shutil
current_dir = os.path.normpath(os.getcwd() + "../../")
def grab_content():
filetocopy = "image%s.png"%image_choice ##(this is a separate variable within the program that determines if it is image1.png, image2.png etc)
file_path = os.path.join(current_dir, "Program folder", "stuff", "content")
images_path = os.path.join(file_path, "images")
new_images_path = os.path.join(current_dir, "My Templates", anothervariable_name, "game", "template", "image_choices")
try:
shutil.copy(images_path + "\\" + filetocopy, new_images_path)
except:
print("error")
(folders listed in this have been checked if existing and placed if not - but not for the new file path due to that needing to be in a specific place within the main folder)
It either works if I have the files set up just right (for my own machine which defeats the purpose) or it doesn't do anything or I get an error saying the path doesn't exist. I have code prior to this that creates the folders needed but I'm trying to grab images from the folder that belongs to the actual program and put them (only ones I specify) into a new folder I create through the program.
Would i use os.walk? I was looking at all the os code but this is my first time dealing with any of it so any advice is helpful.
Related
I have a script that scrapes data, it saves it to my local db and it takes screenshot from webpages and stores it in a local folder. Both the db and screenshots folder are located in a folder on my Desktop.
Here are some lines of code from my script.
connection = sqlite3.connect('NW.db')
if not os.path.exists('nw_' + req[1]):
os.mkdir('nw_' + req[1])
# folder = f'nw_{req[1]}\{Month}_{Hire_Start}'
folder = f'nw_{req[1]}\{Hire_Start}'
if not os.path.exists(folder):
os.mkdir(folder)
folder_w = f'nw_{req[1]}'
if not os.path.exists(folder_w):
os.mkdir(folder_w)
ss.full_Screenshot(driver, save_path=r'.', image_name=folder + "NW_" + req[1] + "_" + days3 + "_" + Hire_Start + ".jpg")
Everything is working fine and it is stored in the same folder where I am running my script from. But when I make a windows scheduled task to run this script at a specific time it stores the db file and the screenshots folder in the following location C:\Windows\System32 instead of my folder on the Desktop
I made a batch file and I added this to my scheduled task
"C:\Users\Administrator\AppData\Local\Programs\Python\Python310\python.exe" "C:\Users\Administrator\Desktop\Python_Kishen\NW_offer.py"
pause
You shouldn't rely on your "current" or "working" directory in a script run from the system. Specify the full path to where you want your output files.
You can use os.chdir() to make your preferred destination your current dir, but that's not the best way. Full paths.
The scheduler process likely has a different location it operates from, and it looks like you're using relative paths. I'd recommend using full paths if you have a place you want this all to go. Then it doesn't matter where the code is run from.
As a side note, it looks like you're doing a lot of repeat string building. Why not do that once and store the result in a variable? Just use the variable in the few places you need that value.
I have 5 files and they exist in 5 different locations
I would like to write a piece of code preferably in Python(I am a newbie) and the code should check all these 5 folders and code should check if the file exists, if it does, it should move over all those files from different locations to a single shared drive location. I tried the below URL and used import shutil, this worked but this is for one or many files from the same location to another location. Any pointers, thoughts, suggestions on how I can do this will be greatly appreciated.
https://linuxhint.com/move-file-to-other-directory-python/
Speaking just about principles on how to solve your problem without writing any code:
From the code you linked, you have a solution for copying one file at a time. Moving that code inside a function will let you easily re-use it for many different input files, and even more than one destination file. You can define the file(s) to be copied and the destination folder as arguments.
Python's os library is great for operating system actions like checking if file exists, making the target directory if needed, just to name a few that might be useful. But this is just one tool for getting there, you'll likely see a few different but valid answers.
Try Pathlib ->
from pathlib import Path
import shutil
inp_file_paths = [Path('input_file_path_1'),
Path('input_file_path_2'),
Path('input_file_path_3')]
destination_path = Path('destination_path')
def check(inp_files):
for file in inp_files:
if file.exists() == False:
print(f'file = {file.name} doesn\'t exists please check the path')
inp_files.remove(file)
return inp_files
def move_files(inp_files):
try:
for file in inp_files:
dest_path = destination_path / file.name
shutil.copy(file, dest_path)
except Exception as e:
print(f'Exception {e} occurred while moving the file {file.name}')
inp_file_paths = check(inp_file_paths)
move_files(inp_file_paths)
I'm writing a script to reorganise a large data set so that it'll be mostly compatible with a database system we're currently implementing. Most of our data is currently not organised in any meaningful way, although some folders are already marked in the way the script will. As such I've implemented a case to catch this.
Let's call this case "INFO". In all but the first instance of this my script works fine, finds the "INFO_example[2,3,4....]" folder at the top level, and moves it to a new "INFO" folder that contains all of these examples. However, for some reason in the first instance "INFO_example1" it instead takes the contents of the folder and dumps them in "INFO" instead.
I've attempted to debug the problem, but can't see any differences between the first and any other instances. The folder path also doesn't appear any differently.
file_path = join(self.path, file)
try:
move(file_path, self.info_path)
except shutil.Error:
print("Trying to move", file_path, " didn't work")
I'm a little stumped to what's actually going on.
I'd expect the "INFO_example1" folder to behave as all the others do, and just be moved into the top level "INFO" folder.
Currently, it's contents are moved into "INFO", and the "INFO_example1" folder appears to be deleted.
My print message also never fires.
So, the issue was that shutil.move would look at say "INFO_example1" and try to move it to "INFO", notice that "INFO" did not exist, and so would create it and put the contents of "INFO_example1" in to it instead of creating another folder within and then moving the contents into there.
Due to the nature of the script creating "INFO" before trying to move files caused more issues, however, if I updated the section that moves the "INFO_example" folders, to look like:
file_path = join(self.path, file)
try:
if not isdir(self.info_path):
mkdir(self.info_path)
move(file_path, self.info_path)
except shutil.Error:
print("Trying to move", file_path, " didn't work")
Checking whether the "INFO" directory exists or not, and if not creating it but only when I'm attempting to move an "INFO_example" style folder gives the expected behaviour, at least for myself.
I'm trying to create a script in order to move files from a list I have. I'd like to create some conditions to that but I'm afraid that's where my Python knowledge fails me. I have a list of names (AAA, BBB, CCC).
For each of those, there are six different files with six different extensions that need to be moved (AAA.1, AAA.2, AAA.3, AAA.4, AAA.5, AAA.6). Those files might be in 3 different folders. Let's suppose, either AAA/AAA or BBB/IOL or BBB/ABC. I want all of those files to be moved to REAL/AAA. The thing is, on the folder AAA/AAA there are some AAAXXX.1 files that I do not want to be moved.
I'm completely lost and new to Python (basically, it's my first week :p).
import os
import shutil
import fnmatch
source = os.listdir(r"\\enterprise\AAA\AAA")
destination = os.listdir(r"\\enterprise\REAL\AAA")
set = {
"AAA",
"BBB",
"CCC"
}
for file in source:
for x in set:
if file.__contains__(str(x)):
print(file)
I don't know how could I specify that AAAXXX, BBBXXX and etc shall not be moved.
I don't how how to insert multiple folders for searching files with conditions (If not in folder AAA/AAA, try BBB/IOL and if not BBB/ABC)
I don't know how could I specify that AAAXXX, BBBXXX and etc shall not
be moved.
The simplest way is something like this:
if 'XXX' in file:
continue # this means skip the rest of the cycle and move to next file
I don't how how to insert multiple folders for searching files with
conditions (If not in folder AAA/AAA, try BBB/IOL and if not BBB/ABC)
The most blunt approach would be
if file_name in os.listdir('first_folder'):
# move from first
elif file_name in os.listdir('second_folder'):
# move from second
# continue adding elif for every folder
else:
print(f'file {file_name) is not found')
But I'd probably just scanned every folder and then moved everything matching given name to the destination. Although this might not be what you want if you've got duplicate names and different file contents, and you have some folder particular folder precedence.
I am trying to write a piece of code that will recursively iterate through the subdirectories of a specific directory and stop only when reaching files with a '.nii' extension, appending these files to a list called images - a form of a breadth first search. Whenever I run this code, however, I keep receiving [Errno 20] Not a directory: '/Volumes/ARLO/ADNI/.DS_Store'
*/Volumes/ARLO/ADNI is the folder I wish to traverse through
*I am doing this in Mac using the Spyder IDE from Anaconda because it is the only way I can use the numpy and nibabel libraries, which will become important later
*I have already checked that this folder directly contains only other folders and not files
#preprocessing all the MCIc files
import os
#import nibabel as nib
#import numpy as np
def pwd():
cmd = 'pwd'
os.system(cmd)
print(os.getcwd())
#Part 1
os.chdir('/Volumes/ARLO')
images = [] #creating an empty list to store MRI images
os.chdir('/Volumes/ARLO/ADNI')
list_sample = [] #just an empty list for an earlier version of
#the program
#Part 2
#function to recursively iterate through folder of raw MRI
#images and extract them into a list
#breadth first search
def extract(dir):
#dir = dir.replace('.DS_Store', '')
lyst = os.listdir(dir) #DS issue
for item in lyst:
if 'nii' not in item: #if item is not a .nii file, if
#item is another folder
newpath = dir + '/' + item
#os.chdir(newpath) #DS issue
extract(newpath)
else: #if item is the desired file type, append it to
#the list images
images.append(item)
#Part 3
adni = os.getcwd() #big folder I want to traverse
#print(adni) #adni is a string containing the path to the ADNI
#folder w/ all the images
#print(os.listdir(adni)) this also works, prints the actual list
"""adni = adni + '/' + '005_S_0222'
os.chdir(adni)
print(os.listdir(adni))""" #one iteration of the recursion,
#works
extract(adni)
print(images)
With every iteration, I wish to traverse further into the nested folders by appending the folder name to the growing path, and part 3 of the code works, i.e. I know that a single iteration works. Why does os keep adding the '.DS_Store' part to my directories in the extract() function? How can I correct my code so that the breadth first traversal can work? This folder contains hundreds of MRI images, I cannot do it without automation.
Thank you.
The .DS_Store files are not being created by the os module, but by the Finder (or, I think, sometimes Spotlight). They're where macOS stores things like the view options and icon layout for each directory on your system.
And they've probably always been there. The reason you didn't see them when you looked is that files that start with a . are "hidden by convention" on Unix, including macOS. Finder won't show them unless you ask it to show hidden files; ls won't show them unless you pass the -a flag; etc.
So, that's your core problem:
I have already checked that this folder directly contains only other folders and not files
… is wrong. The folder does contain at least one regular file; .DS_Store.
So, what can you do about that?
You could add special handling for .DS_Store.
But a better solution is probably to just check each file to see if it's a file or directory, by calling os.path.isdir on it.
Or, even better, use os.scandir instead of listdir, which gives you entries with more information than just the name, so you don't need to make extra calls like isdir.
Or, best of all, just throw out this code and use os.walk to recursively visit every file in every directory underneath your top-level directory.