We Keep Coding

If filename in folder A exists in Database move to folder B, If file does not exist, insert filename into database and move file to folder C

I have 3 Folders
Folder A (Landing Folder)
Folder B (Deletion Folder)
Folder C (Transfer to AWS Folder)
I've written some python code to move files to the landing folder after some data transformation, from the landing folder - i need to make a decision tree for the files:
Check each filename in folder A to see if it exists in the database
if the file exists already:
move the file to Folder B
Else
Insert the file name into the Database & move the file to Folder C for transfer to AWS
I have the SQL statement inside python tested and working, but im not sure on how to move the files to their respective folders.
I'm assuming that i need something along the lines of using shutil.move to Folder B or C depending on outcome, but not 100% sure on the syntax to get there.
Any help appreciated.

You can use os.listdir to get a list of the files in a given path, and as you said, you can use shutil.move() to move the files. So, you could try something like this:
import os
import shutil
folderA='pathfolderA'
folderB='pathfolderB'
folderC='pathfolderC'
files=os.listdir(folderA)
for fil in files:
if fil in database:
shutil.move(fil,folderB)
else:
# insert(fil) into database
shutil.move(fil,folderC)
I'm ignoring the explicity of steps if fil in database and insert(fil) into database, because you said you have the SQL statement successfully tested. You can chekout the this helpful info about shutil and os.listdir: link 1:shutil, link 2:shutil, link 3:shutil, link 4:os.listdir.

How to access a file a path level below from a subfolder

Let's say I have a file called credentials.json in my current directory, an environment variable MY_CREDENTIALS=credentials.json and a script main.py that uses this environment variable via os.getenv('MY_CREDENTIALS').
Now suppose I create a subfolder and put something there like this: /subfolder/my_other_script.py.
If I print os.getenv('MY_CREDENTIALS') then I get indeed credentials.json but I can't use this file as it is in my root directory (not in /subfolder). So, how can I use this file although it is in the root directory? The only thing that works for me is to make a copy of credentials.json in /subfolder, but then I would have multiple copies of this file and I don't want that.
Thanks for your response!

Something like this could work:
from pathlib import Path
import os
FILENAME = os.getenv('MY_CREDENTIALS')
filePath = Path(FILENAME)
if filePath.exists() and filePath.is_file():
print("Success: File exists")
print(filePath.read_text())
else:
print("Error: File does not exist. Getting file from level below.")
print((filePath.absolute().parent.parent / filePath.name).read_text())
Basically, you check whether your file exists in the current folder. This will be the case, if your script is in your root folder. If it is not, you assume that you are in a subfolder. So you try to get the file from one level below (your root).
It's not totally production ready, but for the specific case you mentioned it should work. In production you should think about cases where you might have nested subfolder or your file is missing for good.

You can create symbolic link
os.symlink('<root>', '<subfolder>')
https://docs.python.org/3/library/os.html#os.symlink

Unable to open saved excel file using urllib.request.urlretrieve (Sample link mentioned )

Currently, I'm using Flask with Python 3.
For sample purposes, here is a dropbox link
In order to fetch the file and save it, I'm doing the following.
urllib.request.urlretrieve("https://www.dropbox.com/s/w1h6vw2st3wvtfb/Sample_List.xlsx?dl=0", "Sample_List.xlsx")
The file is saved successfully to my project's root directory, however there is a problem. When I try to open the file, I get this error.
What am I doing wrong over here?
Also, is there a way to get the file name and extension from the URL itself? Example, filename = Sample_List and extension = xlsx...something like this.

Copying file from one directory to another

Does anyone know how I can copy/duplicate a file from one directory into another without specification of src path? I got it to work with "shutil.copy2" but it's not exactly what I am looking for since the src argument asks for the path.
My goal is to be able to copy/duplicate a file from one directory into another by filename. Has anyone done this before, if so can you guide me in the right direction? - Thanks
#----------------------------------------------------------------------------------------------------------------#
# These params will be used for specifying which template you want to copy and where to output
#----------------------------------------------------------------------------------------------------------------#
'''Load file from x directory into current working directory '''
#PullTemplate: Specify which template you want to copy, by directory path
TemplateRepo = ("/home/hadoop/BackupFolders/Case_Project/scripts")
#OutputTemplate: Let's you specify where you want to output the copied template.
#Originally set to your current working directory (u".")
OutputTemplate = (u".")
shutil.copy2(TemplateRepo, OutputTemplate)

Well if you are trying to load a file in the same project you need to have at least the folder name inside that project.
You can use json
Something like this.
import json
#someFiles is just a fold name inside the projects main folder.
with open("someFiles\\file_name", "r") as whatever_u_want:
var_of_choice = json.load(whatever_u_want)
print (var_of_choice)
once the file is open you can save the variable var_of_choice as any file name you wish where you wish using the json dump method.

Click the file you want to copy, create a duplicate of the file by choosing Duplicate key under File ( below the Jupyter logo at the top ).
Choose the file copied(file_copy), and choose Move key under File.
Choose the file path where you want to paste/move the copied file.
Rename the copied file name as you wish.
For more info, you can refer to here: https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781785884870/1/ch01lvl1sec12/basic-notebook-operations

Opening a dynamic file in Django not in the project directory

So I am working on an app that allows users to upload CSV files and then generate graph data corresponding to their files. I have it working in development only when the files that are uploaded are sitting in the project's current working directory.
I discovered that the reason behind this is because in my graph view I am opening the file like so:
data_file = open(new_file, 'rb')
Open expects to find the file within the project directory. If I attempt to upload a file outside of that directory it throws this error:
Errno 2] No such file or directory: 'test.CSV'
I've read about os.path.expanduser and have tried:
data_file = open(os.path.expanduser('~' + new_file), 'rb')
but without success. The above code tries to find the file in C:/Users/test.csv.
Any suggestions on how I can achieve this would be greatly appreciated.
EDIT
My current attempt is now:
file_upload_dir = os.path.join(settings.MEDIA_ROOT, 'Data_Files')
data_file = open(os.path.join(file_upload_dir, new_file), 'rb')
And the error is:
File b'test.CSV' does not exist
Data_Files is a folder within my Media folder.

You need join ~, and file name with directory separator (os.sep). Using os.path.join will do it for you.
data_file = open(os.path.expanduser(os.path.join('~', new_file)), 'rb')

Uploaded files, as well as file generated by your application, have nothing to do in the project's directory (=> source code). You have a setting for where to store them (settings.MEDIA_ROOT), and you have a models.FileField to remember where they are stored and how to access them.