We Keep Coding

Streamlip app not searching files in the good directory

I am trying to run a Streamlit app importing pickle files and a DataFrame. The pathfile for my script is :
/Users/myname/Documents/Master2/Python/Final_Project/streamlit_app.py
And the one for my DataFrame is:
/Users/myname/Documents/Master2/Python/Final_Project/data/metabolic_syndrome.csv
One could reasonably argue that I only need to specify df = pd.read_csv('data/df.csv') yet it does not work as the Streamlit app is unexpectedly not searching in its directory:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/myname/data/metabolic_syndrome.csv'
How can I manage to make the app look for the files in the good directory (the one where it is saved) without having to use absolute pathfiles ?

you can use os.getcwd() to get the Current Working Directory. Read up on what this means exactly, fe here
See the sample script below on how to use it. I'm using os.sep for OS agnostic filepath separators.
import os
print(os.getcwd())
relative_path = "data"
full_path = f"{os.getcwd()}{os.sep}{relative_path}"
print(full_path)
filename = "somefile.csv"
full_file_path = f"{full_path}{os.sep}{filename}"
print(full_file_path)
with open(full_file_path) as infile:
for line in infile.read().splitlines():
print(line)

In which directory are you standing when you are running your code?
From your error message I would assume that you are standing in /Users/myname/ which makes python look for data as a subdirectory of /Users/myname/.
But if you first change directory to /Users/myname/Documents/Master2/Python/Final_Project and then run your code from there I think it would work.

Trying to move data to a new directory after analysis in the same script

So I have code that completes an analysis for me on imaging data. This code is working fine except that the data is not being outputted into the correct folder. Instead, it is being outputted one folder too high. This is something I obviously want to fix in the long run, but as I am under certain time constraints I want to input code into my script that will simply move the files to the correct folder/directory that I have created. I have tried the mv and shutil commands but they don't seem to be working. I would be grateful if someone had a suggestion for how to fix/improve my method of moving these files to the correct location. If someone has a suggestion for why the files are being outputted to the wrong directory as well that would be awesome. I am relatively new to coding and no expert so please forgive any obvious mistakes. Thank you.
This is where I set up my directories
subject_dir = os.getcwd()
dti_dir = os.path.abspath( os.path.join(subject_dir, 'dti'))
dti_input_dir = os.path.abspath( os.path.join(dti_dir, 'input'))
This is where I entered a few shortcuts
eddy_prefix = 'eddy'
input_path = dti_input_dir
output_prefix = 'dtifit'
output_path = '../dti/dtifit'
output_basename = os.path.abspath(os.path.join(dti_dir, output_path))
infinite_path = os.path.join(os.getenv('INFINITE_PATH'), 'infinite')
dti30 = 'dti30.nii.gz'
dti30_brain = 'bet.b0.dti30.nii.gz'
dti30_mask = 'bet.b0.dti30_mask.nii.gz'
This is where I ran my test.
The test is being run, but my data is being outpputed in dti_dir and not output_basename (this is my second question)
dti = fsl.DTIFit()
dti.inputs.dwi = os.path.join(input_path, eddy_prefix + '.nii.gz')
dti.inputs.bvecs = os.path.join(input_path, eddy_prefix + '.eddy_rotated_bvecs')
dti.inputs.bvals = os.path.abspath(os.path.join(infinite_path, 'dti30.bval'))
dti.inputs.base_name = output_basename
dti.inputs.mask = os.path.join(input_path, dti30_mask)
dti.cmdline
Creating output directory if doesn't exist.
This is working fine and the directory is created in the proper location.
if not os.path.exists(output_basename):
os.makedirs(output_basename)
print('DTI Command line')
print(dti.cmdline)
res = dti.run()
exit()
print('DTIFIT Complete!')
Here I try to move the files and I get the error: IOError: [Errno 2] No such file or directory: even though I know the files exist
src = dti_dir
dst = output_basename
files = os.listdir(src)
for f in files:
if (f.startswith("dtifit_")):
shutil.move(f, dst)

Your problem is most likely in the very first line. You may want to change that to be more exact, pointing to an exact directory rather than os.getcwd(). os.getcwd() will point to wherever the process is executing from, wherever you kicked off your python run, so it could change. You probably want to hard-code it somehow.
For example, you could use something like this, to point to the directory that the file lives in:
Find current directory and file's directory
Another thing you could consider doing is printing out dst in your last few lines and seeing if it's what you expect. You could also use PDB to inspect that value live:
https://pymotw.com/2/pdb/
Edit:
Your problem is using a relative path with shutil.move(). You should make sure that the path is absolute, please see this answer for more info:
stackoverflow.com/a/22230227/7299836

Python - Errno 13 Permission denied when trying to copy files

I am attempting to make a program in Python that copies the files on my flash drive (letter D:) to a folder on my hard drive but am getting a PermissionError: [Errno 13] Permission denied: 'D:'.
The problematic part of my code is as follows:
# Copy files to folder in current directory
def copy():
source = getsource()
if source != "failure":
copyfile(source, createfolder())
wait("Successfully backup up drive"
"\nPress 'Enter' to exit the program")
else:
wait("No USB drive was detected"
"\nPress 'Enter' to exit")
# Create a folder in current directory w/ date and time
def createfolder():
name = strftime("%a, %b %d, %Y, %H.%M.%S", gmtime())
dir_path = os.path.dirname(os.path.realpath(__file__))
new_folder = dir_path + "\\" + name
os.makedirs(new_folder)
return new_folder
Everything seems to run fine until the copyfile() function runs, where it returns the error.
I tried replacing getsource() with the destination of the file instead and it returned the same permission error except for the new_folder directory instead.
I have read several other posts but none of them seem to be relevant to my case. I have full admin permissions to both locations as well.
Any help would be greatly appreciated!

As I stated in my comment above, it seems as if you're trying to open the directory, D:, as if it was a file, and that's not going to work because it's not a file, it's a directory.
What you can do is use os.listdir() to list all of the files within your desired directory, and then use shutil.copy() to copy the files as you please.
Here is the documentation for each of those:
os.listdir() (You will be passing the full file path to this function)
shutil.copy() (You will be passing each file to this function)
Essentially you would store all of the files in the directory in a variable, such as all_the_files = os.listdir(/path/to/file), then loop through all_the_files by doing something like for each_file in all_the_files: and then use shutil.copy() to copy them as you please.

If you're looking to copy an entire directory and its contents, then shutil.copytree(source, destination) can be used.

Why can't I open this file for reading in python?

I'm working in Python and I'm confused why I can't open the file I'm trying to. The code is pretty simple. Here it is.
import os
def main():
FILE_NAME = "default_template.csv"
source_path = os.path.join("Documents", FILE_NAME)
file = open(source_path, "r")
At this point I get
IOError: [Errno 2] No such file or directory: 'Documents/FILE_NAME'.
I also decided to try to change directories for whatever reason using os.chdir() and passing just about every high level directory on my computer in seperately and nothing worked. In an attempt to find the fix for opening a file I tried editing the path in a bunch of different ways.
I've tried something like:
os.path.join("/derek/Documents", FILE_NAME)
os.path.join("/Documents", FILE_NAME)
os.path.join("~/derek/Documents", FILE_NAME)
os.path.join("~", FILE_NAME)
If anyone could help me out I would be extremely thankful. I'm still new to using python to navigate and manage files.

Python doesn't expand ~ by default. To make it work try os.path.expanduser:
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
source_path = os.path.join("~/Documents", FILE_NAME)
source_path = os.path.expanduser(source_path)

Shutil.copy: Why is my script looking for unicode instead of the path saved in my variable?

I'm trying to use the shutil module to copy files from one drive to another. Since this is an ArcGIS script, I store the user's choice for folder source and destination locations as:
src = arcpy.GetParameterAsText(0)
dst = arcpy.GetParameterAsText(1)
Using arcpy.AddMessage(src) to print that out gives me:
C:\Folder1\Folder2\File.extension
Which is what I want! However, when I try to use shutil.copy(src,dst), I get:
u'C:\\Folder1\\Folder2\\File.extension'
IOError: [Errno 2] No such file or directory: u'C:\\Folder1\\Folder2'
What is happening here? Since I'm not spelling out the path I can't change the "u" to an "r" for raw input...

Shutil.copy/unicode wasn't the problem here; tyring to copy a non-existent file was.

As a form of file management validation, you should always check to see whether or not the directory exists. You can do this using the os.path.exists(path) method. If your path exists, you should have no problem. If not, then create it before copying your files over.
See example code below:
if not os.path.exists(dst):
os.mkdir(dst)
shutil.copy(src, dst)