I'm trying to load a simple text file with an array of numbers into Python. A MWE is
import numpy as np
BASE_FOLDER = 'C:\\path\\'
BASE_NAME = 'DATA.txt'
fname = BASE_FOLDER + BASE_NAME
data = np.loadtxt(fname)
However, this gives an error while running:
OSError: C:\path\DATA.txt not found.
I'm using VSCode, so in the debug window the link to the path is clickable. And, of course, if I click it the file opens normally, so this tells me that the path is correct.
Also, if I do print(fname), VSCode also gives me a valid path.
Is there anything I'm missing?
EDIT
As per your (very helpful for future reference) comments, I've changed my code using the os module and raw strings:
BASE_FOLDER = r'C:\path_to_folder'
BASE_NAME = r'filename_DATA.txt'
fname = os.path.join(BASE_FOLDER, BASE_NAME)
Still results in error.
Second EDIT
I've tried again with another file. Very basic path and filename
BASE_FOLDER = r'Z:\Data\Enzo\Waste_Code'
BASE_NAME = r'run3b.txt'
And again, I get the same error.
If I try an alternative approach,
os.chdir(BASE_FOLDER)
a = os.listdir()
then select the right file,
fname = a[1]
I still get the error when trying to import it. Even though I'm retrieving it directly from listdir.
>> os.path.isfile(a[1])
False
Using the module os you can check the existence of the file within python by running
import os
os.path.isfile(fname)
If it returns False, that means that your file doesn't exist in the specified fname. If it returns True, it should be read by np.loadtxt().
Extra: good practice working with files and paths
When working with files it is advisable to use the amazing functionality built in the Base Library, specifically the module os. Where os.path.join() will take care of the joins no matter the operating system you are using.
fname = os.path.join(BASE_FOLDER, BASE_NAME)
In addition it is advisable to use raw strings by adding an r to the beginning of the string. This will be less tedious when writing paths, as it allows you to copy-paste from the navigation bar. It will be something like BASE_FOLDER = r'C:\path'. Note that you don't need to add the latest '\' as os.path.join takes care of it.
You may not have the full permission to read the downloaded file. Use
sudo chmod -R a+rwx file_name.txt
in the command prompt to give yourself permission to read if you are using Ubuntu.
For me the problem was that I was using the Linux home symbol in the link (~/path/file). Replacing it with the absolute path /home/user/etc_path/file worked like charm.
Related
I have three files
My python file running in an unimportant different folder: C:\DD\CC\BB\AA\code.py
A playlist file "C:\ZZ\XX\Playlist.pls" which points to ....\mp3\song.mp3
The C:\mp3\song.mp3 file.
What I want is to get the location of the mp3 as an absolute path. But every attemp I try I get everything related to whenever the code.py file is.
import pathlib
plMaster = pathlib.Path(r"C:\ZZ\XX\Playlist.pls")
plSlave = pathlib.Path(r"..\..\mp3\song.mp3")
I have tried plSlave.absolute() and gives me "C:\DD\CC\BB\AA....\mp3\song.mp3"
Using relative_to doesn't work. I feel like I am doing such an easy task but I must be missing something because I can't find any function that lets me set the reference to compute the relative path.
Note: I already have parsed the pls file, and have the string r"....\mp3\song.mp3" extracted. I just need to get the path "C:\mp3\song.mp3" knowing that they are relative to the pls. (Not relative to the code.py)
If you're using a Windows version of Python, this is fairly easy. You can join the directory of plMaster (plMaster.parent) with the relative path of plSlave, then resolve the path using resolve(). You can use strict=False to force the resolve even if the path components aren't found.
This worked for me:
>>> plMaster = pathlib.Path(r"C:\ZZ\XX\Playlist.pls")
>>> plSlave = pathlib.Path(r"..\..\mp3\song.mp3")
>>> plMaster.parent.joinpath(plSlave).resolve(strict=False)
WindowsPath('C:/mp3/song.mp3')
If you're on a Unix version of Python, using Windows paths, I couldn't get this to work no matter what I tried, even using pathlib.PureWindowsPath().
Might well be a better method here, but you can use pathlib.Path.parents and pathlib.Path.parts to extract some useful info here and get where you are going
new_relative_path = r"..\..\mp3\song.mp3" #however you got this from reading your .pls file or whatever
pls_path = pathlib.Path(r'C:\ZZ\XX\Playlist.pls')
relative_save = pathlib.Path(new_relativePath)
n = relative_save.parts.count('..')
new_path = pls_path.parents[n-1].joinpath(*relative_save.parts[n:])
The key thing here is that you are going to navigate up the original path (the pls_path) n times (so n-1 since we start at 0), and then you are going to append to that whatever your new relative path is, stripping the '..' segments from the beginning of it.
Whilst I was waiting for other answers I manage to figure it out ditching pathlib and using os instead.
import os
plMaster = r"C:\ZZ\XX\Playlist.pls"
plSlave = r"..\..\mp3\song.mp3"
os.chdir(os.path.dirname(plMaster))
os.path.abspath(plSlave)
I'm trying to run a script that works without issue when I run using in console, but causes issue if I try to run it from another directory (via IPython %run <script.py>)
The issue comes from this line, where it references a folder called "Pickles".
with open('Pickles/'+name+'_'+date.strftime('%y-%b-%d'),'rb') as f:
obj = pickle.load(f)
In Console:
python script.py <---works!
In running IPython (Jupyter) in another folder, it causes a FileNotFound exception.
How can I make any path references within my scripts more robust, without putting the whole extended path?
Thanks in advance!
Since running in the console the way you show works, the Pickles directory must be in the same directory as the script. You can make use of this fact so that you don't have to hard code the location of the Pickles directory, but also don't have to worry about setting the "current working directory" to be the directory containing Pickles, which is what your current code requires you to do.
Here's how to make your code work no matter where you run it from:
with open(os.path.join(os.path.dirname(__file__), 'Pickles', name + '_' + date.strftime('%y-%b-%d')), 'rb') as f:
obj = pickle.load(f)
os.path.dirname(__file__) provides the path to the directory containing the script that is currently running.
Generally speaking, it's a good practice to always fully specify the locations of things you interact with in the filesystem. A common way to do this as shown here.
UPDATE: I updated my answer to be more correct by not assuming a specific path separator character. I had chosen to use '/' only because the original code in the question already did this. It is also the case that the code given in the original question, and the code I gave originally, will work fine on Windows. The open() function will accept either type of path separator and will do the right thing on Windows.
You have to use absolute paths. Also to be cross platform use join:
First get the path of your script using the variable __file__
Get the directory of this file with os.path.dirname(__file__)
Get your relative path with os.path.join(os.path.dirname(__file__), "Pickles", f"{name}_{date.strftime('%y-%b-%d')}")
it gives you:
with open(os.path.join(os.path.dirname(__file__), "Pickles", f"{name}_{date.strftime('%y-%b-%d')}"), 'rb') as f:
obj = pickle.load(f)
I have been struggling a lot with this piece of code i wrote by referring tutorials:
import os
os.chdir(f'/home/django/image-io/accounts/john_doe/optimized/')
cwd = os.getcwd()
logger.info(f'CURRENT WORKING DIRECOTRY :{cwd}')
subprocess.Popen(f"optimize-images")
What i am trying to do is dynamically change directory on username (john_doe) in this case and run a command optimize-images inside that directory to optimize all images .
It doesn't seem to work.
When i checked the logs , it says:
optimize-images : "No such directory present"
I am not able to understand , where i am doing wrong. any help would be appreciated.
In this code, We are optimizing the images contained in the optimized folder.
import os
import subprocess
os.chdir(r'/home/django/image-io/accounts/john_doe/optimized')
cwd = os.getcwd()
p=subprocess.Popen([r"optimize-images", cwd])
We need to pass the the parameters in the array list format. You can get more details here.
subprocess.Popen([r"optimize-images", cwd])
Note:
r is used to indicate the raw string.
f is used when we need to include the value of Python expressions inside strings.
Is this what you need? Popen error: [Errno 2] No such file or directory
Try subprocess.Popen(f"optimize-images", shell = True)
I want this line to save the csv in my current directory alongside my python file:
df.to_csv(./"test.csv")
My python file is in "C:\Users\Micheal\Desktop\VisualStudioCodes\Q1"
Unfortunately it saves it in "C:\Users\Micheal" instead.
I have tried import os path to use os.curdir but i get nothing but errors with that.
Is there even a way to save the csv alongside the python file using os.curdir?
Or is there a simpler way to just do this in python without importing anything?
import os
directory_of_python_script = os.path.dirname(os.path.abspath(__file__))
df.to_csv(os.path.join(directory_of_python_script, "test.csv"))
And if you want to read same .csv file later,
pandas.read_csv(os.path.join(directory_of_python_script, "test.csv"))
Here, __file__ gives the relative location(path) of the python script being runned. We get the absolute path by os.path.abspath() and then convert it to the name of the parent directory.
os.path.join() joins two paths together considering the operating system defaults for path seperators, '\' for Windows and '/' for Linux, for example.
This kind of an approach should work, I haven't tried, if does not work, let me know.
I have recently begun working on a new computer. All my python files and my data are in the dropbox folder, so having access to the data is not a problem. However, the "user" name on the file has changed. Thus, none of my os.chdir() operations work. Obviously, I can modify all of my scripts using a find and replace, but that won't help if I try using my old computer.
Currently, all the directories called look something like this:
"C:\Users\Old_Username\Dropbox\Path"
and the files I want to access on the new computer look like:
"C:\Users\New_Username\Dropbox\Path"
Is there some sort of try/except I can build into my script so it goes through the various path-name options if the first attempt doesn't work?
Thanks!
Any solution will involve editing your code; so if you are going to edit it anyway - its best to make it generic enough so it works on all platforms.
In the answer to How can I get the Dropbox folder location programmatically in Python? there is a code snippet that you can use if this problem is limited to dropbox.
For a more generic solution, you can use environment variables to figure out the home directory of a user.
On Windows the home directory is location is stored in %UserProfile%, on Linux and OSX it is in $HOME. Luckily Python will take care of all this for you with os.path.expanduser:
import os
home_dir = os.path.expanduser('~')
Using home_dir will ensure that the same path is resolved on all systems.
Thought the file sq.py with these codes(your olds):
C:/Users/Old_Username/Dropbox/Path
for x in range:
#something
def Something():
#something...
C:/Users/Old_Username/Dropbox/Path
Then a new .py file run these codes:
with open("sq.py","r") as f:
for x in f.readlines():
y=x
if re.findall("C:/Users/Old_Username/Dropbox/Path",x) == ['C:/Users/Old_Username/Dropbox/Path']:
x="C:/Users/New_Username/Dropbox/Path"
y=y.replace(y,x)
print (y)
Output is:
C:/Users/New_Username/Dropbox/Path
for x in range:
#something
def Something():
#something...
C:/Users/New_Username/Dropbox/Path
Hope its your solution at least can give you some idea dealing with your problem.
Knowing that eventually I will move or rename my projects or scripts, I always use this code right at the beginning:
import os, inspect
this_dir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
this_script = inspect.stack()[0][1]
this_script_name = this_script.split('/')[-1]
If you call your script not with the full but a relative path, then this_script will also not contain a full path. this_dir however will always be the full path to the directory.