I've been working on a project where I need to import csv files, previously the csv files have been in the same working directory. Now the project is getting bigger so for security and organizational resaons I'd much prefer to keep them in a different directory.
I've had a look at some other questions asking similar things but I couldn't figure out out how to apply them to my code as each time I tried I kept getting the same error message mainly:
IOError: [Errno 2] No such file or directory:
My original attempts all looked something like this:
import csv # Import the csv module
import MySQLdb # Import MySQLdb module
def connect():
login = csv.reader(file('/~/Projects/bmm_private/login_test.txt'))
I changed the path within several times as well by dropping the first / then then the ~ then the / again after that, but each time I got the error message. I then tried another method suggested by several people by importing the os:
import os
import csv # Import the csv module
import MySQLdb # Import MySQLdb module
def connect():
path = r'F:\Projects\bmm_private\login_test.txt'
f = os.path.normpath(path)
login = csv.reader(file(f))
But I got the error message yet again.
Any help here would be much appreciated, if I could request that you use the real path (~/Projects/bmm_private/login_test.txt) in any answers you know of so it's very clear to me what I'm missing out here.
I'm pretty new to python so I may struggle to understand without extra clarity/explanation. Thanks in advance!
The tilde tells me that this is the home folder (e.g. C:\Users\<username> on my Windows system, or /Users/<username> on my Mac). If you want to expand the user, use the os.path.expanduser call:
full_path = os.path.expanduser('~/Projects/bmm_private/login_test.txt')
# Now you can open it
Another approach is to seek for the file in relative to your current script. For example, if your script is in ~/Projects/my_scripts, then:
script_dir = os.path.dirname(__file__) # Script directory
full_path = os.path.join(script_dir, '../bmm_private/login_test.txt')
# Now use full_path
Related
I would like to access the output of another python program that extracts information from a fits file.
I usually do this in the following way:
import sys
sys.path.insert(1, '/software/xray/Python_scripts')
from program2 import results
However, in this case, I receive the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'info.fits'
When I run the program2.py It runs without problem. So, I don't understand why when I call it from program1.py it does not recognize the fits file, therefore it doesn't give the results! Can anybody point me in the right direction?
Thanks.
Seems like your import from program2 import results searching for a file named info.fits which is located most probably in '/software/xray/Python_scripts'.
Basically, sys.path.insert temporarily adds path to PATH, in order to make OS executing/importing script from that place. It doesn't mean that it becomes a file search path as well.
You need to do somethig like:
import os
cwd = os.getcwd()
os.chdir('/software/xray/Python_scripts')
from program2 import results
os.chdir(cwd)
It is certainly a crunch, I would suggest you to create a package out of your program2 module. https://realpython.com/python-modules-packages/
I definitely have placed some files in my trash bin on my Mac but for some reason my code is printing an empty array [] - here is my code in my python file:
import os
from distutils.dir_util import copy_tree
from datetime import datetime
import shutil
bin_location = "/Users/nick/.Trash"
bin_files = os.listdir(bin_location)
for f in bin_files:
print(f)
print(os.listdir('/Users/nick/.Trash'))
I’m using VSCode and have given permission in my system settings to allow the program to access the .Trash directory (to fix a permissions error I initially had). Is there something obvious that I’m doing wrong? I cannot see why it is not listing the files in the bin.
I have a python file, converted from a Jupiter Notebook, and there is a subfolder called 'datasets' inside this file folder. When I'm trying to open a file that is inside that 'datasets' folder, with this code:
import pandas as pd
# Load the CSV data into DataFrames
super_bowls = pd.read_csv('/datasets/super_bowls.csv')
It says that there is no such file or folder. Then I add this line
os.getcwd()
And the output is the top-level folder of the project, and not the subfolder when is this python file. And I think maybe that's the reason why it's not working.
So, how can I open that csv file with relative paths? I don't want to use absolute path because this code is going to be used in another computers.
Why os.getcwd() is not getting the actual folder path?
My observation, the dot (.) notation to move to the parent directory sometimes does not work depending on the operating system. What I generally do to make it os agnostic is this:
import pandas as pd
import os
__location__ = os.path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
super_bowls = pd.read_csv(__location__ + '/datasets/super_bowls.csv')
This works on my windows and ubantu machine equally well.
I am not sure if there are other and better ways to achieve this. Would like to hear back if there are.
(edited)
Per your comment below, the current working directory is
/Users/ivanparra/AprendizajePython/
while the file is in
/Users/ivanparra/AprendizajePython/Jupyter/datasets/super_bowls.csv
For that reason, going to the datasets subfolder of the current working directory (CWD) takes you to /Users/ivanparra/AprendizajePython/datasets which either doesn't exist or doesn't contain the file you're looking for.
You can do one of two things:
(1) Use an absolute path to the file, as in
super_bowls = pd.read_csv("/Users/ivanparra/AprendizajePython/Jupyter/datasets/super_bowls.csv")
(2) use the right relative path, as in
super_bowls = pd.read_csv("./Jupyter/datasets/super_bowls.csv")
There's also (3) - use os.path.join to contact the CWD to the relative path - it's basically the same as (2).
(you can also use
The answer really lies in the response by user2357112:
os.getcwd() is working fine. The problem is in your expectations. The current working directory is the directory where Python is running, not the directory of any particular source file. – user2357112 supports Monica May 22 at 6:03
The solution is:
data_dir = os.path.dirname(__file__)
Try this code
super_bowls = pd.read_csv( os.getcwd() + '/datasets/super_bowls.csv')
I noticed this problem a few years ago. I think it's a matter of design style. The problem is that: your workspace folder is just a folder, not a project folder. Most of the time, your relative reference is based on the current file.
VSCode actually supports the dynamic setting of cwd, but that's not the default. If your work folder is not a rigorous and professional project, I recommend you adding the following settings to launch.json. This is the simplest answer you need.
"cwd": "${fileDirname}"
Thanks to everyone that tried to help me. Thanks to the Roy2012 response, I got a code that works for me.
import pandas as pd
import os
currentPath = os.path.dirname(__file__)
# Load the CSV data into DataFrames
super_bowls = pd.read_csv(currentPath + '/datasets/super_bowls.csv')
The os.path.dirname gives me the path of the current file, and let me work with relative paths.
'/Users/ivanparra/AprendizajePython/Jupyter'
and with that it works like a charm!!
P.S.: As a side note, the behavior of os.getcwd() is quite different in a Jupyter Notebook than a python file. Inside the notebook, that function gives the current file path, but in a python file, gives the top folder path.
I have a python script (db.py) which has a class(Chat) which refers to a json file(cred.json) saved in the same folder(db). This script runs perfectly fine without any error.
When I try to import the class Chat in a python script(wa.py) which is saved in one folder above folder db I get an error saying "[Errno 2] No such file or directory".
My folder structure looks like below
- wa.py
- db
- db.py
- cred.json
Piece of code in db.py where I am referring to the json file
cred_filename = 'creds.json'
with open(cred_filename, 'r') as c:
data = json.load(c)
wa.py looks like this
from db.db import Chat
I tried to find answer on google but couldn't find a relevant explanation for my case. I know there is something fundamentally wrong here but I am not able to figure it out.
Thanks in advance.
Essentially you're trying to open creds.json which is relative to your current working directory (unless changed during the runtime, one you have been in when calling the script) which does not need to be and apparently when this happens is not the one in which db.py resides.
This can only work and worked, as long as you were already in db/ when importing / running db.py.
If you always wanted to get creds.json in the directory where db.py resides, you could say for instance:
from pathlib import Path
...
cred_filename = Path(__file__).with_name("creds.json")
That takes path of the file this module is imported from and replaces the filename part with "creds.json".
I'm trying to load a simple text file with an array of numbers into Python. A MWE is
import numpy as np
BASE_FOLDER = 'C:\\path\\'
BASE_NAME = 'DATA.txt'
fname = BASE_FOLDER + BASE_NAME
data = np.loadtxt(fname)
However, this gives an error while running:
OSError: C:\path\DATA.txt not found.
I'm using VSCode, so in the debug window the link to the path is clickable. And, of course, if I click it the file opens normally, so this tells me that the path is correct.
Also, if I do print(fname), VSCode also gives me a valid path.
Is there anything I'm missing?
EDIT
As per your (very helpful for future reference) comments, I've changed my code using the os module and raw strings:
BASE_FOLDER = r'C:\path_to_folder'
BASE_NAME = r'filename_DATA.txt'
fname = os.path.join(BASE_FOLDER, BASE_NAME)
Still results in error.
Second EDIT
I've tried again with another file. Very basic path and filename
BASE_FOLDER = r'Z:\Data\Enzo\Waste_Code'
BASE_NAME = r'run3b.txt'
And again, I get the same error.
If I try an alternative approach,
os.chdir(BASE_FOLDER)
a = os.listdir()
then select the right file,
fname = a[1]
I still get the error when trying to import it. Even though I'm retrieving it directly from listdir.
>> os.path.isfile(a[1])
False
Using the module os you can check the existence of the file within python by running
import os
os.path.isfile(fname)
If it returns False, that means that your file doesn't exist in the specified fname. If it returns True, it should be read by np.loadtxt().
Extra: good practice working with files and paths
When working with files it is advisable to use the amazing functionality built in the Base Library, specifically the module os. Where os.path.join() will take care of the joins no matter the operating system you are using.
fname = os.path.join(BASE_FOLDER, BASE_NAME)
In addition it is advisable to use raw strings by adding an r to the beginning of the string. This will be less tedious when writing paths, as it allows you to copy-paste from the navigation bar. It will be something like BASE_FOLDER = r'C:\path'. Note that you don't need to add the latest '\' as os.path.join takes care of it.
You may not have the full permission to read the downloaded file. Use
sudo chmod -R a+rwx file_name.txt
in the command prompt to give yourself permission to read if you are using Ubuntu.
For me the problem was that I was using the Linux home symbol in the link (~/path/file). Replacing it with the absolute path /home/user/etc_path/file worked like charm.