I am trying to copy files from a directory based on a list of file names. The code is working fine as long as the file from the list exists in the directory, but if it does not the process will stop and I get the error: No such file or directory
I would like the process to proceed to the next file if it doesnt find the file from the list in the directory. Any ideas how I can achieve this?
This is my code:
# Import libraries
import pandas as pd
import shutil
import os
import subprocess
from joblib import Parallel, delayed
import multiprocessing
# define how many cores should be used
num_cores = 30
os.chdir(source_folder)
# start to copy the tif images to the local drive
tmp = Parallel(n_jobs=num_cores)(delayed(shutil.copy)(f, dst_folder) for f in source_files_list)
cheers!
I've so many different ways and it is still saying file not found
I'm running the code in a Jupyter Notebook.
I'd rather run the file from wherever it is. Here is the infomation for its location
Have I generated the correct code (below is the code).
import numpy
numpy.loadtxt(fname='C:\Desktop\swc-python\data\inflammation-01.csv', delimiter=',')
Also tried this but it did not work:
import numpy
numpy.fname = ('C:\Desktop\swc-python\data\small-01.csv')
openfname = open(fname,'r')
Also, an you save a Jupyter notebook in the same directory as the infomation.
these are some examples with pandas, and os, maybe they could help
they use slash, not backward slash (this option, or the option below)
# import pandas as pd
pd.read_csv("C:/Users/<Insert your user>/Desktop/code/Exercise Files/us_baby_names.csv")
or
(option below), change the current directory,
# import os and pandas library
import os
import pandas as pd
# show current working directory, change it, show it again
os.getcwd()
os.chdir('C:/Users/<Insert your user>/Desktop/code/Exercise Files/')
os.getcwd()
pd.read_csv("us_baby_names.csv")
In my google drive I have a folder called Colab Notebooks/data/, How can I append this path to system, so that I dont have to give the full name of data file ?
My attempt:
from google.colab import drive
drive.mount('/content/drive')
dat_dir = 'drive/My Drive/Colab Notebooks/data/'
# read data
import pandas as pd
pd.read_csv(dat_dir + 'titanic_kaggle_train.csv') # this works
I want this
pd.read_csv('titanic_kaggle_train.csv') # Here, I dont have full path
I tried
import sys
sys.path.append(dat_dir) # did not work
# another attempt (did not work)
!export PYTHONPATH=$HOME/drive/My\ Drive/Colab\ Notebooks/data/:$PYTHONPATH
Question
How can this command work?
pd.read_csv('titanic_kaggle_train.csv')
Is this, what you are searching (change the working directory using os.chdir:
import os
os.chdir(dat_dir)
pd.read_csv('titanic_kaggle_train.csv')
You can also use a one-line magic.
%cd drive/My\ Drive/Colab\ Notebooks/data/
When loading a dataset into Jupyter, I know it requires lines of code to load it in:
from tensorflow.contrib.learn.python.learn.datasets import base
# Data files
IRIS_TRAINING = "iris_training.csv"
IRIS_TEST = "iris_test.csv"
# Load datasets.
training_set = base.load_csv_with_header(filename=IRIS_TRAINING,
features_dtype=np.float32,
target_dtype=np.int)
test_set = base.load_csv_with_header(filename=IRIS_TEST,
features_dtype=np.float32,
target_dtype=np.int)
So why is ther error NotFoundError: iris_training.csv
still thrown? I feel as though there is more to loading data sets on to jupyter, and would be grateful on any help on this topic
I'm following a course through AI adventures, and dont know how to add in the .csv file; the video mentions nothing about how to add it on.
Here is the link: https://www.youtube.com/watch?v=G7oolm0jU8I&list=PLIivdWyY5sqJxnwJhe3etaK7utrBiPBQ2&index=3
The issue is that you either need to use file's absolute path i.e C:\path_to_csv\iris_training.csv for windows and for UNIX/Linux /path_to_csv/iris_training.csv or you will need to place the file in your notebook workspace i.e directory that is being listed in your Jupyter UI which can be found at http://localhost:8888/tree Web UI. If you are having trouble finding the directory then just execute below python code and place the file in the printed location
import os
cwd = os.getcwd()
print(cwd)
Solution A
if you are working with python you can use python lib pandas to import your file .csv using:
import pandas as pd
IRIS_TRAINING = pd.read_csv("../iris_training.csv")
IRIS_TEST = pd.read_csv("../iris_test.csv")
Solution B
import numpy as np
mydata = np.genfromtxt(filename, delimiter=",")
Read More About
python-pandas
Read More About
python-Numpy
I am a little new to Python, and I have been using the Jupyter Notebook through Anaconda. I am trying to import a csv file to make a DataFrame, but I am unable to import the file.
Here is an attempt using the local method:
df = pd.read_csv('Workbook1')
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-11-a2deb4e316ab> in <module>()
----> 1 df = pd.read_csv('Workbook1')
After that I tried using the path (I put user for my username)
df = pd.read_csv('Users/user/Desktop/Workbook1.csv')
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-13-3f2bedd6c4de> in <module>()
----> 1 df = pd.read_csv('Users/user/Desktop/Workbook1.csv')
I am using a Mac, which I am also new to, and I am not 100% sure if I am correctly importing the right path. Can anyone offer some insight or solutions that would allow me to open this csv file.
Instead of providing path, you can set a path using the code below:
import os
import pandas as pd
os.chdir("D:/dataset")
data = pd.read_csv("workbook1.csv")
This will surely work.
Are you sure that the file exists in the location you are specifying to the pandas read_csv method? You can check using the os python built in module:
import os
os.path.isfile('/Users/user/Desktop/Workbook1.csv')
Another way of checking if the file of interest is in the current working directory within a Jupyter notebook is by running ls -l within a cell:
ls -l
I think the problem is probably in the location of the file:
df1 = pd.read_csv('C:/Users/owner/Desktop/contacts.csv')
Having done that, now you can play around with the big file if you have, and create useful data with:
df1.head()
The OS module in python provides functions for interacting with the operating system. OS, comes under Python’s standard utility modules.
import os
import pandas as pd
os.chdir("c:\Pandas")
df=pd.read_csv("names.csv")
df
This might help. :)
The file name is case sensitive, so check your case.
I had the same problem on a Mac too and for some reason it only happened to me there. And I tried to use many tricks but nothing works. I recommend you go directly to the file, right click and then press “alt” key after that the option to “copy route” will appear, and just paste it into your jupyter. For some reason that worked to me.
I believe the issue is that you're not using fully qualified paths. Try this:
Move the data into a suitable project directory. You can do this using the %%bash Magic commands.
%%bash
mkdir -p /project/data/
cp data.csv /project/data/data.csv
You can read the file
f = open("/project/data/data.csv","r")
print(f.read())
f.close()
But it might be most useful to load it into a library.
import pandas as pd
data = pd.read_csv("/project/data/data.csv")
I’ve created a runnable Jupyter notebook with more details here: Jupyter Basics: Reading Files.
Try double quotes, instead of single quotes. it worked for me.
you can open csv files in Jupyter notebook by following these easy steps-
Step 1 - create a directory or folder (you can also use old created folder)
Step 2 - Change your Jupyter working directory to that created directory -
import os
os.chdir('D:/datascience/csvfiles')
Step 3 - Now your directory is changed in Jupyter Notebook. Store your file(s) in that directory.
Step 4 - Open your file -
import pandas as pd
df = pd.read_csv("workbook1.csv")
Now your file is read and stored in a Data Frame variable df, you can display this file content by following
df.head() - display first five rows of this file
df - display all rows of this file
Happy Data Science!
There was a similar problem for me while reading a CSV file in Jupyter notebook from the computer.
I solved it by substituting the "" symbol with "/" in the path like this.
This is what I had:
"C:\Users\RAJ\Desktop\HRPrediction\HRprediction.csv"
This is what I changed it for:
"C:/Users/RAJ/Desktop/HRPrediction/HRprediction.csv".
This is what worked for me. I am using Mac OS.
Save your CSV on a separate folder on your desktop.
When opening a Jupyter notebook press on the same folder that your dataset is currently saved in. Press new notebook in the upper right hand corner.
After opening a new notebook. Code as per usual and read your data using import pandas as pd and pd.read_csv calling to your dataset.
No need to use anything extra just use r in front of the location.
df = pd.read_csv(r'C:/Users/owner/Desktop/contacts.csv'