I have several python files, and I want to run in a different directory where it will search for a particular pattern from 100s of files. Let's say I have the python file in /home/jay directory, and I have 100s of files in /home/jay/data directory.
What can I do to achieve this?
My python code is :
import re
import os
os.chdir(r'/home/jay/data/')
files = open('*')
for line in files :
line = line.rstrip()
if re.search('Model' , line):
print(line)
I'm getting following error:
Traceback (most recent call last):
File "/home/jay/test.py", line 4, in ?
files = open('*')
IOError: [Errno 2] No such file or directory: '*'
You are looking for os.listdir. It will give you a list of all the file names in the specified directory, which defaults to the current directory. The reason that '*' does not work is that it is a command-line construct that is expanded by your shell. You can only really use it in a shell or script that supports that sort of expansion. Since open does not go through the shell, it tries to find a file actually named *. Also, open can only deal with one file at a time.
import os, os.path, re
os.chdir(r'/home/jay/data/')
files = os.listdir()
for name in files:
# Skip directories
if os.path.isdir(name):
continue
with open(name) as file:
for line in file:
line = line.rstrip()
if re.search('Model' , line):
print(line)
That being said, as a matter of personal preference, I generally avoid using os.chdir. Instead, I prefer specifying full paths using os.path.join. Here is your example rewritten to do that:
from os import listdir
from os.path import join, isdir
folder = '/home/jay/data'
files = listdir(folder)
for name in files:
name = join(folder, name)
# Skip directories
if isdir(name):
continue
with open(name) as file:
for line in file:
line = line.rstrip()
if 'Model' in line:
print(line)
I took the liberty of removing the regex completely since it only serves to slow things down in you have many files. If you do use a regex for some more complicated scenario, compile if before you use it using re.compile.
Furthermore, you are free to use relative paths here if you want. For example, if you are always running from /home/jay, you can set folder = 'data' instead of folder = '/home/jay/data' in the second example.
Related
This question already has answers here:
open() gives FileNotFoundError / IOError: '[Errno 2] No such file or directory'
(8 answers)
Closed 7 months ago.
I'm trying to write a simple program to read a file and search for a word then print how many times that word is found in the file. Every time I type in "test.rtf" (which is the name of my document) I get this error:
Traceback (most recent call last):
File "/Users/AshleyStallings/Documents/School Work/Computer Programming/Side Projects/How many? (Python).py", line 9, in <module>
fileScan= open(fileName, 'r') #Opens file
FileNotFoundError: [Errno 2] No such file or directory: 'test.rtf'
In class last semester, I remember my professor saying you have to save the file in a specific place? I'm not sure if he really said that though, but I'm running apple OSx if that helps.
Here's the important part of my code:
fileName= input("Please enter the name of the file you'd like to use.")
fileScan= open(fileName, 'r') #Opens file
If the user does not pass the full path to the file (on Unix type systems this means a path that starts with a slash), the path is interpreted relatively to the current working directory. The current working directory usually is the directory in which you started the program. In your case, the file test.rtf must be in the same directory in which you execute the program.
You are obviously performing programming tasks in Python under Mac OS. There, I recommend to work in the terminal (on the command line), i.e. start the terminal, cd to the directory where your input file is located and start the Python script there using the command
$ python script.py
In order to make this work, the directory containing the python executable must be in the PATH, a so-called environment variable that contains directories that are automatically used for searching executables when you enter a command. You should make use of this, because it simplifies daily work greatly. That way, you can simply cd to the directory containing your Python script file and run it.
In any case, if your Python script file and your data input file are not in the same directory, you always have to specify either a relative path between them or you have to use an absolute path for one of them.
Is test.rtf located in the same directory you're in when you run this?
If not, you'll need to provide the full path to that file.
Suppose it's located in
/Users/AshleyStallings/Documents/School Work/Computer Programming/Side Projects/data
In that case you'd enter
data/test.rtf
as your file name
Or it could be in
/Users/AshleyStallings/Documents/School Work/Computer Programming/some_other_folder
In that case you'd enter
../some_other_folder/test.rtf
As noted above the problem is in specifying the path to your file.
The default path in OS X is your home directory (/Users/macbook represented by ~ in terminal ...you can change or rename the home directory with the advanced options in System Preferences > Users & Groups).
Or you can specify the path from the drive to your file in the filename:
path = "/Users/macbook/Documents/MyPython/"
myFile = path + fileName
You can also catch the File Not Found Error and give another response using try:
try:
with open(filename) as f:
sequences = pick_lines(f)
except FileNotFoundError:
print("File not found. Check the path variable and filename")
exit()
A good start would be validating the input. In other words, you can make sure that the user has indeed typed a correct path for a real existing file, like this:
import os
fileName = input("Please enter the name of the file you'd like to use.")
while not os.path.isfile(fileName):
fileName = input("Whoops! No such file! Please enter the name of the file you'd like to use.")
This is with a little help from the built in module os, That is a part of the Standard Python Library.
You might need to change your path by:
import os
path=os.chdir(str('Here_should_be_the_path_to_your_file')) #This command changes directory
This is what worked for me at least! Hope it works for you too!
Difficult to give code examples in the comments.
To read the words in the file, you can read the contents of the file, which gets you a string - this is what you were doing before, with the read() method - and then use split() to get the individual words.
Split breaks up a String on the delimiter provided, or on whitespace by default. For example,
"the quick brown fox".split()
produces
['the', 'quick', 'brown', 'fox']
Similarly,
fileScan.read().split()
will give you an array of Strings.
Hope that helps!
First check what's your file format(e.g: .txt, .json, .csv etc ),
If your file present in PWD , then just give the name of the file along with the file format inside either single('')or double("") quote and the appropriate operation mode as your requirement
e.g:
with open('test.txt','r') as f: data=f.readlines() for i in data: print(i)
If your file present in other directory, then just give the full path name where is your file is present and file name along with file format of the file inside either single('')or double("") quote and the appropriate operation mode as your requirement.
If it showing unicode error just put either r before quote of file path or else put '/' instead of ''
with open(r'C:\Users\soman\Desktop\test.txt','r') as f: data=f.readlines() for i in data: print(i)
The mistake I did was
my code :
x = open('python.txt')
print(x)
But the problem was in file directory ,I saved it as python.txt instead of just python .
So my file path was
->C:\Users\noob\Desktop\Python\Course 2\python.txt.txt
That is why it was giving a error.
Name your file without .txt it will run.
I was trying to read data from a bunch of textfiles in a directory, but getting an error while opening the file
import os
fileList = os.listdir("Desktop/SLUI")
for txtName in fileList:
#Open the textfile
UIname=str(txtName)
userDTL=open(UIname,'r')
if userDTL.mode=='r':
line=userDTL.readlines()
string1=line[0]
string2=line[1]
string3=line[2]
UserDTL.close()
print(string1)
Here is the error when I try to run this code via cmd.exe
File "C:\Users\*****\Desktop\programName.py", line 24, in <module>
userDTL=open(UIname,'r')
FileNotFoundError: [Errno 2] No such file or directory: 'file1.txt'
It's because os.listdir only displays the name of the files, and to open them you need the whole path.
You need to redefine UIname as such:
UIname=os.path.join("Desktop/SLUI",txtName) # I don't think you need the string conversion.
os.path.join() will properly join two (or more) bits of paths, whichever OS you use.
I'm not really familiar with how Python works on Windows, so you might need to replace "Desktop/SLUI" by the appropriate path to your desktop (C:\Users*****\Desktop).
I'm using Python 3.5 and I'm having some problems opening a CSV file. I've tried entering the entire path but it still doesn't work, but the file is clearly in the folder. (My code is called 'simplecsvtest.py')
Here's the code snippet:
import csv
import sys
file = open(r"C:\python35\files\results.csv", 'rt')
try:
reader = csv.reader(file, delimiter='\t')
... some code here ...
finally:
file.close()
And here's what PowerShell says:
PS C:\python35\files> python simplecsvtest.py
Traceback (most recent call last):
File "simplecsvtest.py", line 20, in
file = open(r"C:\python35\files\results.csv", 'rt')
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\python35\\files\\results.csv'
Well, I'm very certain that 'results.csv' is in that folder: here's the filepath in Windows Explorer:
C:\Python35\files
(Note: The folder has capital 'P' for Python35, and I've tried having both capitalized and uncapitalized 'P' in the code, neither works)
The CSV file is a "Microsoft Excel Comma Separated Values Files", if that matters, but the extension is still csv. Can anyone tell me what's wrong?
I would suggest creating a folder inside your project folder and then use a relative path :
file = open(r".\files\results.csv", 'rt')
. implies that the path is relative to your current directory
I figured out a work-around solution myself:
Somehow, if I copy all the data from the csv and paste it in a new excel spreadsheet and save it as a csv, it works. I don't know why.
Although the file path is absolute it wouldn't work this way.
You have to use "forward" slashes in the path name instead of "backward" slashes. As
file = open(r"C:/python35/files/results.csv", 'rt')
Im trying to create a random set of files from a list of file names in python 3.
#!/usr/local/bin/python3
import random
import sys
random.seed()
print("Reading lines from ", sys.argv[1])
linecount = 0
lines = []
with open(sys.argv[1]) as f:
for line in f:
lines.append(line)
filelist = random.sample(lines, 5);
for newfile in filelist:
print("Touching file ", newfile)
open(newfile.rstrip(), "a+")
This always fails for the first file with:
$ : ./file-gen.py files.txt
Reading lines from files.txt
Touching file 62/6226320DE.pdf
Traceback (most recent call last):
File "./file-gen.py", line 19, in <module>
open(newfile.rstrip(), "a+");
FileNotFoundError: [Errno 2] No such file or directory: '62/6226320DE.pdf'
Any ideas what's missing?
I believe the problem is that the file mode a+ will create the file if its not present, but not the directory. You would have to write custom code to split the line of the filename from the path (or better still use os.path : https://docs.python.org/2/library/os.path.html, perhaps even os.path.dirname(path)
Have a look at: How to check if a directory exists and create it if necessary?
Be wary of security considerations of creating random paths in your system. Validating that the paths are inside a particular sandbox (consider someone putting an entry in your file of ../../ or /etc/passwd to get you to append random user data.. os.path.abspath can be useful - for this reason I am wary of pasting code which will just create random directories that you copy and paste in without considering that effect.
I would suggest as a first step trying to open a specific file rather than a random set of files from the input to rule out permissions and path problems.
You should also try printing os.path.getcwd() to make sure that you have write permissions there.
This question already has answers here:
open() gives FileNotFoundError / IOError: '[Errno 2] No such file or directory'
(8 answers)
Closed 7 months ago.
I'm trying to write a simple program to read a file and search for a word then print how many times that word is found in the file. Every time I type in "test.rtf" (which is the name of my document) I get this error:
Traceback (most recent call last):
File "/Users/AshleyStallings/Documents/School Work/Computer Programming/Side Projects/How many? (Python).py", line 9, in <module>
fileScan= open(fileName, 'r') #Opens file
FileNotFoundError: [Errno 2] No such file or directory: 'test.rtf'
In class last semester, I remember my professor saying you have to save the file in a specific place? I'm not sure if he really said that though, but I'm running apple OSx if that helps.
Here's the important part of my code:
fileName= input("Please enter the name of the file you'd like to use.")
fileScan= open(fileName, 'r') #Opens file
If the user does not pass the full path to the file (on Unix type systems this means a path that starts with a slash), the path is interpreted relatively to the current working directory. The current working directory usually is the directory in which you started the program. In your case, the file test.rtf must be in the same directory in which you execute the program.
You are obviously performing programming tasks in Python under Mac OS. There, I recommend to work in the terminal (on the command line), i.e. start the terminal, cd to the directory where your input file is located and start the Python script there using the command
$ python script.py
In order to make this work, the directory containing the python executable must be in the PATH, a so-called environment variable that contains directories that are automatically used for searching executables when you enter a command. You should make use of this, because it simplifies daily work greatly. That way, you can simply cd to the directory containing your Python script file and run it.
In any case, if your Python script file and your data input file are not in the same directory, you always have to specify either a relative path between them or you have to use an absolute path for one of them.
Is test.rtf located in the same directory you're in when you run this?
If not, you'll need to provide the full path to that file.
Suppose it's located in
/Users/AshleyStallings/Documents/School Work/Computer Programming/Side Projects/data
In that case you'd enter
data/test.rtf
as your file name
Or it could be in
/Users/AshleyStallings/Documents/School Work/Computer Programming/some_other_folder
In that case you'd enter
../some_other_folder/test.rtf
As noted above the problem is in specifying the path to your file.
The default path in OS X is your home directory (/Users/macbook represented by ~ in terminal ...you can change or rename the home directory with the advanced options in System Preferences > Users & Groups).
Or you can specify the path from the drive to your file in the filename:
path = "/Users/macbook/Documents/MyPython/"
myFile = path + fileName
You can also catch the File Not Found Error and give another response using try:
try:
with open(filename) as f:
sequences = pick_lines(f)
except FileNotFoundError:
print("File not found. Check the path variable and filename")
exit()
A good start would be validating the input. In other words, you can make sure that the user has indeed typed a correct path for a real existing file, like this:
import os
fileName = input("Please enter the name of the file you'd like to use.")
while not os.path.isfile(fileName):
fileName = input("Whoops! No such file! Please enter the name of the file you'd like to use.")
This is with a little help from the built in module os, That is a part of the Standard Python Library.
You might need to change your path by:
import os
path=os.chdir(str('Here_should_be_the_path_to_your_file')) #This command changes directory
This is what worked for me at least! Hope it works for you too!
Difficult to give code examples in the comments.
To read the words in the file, you can read the contents of the file, which gets you a string - this is what you were doing before, with the read() method - and then use split() to get the individual words.
Split breaks up a String on the delimiter provided, or on whitespace by default. For example,
"the quick brown fox".split()
produces
['the', 'quick', 'brown', 'fox']
Similarly,
fileScan.read().split()
will give you an array of Strings.
Hope that helps!
First check what's your file format(e.g: .txt, .json, .csv etc ),
If your file present in PWD , then just give the name of the file along with the file format inside either single('')or double("") quote and the appropriate operation mode as your requirement
e.g:
with open('test.txt','r') as f: data=f.readlines() for i in data: print(i)
If your file present in other directory, then just give the full path name where is your file is present and file name along with file format of the file inside either single('')or double("") quote and the appropriate operation mode as your requirement.
If it showing unicode error just put either r before quote of file path or else put '/' instead of ''
with open(r'C:\Users\soman\Desktop\test.txt','r') as f: data=f.readlines() for i in data: print(i)
The mistake I did was
my code :
x = open('python.txt')
print(x)
But the problem was in file directory ,I saved it as python.txt instead of just python .
So my file path was
->C:\Users\noob\Desktop\Python\Course 2\python.txt.txt
That is why it was giving a error.
Name your file without .txt it will run.