The intent is to look in a json file in the directory above the script and load up what it finds in that file. This is what I've got:
import os
import json
settings_file = '/home/me/foo/bar.txt'
root = os.path.dirname(os.path.dirname(os.path.abspath(settings_file))) # '/home/me'
target = os.path.join(root,'.extras.txt') # '/home/me/.extras.txt'
db_file= open(target)
databases = json.load(db_file) # works, returns object
databases2 = json.load(open(target)) # equivalent to above, also works
# try to condense code, lose pointless variables target and file
databases3 = json.load(open(os.path.join(root,'.extras.txt'))) # equivalent (I thought!) to above, doesn't work.
So... why doesn't the all-at-once, no holding variables version work? Oh, the error returned is (now in it's entirety):
$ ./json_test.py
Traceback (most recent call last):
File "./json_test.py", line 69, in <module>
databases = json.load(open(os.path.join(root,'/.extras.txt')))
IOError: [Errno 2] No such file or directory: '/.extras.txt'
And to satisfy S.Lott's well-intentioned advice... it doesn't matter what target is set to. The databases and databases2 populate correctly while databases3 does not. target exists, is readable and contains what json expects to see. I suspect there's something I don't understand about the nature of stringing commands together... I can make the code work, was just wondering why the concise (or complex?) version failed.
Code looks fine, make sure referenced files are in the appropriate places. Given your code that includes target/file variable assignment, full path to .extras.txt is
/home/me/.extras.txt
You need to do:
file = open(target, 'w')
because by default open will try to open the file in read mode (r) but you need to open it in w (write) mode if you want it to be created.
Also, I would not use the variable name file since it is also a type (<type 'file'>) in python.
You could add the write-mode flag to this line as well:
databases = json.load(open(os.path.join(root,'.extras.txt'), 'w'))
because from the limited information we have in the question it appears your /.extras file does not previously exist.
Final note, you are losing the handle to your open file in this line (since you are not storing it in your file variable):
databases = json.load(open(os.path.join(root,'.extras.txt')))
How do you intend to close the file when you're finished with it?
You could do this with a context manager (python >=2.6 or 2.5 if import with_statement used):
with open(os.path.join(root,'.extras.txt'), 'w') as f:
databases = json.load(f)
which will take care of closing the file for you.
Related
I have code like this:
with open('foo.txt') as file:
...do something with file...
...move foo.txt to another place while it's still open...
Are there any problems with that?
This depends on the operating system. In Unix-based systems this is generally safe (you can move or even delete file while still having it open). Windows will produce Access Denied error if you try to move/delete an opened file.
So yes, the safest and portable way is to close the file first. Note that with clause closes the file automatically, so all you need is to perform the move outside of with block.
On Windows:
>>> import os
>>> with open('old.txt') as file:
os.rename('old.txt', 'new.txt')
Traceback (most recent call last):
File "<pyshell#4>", line 2, in <module>
os.rename('test.txt', 'newtest.txt')
WindowsError: [Error 32] The process cannot access the file because it is being used by another process
You can't move the file, because someone(you) is already holding it. You need to close the file before you can move it.
Best practice with file is close and move, because if you will not close then it might be create problem in some OS, like Windows.
To make your code more portable, close file before move.
yes , I was getting same issue,
I was doing some operation in for loop(if rows/content is valid then move destination folder)
outside for loop, if no header/no content/no rows then move to error folder
and I was getting error
I could resolve it using "read_obj.close" jus before filemove method
here is the snippet
with open(csv_file, 'r') as read_obj:
# pass the file object to reader() to get the reader object
csv_dic_reader = csv.DictReader(read_obj)
data = [row for row in csv_dic_reader]
for row in data:
read_obj.close()
filesMove(csv_file, destination_file_path, src_file_path)
if len(data) <= 1: # this will check no rows/content
read_obj.close()
filesMove(csv_file, error_file_path, src_file_path)
I'm supposed to upload a database for an assignment, but I'm having a problem. These are the instructions:
This application will read the mailbox data (mbox.txt) count up the
number email messages per organization (i.e. domain name of the email
address) using a database with the following schema to maintain the
counts.
CREATE TABLE Counts (org TEXT, count INTEGER) When you have run the
program on mbox.txt upload the resulting database file above for
grading. If you run the program multiple times in testing or with
different files, make sure to empty out the data before each run.
The data file for this application is the same as in previous
assignments: http://www.pythonlearn.com/code/mbox.txt.
Because the sample code is using an UPDATE statement and committing
the results to the database as each record is read in the loop, it
might take as long as a few minutes to process all the data. The
commit insists on completely writing all the data to disk every time
it is called.
The error message that it keeps sending me is:
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
fh = open(fname)
IOError: [Errno 2] No such file or directory: 'mbox.txt'
I saved them both in the same folder.
Can anybody help with this?
The code that I entered is here
Your code is having problem to find the file mbox.txt. It has nothing to do
with anything in the database as you did not run so far yet.
Good practice (at least during development) is to make sure, the things you
hope are true are really true. For this purpose I would use following code
which makes sure, the file really exists.
import os.path
fname = "mbox.txt"
assert os.path.exists(fname), "The file shall exist"
If you happen to run the code in situation, the file does not exist, it will throw an
AssertionError telling you what went wrong.
This exception is very practical as it will tell you quickly what assumption does not holds true and
you know, what to fix.
Your code is looking for a file called mbox.txt and not finding it. My guess is that open(fname) is looking for mbox.txt in the current directory, but the code is being run from a different directory.
Something like this might help resolve your issue:
import os
# figure out directory of the Python file
mdir = os.path.dirname(os.path.abspath(__file__))
# assuming that mbox.txt is in the same folder as the Python file,
# get the path to that file
mpath = os.path.join(mdir, 'mbox.txt')
# open the file
with open(mpath, 'r') as fh:
# ...
Another approach is using command line arguments. Perhaps there are other files like mbox.txt that you want to work with. In these cases, you could accept the path to mbox.txt as a command line option:
import argparse
argp = argparse.ArgumentParser(description='foo the mbox')
argp.add_argument('mbox_path', help='Path to mbox file')
opts = argp.parse_args()
with open(opts.mbox_path, 'r') as fh:
# ...
Or get fancier and use argparse.FileType for the type argument to argparse.add_argument.
Answer Has Been Decided
The problem was not listing the entire file listing (i.e. DRIVE1\DRIVE2\...\fileName.txt).
Also, note that the backslashes must be turned into forwardslashes in order for
Python to read it (backslashes are outside of the unicode listing).
Thus, using: addy = 'C:/Users/Tanner/Desktop/Python/newfile.txt'
returns the desired results.
It's been a while since I have played with Python, and for my most recent class, we are required to make a BFS search that does a Word Puzzle that the Alice in Wonderland author created. I am just stating this, as the algorithm is the homework, which I have completed. In other words, my question does not apply to the answer to my homework question.
With that out of the way, I am in need of help on how to open, edit, read, create some form of text files in Python. My real problem is to place a list of words that I have inside of a .txt file, into a Dictionary dictionary. but I would much rather do this myself. Thus, I am left with how to do the said to text files.
NOTE:
I am running v3.3.
All documentation that I have found while searching how to solve this simple problem
is in regards to 2.7 or older.
I have tried to use:
>>> import sys from argv
>>> script, filename = argv
Traceback (most recent call last):
File "<pyshell#15>", line 1, in <module>
script, filename = argv
ValueError: need more than 1 value to unpack
I have also tried to use:
>>> f = open(newfile.txt, 'r')
But again, I get this error:
File "<pyshell#8>", line 1, in <module>
f = open (filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'newfile.txt'
However, I am positive that this file does exist. All of this being said, I am not sure if this is a directory problem, a problem understanding, or what... That is, anything would help!
First, if you want to retrieve a file name which is passed as the first argument to your script, use code like this:
import sys
if len(sys.argv) > 2:
filename = sys.argv[1]
else:
# set a default filename or print an error
Secondly, the error clearly indicates that the script can't find the file newfile.txt. So it is either not in the current directory, you don't have the permission to read it, etc...
To open a file for reading, use with command as follows
# This is python 3 code
with open('yourfile.txt', 'r') as f:
for line in f:
print(line)
with command used together with open command has already the raising exception process.
I saved a file as DictionaryE.txt in a Modules folder I created within Python. Then I type:
fh = open("DictionaryE.txt")
I get this error message:
Traceback (most recent call last):
File "<pyshell#17>", line 1, in <module>
fh = open("DictionaryE.txt")
IOError: [Errno 2] No such file or directory: 'DictionaryE.txt'
What am I doing wrong? Could someone please describe a specific, detailed step-by-step instruction on what to do? Thanks.
As other answers suggested, you need to specify the file's path, not just the name.
For example, if you know the file is in C:\Blah\Modules, use
fh = open('c:/Blah/Modules/DictionaryE.txt')
Note that I've turned the slashes "the right way up" (Unix-style;-) rather than "the Windows way". That's optional, but Python (and actually the underlying C runtime library) are perfectly happy with it, and it saves you trouble on many occasions (since \, in Python string literals just as in C ones, is an "escape marker", once in a while, if you use it, the string value you've actually entered is not the one you think -- with '/' instead, zero problems).
Use the full path to the file? You are trying to open the file in the current working directory.
probably something like:
import os
dict_file = open(os.path.join(os.path.dirname(__file__), 'Modules', 'DictionaryE.txt'))
It's hard to know without knowing your project structure and the context of your code. Fwiw, when you just "open" a file, it will be looking in whatever directory you're running the python program in, and __file__is the full path to ... the python file.
To complement Alex's answer, you can be more specific and explicit with what you want to do with DictionaryE.txt. The basics:
READ (this is default):
fh = open("C:/path/to/DictionaryE.txt", "r")
WRITE:
fh = open("C:/path/to/DictionaryE.txt", "w")
APPEND:
fh = open("C:/path/to/DictionaryE.txt", "a")
More info can be found here: Built-in Functions - open()
I hope I framed the question right. I am trying to force myself to be a better programmer. By better I mean efficient. I want to write a program to identify the files in a directory and read each file for further processing. After some shuffling I got to this:
for file in os.listdir(dir):
y=open(dir+'\\'+file,'r').readlines()
for line in y:
pass
y.close()
It should be no surprise that I get an AttributeError since y is a list. I didn't think about that when I wrote the snippet.
I am thinking about this and am afraid that I have five open files (there are five files in the directory specified by dir.
I can fix the code so it runs and I explicitly close the files after opening them. I am curious if I need to or if Python handles closing the file in the next iteration of the loop. If so then I only need to write:
for file in os.listdir(dir):
y=open(dir+'\\'+file,'r').readlines()
for line in y:
pass
I am guessing that it(python) does handle this effortlessly. The reason I think that this might be handled is that I have changed the object/thing that y is referencing. When I start the second iteration there are no more memory references to the file that was opened and read using the readlines method.
Python will close open files when they get garbage-collected, so generally you can forget about it -- particularly when reading.
That said, if you want to close explicitely, you could do this:
for file in os.listdir(dir):
f = open(dir+'\\'+file,'r')
y = f.readlines()
for line in y:
pass
f.close()
However, we can immediately improve this, because in python you can iterate over file-like objects directly:
for file in os.listdir(dir):
y = open(dir+'\\'+file,'r')
for line in y:
pass
y.close()
Finally, in recent python, there is the 'with' statement:
for file in os.listdir(dir):
with open(dir+'\\'+file,'r') as y:
for line in y:
pass
When the with block ends, python will close the file for you and clean it up.
(you also might want to look into os.path for more pythonic tools for manipulating file names and directories)
Don't worry about it. Python's garbage collector is good, and I've never had a problem with not closing file-pointers (for read operations at least)
If you did want to explicitly close the file, just store the open() in one variable, then call readlines() on that, for example..
f = open("thefile.txt")
all_lines = f.readlines()
f.close()
Or, you can use the with statement, which was added in Python 2.5 as a from __future__ import, and "properly" added in Python 2.6:
from __future__ import with_statement # for python 2.5, not required for >2.6
with open("thefile.txt") as f:
print f.readlines()
# or
the_file = open("thefile.txt")
with the_file as f:
print f.readlines()
The file will automatically be closed at the end of the block.
..but, there are other more important things to worry about in the snippets you posted, mostly stylistic things.
Firstly, try to avoid manually constructing paths using string-concatenation. The os.path module contains lots of methods to do this, in a more reliable, cross-platform manner.
import os
y = open(os.path.join(dir, file), 'r')
Also, you are using two variable names, dir and file - both of which are built-in functions. Pylint is a good tool to spot things like this, in this case it would give the warning:
[W0622] Redefining built-in 'file'