I'm supposed to upload a database for an assignment, but I'm having a problem. These are the instructions:
This application will read the mailbox data (mbox.txt) count up the
number email messages per organization (i.e. domain name of the email
address) using a database with the following schema to maintain the
counts.
CREATE TABLE Counts (org TEXT, count INTEGER) When you have run the
program on mbox.txt upload the resulting database file above for
grading. If you run the program multiple times in testing or with
different files, make sure to empty out the data before each run.
The data file for this application is the same as in previous
assignments: http://www.pythonlearn.com/code/mbox.txt.
Because the sample code is using an UPDATE statement and committing
the results to the database as each record is read in the loop, it
might take as long as a few minutes to process all the data. The
commit insists on completely writing all the data to disk every time
it is called.
The error message that it keeps sending me is:
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
fh = open(fname)
IOError: [Errno 2] No such file or directory: 'mbox.txt'
I saved them both in the same folder.
Can anybody help with this?
The code that I entered is here
Your code is having problem to find the file mbox.txt. It has nothing to do
with anything in the database as you did not run so far yet.
Good practice (at least during development) is to make sure, the things you
hope are true are really true. For this purpose I would use following code
which makes sure, the file really exists.
import os.path
fname = "mbox.txt"
assert os.path.exists(fname), "The file shall exist"
If you happen to run the code in situation, the file does not exist, it will throw an
AssertionError telling you what went wrong.
This exception is very practical as it will tell you quickly what assumption does not holds true and
you know, what to fix.
Your code is looking for a file called mbox.txt and not finding it. My guess is that open(fname) is looking for mbox.txt in the current directory, but the code is being run from a different directory.
Something like this might help resolve your issue:
import os
# figure out directory of the Python file
mdir = os.path.dirname(os.path.abspath(__file__))
# assuming that mbox.txt is in the same folder as the Python file,
# get the path to that file
mpath = os.path.join(mdir, 'mbox.txt')
# open the file
with open(mpath, 'r') as fh:
# ...
Another approach is using command line arguments. Perhaps there are other files like mbox.txt that you want to work with. In these cases, you could accept the path to mbox.txt as a command line option:
import argparse
argp = argparse.ArgumentParser(description='foo the mbox')
argp.add_argument('mbox_path', help='Path to mbox file')
opts = argp.parse_args()
with open(opts.mbox_path, 'r') as fh:
# ...
Or get fancier and use argparse.FileType for the type argument to argparse.add_argument.
Related
I cannot figure out what I am doing wrong. I am running some tests and writing the results to a file. The portion of the code which writes to a file is the following (in a class called Tester):
#staticmethod
def printHeader(resultsFileName):
if not os.path.isfile(resultsFileName):
# The file does not exist, thus
# we need to print the header
# Opens the results file
with open(resultsFileName,"a") as file:
# Prints the header
file.write("A long header")
#staticmethod
def printResults(resultsFileName,otherArguments):
# Prints the header if it does not exist
Tester.printHeader(resultsFileName)
# Prints the results
with open(resultsFileName,"a") as file:
file.write(otherArguments)
Sometimes I get this error:
Traceback (most recent call last):
File "main.py", line 74, in <module>
File "tester.py", line 88, in methodOne
File "tester.py", line 441, in printResults
File "tester.py", line 428, in printHeader
IOError: [Errno 13] Permission denied: 'results.txt'
while other times it runs smoothly. I cannot figure out where the problem is. Any ideas?
NOTE1: I have rwxpermission on the directory where the file is written.
NOTE2: The error happens after several lines of results have already been written. Thus, it happens when the code is checking whether the header should be printed or not (but it is not supposed to print it, since the file exists).
UPDATE 1:
As suggested, I have changed my code to avoid opening and closing the file multiple times. Now it writes everything in one shot. This is the updated code:
#staticmethod
def printResults(resultsFileName,otherArguments):
# Prints the header if it does not exist
if not os.path.exists(resultsFileName):
# The file does not exist, thus
# we need to print the header
# Opens the results file
# HERE IS WHERE ERRNO 13 HAPPENS
# STRANGELY, THE FILE DOES EXIST
# AS SEVERAL LINES OF RESULTS
# HAVE ALREADY BEEN WRITTEN
with open(resultsFileName,"w") as file:
# Prints the header
file.write("A suitable header")
# Prints the results
file.write(otherArguments)
else:
# Prints the results
with open(resultsFileName,"a") as file:
file.write(otherArguments)
It seems that os.path.exists() at some point returns FALSEeven if the file does exist. Probably, there is something revoking me permission to write (perhaps the file is not properly closed after writing?).
The explanation of os.path.exists() says that:
On some platforms, this function may return False if permission is not granted to execute os.stat() on the requested file, even if the path physically exists.
UPDATE 2
I have changed my code to the following, to avoid os.path.isfile():
# Opens the results file
with open(resultsFileName,"a") as file:
if file.tell() == 0:
# Prints the header
file.write("Header")
# Prints the results
file.write(otherArguments)
file.close()
else:
# Prints the results
file.write(otherArguments)
file.close()
Nevertheless, ERRNO 13 happens at with open(resultsFileName,"a") as file:.
I have rwpermissions both on the folder and on the file, on which several lines are written before the error happens. The OS is Linux.
Try closing the file result.txt before running the program (I'd its open).
Check the writing permissions.
Another cause might be writing into a .txt file in a directory without those permissions. Try running python from the specified directory.
os.path.isfile(path) returns True if path exists AND is a file AND you have (at least) read permissions on it. IOW, it returns False if path exists but is a directory and/or you don't have permissions on it. So your test is wrong from the start - os.path.exists() would be a better choice. But anyway: those operations are not atomic so the file could well be created between the moment where you test for it's existence and the moment where you try to open it, so the whole design is actually unsafe. Also, stating, opening and closing a file are all costly operations, so I suggest - if possible - that you rethink your whole design to open a file only once and only close it once when you're done.
You can check if the file is open with another program. If so, you can try closing them all.
Cheers!
Surely it will work
I am attempting to write a short piece of code which will allow me to write the serial number of a Raspberry Pi and the date/time into a text file. This information will be written to a USB stick every time it is plugged into my Raspberry Pi. This script will be used on multiple Raspberry Pi's with different USB sticks so I have been trying to be as general as possible because paths change, etc. between the pieces of hardware.
The first part of my script works fine:
#import packages
from datetime import datetime
import os
import os.path
from shutil import copyfile
usblist = [x for x in usblist if not x.startswith('SETTINGS')] #For some reason I have some folders starting with SETTINGS that I don't want to delete but simply ignore in my list
#create a new list with the paths for each USB
usbpathlist = os.path.abspath(usblist[0]),os.path.abspath(usblist[1]) #I will only have two USB sticks inserted at a time
#for some reason my paths show as in the /home/ directory when it appears that they are mounted in /media/
usbpathlist = [w.replace('home', 'media') for w in usbpathlist] #this fixes the paths
I then pull the information that is unique to the Raspberry Pi and the date/time and save them as variables
def getserial():
#Extract the serial number from the cpuinfo file
cpuserial = "0000000000000000"
try:
f = open("/proc/cpuinfo", "r")
for line in f.readlines():
if line[0:6] == "Serial":
cpuserial = line[10:26]
f.close()
except:
cpuserial = "ERROR00000000000"
return cpuserial
rpi_serial = getserial()
time_experiment = str(datetime.now())
Now I write this information as a text file
with open("rpi_information.txt", "w") as rpi_information:
rpi_information.write("Raspberry Pi Serial Number: " + rpi_serial + "\n") #Write the serial number
rpi_information.write("Time written: " + time_experiment + "\n") #Write the date
rpi_information.close()
Finally the problematic part of my script is where I try to save this file to the paths of my USB sticks that I have saved above as 'usbpathlist'
for d in usbpathlist:
copyfile(rpi_information, d)
From here is where my problem lies- I have spent quite a bit of time googling and searching this site but I have been unable to figure out how to save this file to each USB stick after it has been created. A lot of the information online suggests that I cannot actually save text files to multiple locations at once, but that seems unlikely. If I manually insert the path in front of "rpi_information.txt" above, it saves without issue. My problem is that because my USB sticks will be different, these paths will change so I don't think this is a good solution. The errors that I have been receiving from this last part of the script (as is probably expected) are:
Traceback (most recent call last):
File "home/pi/test.py", line 55 in <module> #test.py is my filename
copyfile(rpi_information, d)
File "/usr/lib/python3.5/shutil.py", line 103, in copyfile
if _samefile(src,dst):
File "/usr/lib/python3.5/shutil.py", line 88, in samefile
return os.path.samefile(src,dst)
File "/usr/lib/python3.5/genericpath.py", line 90, in samefile
s1 = os.stat(f1)
TypeError: argument should be string, bytes, or integer, not _io.TextWrapper
Ultimately, I would like to find a way to pass this file to both USB sticks without issue. I'd like to keep this information as a text file because I will want to refer to it later on another computer. Any insight into how to make this work would be appreciated or any comments on how to improve my code would also be appreciated. This is my first python script and my first question on StackOverFlow (even though I have lurked for years), so please go easy on me! I have read a lot of documentation about this issue as well so I'm just baffled. Thanks for your assistance in advance.
---EDIT---
User Vulcan below suggested that my error was a failure to include ".txt" in the copyfile function. I had tried this previously to no avail either. This unfortunately produces the error:
Traceback (most recent call last):
File "home/pi/test.py", line 55 in <module> #test.py is my filename
copyfile(rpi_information.txt, d)
AttributeError: '_io.TextWrapper' object has no attribute 'txt'
The error trace highlights your issue here: argument should be string... not _io.TextWrapper. When you call copyfile(rpi_information, d), the rpi_information member is a file handle, rather than a file path, which one of the file types that copyfile expects. Pass the file name:
for d in usbpathlist:
copyfile('rpi_information.txt', d)
Answer Has Been Decided
The problem was not listing the entire file listing (i.e. DRIVE1\DRIVE2\...\fileName.txt).
Also, note that the backslashes must be turned into forwardslashes in order for
Python to read it (backslashes are outside of the unicode listing).
Thus, using: addy = 'C:/Users/Tanner/Desktop/Python/newfile.txt'
returns the desired results.
It's been a while since I have played with Python, and for my most recent class, we are required to make a BFS search that does a Word Puzzle that the Alice in Wonderland author created. I am just stating this, as the algorithm is the homework, which I have completed. In other words, my question does not apply to the answer to my homework question.
With that out of the way, I am in need of help on how to open, edit, read, create some form of text files in Python. My real problem is to place a list of words that I have inside of a .txt file, into a Dictionary dictionary. but I would much rather do this myself. Thus, I am left with how to do the said to text files.
NOTE:
I am running v3.3.
All documentation that I have found while searching how to solve this simple problem
is in regards to 2.7 or older.
I have tried to use:
>>> import sys from argv
>>> script, filename = argv
Traceback (most recent call last):
File "<pyshell#15>", line 1, in <module>
script, filename = argv
ValueError: need more than 1 value to unpack
I have also tried to use:
>>> f = open(newfile.txt, 'r')
But again, I get this error:
File "<pyshell#8>", line 1, in <module>
f = open (filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'newfile.txt'
However, I am positive that this file does exist. All of this being said, I am not sure if this is a directory problem, a problem understanding, or what... That is, anything would help!
First, if you want to retrieve a file name which is passed as the first argument to your script, use code like this:
import sys
if len(sys.argv) > 2:
filename = sys.argv[1]
else:
# set a default filename or print an error
Secondly, the error clearly indicates that the script can't find the file newfile.txt. So it is either not in the current directory, you don't have the permission to read it, etc...
To open a file for reading, use with command as follows
# This is python 3 code
with open('yourfile.txt', 'r') as f:
for line in f:
print(line)
with command used together with open command has already the raising exception process.
The intent is to look in a json file in the directory above the script and load up what it finds in that file. This is what I've got:
import os
import json
settings_file = '/home/me/foo/bar.txt'
root = os.path.dirname(os.path.dirname(os.path.abspath(settings_file))) # '/home/me'
target = os.path.join(root,'.extras.txt') # '/home/me/.extras.txt'
db_file= open(target)
databases = json.load(db_file) # works, returns object
databases2 = json.load(open(target)) # equivalent to above, also works
# try to condense code, lose pointless variables target and file
databases3 = json.load(open(os.path.join(root,'.extras.txt'))) # equivalent (I thought!) to above, doesn't work.
So... why doesn't the all-at-once, no holding variables version work? Oh, the error returned is (now in it's entirety):
$ ./json_test.py
Traceback (most recent call last):
File "./json_test.py", line 69, in <module>
databases = json.load(open(os.path.join(root,'/.extras.txt')))
IOError: [Errno 2] No such file or directory: '/.extras.txt'
And to satisfy S.Lott's well-intentioned advice... it doesn't matter what target is set to. The databases and databases2 populate correctly while databases3 does not. target exists, is readable and contains what json expects to see. I suspect there's something I don't understand about the nature of stringing commands together... I can make the code work, was just wondering why the concise (or complex?) version failed.
Code looks fine, make sure referenced files are in the appropriate places. Given your code that includes target/file variable assignment, full path to .extras.txt is
/home/me/.extras.txt
You need to do:
file = open(target, 'w')
because by default open will try to open the file in read mode (r) but you need to open it in w (write) mode if you want it to be created.
Also, I would not use the variable name file since it is also a type (<type 'file'>) in python.
You could add the write-mode flag to this line as well:
databases = json.load(open(os.path.join(root,'.extras.txt'), 'w'))
because from the limited information we have in the question it appears your /.extras file does not previously exist.
Final note, you are losing the handle to your open file in this line (since you are not storing it in your file variable):
databases = json.load(open(os.path.join(root,'.extras.txt')))
How do you intend to close the file when you're finished with it?
You could do this with a context manager (python >=2.6 or 2.5 if import with_statement used):
with open(os.path.join(root,'.extras.txt'), 'w') as f:
databases = json.load(f)
which will take care of closing the file for you.
I saved a file as DictionaryE.txt in a Modules folder I created within Python. Then I type:
fh = open("DictionaryE.txt")
I get this error message:
Traceback (most recent call last):
File "<pyshell#17>", line 1, in <module>
fh = open("DictionaryE.txt")
IOError: [Errno 2] No such file or directory: 'DictionaryE.txt'
What am I doing wrong? Could someone please describe a specific, detailed step-by-step instruction on what to do? Thanks.
As other answers suggested, you need to specify the file's path, not just the name.
For example, if you know the file is in C:\Blah\Modules, use
fh = open('c:/Blah/Modules/DictionaryE.txt')
Note that I've turned the slashes "the right way up" (Unix-style;-) rather than "the Windows way". That's optional, but Python (and actually the underlying C runtime library) are perfectly happy with it, and it saves you trouble on many occasions (since \, in Python string literals just as in C ones, is an "escape marker", once in a while, if you use it, the string value you've actually entered is not the one you think -- with '/' instead, zero problems).
Use the full path to the file? You are trying to open the file in the current working directory.
probably something like:
import os
dict_file = open(os.path.join(os.path.dirname(__file__), 'Modules', 'DictionaryE.txt'))
It's hard to know without knowing your project structure and the context of your code. Fwiw, when you just "open" a file, it will be looking in whatever directory you're running the python program in, and __file__is the full path to ... the python file.
To complement Alex's answer, you can be more specific and explicit with what you want to do with DictionaryE.txt. The basics:
READ (this is default):
fh = open("C:/path/to/DictionaryE.txt", "r")
WRITE:
fh = open("C:/path/to/DictionaryE.txt", "w")
APPEND:
fh = open("C:/path/to/DictionaryE.txt", "a")
More info can be found here: Built-in Functions - open()