I have not had much luck hunting for finding a good explanation of what invalid argument errors are and what would cause them.
My current sample I am working with is
import sys
mylog="mylog.log"
sys.stdout = open(mylog,'w')
#lots of code
#.
#.
#.
#End of lots of code
from time import clock
print "blablabla",clock()
I receive an IOError Invalid Argument error on the clock line. I have also tried
print "blablabla\t%s"%clock()
Any information about this error would be great help. Those lines work perfectly fine on short runs, it just after running the code for a while it breaks. I have tried to setting the buffer size to something low like 45-100 lines.
I can't reproduce this exact problem on my own computer, so I can't give specific advice, but here is some general commentary on how to debug this sort of thing.
When you see "Invalid argument" in an IOError or OSError exception from python, that means the interpreter tried to make a system call, which failed and set errno to the code EINVAL. (Tangentially, python really shouldn't print the numeric values for errno codes - the symbolic names are standardized but the numbers aren't.) The first thing you need to do is find out which system call it was, and the easiest way to do that is run your program under the strace utility, like this:
$ strace -f -o strace.log python yourscript.py [arguments...]
Wait for it to fail, then search the file strace.log for "-1 E" (exactly that string). You will find something like this:
times({tms_utime=162, tms_stime=123, tms_cutime=0, tms_cstime=0}) = 1718279979
write(1, "2.85\n", 5) = -1 EINVAL (Invalid argument)
You then read the man page for the system call that failed ("man 2 write" in this case) and look for the errno code name (EINVAL in this case), and see what it says has gone wrong.
In this case I strongly suspect you have found a bug in either the Python interpreter or the operating system. "Invalid argument" means what it says - one of the input arguments to the system call had an invalid value. You're not doing anything tricky in your script, so either the interpreter is messing up its system calls, or the kernel misunderstood what the interpreter wanted.
Related
I tend to write a lot of command line utility programs and was wondering if
there is a standard way of messaging the user in Python. Specifically, I would like to print error and warning messages, as well as other more conversational output in a manner that is consistent with Unix conventions. I could produce these myself using the built-in print function, but the messages have a uniform structure so it seems like it would be useful to have a package to handle this for me.
For example, for commands that you run directly in the command line you might
get messages like this:
This is normal output.
error: no files given.
error: parse.c: no such file or directory.
error: parse.c:7:16: syntax error.
warning: /usr/lib64/python2.7/site-packages/simplejson:
not found, skipping.
If the commands might be run in a script or pipeline, they should include their name:
grep: /usr/dict/words: no such file or directory.
It would be nice if could handle levels of verbosity.
These things are all relatively simple in concept, but can result in a lot of
extra conditionals and complexity for each print statement.
I have looked at the logging facility in Python, but it seems overly complicated and more suited for daemons than command line utilities.
I can recommend Inform. It is the only package I have seen that seems to address this need. It provides a variety of print functions that print in different circumstances or with different headers. For example:
log() -- prints to log file, no header
comment() -- prints if verbose, no header
display() -- prints if not quiet, no header
output() -- always prints, no header
warning() -- always prints with warning header
error() -- always prints with error header
fatal() -- always prints with error header, terminates program.
Inform refers to these functions as 'informants'. Informants are very similar to the Python print function in that they take any number of arguments and builds the message by joining them together. It also allows you to specify a culprit, which is added to the front of the message.
For example, here is a simple search and replace program written using Inform.
#!/usr/bin/env python3
"""
Replace a string in one or more files.
Usage:
replace [options] <target> <replacement> <file>...
Options:
-v, --verbose indicate whether file is changed
"""
from docopt import docopt
from inform import Inform, comment, error, os_error
from pathlib import Path
# read command line
cmdline = docopt(__doc__)
target = cmdline['<target>']
replacement = cmdline['<replacement>']
filenames = cmdline['<file>']
Inform(verbose=cmdline['--verbose'], prog_name=True)
for filename in filenames:
try:
filepath = Path(filename)
orig = filepath.read_text()
new = orig.replace(target, replacement)
comment('updated' if orig != new else 'unchanged', culprit=filename)
filepath.write_text(new)
except OSError as e:
error(os_error(e))
Inform() is used to specify your preferences; comment() and error() are the
informants, they actually print the messages; and os_error() is a useful utility that converts OSError exceptions into a string that can be used as an error message.
If you were to run this, you might get the following output:
> replace -v tiger toe eeny meeny miny moe
eeny: updated
meeny: unchanged
replace error: miny: no such file or directory.
replace error: moe: no such file or directory.
Hopefully this gives you an idea of what Inform does. There is a lot more power there. For example, it provides a collection of utilities that are useful when printing messages. An example is os_error(), but there are others. You can also define your own informants, which is a way of handling multiple levels of verbosity.
import logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)s %(message)s')
level specified above controls the verbosity of the output.
You can attach handlers (this is where the complexity outweighs the benefit in my case) to the logging to send output to different places (https://docs.python.org/2/howto/logging-cookbook.html#multiple-handlers-and-formatters) but I haven't needed more than command line output to date.
To produce output you must specify it's verbosity as you log it:
logging.debug("This debug message will rarely appeal to end users")
I hadn't read your very last line, the answer seemed obvious by then and I wouldn't have imagined that single basicConfig line could be described as "overly complicated". It's all I use the 60% of the time when print is not enough.
I'm making a Network sniffing tool for personal use, and I can't find the syntax error within my code, this is Python 2.7.9 by the way.
Here's the code;
def main():
global listen
global port
global command
global execute
global upload_destination
global target
if not len(sys.argv[1:]):
usage()
#read the commandline options
It says the error is featured below in the next 3 lines, any ideas?
try:
opts, args = getopt.getopt(sys.argv[1:],"hle:t:p:cu:", ¬ ["help","listen","execute","target","port","command","upload"])
except getopt.GetoptError as err:
print str(err)
usage()
I feel there's been a mix up between Python 2 and 3 but I'm not sure.
¬ ["help","listen","execute","target","port","command","upload"])
"¬" This is not valid Python syntax. Removing it should solve the issue.
Also in the future maybe post the actual error which is being shown in the output.
First, this is not valid in programs: ¬. This is Unicode, which basically doesn't work where you placed it all.. Since when does Python allow Unicode as commands in programs? It is not valid and in the wrong place. Now doing this will work:
print "¬"
It's a string so nothing wrong but the usage in your program makes that a Syntax error as there is no such command called ¬. Also, in the try statement, you have an indention of 8 spaces. You can only use 4 or 2-space indention in your programs.
EDIT: Okay, you can use 8-space indention in programs but you need to use 8 (or a multiple of 8) spaces every single line you need to indent. Since your indention is non-consistent, that could also be the reason you are getting an error.
Windows environment, python 2.7, latest nosetest.
Looking at nosetest docs, and googling around, nowhere do I see that nosetest sets the cmd line errorlevel on test failure.
We need this so that our build system can detect test failure.
Questions are:
Does Nosetest set the cmd line, errorlevel? (if so, where are docs)
If not, what is the appropriate way to handle this? (must my build parse some log output, or?)
%errorlevel% on windows is the return code of the application, typically the argument given to the exit(int) call (exit code). These return codes are the same as unittest, but the documentation is not very explicit:
The testRunner argument can either be a test runner class or an already created instance of it. By default main calls sys.exit() with an exit code indicating success or failure of the tests run.
In the above sentence By default is to understand as if the call argument exit is not set to False:
main supports being used from the interactive interpreter by passing in the argument exit=False. This displays the result on standard output without calling sys.exit()
(New in 2.7 and 3.1. In older version, sys.exit is always called.)
I found no special documentation about the return code, but looking at the source, one can find that exit code is 0 for success, 1 for error (same for unittest alone) and 2 if the usage help has to be printed (given arguments when calling as standalone program are incorrect). Specific for nose, when program is asked to display version or list plugins, exit code is 0 too.
Ive been using MSVC2010 for some time now combined with Python Tools 1.0
I'm currently working on a python project and my solution looks like this:
Solution
- Pyproj
- EmptyTestRunner
The pyproj contains a MainTest.py file that runs all my tests,
the EmptyTestRunner is just an empty project with a "Post-Build" action that runs the MainTest.py (thus running all my tests written in PyUnit on my python code each time i "re-build")
My problem is that running the python file, it produces normal python output - which because i'm running it as a post-build event, refers it to the MSVC output window.
MSVC2010 doesn't recognize this output as a "compile / linker" error (for example when an exception is raised in my python code which isn't caught) so it doesn't add it to the error window.
Is there a tool that already does this?
Is there a flag passed to the python.exe that can do it?
I've googled the subject throghouly but found nothing resembling it.
I'm about to just simply write a python script that will re-format the output of my pyproj to the msvc format.
thx in advance for any help!
I've done this with Visual Studio 9 (2008), so I don't know if the trick works for 2010, but anyway: the linking between the output window and the error manager in VS is automagical. All it expects is a line of the form:
path\to\file\name(lineno) : error : text of the error
e.g.
c:\dev\myfile.ext(17) : error : 'foo' is not a valid configuration directive
if your script somehow reads a configuration file.
Then when you double click this line, it opens the file at the specified line. The error message ('text of the error' here) also appears in the Error list window.
All other lines of your script's output, if they do not have this format, are not affected, so you may very well throw a lengthy explanation after the magical line - just as the C++ compiler does for template errors.
Whether you choose to modify your script to translate Python errors to a VS-compatible format or use a wrapper script is your decision - I'd favor the first one since you can catch all errors in the __main__ function, and VS-print them (given that you can still print the full Python trace just afterwards).
Where can I find information about meaning of exit codes of "python" process on Unix? For instance, if I do "python thisfiledoesntexist.py", I get exit code 2
Summary:
from errno import errorcode
print errorcode[2]
As stated, mostly the error codes come from the executed script and sys.exit().
The example with a non-existing file as an argument to the interpreter fall in a different category. Though it's stated nowhere I would guess, that these exit codes are the "standard" Linux error codes. There is a module called errno that provides these error numbers (the exit codes come from linux/include/errno.h.
I.e.: errno.ENOENT (stands for for "No such file or directory") has the number 2 which coincides with your example.
The Python manual states this regarding its exit codes:
Some systems have a convention for assigning specific meanings to specific exit codes, but these are generally underdeveloped; Unix programs generally use 2 for command line syntax errors and 1 for all other kind of errors.
So, since you specified thisfiledoesntexist.py as a command line argument, you get a return code of 2 (assuming the file does not, in fact, exist. In that case I'd recommend renaming it to thisfiledoesexist.py. ;) )
Other that such parsing errors, the return code is determined by the Python program run. 0 is returned unless you specify another exit code with sys.exit. Python itself does not interfere.
Maybe exit code constants from os module can help you. Also have a look at sys.exit documentation.
Unfortunately, there is no 100% guarantee that Python's exit codes will be what the documentation claims they will be: os._exit allows the Python programmer to define which exit code is supposed to be used, which means python file_exists_but_claims_that_it_does_not.py could exit with os.EX_DATAERR.