We Keep Coding

Input variable from command line in Python

What I want to do is very simple but I can't figure out a good and not too complex solution for this. Basically I want to define some global variables that will be used for example as a folder name
global folder = "C:\\TEMP\\" + foldername
And what I want is to set the foldername value as input when running the script, something like:
python myscript.py --folder somebeautifulfoldername
so when running my script, the folder will become C:\TEMP\somebeautifulfoldername

You can pass arguments to Python script like following:
python test.py arg1 arg2 arg3
And this is what you get
Argument List: ['test.py', 'arg1', 'arg2', 'arg3']
In your case:
python myscript.py somebeautifulfoldername
folder = "C:\\TEMP\\" + sys.argv[1]

You can use the built-in argparse module for this combined with getting the command line arguments from sys.argv:
import argparse
import sys
if __name__ == '__main__':
parser = argparse.ArgumentParser(description = 'My script')
parser.add_argument('--folder', help = "Subfolder of C:\TEMP\ to manipulate")
args = parser.parse_args(sys.argv[1:])
folder = "C:\\TEMP\\"+args.folder
print folder
Here I added just a very simple argument with some basic help string, but you can do quite a lot with this like giving a default value, allowing a list of files instead of a single file, specify the type, ... . See the manual for more details and examples.
Usage:
>python myscript.py --folder somebeautifulfoldername
C:\TEMP\somebeautifulfoldername
>python myscript.py --help
usage: tmp.py [-h] [--folder FOLDER]
My script
optional arguments:
-h, --help show this help message and exit
--folder FOLDER Subfolder of C:\TEMP\ to manipulate

import sys
folder = "none"
if("--folder" in sys.argv):
folder = sys.argv[sys.argv.index("--folder") + 1]
print folder
If you run it the way you want:
python myscript.py --folder "HELLOFOLDER"
It will give: HELLOFOLDER

There are a number of options to parse command line arguments into a Python script. There's the standard library's optparse and argparse for instance.
A really nice 3rd party tool is docopt, which allows you to write the logic described above very easily by describing the script usage directly as documentation of your script like this:
"""My script
Usage:
myscript.py --folder=<folder>
Options:
-h --help Show this screen.
--version Show version.
--folder=<folder> Choose folder.
"""
from docopt import docopt
if __name__ == '__main__':
arguments = docopt(__doc__, version='myscript 1.0')
folder = "C:\\TEMP\\" + arguments["--folder"]
print(folder)
That said, you may also want to look into tempfile for generating temporary files to make the script more cross-platform. Hard-coding Windows-specific paths is rarely a good idea.

is there a way to clear python argparse?

Consider the following script:
import argparse
parser1 = argparse.ArgumentParser()
parser1.add_argument('-a')
args1 = parser1.parse_args()
parser2 = argparse.ArgumentParser()
parser2.add_argument('-b')
args2 = parser2.parse_args()
I have several questions:
Is parse_args a one-time method or is there a way to clear the
arguments before adding new ones? (e.g. something like
args1.clear() or parser1.clear())
The result of this script is unusable. Although this script accepts
the -a argument, it does not accept any value for 'a'. Nor does it
accept any -b argument. Is there some way to make any of the arguments really work?
This is my actual scenario: I have 2 scripts. Both import the same
file which has initialization code (load config files, create
loggers, etc.), lets call it init.py This init.py file also parses
the arguments only because it needs one value from it. The problem
is that I need one of the scripts to accept other arguments as well.
Since init.py does something with one argument, I cannot wait with
parse_args. How can I make it work?
Edit:
Here is the output of my script:
[prompt]# python2.7 myscript.py -a
usage: a.py [-h] [-a A]
myscript.py: error: argument -a: expected one argument
[prompt]# python2.7 myscript.py -a 1
Namespace(a='1')
usage: a.py [-h] [-b B]
myscript.py: error: unrecognized arguments: -a 1

Your scenario is quite unclear, but I guess what you're looking for is parse_known_args
Here I guessed that you called init.py from the other files, say caller1.py and caller2.py
Also suppose that init.py only parses -a argument, while the original script will parse the rest.
You can do something like this:
in init.py put this in do_things method:
parser = argparse.ArgumentParser()
parser.add_argument('-a')
parsed = parser.parse_known_args(sys.argv)
print 'From init.py: %s' % parsed['a']
In caller1.py:
init.do_things(sys.argv)
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parsed = parser.parse_known_args(sys.argv)
print 'From caller1.py: %s' % parsed['b']
If you call caller1.py as follows: python caller1.py -a foo -b bar, the result will be:
From init.py: foo
From caller1.py: bar
But if your scenario is not actually like this, I would suggest to use #Michael0x2a answer, which is just to use single ArgumentParser object in caller1.py and pass the value appropriately for init.py

This doesn't really make sense, because for all intents and purposes, the parser object is stateless. There's nothing to clear, since all it does is takes in the console arguments, and returns a Namespace object (a pseudo-dict) without ever modifying anything in the process.
Therefore, you can consider parse_args() to be idempotent. You can repeatedly call it over and over, and the same output will occur. By default, it will read the arguments from sys.argv, which is where the console arguments are stored.
However, note that you can pipe in custom arguments by passing in a list to the parse_args function so that the parser will using something other then sys.argv as input.
I'm not sure what you mean. If you call python myscript.py -a 15, args1 will equal Namespace(a='15'). You can then do args1['a'] to obtain the value of 15. If you want to make the flag act as a toggle, call parser.add_argument('-a', action='store_true'). Here is a list of all available actions.
I would try and confine all the console/interface code into a single module and into a single parser. Basically, remove the code to parse the command line from init.py and the second file into an independent little section. Once you run the parser, which presents a unified interface for everything in your program, pass in the appropriate variables to functions inside init.py. This has the added advantage of keeping the UI separate and more easily interchangeable with the rest of the code.

Option multiplicity with docopt

I would like to using docopt for parsing a command line that can receive the same option multiple times. Could somebody explain me how to do it?
A test example:
#!/usr/bin/env python
"""
Test program.
Usage:
test.py -v
Options:
-v Flag that should be counted
"""
import docopt
print docopt.docopt(__doc__)
If I run this with test.py -v, I get:
{'-v': True}
Where as if I run this with test.py -vv, it displays the usage message (indicating the command line is not valid).
I'd like to tweak the option documentation so that docopt returns me:
{'-v': 1}
When only 1 -v was passed and:
{'-v': 3}
If, say, the user passed -vvv. This is pretty much the same functionality the count action in argparse.

After digging the docopt (closed) issue list, I have found that the right way to represent this would be:
#!/usr/bin/env python
"""
Test program.
Usage:
test.py (-v ...)
Options:
-v Flag that should be counted
"""
import docopt
print docopt.docopt(__doc__)
That is, one must use the symbol "..." to signify that an option may appear multiple times. In this case, the option will be correctly counted. If the above program is called with test.py -vvv, it will correctly print:
{'-v': 3}
The symbol "..." can also be used with arguments and options that take arguments pretty much the same way, just hit the link above for an example of that.

(This is just a comment to the above, but would get awkward as a comment.)
And this can be extended to passing a list as an argument:
"""
Usage:
program (--opt=OPT ...)
Options:
--opt=OPT An option that can be specified multiple times to form a list
"""
import docopt
print docopt.docopt(__doc__)
And we run this as
python test.py --opt=first_option
{'--opt': ['first_option']}
python test.py --opt=first_option --opt="second in line"
{'--opt': ['first_option', 'second in line']}
And so on.

Custom tab completion in python argparse

How to get shell tab completion cooperating with argparse in a Python script?
#!/usr/bin/env python
import argparse
def main(**args):
pass
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('positional', choices=['spam', 'eggs'])
parser.add_argument('--optional', choices=['foo1', 'foo2', 'bar'])
args = parser.parse_args()
main(**vars(args))
With an executable flag set on the .py file, the expected results should be something like:
$ ./example.py sp<tab>
-> completes to "./example.py spam"
$ ./example.py --op<tab>
-> completes to "./example.py --optional"
$ ./example.py --optional b<tab>
-> completes to "./example.py --optional bar"
$ ./example.py --optional f<tab>
-> completes to "./example.py --optional foo"
and, additionally, prints "foo1 foo2" choices on stdout on a new line

Have a look at argcomplete by Andrey Kislyuk.
Install it with:
pip install argcomplete
Import the module and add one line in your source before calling parser.parse_args():
#!/usr/bin/env python
import argparse as ap
import argcomplete
def main(**args):
pass
if __name__ == '__main__':
parser = ap.ArgumentParser()
parser.add_argument('positional', choices=['spam', 'eggs'])
parser.add_argument('--optional', choices=['foo1', 'foo2', 'bar'])
argcomplete.autocomplete(parser)
args = parser.parse_args()
main(**vars(args))
and to make sure that bash knows about this script, you use
eval "$(register-python-argcomplete your_script)"
you should put that line in your ~/.bashrc or follow argcomplete's docs and activate 'global' completion.
After that you completion works as requested.
The way this works is that the eval line creates a function _python_argcomlete which is registered using complete. (Run register-python-argcomplete your_script to just have a look at what gets eval-ed into bash). The autocomplete function looks for environment variables set by the bash completion mechanism to see if it needs to act. If it acts, it exits the program. If it doesn't act, this is a normal call to the program that function does nothing and the normal flow of the program continues.

For auto-complete to work you need a bash function to generate the possible options, and then you need to run complete -F <function_name> <program_name>
The best way of doing this is to have the program generate the completion function based on it's own parsing algorithm to avoid duplication. However, at a quick glance on argparse, I could not find a way to access it's internal structure, but I suggest you look for it.
Here is a bash function that will do for the above program:
function _example_auto() {
local cur=${COMP_WORDS[COMP_CWORD]}
local prev=${COMP_WORDS[COMP_CWORD-1]}
case "$prev" in
--optional )
COMPREPLY=( $(compgen -W "foo1 foo2 bar" -- $cur) )
return 0
;;
*)
COMPREPLY=( $(compgen -W "--optional spam eggs" -- $cur) )
return 0
;;
esac
}

Is it possible to only test specific functions with doctest in a module?

I am trying to get into testing in Python using the doctest module. At the moment I do
Write the tests for the functions.
implement the functions code.
If Tests pass, write more tests and more code.
When the function is done move on to the next function to implement.
So after 3 or 4 (independent) functions in the same module with many tests I get a huge output by doctest. And it is a little annoysing.
Is there a way to tell doctest "don't test functions a(), b() and c()", so that it runs only the unmarked functions?
I only found the doctest.SKIP flag, which is not sufficient for my needs. I would have to place this flag in a lot of lines. And if I would want to check a marked function again, I would have to go manually through the code and remove any flag I set inside.

looks like you could pass the function to run_docstring_examples:
def f(a, b, c):
'''
>>> f(1,2,3)
42
'''
if __name__ == '__main__':
import doctest
# doctest.testmod()
doctest.run_docstring_examples(f, globals())
example found via google.

I put together a helper script to make this a little less painful. It can be installed using:
pip install doctestfn
It can then be used as follows:
usage: doctestfn [-h] [-v] module function
Run doctests for one function
positional arguments:
module Module to load
function Function to test
optional arguments:
-h, --help show this help message and exit
-v, --verbose Enable verbose doctest output