In a shell script I have:
/usr/local/bin/pybot --variablefile variables.py:$var1:$var2 test_cases.tsv
inside variables.py how can I access var1 and var2 arguments?
I have tried:
import sys
var1 = sys.argv[1]
var1 = sys.argv[2]
it seems like this doesn't work.
For you to access the variables, your variable file must define the function get_variables, which will be given the arguments passed from the command line. This function needs to return a dictionary where the keys are the robot variable names.
For example:
def get_variables(arg1, arg2):
variables = {
"var1": arg1,
"var2": arg2
}
return variables
If your variable file is based on a class, the class needs to have the get_variables method.
For example:
# variables.py
class variables(object):
def get_variables(self, arg1, arg2):
variables = {
"var1": arg1,
"var2": arg2
}
return variables
When you do the above, your test will have two variables set: ${var1} and ${var2} which will have the values that were passed via the --variablefile argument.
Here is a test that can be used to verify the above:
# example.robot
*** Test cases ***
Example
should be equal ${var1} hello
should be equal ${var2} world
Here is how to run the test in order for it to pass:
$ var1=hello
$ var2=world
$ /usr/local/bin/pybot --variablefile variables.py:$var1:$var2 example.robot
Of course, var1 and var2 are completely arbitrary. You can pass raw strings, too:
$ /usr/local/bin/pybot --variablefile variables.py:hello:world example.robot
Passing arguments is described in the user guide section titled Getting variables from a special function
sys reads the arguments fron the command line, as they appears to it:
sys.argv[0] contains the script name
sys.argv[1], the first argument (whatever it is)
sys.argv[2], the second, and so on.
You should use argparse, it helps to build comprehensive CLIs. A nice tutorial exists on the Python website.
You seem to make assumptions about how the arguments are parsed which are not true. Here's how these arguments are passed from the shell to Python:
sys.argv[0] is /usr/local/bin/pybot
sys.argv[1] is --variablefile
sys.argv[2] is variables.py:$var1:$var2 where the values of the shell variables var1 and var2 are substituted.
sys.argv[n] is test_cases.tsv
The last one is [n] because without quotes around the argument, sys.argv[2] might actually be split into multiple values. For example, if var1 contains = foo * bar= then actually
sys.argv[2] is variables.py:=
sys.argv[3] is foo
sys.argv[4..n-2] is a list of files in the current directory, and
sys.argv[n-1] is =bar:$var2 where similar further processing for the value of var2 may take place.
There are Python argument parsing modules which assign further semantics e.g. to arguments which start with a dash (these will be interpreted as options) but by itself, Python does no such thing. If that's what you want, maybe look at argparse or one of its replacements; but you still need to understand how the basic mechanics work. A common arrangement is to avoid internal structure in arguments, and instead require the user to pass each value as a separate argument -- so perhaps
--variablefile variables.py --variablefile "$var1" --variablefile "$var2"
with quoting to prevent the shell from attempting to perform whitespace tokenization and wildcard expansion on the variable values, and then probably in your script an argparse definition which says to merge multiple option arguments into a list.
parser = argparse.ArgumentParser()
parser.add_argument('--variablefile', action='append')
Related
I am trying to run a windows executable via python3 and have below code snippet. The arguments can be of the form key-value pair but its possible that few arguments may not have value like arg1 below. arg2 needs to have a value which is passed as creating arg2 variable below. The data.filepath needs to be used to construct arg2
# data.filepath resolves to \\server1\data\inputs\filename.txt
arg2 = "--arg2 {}".format(data.filepath)
child = subprocess.Popen([output_path, "--arg1", arg2, "--arg3 val3", "--arg4 val4"],
shell=False, stderr=subprocess.STDOUT)
child.communicate()[0]
rc = child.returncode
But seems that i am not following correct syntax and getting error as below
Throw location unknown (consider using BOOST_THROW_EXCEPTION)
Dynamic exception type: class boost::exception_detail::clone_impl<struct boost::exception_detail::error_info_injector<class boost::program_options::unknown_option> >
std::exception::what: unrecognised option '--arg2 \\server1\data\inputs\filename.txt'
Please let me know the right syntax in python to pass arguments to executables properly.
Apparently, the program you are running expects to receive an argument and its value as separate strings (which makes a lot of sense). You can do something like
if phase_of_moon() == 'waxing gibbous':
arg2 = ['--arg2', data.filepath]
else:
arg2 = []
x = Popen([output_path, '--arg1', *arg2, '--arg3', val3])
using iterable unpacking to expand arg2.
This is my script mytest.py.
import argparse
parser = argparse.ArgumentParser(description="Params")
parser.add_argument(
"--value")
def test(args):
print(args.value)
args = parser.parse_args()
test(args)
I want to pass argument store in variable val
val =1
!python mytest.py --value val
instead of printing 1 it print val. How to send 1 stored in variable val.
argparse always get argument as string, or list of strings on default, and what you do on your shell is irrelevant with python program. It is no wonder val is printed.
Use file that contains "1" and read that file to do what you intended to.
As jueon park said naming a variable in commandline wont work
It would create an error like the above one.If you are calling the command from any programer will work but in cmd it won't work
I'm very late for this but your python code works just fine. The problem you have is that you are not passing the arguments correctly.
For this to work first you need to correctly set the variable:
val=1
(note that the "=" must be next to both the variable name and the value)
and the you can simply use $ to get the value from the variable. So:
python mytest.py --value $val
How do I pass in parameters to Luigi? if I have a python file called FileFinder.py with a class named getFIles:
class getFiles(luigi.Task):
and I want to pass in a directory to this class such as:
C://Documents//fileName
and then use this parameter in my run method
def run(self):
how do I run this in command line and add the parameter for use in my code? I am accustomed to running this file in command line like this:
python FileFinder.py getFiles --local-scheduler
What do I add to my code to use a parameter, and how do I add that parameter to the command line argument?
Also, as an extension of this question, how would I use multiple arguments? or arguments of different data types such as strings or lists?
As you have already figured out, you can pass arguments to luigi via
--param-name param-value
in the command line. Inside your code, you have to declare these variables by instantiating the Parameter class or one of it's subclasses. The subclasses are used to tell luigi if the variable has a data-type that is not string. Here is an example which uses two command line arguments, one Int and one List:
import luigi
class testClass(luigi.Task):
int_var = luigi.IntParameter()
list_var = luigi.ListParameter()
def run(self):
print('Integer Param + 1 = %i' % (self.int_var + 1))
list_var = list(self.list_var)
list_var.append('new_elem')
print('List Param with added element: ' + str(list_var))
Note that ListParams actually get converted to tuples by luigi, so if you want to do list operations on them, you have to convert them back first (This is a known issue, but doesn't look like it will be fixed soon).
You can invoke the above module from the command line like this (i have saved the code as a file called "testmodule.py" and made the call from inside the same directory):
luigi --module testmodule testClass --int-var 3 --list-var '[1,2,3]' --local-scheduler
Note here that for variables containing a _, this has to be replaced by -.
The call yields (along with many status messages):
Integer Param + 1 = 4
List Param with added element: [1, 2, 3, 'new_elem']
So I think this works, in the code I added:
fileName = luigi.Parameter()
if i run this in the command line:
python FileFinder.py getFiles --local-scheduler --getFiles-fileName C://Documents//fileName
but if anyone has any advice on parameters of different types and how to use them, especially numbers and lists, please let me know.
Adding to Toterich's answer.
While passing a list of string arguments as a ListParameter():
python file_name.py --local-scheduler TaskName --arg '["a","b"]'
The string arguments must be enclosed in double-quotes and not single quotes otherwise it'll give a JSONParsing error.
I have a use case where I'd like the user to be able to provide, as an argument to argparse, EITHER a single string OR a filename where each line has a string.
Assume the user launches ./myscript.py -i foobar
The logical flow I'm looking for is something like this:
The script determines whether the string foobar is a readable file.
IF it is indeed a readable file, we call some function from the script, passing each line in foobar as an argument to that function. If foobar is not a readable file, we call the same function but just use the string foobar as the argument and return.
I have no ability to guarantee that a filename argument will have a specific extension (or even an extension at all).
Is there a more pythonic way to do this OTHER than just coding up the logic exactly as I've described above? I looked through the argparse tutorial and didn't see anything, but it also seems reasonable to think that there would be some specific hooks for filenames as arguments, so I figured I'd ask.
A way would be:
Let's say that you have created a parser like this:
parser.add_argument('-i',
help='...',
type=function)
Where type points to the function which will be an outer function that evaluates the input of the user and decides if it is a string or a filename
More information about type you can find in the documentation.
Here is a minimal example that demonstrates this use of type:
parser.add_argument('-d','--directory',
type=Val_dir,
help='...')
# ....
def Val_dir(dir):
if not os.path.isdir(dir):
raise argparse.ArgumentTypeError('The directory you specified does not seem to exist!')
else:
return dir
The above example shows that with type we can control the input at parsing time. Of course in your case the function would implement another logic - evaluate if the input is a string or a filename.
This doesn't look like an argparse problem, since all you want from it is a string. That string can be a filename or a function argument. To a parser these will look the same. Also argparse isn't normally used to run functions. It is used to parse the commandline. Your code determines what to do with that information.
So here's a script (untested) that I think does your task:
import argparse
def somefunction(*args):
print(args)
if __name__=='__main__':
parser=argparse.ArgumentParser()
parser.add_argument('-i','--input')
args = parser.parse_args()
try:
with open(args.input) as f:
lines = f.read()
somefunction(*lines)
# or
# for line in lines:
# somefuncion(line.strip())
except:
somefunction(arg.input)
argparse just provides the args.input string. It's the try/except block that determines how it is used.
================
Here's a prefix char approach:
parser=argparse.ArgumentParser(fromfile_prefix_chars='#',
description="use <prog -i #filename> to load values from file")
parser.add_argument('-i','--inputs')
args=parser.parse_args()
for arg in args.inputs:
somefunction(arg)
this is supposed to work with a file like:
one
two
three
https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars
Using python optparse.py, is there a way to work out whether a specific option value was set from the command line or from the default value.
Ideally I would like to have a dict just like defaults, but containing the options actually supplied from command line
I know that you could compare the value for each option with defaults, but this wouldn't distinguish a value was passed through command line which matched the default.
Thanks!
EDIT
Sorry my original phrasing wasn't very clear.
I have a large number of scripts which are called from batch files. For audit purposes, I would like to report on the options being passed, and whether they are passed from command line, default, or some other means, to a log file.
Using defaults you can tell whether an option matches a default value, but that still doesn't tell you whether it was actually supplied from command line. This can be relevant: if an option is passed from command line and agrees with the default, if you then change the default in the code the script will still get the same value.
To me it would feel quite natural to have an equivalent to defaults, containing the values actually supplied.
To make the question concrete, in this example:
>>> sys.argv = ['myscript.py','-a','xxx']
>>> import optparse
>>> parser = optparse.OptionParser()
>>> parser.add_option('-a', default = 'xxx')
>>> parser.add_option('-b', default = 'yyy')
How do I know that option a was passed from command line. Is the only way to parse the command line manually?
(I know this is a fairly minor point, but I thought it would be worth asking in case I'm missing smthing on optparse)
Thanks again
Instead of the below boilerplate code:
opts, args = parser.parse_args()
if you use the below code, you will have an additional values object opts_no_defaults with options that were explicitly specified by the user:
opts_no_defaults = optparse.Values()
__, args = parser.parse_args(values=opts_no_defaults)
opts = Values(parser.get_default_values().__dict__)
opts._update_careful(opts_no_defaults.__dict__)
At the end, opts should be identical as in the initial boilerplate code.
print opts_no_defaults.__dict__
print opts.__dict__
for opt in parser._get_all_options():
if opt.dest:
print "Value of %s: %s" % (opt._long_opts[0], getattr(opts, opt.dest))
print "Is %s specified by user? %s" % (opt._long_opts[0], hasattr(opt_no_defaults, opt.dest))
Not knowing you code is impossible to give the better answer, but...
simply don't pass defaults to the parser and check for None values. A None value is a default for optparse lib so you can retrieve your own default and act as usually;
extend optparse to specialize it.
I don't know your program but usually it is not a good design changing behavior when the configuration is the same.
def is_opt_provided (parser, dest):
if any (opt.dest == dest and (opt._long_opts[0] in sys.argv[1:] or opt._short_opts[0] in sys.argv[1:]) for opt in parser._get_all_options()):
return True
return False
Usage:
parser = OptionsParser()
parser.add_option('-o', '--opt', dest='opt_var', ...)
if is_opt_provided(parser, 'opt_var'):
print "Option -o or --opt has been provided"
It would be great if Python maintainers included the suggested function to OptionParser class.