I am using Python RPyC to communicate between two machines. Since the link may be prone to errors I would like to have a generic wrapper function which takes a remote function name plus that function's parameters as its input, does some status checking, calls the function with the parameters, does a little more status checking and then returns the result of the function call. The wrapper should have no knowledge of the function, its parameters/parameter types or the number of them, or the return value for that matter, the user has to get that right; it should just pass them transparently through.
I get the getattr(conn.root, function)() pattern to call the function but my Python expertise runs out at populating the parameters. I have read various posts on the use of *arg and **kwarg, in particular this one, which suggests that it is either difficult or impossible to do what I want to do. Is that correct and, if so, might there be a scheme which would work if I, say, ensured that all the function parameters were keyword parameters?
I do own both ends of this interface (the caller and the called) so I could arrange to dictionary-ise all the function parameters but I'd rather not make my API too peculiar if I could possibly avoid it.
Edit: the thing being called, at the remote end of the link, is a class with very ordinary methods, e.g.;
def exposed_a(self)
def exposed_b(self, thing1)
def exposed_c(self, thing1=None)
def exposed_d(self, thing1=DEFAULT_VALUE1, thing2=None)
def exposed_e(self, thing1, thing2, thing3=DEFAULT_VALUE1, thing4=None)
def exposed_f(self, thing1=None, thing2=None)
...where the types of each argument (and the return values) could be string, dict, number or list.
And it is indeed, trivial, my Goggle fu had simply failed me in finding the answer. In the hope of helping anyone else who is inexperienced in Python and is having a Google bad day:
One simply takes *arg and **kwarg as parameters and passes them directly on, with the asterisks attached. So in my case, to do my RPyC pass-through, where conn is the RPyC connection:
def my_passthru(conn, function_name, *function_args, **function_kwargs):
# Do a check of something or other here
return_value = getattr(conn.root, function_name)(*function_args, **function_kwargs)
# Do another check here
return return_value
Then, for example, a call to my exposed_e() method above might be:
return_value = my_passthru(conn, e, thing1, thing2, thing3)
(the exposed_ prefix being added automagically by RPyC in this case).
And of course one could put a try: / except ConnectionRefusedError: around the getattr() call in my_passthru() to generically catch the case where the connection has dropped underneath RPyC, which was my main purpose.
For example, I'd like to do something like: greet(,'hola'), where greet is:
def greet(person='stranger', greeting='hello')
This would help greatly for testing while writing code
Upon calling a function you can use the variable names to make it even more clear what variable will assume which value. At the same time, if defaults are provided in the function definition, skipping variables when calling the function does not raise any errors. So, in short you can just do this:
def greet(person='stranger', greeting='hello')
print('{} {}'.format(greeting, person))
return
greet(greeting='hola') # same as greet(person='stranger', greeting='hola')
# returns 'hola stranger'
Note that, as I said above this would not work if for example your function definition was like this:
def greet(person, greeting)
print('{} {}'.format(greeting, person))
return
Since in this case, Python would complain saying that it does not know what to do with person; no default is supplied..
And by the way, the problem you are describing is most likely the very reason defaults are used in the first place
Without knowing the other parameters, and only knowing that the parameter you want to change is in second position you could use the inspect module to get function signature & associated default values.
Then make a copy of the default values list and change the one at the index you want:
import inspect
def greet(person='stranger', greeting='hello'):
print(person,greeting)
argspec = inspect.getargspec(greet)
defaults = list(argspec.defaults)
defaults[1] = "hola" # change second default parameter
greet(**dict(zip(argspec.args,defaults)))
Assuming that all parameters have default values (else it shifts the lists an that fails) that prints:
stranger hola
So, I would like to modify the json.loads() function to accept a new keyword parameter, but not have it just be a part of kwargs. In other words, I want it be an explicit part of the function's signature.
Here's my guess on how to do this. Are there better ways of doing this?
def json_to_python_syntax(json_method):
"""
Translate JSON-conforming key names to Pythonic standards on dict.
The goal of this decorator is to add a standard keyword parameter
'convert_syntax' onto the method. But, I'm not sure how to do this.
"""
#wraps(json_method)
def wrapper(json_string, convert_syntax=False, **kwargs):
pythonic_dict = dict()
json_syntax_dict = json_method(json_string, **kwargs)
if not convert_syntax:
return json_syntax_dict
for key, value in json_syntax_dict.iteritems():
for json_syntax in re.finditer(r'[A-Z]', key):
key = key.replace(
json_syntax.group(), '_' + json_syntax.group()[0].lower())
pythonic_dict[key] = value
return pythonic_dict
return wrapper
My concern with this method is that it this monkeys with the expected order of keyword parameters in json.loads (It makes convert_syntax the first expected parameter after the json string) and could mess up other calls to json.loads within the larger program that assume the standard order.
Seeing as your change breaks the expected signature of json.loads and you're concerned with it breaking other code that depends on the original signature, I'd agree with Simeon Visser, seems like you shouldn't do this at all.
Only code you write will be able to properly call your new method, so why not give your method a different name, rather than decorating an existing method? If you want to prevent other code from calling this method without the convert_syntax flag, just avoid importing the json module at all, and instead import your json library that wraps the json module.
What are [function] arguments? What are they used for?
I started learning Python very recently; I'm new to programming and I apologize for this basic question.
In every Python tutorial I go through they talk about arguments. I have looked for the answer to this question and have found many answers but they are just a little too hard for me to understatnd. I may just be missing some conceptual background.
So... when I define a function, what are the things in parenthesis used for?
Example:
def hi( This is the part that i dont get):
print 'hi'
Edit:
Two follow-up questions related to this one were later closed and merged here, hence the partial out-of-context trait of some of the answers.
The follow-up questions were: [paraphrased]
Can arguments only be used for input ?
what are other examples of the use of arguments ?
Why use arguments, rather than having the function call raw_input ?
Why is the concept of argument passing described as such a powerful thing? it seems to me we're merely using them to replace stuff the user could have type on the keyboard.
In a few words, they're data that gets "passed into" the function to tell it what to do. Wikipedia has details.
http://en.wikipedia.org/wiki/Function_argument
For instance, your hi() function might need to know who to say hello to:
def hi(person):
print "Hi there " + person + ", how are you?"
Or a mathematical function might need a value to operate on:
def square(x):
return x * x
This is not a Python question, but rather a generic programming question. A very basic one.
Before answering the question about arguments, and in view of the other questions you asked, it is useful to discuss the concept of variables.
A variable is a named piece of memory where information of interest to the underlying program can be stored and retrieved. In other words, it is a symbolic name, chosen by the programmer, that is associated to its contents. Using various language constructs generally known as assignments, the programmer can read or write the contents of a variable.
It is important to note that the value (i.e. the content) of a variable needn't be defined when the program is written. It is only necessary at run-time. This allows the program to describe actions to be performed on symbolic elements without knowing exactly the value these elements have. Consider this snippet, part of a bigger program:
# ... some logic above
ball_volume = 4.0 / 3 * math.pi * ball_radius
if ball_volume > 200:
print ("Man, that's a big ball")
# ... more logic below
At the time the program is written one doesn't need to know the actual value of ball_radius; yet, with the assumption that this variable will contain the numeric value of some hypothetical ball, the snippet is capable of describing how to compute the ball's volume. In this fashion, when the program is running, and somehow (more on this later) the ball_radius variable has been initialized with some appropriate value, the variable ball_volume can too be initialized and used, here in the conditional statement (if), and possibly below. (At some point the variable may go out-of-scope, but this concept which controls when particular variables are accessible to the program is well beyond this primer).
In some languages the type of data that may be associated with a particular variable needs to be explicitly defined and cannot change. For example some variables could hold only integer values, other variables string values (text) etc. In Python there is no such restriction, a variable can be assigned and re-assigned to any type of data, but of course, the programmer needs to keep track of this for example to avoid passing some text data to a mathematical function.
The data stored inside variable may come from very different sources. Many of the examples provided in tutorials and introductory documentation have this data coming from keyboard input (as when using raw_input as mentioned in some of your questions). That is because it allows interactive tests by the people trying out these tutorial snippets. But the usefulness of programs would be rather limited if variables only get their data from interactive user input. There are many other sources and this is what makes programming so powerful: variables can be initialized with data from:
databases
text files or files various text-base formats (XML, JSON, CSV..)
binary files with various formats
internet connections
physical devices: cameras, temperature sensors...
In a nutshell, Arguments, also called Parameters, are variables passed to the function which [typically] are used to provide different output and behavior from the function. For example:
>>> def say_hello(my_name):
... print("Hello,", my_name, "!")
>>> say_hello("Sam")
Hello, Sam !
>>> customer_name = "Mr Peter Clark" #imagine this info came from a database
>>> # ...
>>> say_hello(customer_name)
Hello, Mr Peter Clark !
>>>
In the example above, my_name is just like any local variable of the say_hello function; this allows the function to define what it will do with the underlying value when the function is called, at run-time.
At run-time, the function can be called with an immediate value (a value that is "hard-coded" in the logic, such as "Sam" in the example), or with [the value of] another variable (such as customer_name). In both cases the value of the function's my_name variable gets assigned some value, "Sam" and "Mr Peter Clark" respectively. In the latter case, this value is whatever the customer_name variable contains. Note that the names of the variables used inside the function (my_name) and when the function is called (customer_name) do not need to be the same. (these are called the "formal parameter(s)" and the "actual parameters" respectively)
Note that while typically most arguments as passed as input to a function, in some conditions, they can be used as output, i.e. to provide new/modified values at the level of the logic which called the function. Doing so requires using, implicitly or explicitly, the proper calling convention specification (See Argument passing conventions below)
Now... beyond this very basic understanding of the purpose of parameters, things get a little more complicated than that (but not much). I'll discuss these additional concepts in general and illustrate them as they apply to Python.
Default values for arguments (aka "optional" arguments)
When the function is declared it may specify the default value for some parameters. These values are used for the parameters which are not specified when the function is called. For obvious reasons these optional parameters are found at the end of the parameter list (otherwise the language compiler/interpreter may have difficulty figuring out which parameter is which...)
>>> def say_hello(dude = "Sir"):
... print("Hello,", dude, "!")
...
>>> say_hello()
Hello, Sir !
>>> say_hello("William Gates")
Hello, Bill ! #just kidding ;-)
Hello, William Gates ! # but indeed. works as the original function when param
# is specified
Variable number of parameters
In some cases it may be handy to define a function so that it may accept a variable number of parameters. While such lists of parameter values ultimately get passed in some kind of container (list, array, collection...) various languages offers convenient ways of accessing such parameter values.
>>> def add_many(operand1, *operands):
... Sum = operand1
... for op in operands:
... Sum += op
... return Sum
...
>>> add_many(1, 3, 5, 7, 20)
36
>>> add_many(1, 3)
4
Named Arguments (Keyword Arguments)
With Python and a few other languages, it is possible to explicitly name the arguments when calling the function. Whereby argument passing is by default based a positional basis ("1st argument, 2nd argument etc.), Python will let you name the arguments and pass them in any order. This is mostly a syntactic nicety, but can be useful, in combination with default arguments for functions that accept very many arguments. It is also a nice self-documenting feature.
>>> def do_greetings(greeting, person):
... print (greeting, "dear", person, "!")
...
>>> do_greetings(person="Jack", greeting="Good evening")
Good evening dear Jack !
In Python, you can even pass a dictionary in lieu of several named arguments for example, with do_greetingsas-is, imagine you have a dictionary like:
>>> my_param_dict = {"greeting":"Aloha", "person":"Alan"}
>>> do_greetings(**my_param_dict)
Aloha dear Alan !
In closing, and while the fancy ways of passing arguments, and the capability for methods to handle variable number of arguments are useful features of various languages, two key concepts need to be mentioned:
Argument passing convention : by value or by reference
So far all the functions we used didn't alter the value of the parameters passed to them. We can imagine however many instances when functions may want to do this, either to perform some conversion or computation on the said values, for its own internal use, or to effectively change the value of the variable so that the changes are reflected at the level of logic which called the function. That's where argument passing conventions come handy...
arguments which are passed by value may be altered by the function for its own internal computations but are not changed at the level of the calling method.
arguments which are passed by reference will reflect changes made to them, at the level of the calling method.
Each language specifies the ways that arguments are passed. A typical convention is to pass integers, numberic values and other basic types by value and to pass objects by reference. Most language also offer keyword that allow altering their default convention.
In python all arguments are passed by reference. However a few variables types are immutable (numbers, strings, tuples...) and they can therefore not be altered by the function.
Implicit "self" or "this" argument of class methods
In object oriented languages, methods (i.e. functions within a class) receive an extra argument that is the value of underlying object (the instance of the class), allowing the method to use various properties members of the class in its computation and/or to alter the value of some of these properties.
in Python, this argument is declared at the level of the method definition, but is passed implicitly. Being declared, it may be named most anything one wishes, although by convention this is typically called self.
>>> class Accumulator:
... def __init__(self, initialValue = 0):
... self.CurValue = initialValue
... def Add(self, x):
... self.CurValue += x
... return self.CurValue
...
>>> my_accu = Accumulator(10)
>>> my_accu.Add(5)
15
>>> my_accu.Add(3)
18
In that case for using arguments, it is simply a demo on how to use them, not the most effective perhaps as you demonstrated. Functions are very useful. for example if i wanted to add two numbers:
def add(num1, num2):
x = num1 + num2
return x
add(1,3)
Functions are useful for performing repetitive tasks, let's say in your example you had to say hello to hundreds of names, instead of having to do a loop of that raw_input() function to read their name and add some text to it you could simply call a function to perform the task and pass arguments (the persons name to it).
Per your second question, arguments are just variables passed to the function, so whatever variable you pass to it from the outside, for example I pass numbers 1 and 3 to my function add on the inside of that function they are simply referred to as num1 num2.
In your case with passing too many arguments would yield this:
>>> add(1,2)
3
>>> add(1,2,3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: add() takes exactly 2 arguments (3 given)
>>>
Best of luck! and feel free to shoot me an email if you need further help (sbrichards (at) mit.edu)
The stuff in the parentheses are called arguments. Basically these are variables that you want the function to work with. For example, say you have a function, and you want it to print a word when you call it. With arguments, you can define what that word will be. Here is an example:
def hi(word):
print word
now, if I do something like this:
hi('Hello!')
it will print:
'Hello!'
hi() gets passed 'Hello!' as a variable called word, and then it prints that word.
In your example they are not used.
If your function needs to behave differently depending on what arguments you give it, then what's arguments are for.
def hi_name(name):
print 'Hi ' + name
hi_name("John doe")
This prints "Hi John Doe".
Now back to the basics about functions.
An argument is a special variable that exists only in that function.
In your example you have a function that takes 2 arguments:
def add(num1,num2):
x = num1 + num2
return x
When I call this function with add(), I have to add in the parentheses what I want num1 and num2 to be. In your case, you have 1 and 3, so you call it like this add(1,3).
What you're saying there is that you want to call add() and you want the first argument, num1 to be equal to 1, and the second argument, num2, to be equal to 3.
def main(a, b):
print(a)
print(b)
main(3, 5)
This is a basic example of a function with parameters, check for more information here.
output:
3
5
Functions would be useless if you can't give them information they should handle.
Arguments are such information.
def GiveMeANumberAndIDuplicateIt(x):
return x * 2
def DontGiveMeAnythingAtAll():
return None
i have see some code like bellow:
params = {
'username': username,
'password': password,
'attended': attended,
'openid_identifier': openid_identifier,
'multistage': (stage and True) or None
}
ret = authmethod.login(request, userobj, **params)
login is implemented like this
def login(self,request,user_obj,**kw):
username = kw.get('username')
password = kw.get('password')
so we know that kw is a dictionary , but i don't know the ** meaning , is it something like pointer in C language ? is it used to input the dictionary as a reference .
thank you if you could answer me .
Basically this is what it means:
Without using **kw, you would need to list all the input parameters for login in it's signature.
Now, you are calling login function and you know what variable names login's parameters are. So if you have a lot of parameters, it's difficult to always remember the order of the parameters.
Therefore, you call login on parameters by naming the parameters' variable names and setting them equal to the value that you want to pass it. Think of it like this:
Without using **kw, you'd do this:
def say(phrase):
print phrase
say("Hello, World!")
But, by using **kw, you can do this:
def say(**kw):
phrase = kw.get('say_what')
print phrase
say(**{'say_what':"Hello, World!"})
Now what happens is that using ** "unpacks the dictionary in such a way that it tells say that what it expects as the input parameter named say_what will have the value "Hello, World!".
The above example is not the best place to use **kw because there is only one input parameter. But if you have a function with a long signature, then it would be unreasonable to expect any programmer to remember exactly what parameters must be passed in what order to this function.
If (you and) the programmer were to use **kw, then the programmer can specify a dictionary which maps input the parameters' variable names (as strings) to their values. The function takes care of the rest and the programmer doesn't have to concern himself with the order in which he passes the parameters to the function
Hope this helps
These are called keyword arguments and they're described in lots of places, like the Python manual and in blog posts.