Related
I was testing some code to answer Possible to turn an input string into a callable function object in Python? and the question got closed as duplicate but I could not find the answer to that question in the source answers.
There are many answers on SO similar to this but none address the issue how to return a function from string. exec() doesn't return a function. It just executes arbitrary code.
my question is, is this right approach to convert string to function and return as a function object?
my_string = "def add5(x):return x + 5"
def string_to_func(my_string):
exec(my_string)
for var_name, var in locals().items():
if callable(var):
return var
print("no callable in string")
#fallback code.
add5 = string_to_func(my_string)
print(add5(3))
P.S.: I will delete this question if the original question gets reopened.
Roughly speaking, if you assume that inputs are fully trusted, and where the provided string is of valid Python syntax that will produce a single callable function, and if exec cannot be used, something like this can be provided
import ast
import types
def string_to_function(source):
tree = ast.parse(source)
if len(tree.body) != 1 or not isinstance(tree.body[0], ast.FunctionDef):
raise ValueError('provided code fragment is not a single function')
co = compile(tree, 'custom.py', 'exec')
# first constant should be the code object for the function
return types.FunctionType(co.co_consts[0], {})
Example:
f = string_to_function("""
def my_function(x):
return x + 5
"""
)
print('f = %d' % f(5))
Output
f = 10
The code ensures that the provided source is of a single function definition, and makes assumption of the organisation of the generated bytecode (i.e. only works for the compiler built into the current versions of Python, where the generated bytecode places the code object for the single function that was defined in the source in the 0th element in co_consts). The previous version to this answer made use of exec (which questioner "is not a big fan of exec anyway"), done in a way that binds the results into a specific target and this method should be more reliable as it does not touch this lower level structure, though the sanity checks using ast module used here could be included instead with that original version.
Also note that this answer is only applicable for Python 3+.
Further Edit:
I am actually still a little miffed by the remark on the usage of exec being assumed to execute arbitrary code (on fixed inputs) simply because what it actually does is often very misunderstood. In this particular case, if it is verified that only specific source is accepted, it doesn't necessarily mean every statement is executed immediately. This is especially true for this case (which isn't properly guaranteed in my original lazy answer, but this is where actual understanding of what the framework is actually doing is important for doing anything that involves dynamic compilation of code within the language framework, and given that more level of "safety" is desired (executing function immediately after the fact negates it hence I didn't implemented originally) using ast is done in the edit and it is now objectively better).
So what exactly does calling exec(co) do like essentially what the original example did when given the source input of an single function definition? It can be determined by looking at the bytecode like so:
>>> dis.dis(co)
2 0 LOAD_CONST 0 (<code object my_function at 0x7fdbec44c420, file "custom.py", line 2>)
2 LOAD_CONST 1 ('my_function')
4 MAKE_FUNCTION 0
6 STORE_NAME 0 (my_function)
8 LOAD_CONST 2 (None)
10 RETURN_VALUE
All it does is to load the code object, make it into a proper function and assign the result to my_function (on the currently relevant scope), and essentially that's it. So yes, the correct way is to verify that the source is definitely a function definition like so here (as verification through checking the AST is more safe than a more naive verification that only one statement is present), then running exec on a specific dict and extract the assignment from there. Using exec this way is not inherently less (or more) safe in this instance, given that any function that was provided would be executed immediately anyway.
When forgetting to pass certain arguments to a function, Python gives the only-somewhat-helpful message "myfunction() takes X arguments (Y given)". Is there a way to figure out the names of the missing arguments, and tell the user? Something like:
try:
#begin blackbox
def f(x,y):
return x*y
f(x=1)
#end blackbox
except Exception as e:
#figure out the missing keyword argument is called "y" and tell the user so
Assuming that the code between begin blackbox and end blackbox is unknown to the exception handler.
Edit: As its been pointed out to me below, Python 3 already has this functionality built in. Let me extend the question then, is there a (probably ugly and hacky) way to do this in Python 2.x?
A much cleaner way to do this would be to wrap the function in another function, pass through the *args, **kwargs, and then use those values when you need them, instead of trying to reconstruct them after the fact. But if you don't want to do that…
In Python 3.x (except very early versions), this is easy, as poke's answer explains. Even easier with 3.3+, with inspect.signature, inspect.getargvalues, and inspect.Signature.bind_partial and friends.
In Python 2.x, there is no way to do this. The exception only has the string 'f() takes exactly 2 arguments (1 given)' in its args.
Except… in CPython 2.x specifically, it's possible with enough ugly and brittle hackery.
You've got a traceback, so you've got its tb_frame and tb_lineno… which is everything you need. So as long as the source is available, the inspect module makes it easy to get the actual function call expression. Then you just need to parse it (via ast) to get the arguments passed, and compare to the function's signature (which, unfortunately, isn't nearly as easy to get in 2.x as in 3.3+, but between f.func_defaults, f.func_code.co_argcount, etc., you can reconstruct it).
But what if the source isn't available? Well, between tb_frame.f_code and tb_lasti, you can find out where the function call was in the bytecode. And the dis module makes that relatively easy to parse. In particular, right before the call, the positional arguments and the name-value pairs for keyword arguments were all pushed on the stack, so you can easily see which names got pushed, and how many positional values, and reconstruct the function call that way. Which you compare to the signature in the same way.
Of course that relies on the some assumptions about how CPython's compiler builds bytecode. It would be perfectly legal to do things in all kinds of different orders as long as the stack ended up with the right values. So, it's pretty brittle. But I think there are already better reasons not to do it.
I would argue that doing this doesn’t really make that much sense. Such an exception is thrown because the programmer missed specifying the argument. So if you knowingly catch the exception, then you could just as well just fix it in the first place.
That being said, in current Python 3 versions, the TypeError that is being thrown does mention which arguments are missing from the call:
"f() missing 1 required positional argument: 'y'"
Unfortunately, the argument name is not mentioned separately, so you would have to extract it from the string:
try:
f(x=1)
except TypeError as e:
if 'required positional argument' in e.args[0]:
argumentNames = e.args[0].split("'")[1::2]
print('Missing arguments are ' + argumentNames)
else:
raise # Re-raise other TypeErrors
As Joran Beasley pointed out in the comments, Python 2 does not tell you which arguments are missing but just how many are missing. So there is no way to tell from the exception which arguments were missing in the call.
except TypeError as e:
import inspect
got_args = int(re.search("\d+.*(\d+)",str(e)).groups()[0])
print "missing args:",inspect.getargspec(f).args[got_args:]
a better method would be a decorator
def arg_decorator(fn):
def func(*args,**kwargs):
try:
return fn(*args,**kwargs)
except TypeError:
arg_spec = inspect.getargspec(fn)
missing_named = [a for a in arg_spec.args if a not in kwargs]
if arg_spec.defaults:
missing_args = missing_named[len(args): -len(arg_spec.defaults) ]
else:
missing_args = missing_named[len(args):]
print "Missing:",missing_args
return func
#arg_decorator
def fn1(x,y,z):
pass
def fn2(x,y):
pass
arged_fn2 = arg_decorator(fn2)
fn1(5,y=2)
arged_fn2(x=1)
With purely the exception to deal with it is not possible to do what you want and handle keyword arguments. This is of course wrt Python 2.7.
The code that generates this message in Python is:
PyErr_Format(PyExc_TypeError,
"%.200s() takes %s %d "
"argument%s (%d given)",
PyString_AsString(co->co_name),
defcount ? "at most" : "exactly",
co->co_argcount,
co->co_argcount == 1 ? "" : "s",
argcount + kwcount);
Taken from lines 3056-3063 from http://hg.python.org/cpython/file/0e5df5b62488/Python/ceval.c
As you can see, there is just not enough information given to the exception as to what arguments are missing. co in this context is the PyCodeObject being called. The only thing given is a string (which you could parse if you like) with the function name, whether or not there is a vararg, how many arguments are expected, and how many arguments were given. As has been pointed out, this does not give you sufficient information as to what argument(s) were not given (in the case of keyword arguments).
Something like inspect or the other debugging modules might be able to give you enough information as to what function was called and how it was called, from which you could figure out what arguments were not given.
I should also mention however that almost certainly, whatever solution you come up with will not be able to handle at least some extension module methods (those written in C) because they don't provide argument information as part of their object.
What are [function] arguments? What are they used for?
I started learning Python very recently; I'm new to programming and I apologize for this basic question.
In every Python tutorial I go through they talk about arguments. I have looked for the answer to this question and have found many answers but they are just a little too hard for me to understatnd. I may just be missing some conceptual background.
So... when I define a function, what are the things in parenthesis used for?
Example:
def hi( This is the part that i dont get):
print 'hi'
Edit:
Two follow-up questions related to this one were later closed and merged here, hence the partial out-of-context trait of some of the answers.
The follow-up questions were: [paraphrased]
Can arguments only be used for input ?
what are other examples of the use of arguments ?
Why use arguments, rather than having the function call raw_input ?
Why is the concept of argument passing described as such a powerful thing? it seems to me we're merely using them to replace stuff the user could have type on the keyboard.
In a few words, they're data that gets "passed into" the function to tell it what to do. Wikipedia has details.
http://en.wikipedia.org/wiki/Function_argument
For instance, your hi() function might need to know who to say hello to:
def hi(person):
print "Hi there " + person + ", how are you?"
Or a mathematical function might need a value to operate on:
def square(x):
return x * x
This is not a Python question, but rather a generic programming question. A very basic one.
Before answering the question about arguments, and in view of the other questions you asked, it is useful to discuss the concept of variables.
A variable is a named piece of memory where information of interest to the underlying program can be stored and retrieved. In other words, it is a symbolic name, chosen by the programmer, that is associated to its contents. Using various language constructs generally known as assignments, the programmer can read or write the contents of a variable.
It is important to note that the value (i.e. the content) of a variable needn't be defined when the program is written. It is only necessary at run-time. This allows the program to describe actions to be performed on symbolic elements without knowing exactly the value these elements have. Consider this snippet, part of a bigger program:
# ... some logic above
ball_volume = 4.0 / 3 * math.pi * ball_radius
if ball_volume > 200:
print ("Man, that's a big ball")
# ... more logic below
At the time the program is written one doesn't need to know the actual value of ball_radius; yet, with the assumption that this variable will contain the numeric value of some hypothetical ball, the snippet is capable of describing how to compute the ball's volume. In this fashion, when the program is running, and somehow (more on this later) the ball_radius variable has been initialized with some appropriate value, the variable ball_volume can too be initialized and used, here in the conditional statement (if), and possibly below. (At some point the variable may go out-of-scope, but this concept which controls when particular variables are accessible to the program is well beyond this primer).
In some languages the type of data that may be associated with a particular variable needs to be explicitly defined and cannot change. For example some variables could hold only integer values, other variables string values (text) etc. In Python there is no such restriction, a variable can be assigned and re-assigned to any type of data, but of course, the programmer needs to keep track of this for example to avoid passing some text data to a mathematical function.
The data stored inside variable may come from very different sources. Many of the examples provided in tutorials and introductory documentation have this data coming from keyboard input (as when using raw_input as mentioned in some of your questions). That is because it allows interactive tests by the people trying out these tutorial snippets. But the usefulness of programs would be rather limited if variables only get their data from interactive user input. There are many other sources and this is what makes programming so powerful: variables can be initialized with data from:
databases
text files or files various text-base formats (XML, JSON, CSV..)
binary files with various formats
internet connections
physical devices: cameras, temperature sensors...
In a nutshell, Arguments, also called Parameters, are variables passed to the function which [typically] are used to provide different output and behavior from the function. For example:
>>> def say_hello(my_name):
... print("Hello,", my_name, "!")
>>> say_hello("Sam")
Hello, Sam !
>>> customer_name = "Mr Peter Clark" #imagine this info came from a database
>>> # ...
>>> say_hello(customer_name)
Hello, Mr Peter Clark !
>>>
In the example above, my_name is just like any local variable of the say_hello function; this allows the function to define what it will do with the underlying value when the function is called, at run-time.
At run-time, the function can be called with an immediate value (a value that is "hard-coded" in the logic, such as "Sam" in the example), or with [the value of] another variable (such as customer_name). In both cases the value of the function's my_name variable gets assigned some value, "Sam" and "Mr Peter Clark" respectively. In the latter case, this value is whatever the customer_name variable contains. Note that the names of the variables used inside the function (my_name) and when the function is called (customer_name) do not need to be the same. (these are called the "formal parameter(s)" and the "actual parameters" respectively)
Note that while typically most arguments as passed as input to a function, in some conditions, they can be used as output, i.e. to provide new/modified values at the level of the logic which called the function. Doing so requires using, implicitly or explicitly, the proper calling convention specification (See Argument passing conventions below)
Now... beyond this very basic understanding of the purpose of parameters, things get a little more complicated than that (but not much). I'll discuss these additional concepts in general and illustrate them as they apply to Python.
Default values for arguments (aka "optional" arguments)
When the function is declared it may specify the default value for some parameters. These values are used for the parameters which are not specified when the function is called. For obvious reasons these optional parameters are found at the end of the parameter list (otherwise the language compiler/interpreter may have difficulty figuring out which parameter is which...)
>>> def say_hello(dude = "Sir"):
... print("Hello,", dude, "!")
...
>>> say_hello()
Hello, Sir !
>>> say_hello("William Gates")
Hello, Bill ! #just kidding ;-)
Hello, William Gates ! # but indeed. works as the original function when param
# is specified
Variable number of parameters
In some cases it may be handy to define a function so that it may accept a variable number of parameters. While such lists of parameter values ultimately get passed in some kind of container (list, array, collection...) various languages offers convenient ways of accessing such parameter values.
>>> def add_many(operand1, *operands):
... Sum = operand1
... for op in operands:
... Sum += op
... return Sum
...
>>> add_many(1, 3, 5, 7, 20)
36
>>> add_many(1, 3)
4
Named Arguments (Keyword Arguments)
With Python and a few other languages, it is possible to explicitly name the arguments when calling the function. Whereby argument passing is by default based a positional basis ("1st argument, 2nd argument etc.), Python will let you name the arguments and pass them in any order. This is mostly a syntactic nicety, but can be useful, in combination with default arguments for functions that accept very many arguments. It is also a nice self-documenting feature.
>>> def do_greetings(greeting, person):
... print (greeting, "dear", person, "!")
...
>>> do_greetings(person="Jack", greeting="Good evening")
Good evening dear Jack !
In Python, you can even pass a dictionary in lieu of several named arguments for example, with do_greetingsas-is, imagine you have a dictionary like:
>>> my_param_dict = {"greeting":"Aloha", "person":"Alan"}
>>> do_greetings(**my_param_dict)
Aloha dear Alan !
In closing, and while the fancy ways of passing arguments, and the capability for methods to handle variable number of arguments are useful features of various languages, two key concepts need to be mentioned:
Argument passing convention : by value or by reference
So far all the functions we used didn't alter the value of the parameters passed to them. We can imagine however many instances when functions may want to do this, either to perform some conversion or computation on the said values, for its own internal use, or to effectively change the value of the variable so that the changes are reflected at the level of logic which called the function. That's where argument passing conventions come handy...
arguments which are passed by value may be altered by the function for its own internal computations but are not changed at the level of the calling method.
arguments which are passed by reference will reflect changes made to them, at the level of the calling method.
Each language specifies the ways that arguments are passed. A typical convention is to pass integers, numberic values and other basic types by value and to pass objects by reference. Most language also offer keyword that allow altering their default convention.
In python all arguments are passed by reference. However a few variables types are immutable (numbers, strings, tuples...) and they can therefore not be altered by the function.
Implicit "self" or "this" argument of class methods
In object oriented languages, methods (i.e. functions within a class) receive an extra argument that is the value of underlying object (the instance of the class), allowing the method to use various properties members of the class in its computation and/or to alter the value of some of these properties.
in Python, this argument is declared at the level of the method definition, but is passed implicitly. Being declared, it may be named most anything one wishes, although by convention this is typically called self.
>>> class Accumulator:
... def __init__(self, initialValue = 0):
... self.CurValue = initialValue
... def Add(self, x):
... self.CurValue += x
... return self.CurValue
...
>>> my_accu = Accumulator(10)
>>> my_accu.Add(5)
15
>>> my_accu.Add(3)
18
In that case for using arguments, it is simply a demo on how to use them, not the most effective perhaps as you demonstrated. Functions are very useful. for example if i wanted to add two numbers:
def add(num1, num2):
x = num1 + num2
return x
add(1,3)
Functions are useful for performing repetitive tasks, let's say in your example you had to say hello to hundreds of names, instead of having to do a loop of that raw_input() function to read their name and add some text to it you could simply call a function to perform the task and pass arguments (the persons name to it).
Per your second question, arguments are just variables passed to the function, so whatever variable you pass to it from the outside, for example I pass numbers 1 and 3 to my function add on the inside of that function they are simply referred to as num1 num2.
In your case with passing too many arguments would yield this:
>>> add(1,2)
3
>>> add(1,2,3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: add() takes exactly 2 arguments (3 given)
>>>
Best of luck! and feel free to shoot me an email if you need further help (sbrichards (at) mit.edu)
The stuff in the parentheses are called arguments. Basically these are variables that you want the function to work with. For example, say you have a function, and you want it to print a word when you call it. With arguments, you can define what that word will be. Here is an example:
def hi(word):
print word
now, if I do something like this:
hi('Hello!')
it will print:
'Hello!'
hi() gets passed 'Hello!' as a variable called word, and then it prints that word.
In your example they are not used.
If your function needs to behave differently depending on what arguments you give it, then what's arguments are for.
def hi_name(name):
print 'Hi ' + name
hi_name("John doe")
This prints "Hi John Doe".
Now back to the basics about functions.
An argument is a special variable that exists only in that function.
In your example you have a function that takes 2 arguments:
def add(num1,num2):
x = num1 + num2
return x
When I call this function with add(), I have to add in the parentheses what I want num1 and num2 to be. In your case, you have 1 and 3, so you call it like this add(1,3).
What you're saying there is that you want to call add() and you want the first argument, num1 to be equal to 1, and the second argument, num2, to be equal to 3.
def main(a, b):
print(a)
print(b)
main(3, 5)
This is a basic example of a function with parameters, check for more information here.
output:
3
5
Functions would be useless if you can't give them information they should handle.
Arguments are such information.
def GiveMeANumberAndIDuplicateIt(x):
return x * 2
def DontGiveMeAnythingAtAll():
return None
What are [function] arguments? What are they used for?
I started learning Python very recently; I'm new to programming and I apologize for this basic question.
In every Python tutorial I go through they talk about arguments. I have looked for the answer to this question and have found many answers but they are just a little too hard for me to understatnd. I may just be missing some conceptual background.
So... when I define a function, what are the things in parenthesis used for?
Example:
def hi( This is the part that i dont get):
print 'hi'
Edit:
Two follow-up questions related to this one were later closed and merged here, hence the partial out-of-context trait of some of the answers.
The follow-up questions were: [paraphrased]
Can arguments only be used for input ?
what are other examples of the use of arguments ?
Why use arguments, rather than having the function call raw_input ?
Why is the concept of argument passing described as such a powerful thing? it seems to me we're merely using them to replace stuff the user could have type on the keyboard.
In a few words, they're data that gets "passed into" the function to tell it what to do. Wikipedia has details.
http://en.wikipedia.org/wiki/Function_argument
For instance, your hi() function might need to know who to say hello to:
def hi(person):
print "Hi there " + person + ", how are you?"
Or a mathematical function might need a value to operate on:
def square(x):
return x * x
This is not a Python question, but rather a generic programming question. A very basic one.
Before answering the question about arguments, and in view of the other questions you asked, it is useful to discuss the concept of variables.
A variable is a named piece of memory where information of interest to the underlying program can be stored and retrieved. In other words, it is a symbolic name, chosen by the programmer, that is associated to its contents. Using various language constructs generally known as assignments, the programmer can read or write the contents of a variable.
It is important to note that the value (i.e. the content) of a variable needn't be defined when the program is written. It is only necessary at run-time. This allows the program to describe actions to be performed on symbolic elements without knowing exactly the value these elements have. Consider this snippet, part of a bigger program:
# ... some logic above
ball_volume = 4.0 / 3 * math.pi * ball_radius
if ball_volume > 200:
print ("Man, that's a big ball")
# ... more logic below
At the time the program is written one doesn't need to know the actual value of ball_radius; yet, with the assumption that this variable will contain the numeric value of some hypothetical ball, the snippet is capable of describing how to compute the ball's volume. In this fashion, when the program is running, and somehow (more on this later) the ball_radius variable has been initialized with some appropriate value, the variable ball_volume can too be initialized and used, here in the conditional statement (if), and possibly below. (At some point the variable may go out-of-scope, but this concept which controls when particular variables are accessible to the program is well beyond this primer).
In some languages the type of data that may be associated with a particular variable needs to be explicitly defined and cannot change. For example some variables could hold only integer values, other variables string values (text) etc. In Python there is no such restriction, a variable can be assigned and re-assigned to any type of data, but of course, the programmer needs to keep track of this for example to avoid passing some text data to a mathematical function.
The data stored inside variable may come from very different sources. Many of the examples provided in tutorials and introductory documentation have this data coming from keyboard input (as when using raw_input as mentioned in some of your questions). That is because it allows interactive tests by the people trying out these tutorial snippets. But the usefulness of programs would be rather limited if variables only get their data from interactive user input. There are many other sources and this is what makes programming so powerful: variables can be initialized with data from:
databases
text files or files various text-base formats (XML, JSON, CSV..)
binary files with various formats
internet connections
physical devices: cameras, temperature sensors...
In a nutshell, Arguments, also called Parameters, are variables passed to the function which [typically] are used to provide different output and behavior from the function. For example:
>>> def say_hello(my_name):
... print("Hello,", my_name, "!")
>>> say_hello("Sam")
Hello, Sam !
>>> customer_name = "Mr Peter Clark" #imagine this info came from a database
>>> # ...
>>> say_hello(customer_name)
Hello, Mr Peter Clark !
>>>
In the example above, my_name is just like any local variable of the say_hello function; this allows the function to define what it will do with the underlying value when the function is called, at run-time.
At run-time, the function can be called with an immediate value (a value that is "hard-coded" in the logic, such as "Sam" in the example), or with [the value of] another variable (such as customer_name). In both cases the value of the function's my_name variable gets assigned some value, "Sam" and "Mr Peter Clark" respectively. In the latter case, this value is whatever the customer_name variable contains. Note that the names of the variables used inside the function (my_name) and when the function is called (customer_name) do not need to be the same. (these are called the "formal parameter(s)" and the "actual parameters" respectively)
Note that while typically most arguments as passed as input to a function, in some conditions, they can be used as output, i.e. to provide new/modified values at the level of the logic which called the function. Doing so requires using, implicitly or explicitly, the proper calling convention specification (See Argument passing conventions below)
Now... beyond this very basic understanding of the purpose of parameters, things get a little more complicated than that (but not much). I'll discuss these additional concepts in general and illustrate them as they apply to Python.
Default values for arguments (aka "optional" arguments)
When the function is declared it may specify the default value for some parameters. These values are used for the parameters which are not specified when the function is called. For obvious reasons these optional parameters are found at the end of the parameter list (otherwise the language compiler/interpreter may have difficulty figuring out which parameter is which...)
>>> def say_hello(dude = "Sir"):
... print("Hello,", dude, "!")
...
>>> say_hello()
Hello, Sir !
>>> say_hello("William Gates")
Hello, Bill ! #just kidding ;-)
Hello, William Gates ! # but indeed. works as the original function when param
# is specified
Variable number of parameters
In some cases it may be handy to define a function so that it may accept a variable number of parameters. While such lists of parameter values ultimately get passed in some kind of container (list, array, collection...) various languages offers convenient ways of accessing such parameter values.
>>> def add_many(operand1, *operands):
... Sum = operand1
... for op in operands:
... Sum += op
... return Sum
...
>>> add_many(1, 3, 5, 7, 20)
36
>>> add_many(1, 3)
4
Named Arguments (Keyword Arguments)
With Python and a few other languages, it is possible to explicitly name the arguments when calling the function. Whereby argument passing is by default based a positional basis ("1st argument, 2nd argument etc.), Python will let you name the arguments and pass them in any order. This is mostly a syntactic nicety, but can be useful, in combination with default arguments for functions that accept very many arguments. It is also a nice self-documenting feature.
>>> def do_greetings(greeting, person):
... print (greeting, "dear", person, "!")
...
>>> do_greetings(person="Jack", greeting="Good evening")
Good evening dear Jack !
In Python, you can even pass a dictionary in lieu of several named arguments for example, with do_greetingsas-is, imagine you have a dictionary like:
>>> my_param_dict = {"greeting":"Aloha", "person":"Alan"}
>>> do_greetings(**my_param_dict)
Aloha dear Alan !
In closing, and while the fancy ways of passing arguments, and the capability for methods to handle variable number of arguments are useful features of various languages, two key concepts need to be mentioned:
Argument passing convention : by value or by reference
So far all the functions we used didn't alter the value of the parameters passed to them. We can imagine however many instances when functions may want to do this, either to perform some conversion or computation on the said values, for its own internal use, or to effectively change the value of the variable so that the changes are reflected at the level of logic which called the function. That's where argument passing conventions come handy...
arguments which are passed by value may be altered by the function for its own internal computations but are not changed at the level of the calling method.
arguments which are passed by reference will reflect changes made to them, at the level of the calling method.
Each language specifies the ways that arguments are passed. A typical convention is to pass integers, numberic values and other basic types by value and to pass objects by reference. Most language also offer keyword that allow altering their default convention.
In python all arguments are passed by reference. However a few variables types are immutable (numbers, strings, tuples...) and they can therefore not be altered by the function.
Implicit "self" or "this" argument of class methods
In object oriented languages, methods (i.e. functions within a class) receive an extra argument that is the value of underlying object (the instance of the class), allowing the method to use various properties members of the class in its computation and/or to alter the value of some of these properties.
in Python, this argument is declared at the level of the method definition, but is passed implicitly. Being declared, it may be named most anything one wishes, although by convention this is typically called self.
>>> class Accumulator:
... def __init__(self, initialValue = 0):
... self.CurValue = initialValue
... def Add(self, x):
... self.CurValue += x
... return self.CurValue
...
>>> my_accu = Accumulator(10)
>>> my_accu.Add(5)
15
>>> my_accu.Add(3)
18
In that case for using arguments, it is simply a demo on how to use them, not the most effective perhaps as you demonstrated. Functions are very useful. for example if i wanted to add two numbers:
def add(num1, num2):
x = num1 + num2
return x
add(1,3)
Functions are useful for performing repetitive tasks, let's say in your example you had to say hello to hundreds of names, instead of having to do a loop of that raw_input() function to read their name and add some text to it you could simply call a function to perform the task and pass arguments (the persons name to it).
Per your second question, arguments are just variables passed to the function, so whatever variable you pass to it from the outside, for example I pass numbers 1 and 3 to my function add on the inside of that function they are simply referred to as num1 num2.
In your case with passing too many arguments would yield this:
>>> add(1,2)
3
>>> add(1,2,3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: add() takes exactly 2 arguments (3 given)
>>>
Best of luck! and feel free to shoot me an email if you need further help (sbrichards (at) mit.edu)
The stuff in the parentheses are called arguments. Basically these are variables that you want the function to work with. For example, say you have a function, and you want it to print a word when you call it. With arguments, you can define what that word will be. Here is an example:
def hi(word):
print word
now, if I do something like this:
hi('Hello!')
it will print:
'Hello!'
hi() gets passed 'Hello!' as a variable called word, and then it prints that word.
In your example they are not used.
If your function needs to behave differently depending on what arguments you give it, then what's arguments are for.
def hi_name(name):
print 'Hi ' + name
hi_name("John doe")
This prints "Hi John Doe".
Now back to the basics about functions.
An argument is a special variable that exists only in that function.
In your example you have a function that takes 2 arguments:
def add(num1,num2):
x = num1 + num2
return x
When I call this function with add(), I have to add in the parentheses what I want num1 and num2 to be. In your case, you have 1 and 3, so you call it like this add(1,3).
What you're saying there is that you want to call add() and you want the first argument, num1 to be equal to 1, and the second argument, num2, to be equal to 3.
def main(a, b):
print(a)
print(b)
main(3, 5)
This is a basic example of a function with parameters, check for more information here.
output:
3
5
Functions would be useless if you can't give them information they should handle.
Arguments are such information.
def GiveMeANumberAndIDuplicateIt(x):
return x * 2
def DontGiveMeAnythingAtAll():
return None
I was working with generator functions and private functions of a class. I am wondering
Why when yielding (which in my one case was by accident) in __someFunc that this function just appears not to be called from within __someGenerator. Also what is the terminology I want to use when referring to these aspects of the language?
Can the python interpreter warn of such instances?
Below is an example snippet of my scenario.
class someClass():
def __init__(self):
pass
#Copy and paste mistake where yield ended up in a regular function
def __someFunc(self):
print "hello"
#yield True #if yielding in this function it isn't called
def __someGenerator (self):
for i in range(0, 10):
self.__someFunc()
yield True
yield False
def someMethod(self):
func = self.__someGenerator()
while func.next():
print "next"
sc = someClass()
sc.someMethod()
I got burned on this and spent some time trying to figure out why a function just wasn't getting called. I finally discovered I was yielding in function I didn't want to in.
A "generator" isn't so much a language feature, as a name for functions that "yield." Yielding is pretty much always legal. There's not really any way for Python to know that you didn't "mean" to yield from some function.
This PEP http://www.python.org/dev/peps/pep-0255/ talks about generators, and may help you understand the background better.
I sympathize with your experience, but compilers can't figure out what you "meant for them to do", only what you actually told them to do.
I'll try to answer the first of your questions.
A regular function, when called like this:
val = func()
executes its inside statements until it ends or a return statement is reached. Then the return value of the function is assigned to val.
If a compiler recognizes the function to actually be a generator and not a regular function (it does that by looking for yield statements inside the function -- if there's at least one, it's a generator), the scenario when calling it the same way as above has different consequences. Upon calling func(), no code inside the function is executed, and a special <generator> value is assigned to val. Then, the first time you call val.next(), the actual statements of func are being executed until a yield or return is encountered, upon which the execution of the function stops, value yielded is returned and generator waits for another call to val.next().
That's why, in your example, function __someFunc didn't print "hello" -- its statements were not executed, because you haven't called self.__someFunc().next(), but only self.__someFunc().
Unfortunately, I'm pretty sure there's no built-in warning mechanism for programming errors like yours.
Python doesn't know whether you want to create a generator object for later iteration or call a function. But python isn't your only tool for seeing what's going on with your code. If you're using an editor or IDE that allows customized syntax highlighting, you can tell it to give the yield keyword a different color, or even a bright background, which will help you find your errors more quickly, at least. In vim, for example, you might do:
:syntax keyword Yield yield
:highlight yield ctermbg=yellow guibg=yellow ctermfg=blue guifg=blue
Those are horrendous colors, by the way. I recommend picking something better. Another option, if your editor or IDE won't cooperate, is to set up a custom rule in a code checker like pylint. An example from pylint's source tarball:
from pylint.interfaces import IRawChecker
from pylint.checkers import BaseChecker
class MyRawChecker(BaseChecker):
"""check for line continuations with '\' instead of using triple
quoted string or parenthesis
"""
__implements__ = IRawChecker
name = 'custom_raw'
msgs = {'W9901': ('use \\ for line continuation',
('Used when a \\ is used for a line continuation instead'
' of using triple quoted string or parenthesis.')),
}
options = ()
def process_module(self, stream):
"""process a module
the module's content is accessible via the stream object
"""
for (lineno, line) in enumerate(stream):
if line.rstrip().endswith('\\'):
self.add_message('W9901', line=lineno)
def register(linter):
"""required method to auto register this checker"""
linter.register_checker(MyRawChecker(linter))
The pylint manual is available here: http://www.logilab.org/card/pylint_manual
And vim's syntax documentation is here: http://www.vim.org/htmldoc/syntax.html
Because the return keyword is applicable in both generator functions and regular functions, there's nothing you could possibly check (as #Christopher mentions). The return keyword in a generator indicates that a StopIteration exception should be raised.
If you try to return with a value from within a generator (which doesn't make sense, since return just means "stop iteration"), the compiler will complain at compile-time -- this may catch some copy-and-paste mistakes:
>>> def foo():
... yield 12
... return 15
...
File "<stdin>", line 3
SyntaxError: 'return' with argument inside generator
I personally just advise against copy and paste programming. :-)
From the PEP:
Note that return means "I'm done, and have nothing interesting to
return", for both generator functions and non-generator functions.
We do this.
Generators have names with "generate" or "gen" in their name. It will have a yield statement in the body. Pretty easy to check visually, since no method is much over 20 lines of code.
Other methods don't have "gen" in their name.
Also, we do not every use __ (double underscore) names under any circumstances. 32,000 lines of code. Non __ names.
The "generator vs. non-generator" method function is entirely a design question. What did the programmer "intend" to happen. The compiler can't easily validate your intent, it can only validate what you actually typed.