We Keep Coding

Trying to understand functions and how they are called with an exercise

I am interested in Python and started reading Think Python by Allen B. Downey. I am a complete beginner and do not know anyone in my circle of friends who programs so I decided I will ask my question here, however simple it may be.
I am currently in the section related to Functions and can not make sense of an example/exercise which involves concatenation:
def cat_twice(part1, part2):
cat = part1 + part2
print_twice(cat)
In the book this is described as storing a local variable in a function. In previous chapters of the same book all exercises are done in the Python Shell IDLE. As far as I understood in order to call a function after something is stored in it you do this:
cat_twice()
Yet when I do this I get:
Traceback (most recent call last):
File "<pyshell#63>", line 1, in <module>
cat_twice()
TypeError: cat_twice() missing 2 required positional arguments: 'part1' and 'part2'.
What exactly am I not getting or doing wrong? why isn't the program running?

What you are doing here is, your function is expecting two arguments, i.e., part1 and part2, compulsorily. When you are calling your function, you have to call as follows:
cat_twice(1,2) # assuming are trying to add two numbers in your function
If you are trying string concatenation, I would recommend you to look up join() function in python.
If you want to have a variadic function which can take either of part1 or part2 as an argument, or both or none of them, you can assign certain default value in method signature, for example,
def cat_twice(part1=None, part2=None)
...
...
...
if __name__ == '__main__':
cat_twice(part1=<something>)
cat_twice(part2=<somethingElse>)
cat_twice(part2=<some>, part1=<someMore>)
cat_twice(1,2) # here 1 will be passed as part1 and 2 as part2
Keep in mind that if you have assigned None as default to function parameters, the have a None check before performing addition operation or you may lead into TypeError for adding unsupported operand types.
I hope this is helpful.

Here a Simple Example, i have defined a function cat_twice which will print two values that are passed to it.
Then i passed two values by calling it like cat_twice() simple :) .
def cat_twice(part1,part2):
print(part1+part2) #this is the code in the function that will execute
cat_twice(1,2) #this is known as calling function

The "cat_twice" function contains the function "print_twice".
Be sure that python has run this function first, so that "cat_twice" can reach "print_twice" when it needs to.
It can be included at the start of the block.
Try:
def print_twice(bruce):
print(bruce)
print(bruce)
"""Defines the print_twice function"""
def cat_twice(part1,part2):
cat=part1+part2
"""the print_twice function now feeds into line 9"""
print_twice(cat)
line1='bingtiddle'
line2='tiddlebang'
cat_twice(line1,line2)
I am also alone in learning python, I did a few bootcamp course on Udemy.
Think Python is a far better book for actual learning.
This is the first advice I have felt confident enough to give, so I hope it works.
I have reached chapter 4 so far.
Good luck

Python function parameter as introspectable string

I'm currently building up a library of personal debugging functions, and wanted to make a debugging print function that would provide more quality-of-life functionality than adding print(thing) lines everywhere. My current goal is to have this pseudocode function translated to Python:
def p(*args, **kwargs)
for arg in args:
print(uneval(arg), ": ", arg)
return args
The types of **kwargs and how they'd effect the output isn't as important to this question, hence why I don't use them here.
My main problem comes from trying to have some method of uneval-ing the arguments, such that I can get the string of code that actually produced them. As a use case:
>>> p(3 + 5) + 1
3 + 5: 8 # printed by p
9 # Return value of the entire expression
There's a couple of ways that I see that I can do this, and not only am I not sure if any of them work, I'm also concerned about how Pythonic any solution that actually implements these could possibly be.
Pass the argument as a string, eval it using the previous context's local identifier dictionary (how could I get that without passing it as another argument, which I definitely don't want to do?)
Figure out which line it's being run from in the .py file to extract the uneval'd strings that way (is that even possible?)
Find some metadata magic that has the information I need already in it (if it exists, which is unlikely at best).

How to convert to multithreading subprocess

I have a method in my python (2.7.6) code that I am looking to use multithreading subprocess on by following the advice given in another SO question
This is how the code is currently:
return self.capi(roi_rgb,"",False)
This is how I converted it:
pool = multiprocessing.Pool(None)
result = ""
r = pool.map_async(self.capi(roi_rgb,"",False), callback=result)
r.wait()
return result
but I'm getting errors with the above on the call to pool.map_async
TypeError: map_async() takes at least 3 arguments (3 given)

According to https://docs.python.org/2/library/multiprocessing.html you need to give at least 2 positional arguments where as you gave it one positional and one keyword argument. (The third implicit arg is self)
So you need to pass the method a function and an iterable along with the callback.
P.s.that is a pretty useless error message isn't it?

python argparse: how to use other parsed argument as parameter at calling function in type keyword?

I am trying to create an user interface using argparse module.
One of the argument need to be converted, so I use the type keyword:
add_argument('positional', ..., type=myfunction)
and there is another optional argument:
add_argument('-s', dest='switch', ...)
in addition, I have
parsed_argument=parse_args()
However, in myfunction, I hope I can use an additional parameter to control the behavior, which is the optional argument above, i.e.
def myfunction(positional, switch=parsed_argument.switch):
...
How can I achieve that?

Simple answer: You can’t. The arguments are parsed separately, and there is no real guarantee that some order is maintained. Instead of putting your logic into the argument type, just store it as a string and do your stuff after parsing the command line:
parser.add_argument('positional')
parser.add_argument('-s', '--switch')
args = parser.parse_args()
myfunction(args.positional, switch=args.switch)

I'm not sure I did understand correctly what you want to achieve, but if what you want to do is something that looks like:
myprog.py cmd1 --switcha
myprog.py cmd2 --switchb
yes you can, you need to use subparsers. I wrote a good example of it for a little PoC I wrote to access stackoverflow's API from CLI. The whole logic is a bit long to put thoroughly here, but mainly the idea is:
create your parser using parser = argparse.ArgumentParser(...)
create the subparsers using subparsers = parser.add_subparsers(...)
add the commands with things like `subparser.add_parser('mycommand', help='Its only a command').set_defaults(func=mycmd_fn) where
mycmd_fn takes args as parameters where you have all the switches you issued to the command!
the difference from what you ask, is that you'll need one function per command, and not one function with the positional argument as first argument. But you can leverage that easily by having mycmd_fn being like: mycmd_fn = lambda *args: myfunction('mycmd', *args)
HTH

From the documentation:
type= can take any callable that takes a single string argument and returns the converted value:
Python functions like int and float are good examples of a type function should be like. int takes a string and returns a number. If it can't convert the string it raises a ValueError. Your function could do the same. argparse.ArgumentTypeError is another option. argparse isn't going to pass any optional arguments to it. Look at the code for argparse.FileType to see a more elaborate example of a custom type.
action is another place where you can customize behavior. The documentation has an example of a custom Action. Its arguments include the namespace, the object where the parser is collecting the values it will return to you. This object contains any arguments have already been set. In theory your switch value will be available there - if it occurs first.
There are many SO answers that give custom Actions.
Subparsers are another good way of customizing the handling of arguments.
Often it is better to check for the interaction of arguments after parse_args. In your case 'switch' could occur after the positional and still have effect. And argparse.Error lets you use the argparse error mechanism (e.g. displaying the usage)

How to figure out which Python keyword argument is missing?

When forgetting to pass certain arguments to a function, Python gives the only-somewhat-helpful message "myfunction() takes X arguments (Y given)". Is there a way to figure out the names of the missing arguments, and tell the user? Something like:
try:
#begin blackbox
def f(x,y):
return x*y
f(x=1)
#end blackbox
except Exception as e:
#figure out the missing keyword argument is called "y" and tell the user so
Assuming that the code between begin blackbox and end blackbox is unknown to the exception handler.
Edit: As its been pointed out to me below, Python 3 already has this functionality built in. Let me extend the question then, is there a (probably ugly and hacky) way to do this in Python 2.x?

A much cleaner way to do this would be to wrap the function in another function, pass through the *args, **kwargs, and then use those values when you need them, instead of trying to reconstruct them after the fact. But if you don't want to do that…
In Python 3.x (except very early versions), this is easy, as poke's answer explains. Even easier with 3.3+, with inspect.signature, inspect.getargvalues, and inspect.Signature.bind_partial and friends.
In Python 2.x, there is no way to do this. The exception only has the string 'f() takes exactly 2 arguments (1 given)' in its args.
Except… in CPython 2.x specifically, it's possible with enough ugly and brittle hackery.
You've got a traceback, so you've got its tb_frame and tb_lineno… which is everything you need. So as long as the source is available, the inspect module makes it easy to get the actual function call expression. Then you just need to parse it (via ast) to get the arguments passed, and compare to the function's signature (which, unfortunately, isn't nearly as easy to get in 2.x as in 3.3+, but between f.func_defaults, f.func_code.co_argcount, etc., you can reconstruct it).
But what if the source isn't available? Well, between tb_frame.f_code and tb_lasti, you can find out where the function call was in the bytecode. And the dis module makes that relatively easy to parse. In particular, right before the call, the positional arguments and the name-value pairs for keyword arguments were all pushed on the stack, so you can easily see which names got pushed, and how many positional values, and reconstruct the function call that way. Which you compare to the signature in the same way.
Of course that relies on the some assumptions about how CPython's compiler builds bytecode. It would be perfectly legal to do things in all kinds of different orders as long as the stack ended up with the right values. So, it's pretty brittle. But I think there are already better reasons not to do it.

I would argue that doing this doesn’t really make that much sense. Such an exception is thrown because the programmer missed specifying the argument. So if you knowingly catch the exception, then you could just as well just fix it in the first place.
That being said, in current Python 3 versions, the TypeError that is being thrown does mention which arguments are missing from the call:
"f() missing 1 required positional argument: 'y'"
Unfortunately, the argument name is not mentioned separately, so you would have to extract it from the string:
try:
f(x=1)
except TypeError as e:
if 'required positional argument' in e.args[0]:
argumentNames = e.args[0].split("'")[1::2]
print('Missing arguments are ' + argumentNames)
else:
raise # Re-raise other TypeErrors
As Joran Beasley pointed out in the comments, Python 2 does not tell you which arguments are missing but just how many are missing. So there is no way to tell from the exception which arguments were missing in the call.

except TypeError as e:
import inspect
got_args = int(re.search("\d+.*(\d+)",str(e)).groups()[0])
print "missing args:",inspect.getargspec(f).args[got_args:]
a better method would be a decorator
def arg_decorator(fn):
def func(*args,**kwargs):
try:
return fn(*args,**kwargs)
except TypeError:
arg_spec = inspect.getargspec(fn)
missing_named = [a for a in arg_spec.args if a not in kwargs]
if arg_spec.defaults:
missing_args = missing_named[len(args): -len(arg_spec.defaults) ]
else:
missing_args = missing_named[len(args):]
print "Missing:",missing_args
return func
#arg_decorator
def fn1(x,y,z):
pass
def fn2(x,y):
pass
arged_fn2 = arg_decorator(fn2)
fn1(5,y=2)
arged_fn2(x=1)

With purely the exception to deal with it is not possible to do what you want and handle keyword arguments. This is of course wrt Python 2.7.
The code that generates this message in Python is:
PyErr_Format(PyExc_TypeError,
"%.200s() takes %s %d "
"argument%s (%d given)",
PyString_AsString(co->co_name),
defcount ? "at most" : "exactly",
co->co_argcount,
co->co_argcount == 1 ? "" : "s",
argcount + kwcount);
Taken from lines 3056-3063 from http://hg.python.org/cpython/file/0e5df5b62488/Python/ceval.c
As you can see, there is just not enough information given to the exception as to what arguments are missing. co in this context is the PyCodeObject being called. The only thing given is a string (which you could parse if you like) with the function name, whether or not there is a vararg, how many arguments are expected, and how many arguments were given. As has been pointed out, this does not give you sufficient information as to what argument(s) were not given (in the case of keyword arguments).
Something like inspect or the other debugging modules might be able to give you enough information as to what function was called and how it was called, from which you could figure out what arguments were not given.
I should also mention however that almost certainly, whatever solution you come up with will not be able to handle at least some extension module methods (those written in C) because they don't provide argument information as part of their object.