Test code such that no function calls kwarg as an arg - python

In python, it is possible to check whether a function makes function calls where named arguments are called as positional arguments?
For example:
def a(pos_arg, nam_arg=None, nam_arg2=None):
return "whatever"
def b():
return a(1, 2, nam_arg2="whatever")
Here, the function b calls the function a, but the second argument is a named argument, which is being called as a positional argument. This might cause confusing problems when object inheritance comes into play.
Better would have been:
def b():
return a(1, nam_arg=2, nam_arg2="whatever")
For testing purposes (as this is generally bad code in my project), I would like to find out whether the function b() makes such calls.
Is this possible in python?

You tagged your question as python-2.7 but for the sake of future viewers I would say that in python 3 we now have keyword-only arguments feature which allows to define a function like that:
def func(pos_arg, *, kw_only_arg):
pass
In that case kw_only_arg is allowed to be used only as keyword argument.
If you can't use python 3 then there is a recipe that can help you with that:
http://code.activestate.com/recipes/578993-keyword-only-arguments-in-python-2x/

Related

Python imitate class's constructor only using function dynamically

This is a little bit weird. I want to dynamic initialize part of function's parameters before I call it. But I don't want to use class for some reason. Let's say I have a function:
def inner(a,b,c):
"a,b,c do something"
return result
Before I formally call it, I'd like to initialize it somewhere:
partInitalAbc=inner(,b,c)
Then I'll use it as:
result=partInitialAbc(a)
Please notice I want to do this dynamically. Just as when you initial a class using the constructor, so a decorator may be not appropriate at this time...
Any one have some idea?
It sounds like you're looking for functools.partial:
Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords.
If you pass the arguments to partial as positional arguments, they'll appear at the beginning of the argument list, which isn't exactly what you want. Keyword arguments give you more control over which parameters are passed:
partInitialAbc = functools.partial(inner, b=b_value, c=c_value);

How to assure applying order of function decorators in Python?

Some decorators should only be used in the outermost layer.
A decorator that augments the original function and add a configure parameter is one example.
from functools import wraps
def special_case(f):
#wraps(f)
def _(a, b, config_x=False):
if config_x:
print "Special case here"
return
return f(a, b)
How can I avoid decorators like this getting decorated by another decorator?
EDIT
It is really disgusting to let everyone trying to apply a new decorator worry about the application order.
So, is it possible to avoid this kind of situation? Is it possible to add a config option without introducing a new parameter?
There isn't any way to stop it from being decorated. You just have to document that it needs to apply last and tell people not to use it inside another decorator.
Edit responding to your edit: In Python 3 you can give your function a keyword-only argument. This drastically reduces the impact that the change will have on existing uses of the function. Unfortunately this only works in Python 3.
Ultimately, applying a decorator to a function just means passing the decorated function as an argument to another function. There's no way for a function (or any object) to even know that it's being passed as an argument, let alone what it's being passed to. The reason you can't know about later decorators is the same reason that in an ordinary function call like f(g(x)), the function g can't know that it will later be called by f.
This is one reason writing decorators is tricky. Code that relies on heavy use of decorators that pass explicit arguments to their wrapped functions (as yours passes a and b) is inherently going to be fragile. Fortunately, a lot of the time you can write a decorator that uses *args and **kwargs so it can pass all the arguments it doesn't use along to the decorated function.
If someone takes the code you provide, and writes another decorator that explicitly accepts only a and b as arguments, and then calls the decorated function as f(a, b, True), it's their own fault if it fails. They should have known that other decorators they used might have changed the function signature.
Normally, when one write a decorator to be used generically, one does not estrict the number or name of arguments for the function it is wrapping.
Most decorators out there accept a list o positional arguments, and amapping of keyword arguments as parameters for their wrapper, and pass those, as received, to the decorated function:
def deco(func):
def wrapper(*args, **kwargs):
... decorator stuff here ...
return func(*args, **kwargs)
Threfore, if the decorator is to receive a parameter that it should "consume" - like the config_x you mention, all you have to do is to document it, have it as a keyword parameter, and pick it from kwargs. To avoid name clashes on parameters, one can, for example, prefix this parameter name with the decorator's own name or other distinct name:
def deco(func):
def wrapper(*args, **kwargs):
if "deco_config_x" in kwargs):
config_x = kwargs.pop(deco_config_x)
... decorator stuff here ...
return func(*args, **kwargs)
This way, the decorator may be put anywhere on a "decorator stack" - it will pick the parameter(s) addressed to it, and those bellow it won't get any stranger parameter. Theonly requirement is that your functions and decorators as a whole juts let keyword parametrs they don't know about to pass through.

What is the "**params" syntax in a Python method definition?

So I am trying out the new python code for the google app engine search library and I came across a weird syntax. This was:
cls_createDocument(**params)
where params was a dictionary.
The function this refers to is:
#classmethod
def _createDocument(
cls, pid=None, category=None, name=None, description=None,
category_name=None, price=None, **params)
My questions is, what does the **params signify and what does that do to the object?
Thanks!
Jon
Consider a function with default arguments:
def func(foo=3):
print(foo)
The structure of the arguments is (in principle) very similar to a dictionary. The function foo has (essentially) a dictionary of default arguments (in this case {'foo':3}). Now, lets say that you don't want to use the keyword in the function call, but you want to use a dictionary instead -- then you can call foo as:
d = {"foo":8}
func(**d)
This allows you to dynamically change what arguments you are passing to the function func.
This become a little more interesting if you try the following:
d = {"foo":8, "bar":12}
func(**d)
This doesn't work (it is equivalent to foo(foo=8, bar=12), but since bar isn't a valid argument, it fails).
You can get around that problem by giving those extra arguments a place to go inside the definition of foo.
def func( foo=3, **kwargs ):
print(foo,kwargs)
Now, try:
d = {"foo":8, "bar":12}
func(**d) #prints (8, {'bar':12})
All the extra keyword arguments go into the kwargs dictionary inside the function.
This can also be called as:
func(foo=8, bar=12)
with the same result.
This is often useful if funcA calls funcB and you want funcA to accept all of the keywords of funcB (plus a few extra) which is a very common thing when dealing with classes and inheritance:
def funcA(newkey=None,**kwargs):
funcB(**kwargs)
Finally, here is a link to the documentation
The **params parameter represents all the keyword arguments passed to the function as a dictionary.

what constitutes a member of a class? What constitutes a method?

It seems that an acceptable answer to the question
What is a method?
is
A method is a function that's a member of a class.
I disagree with this.
class Foo(object):
pass
def func():
pass
Foo.func = func
f = Foo()
print "fine so far"
try:
f.func()
except TypeError:
print "whoops! func must not be a method after all"
Is func a member of Foo?
Is func a method of Foo?
I am well aware that this would work if func had a self argument. That's obvious. I'm interested in if it's a member of foo and in if it's a method as presented.
You're just testing it wrong:
>>> class Foo(object): pass
...
>>> def func(self): pass
...
>>> Foo.func = func
>>> f = Foo()
>>> f.func()
>>>
You error of forgetting to have in the def the self argument has absolutely nothing to do with f.func "not being a method", of course. The peculiar conceit of having the def outside the class instead of inside it (perfectly legal Python of course, but, as I say, peculiar) has nothing to do with the case either: if you forget to have a first argument (conventionally named self) in your def statements for methods, you'll get errors in calling them, of course (a TypeError is what you get, among other cases, whenever the actual arguments specified in the call can't match the formal arguments accepted by the def, of course).
The type error wouldn't be thrown if func had a self argument, like any other instance method.
That's because when you evaluate f.func, you're actually binding f to the first argument of the function -- it then becomes a partial application which you can provide further arguments to.
If you want it to be a static method, then you need the staticmethod decorator which just throws away the first parameter and passes the rest into the original function.
So 2 ways of making it work:
def func(self): pass
-- or --
Foo.func = staticmethod(func)
depending on what you're aiming for.
As written, func is a member of Foo and a method of Foo instances such as your f. However, it can't be successfully called because it doesn't accept at least one argument.
func is in f's namespace and has a _call_() method. In other words, it is enough of a method that Python tries to call it like one when you invoke it like one. If it quacks like a duck, in other words...
That this call doesn't succeed is neither here nor there, in my opinion.
But perhaps a proper response to "But Doctor! When I don't accept any arguments in a function defined in a class, I get confused about whether it's really a method or not!" is simply "Don't do that." :-)

"self" inside plain function?

I've got a bunch of functions (outside of any class) where I've set attributes on them, like funcname.fields = 'xxx'. I was hoping I could then access these variables from inside the function with self.fields, but of course it tells me:
global name 'self' is not defined
So... what can I do? Is there some magic variable I can access? Like __this__.fields?
A few people have asked "why?". You will probably disagree with my reasoning, but I have a set of functions that all must share the same signature (accept only one argument). For the most part, this one argument is enough to do the required computation. However, in a few limited cases, some additional information is needed. Rather than forcing every function to accept a long list of mostly unused variables, I've decided to just set them on the function so that they can easily be ignored.
Although, it occurs to me now that you could just use **kwargs as the last argument if you don't care about the additional args. Oh well...
Edit: Actually, some of the functions I didn't write, and would rather not modify to accept the extra args. By "passing in" the additional args as attributes, my code can work both with my custom functions that take advantage of the extra args, and with third party code that don't require the extra args.
Thanks for the speedy answers :)
self isn't a keyword in python, its just a normal variable name. When creating instance methods, you can name the first parameter whatever you want, self is just a convention.
You should almost always prefer passing arguments to functions over setting properties for input, but if you must, you can do so using the actual functions name to access variables within it:
def a:
if a.foo:
#blah
a.foo = false
a()
see python function attributes - uses and abuses for when this comes in handy. :D
def foo():
print(foo.fields)
foo.fields=[1,2,3]
foo()
# [1, 2, 3]
There is nothing wrong with adding attributes to functions. Many memoizers use this to cache results in the function itself.
For example, notice the use of func.cache:
from decorator import decorator
#decorator
def memoize(func, *args, **kw):
# Author: Michele Simoniato
# Source: http://pypi.python.org/pypi/decorator
if not hasattr(func, 'cache'):
func.cache = {}
if kw: # frozenset is used to ensure hashability
key = args, frozenset(kw.iteritems())
else:
key = args
cache = func.cache # attribute added by memoize
if key in cache:
return cache[key]
else:
cache[key] = result = func(*args, **kw)
return result
You can't do that "function accessing its own attributes" correctly for all situations - see for details here how can python function access its own attributes? - but here is a quick demonstration:
>>> def f(): return f.x
...
>>> f.x = 7
>>> f()
7
>>> g = f
>>> g()
7
>>> del f
>>> g()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "<interactive input>", line 1, in f
NameError: global name 'f' is not defined
Basically most methods directly or indirectly rely on accessing the function object through lookup by name in globals; and if original function name is deleted, this stops working. There are other kludgey ways of accomplishing this, like defining class, or factory - but thanks to your explanation it is clear you don't really need that.
Just do the mentioned keyword catch-all argument, like so:
def fn1(oneArg):
// do the due
def fn2(oneArg, **kw):
if 'option1' in kw:
print 'called with option1=', kw['option1']
//do the rest
fn2(42)
fn2(42, option1='something')
Not sure what you mean in your comment of handling TypeError - that won't arise when using **kw. This approach works very well for some python system functions - check min(), max(), sort(). Recently sorted(dct,key=dct.get,reverse=True) came very handy to me in CodeGolf challenge :)
Example:
>>> def x(): pass
>>> x
<function x at 0x100451050>
>>> x.hello = "World"
>>> x.hello
"World"
You can set attributes on functions, as these are just plain objects, but I actually never saw something like this in real code.
Plus. self is not a keyword, just another variable name, which happens to be the particular instance of the class. self is passed implicitly, but received explicitly.
if you want globally set parameters for a callable 'thing' you could always create a class and implement the __call__ method?
There is no special way, within a function's body, to refer to the function object whose code is executing. Simplest is just to use funcname.field (with funcname being the function's name within the namespace it's in, which you indicate is the case -- it would be harder otherwise).
This isn't something you should do. I can't think of any way to do what you're asking except some walking around on the call stack and some weird introspection -- which isn't something that should happen in production code.
That said, I think this actually does what you asked:
import inspect
_code_to_func = dict()
def enable_function_self(f):
_code_to_func[f.func_code] = f
return f
def get_function_self():
f = inspect.currentframe()
code_obj = f.f_back.f_code
return _code_to_func[code_obj]
#enable_function_self
def foo():
me = get_function_self()
print me
foo()
While I agree with the the rest that this is probably not good design, the question did intrigue me. Here's my first solution, which I may update once I get decorators working. As it stands, it relies pretty heavily on being able to read the stack, which may not be possible in all implementations (something about sys._getframe() not necessarily being present...)
import sys, inspect
def cute():
this = sys.modules[__name__].__dict__.get(inspect.stack()[0][3])
print "My face is..." + this.face
cute.face = "very cute"
cute()
What do you think? :3
You could use the following (hideously ugly) code:
class Generic_Object(object):
pass
def foo(a1, a2, self=Generic_Object()):
self.args=(a1,a2)
print "len(self.args):", len(self.args)
return None
... as you can see it would allow you to use "self" as you described. You can't use an "object()" directly because you can't "monkey patch(*)" values into an object() instance. However, normal subclasses of object (such as the Generic_Object() I've shown here) can be "monkey patched"
If you wanted to always call your function with a reference to some object as the first argument that would be possible. You could put the defaulted argument first, followed by a *args and optional **kwargs parameters (through which any other arguments or dictionaries of options could be passed during calls to this function).
This is, as I said hideously ugly. Please don't ever publish any code like this or share it with anyone in the Python community. I'm only showing it here as a sort of strange educational exercise.
An instance method is like a function in Python. However, it exists within the namespace of a class (thus it must be accessed via an instance ... myobject.foo() for example) and it is called with a reference to "self" (analagous to the "this" pointer in C++) as the first argument. Also there's a method resolution process which causes the interpreter to search the namespace of the instance, then it's class, and then each of the parent classes and so on ... up through the inheritance tree.
An unbound function is called with whatever arguments you pass to it. There can't bee any sort of automatically pre-pended object/instance reference to the argument list. Thus, writing a function with an initial argument named "self" is meaningless. (It's legal because Python doesn't place any special meaning on the name "self." But meaningless because callers to your function would have to manually supply some sort of object reference to the argument list and it's not at all clear what that should be. Just some bizarre "Generic_Object" which then floats around in the global variable space?).
I hope that clarifies things a bit. It sounds like you're suffering from some very fundamental misconceptions about how Python and other object-oriented systems work.
("Monkey patching" is a term used to describe the direct manipulation of an objects attributes -- or "instance variables" by code that is not part of the class hierarchy of which the object is an instance).
As another alternative, you can make the functions into bound class methods like so:
class _FooImpl(object):
a = "Hello "
#classmethod
def foo(cls, param):
return cls.a + param
foo = _FooImpl.foo
# later...
print foo("World") # yes, Hello World
# and if you have to change an attribute:
foo.im_self.a = "Goodbye "
If you want functions to share attribute namespaecs, you just make them part of the same class. If not, give each its own class.
What exactly are you hoping "self" would point to, if the function is defined outside of any class? If your function needs some global information to execute properly, you need to send this information to the function in the form of an argument.
If you want your function to be context aware, you need to declare it within the scope of an object.

Categories