I'm trying to use PyContracts within a web application, so I have lots of custom-defined classes being passed around that I simply want to type check alongside other more traditional argument types. I'd like to use contractual programming (PyContracts) to accomplish this, for the sake of cleanliness and forced documentation.
When I reference a locally visible class by name, PyContracts doesn't seem to be aware of the type. For example:
from contracts import contract
class SomeClass:
pass
#contract
def f(a):
"""
:param a: Just a parameter
:type a: SomeClass
"""
print(a)
my_a = SomeClass()
f(my_a)
Raises the following error:
ContractSyntaxError: Unknown identifier 'SomeClass'. Did you mean 'np_complex64'? (at char 0), (line:1, col:1)
I know I can use new_contract to custom-define names and bind them to classes, but that's a lot of hassle to do for every type. I want to use the docstring syntax for PyContracts if at all possible, and I definitely need to use the string-defined contract format since I'm using boolean type logic ("None|str|SomeClass"). How do I accomplish this with local types and minimal intrusion into the rest of my codebase?
I hacked together a magic decorator that adds types before creating the contract. For anyone that's interested, it seems to work, but it's probably pretty slow if you define a large number of functions:
def magic_contract(*args, **kwargs):
# Check if we got called without arguments, just the function
func = None
if len(args) == 1 and len(kwargs) == 0 and callable(args[0]):
func = args[0]
args = tuple()
def inner_decorator(f):
for name, val in f.__globals__.items():
if isinstance(val, type):
new_contract(name, val)
return contract(*args, **kwargs)(f)
if func:
return inner_decorator(func)
return inner_decorator
And some test runs:
In [3]: class SomeClass:
...: pass
...:
In [4]: #magic_contract
...: def f(a):
...: """
...:
...: :param a: Some parameter
...: :type a: None|SomeClass
...: """
...: print(a)
...:
In [5]: f(None)
None
In [6]: f(SomeClass())
<__main__.SomeClass object at 0x7f1fa17c8b70>
In [7]: f(2)
...
ContractNotRespected: Breach for argument 'a' to f().
...
In [8]: #magic_contract(b='int|SomeClass')
...: def g(b):
...: print(type(b))
...:
In [9]: g(2)
<class 'int'>
In [10]: g(SomeClass())
<class '__main__.SomeClass'>
In [11]: g(None)
...
ContractNotRespected: Breach for argument 'b' to g().
...
This is a question about scope and closures in Python, motivated by an exercise in SICP. Much thanks for your time if you read this!
A question (3.2) in SICP asks one to create a procedure "make-monitored", that takes in a function f (of one parameter) as input and returns a procedure that keeps track of how many times f has been called. (If the input to this new procedure is "num-calls" it returns the number of times f has been called, if it is "reset" it resets counter to 0 and anything else, it applies f to the input and returns the result (after appropriately incrementing the counter).
Here is code in Scheme that I wrote that works:
(define (make-monitored f)
(let ((counter 0))
(define (number-calls) counter)
(define (reset-count)
(set! counter 0))
(define (call-f input)
(begin (set! counter (+ 1 counter))
(f input)))
(define (dispatch message)
(cond ((eq? message 'num-calls) (number-calls))
((eq? message 'reset) (reset-count))
(else (call-f message))))
dispatch))
My question however is about how to write this in a "pythonic" way. My attempt below is obviously a direct translation of my Scheme code and I realize that though it is fine for an impure functional language (like Scheme) it's probably not the cleanest or best way to do it in Python. How does one solve a general problem like this in Python where you want a higher order procedure to dispatch on type and remember local state?
Below is my noobish attempt that works (earlier I had said it did not but the problem was that an earlier version of the program was still in the terminal's memory) (In 2 it seems hard to make nonlocal variable binding)
def make_monitored(func):
counter = 0
def dispatch(message):
if message == "num-calls":
return num_calls()
elif message == "reset":
reset()
else:
nonlocal counter
counter += 1
return func(message)
def num_calls():
nonlocal counter
return counter
def reset():
nonlocal counter
counter = 0
return dispatch
PS: This question is related to this same set of exercises in SICP but my question is really about Python best practice and not the concept of closures or Scheme...
I think writing a decorator wrapping the function in a class would be more pythonic:
from functools import wraps
def make_monitored(func):
class wrapper:
def __init__(self, f):
self.func = f
self.counter = 0
def __call__(self, *args, **kwargs):
self.counter += 1
return self.func(*args, **kwargs)
return wraps(func)(wrapper(func))
This has the advantage that it mimics the original function as close as possible, and just adds a counter field to it:
In [25]: msqrt = make_monitored(math.sqrt)
In [26]: msqrt(2)
Out[26]: 1.4142135623730951
In [29]: msqrt.counter
Out[29]: 1
In [30]: msqrt(235)
Out[30]: 15.329709716755891
In [31]: msqrt.counter
Out[31]: 2
In [32]: #make_monitored
...: def f(a):
...: """Adding the answer"""
...: return a + 42
In [33]: f(0)
Out[33]: 42
In [34]: f(1)
Out[34]: 43
In [35]: f.counter
Out[35]: 2
In [36]: f.__name__
Out[36]: 'f'
In [37]: f.__doc__
Out[37]: 'Adding the answer'
For f, you also see the usage as a decorator, and how the wrapper keeps the original name and docstring (which would not be the case without functools.wraps).
Defining reset is left as an exercise to the reader, but quite trivial.
I've noticed that a common pattern I use is to assign SomeClass.__init__() arguments to self attributes of the same name. Example:
class SomeClass():
def __init__(self, a, b, c):
self.a = a
self.b = b
self.c = c
In fact it must be a common task for others as well as PyDev has a shortcut for this - if you place the cursor on the parameter list and click Ctrl+1 you're given the option to Assign parameters to attributes which will create that boilerplate code for you.
Is there a different, short and elegant way to perform this assignment?
You could do this, which has the virtue of simplicity:
>>> class C(object):
def __init__(self, **kwargs):
self.__dict__ = dict(kwargs)
This leaves it up to whatever code creates an instance of C to decide what the instance's attributes will be after construction, e.g.:
>>> c = C(a='a', b='b', c='c')
>>> c.a, c.b, c.c
('a', 'b', 'c')
If you want all C objects to have a, b, and c attributes, this approach won't be useful.
(BTW, this pattern comes from Guido his own bad self, as a general solution to the problem of defining enums in Python. Create a class like the above called Enum, and then you can write code like Colors = Enum(Red=0, Green=1, Blue=2), and henceforth use Colors.Red, Colors.Green, and Colors.Blue.)
It's a worthwhile exercise to figure out what kinds of problems you could have if you set self.__dict__ to kwargs instead of dict(kwargs).
I sympathize with your sense that boilerplate code is a bad thing. But in this case, I'm not sure there even could be a better alternative. Let's consider the possibilities.
If you're talking about just a few variables, then a series of self.x = x lines is easy to read. In fact, I think its explicitness makes that approach preferable from a readability standpoint. And while it might be a slight pain to type, that alone isn't quite enough to justify a new language construct that might obscure what's really going on. Certainly using vars(self).update() shenanigans would be more confusing than it's worth in this case.
On the other hand, if you're passing nine, ten, or more parameters to __init__, you probably need to refactor anyway. So this concern really only applies to cases that involve passing, say, 5-8 parameters. Now I can see how eight lines of self.x = x would be annoying both to type and to read; but I'm not sure that the 5-8 parameter case is common enough or troublesome enough to justify using a different method. So I think that, while the concern you're raising is a good one in principle, in practice, there are other limiting issues that make it irrelevant.
To make this point more concrete, let's consider a function that takes an object, a dict, and a list of names, and assigns values from the dict to names from the list. This ensures that you're still being explicit about which variables are being assigned to self. (I would never suggest a solution to this problem that didn't call for an explicit enumeration of the variables to be assigned; that would be a rare-earth bug magnet):
>>> def assign_attributes(obj, localdict, names):
... for name in names:
... setattr(obj, name, localdict[name])
...
>>> class SomeClass():
... def __init__(self, a, b, c):
... assign_attributes(self, vars(), ['a', 'b', 'c'])
Now, while not horribly unattractive, this is still harder to figure out than a straightforward series of self.x = x lines. And it's also longer and more trouble to type than one, two, and maybe even three or four lines, depending on circumstances. So you only get certain payoff starting with the five-parameter case. But that's also the exact moment that you begin to approach the limit on human short-term memory capacity (= 7 +/- 2 "chunks"). So in this case, your code is already a bit challenging to read, and this would only make it more challenging.
Mod for #pcperini's answer:
>>> class SomeClass():
def __init__(self, a, b=1, c=2):
for name,value in vars().items():
if name != 'self':
setattr(self,name,value)
>>> s = SomeClass(7,8)
>>> print s.a,s.b,s.c
7 8 2
Your specific case could also be handled with a namedtuple:
>>> from collections import namedtuple
>>> SomeClass = namedtuple("SomeClass", "a b c")
>>> sc = SomeClass(1, "x", 200)
>>> print sc
SomeClass(a=1, b='x', c=200)
>>> print sc.a, sc.b, sc.c
1 x 200
Decorator magic!!
>>> class SomeClass():
#ArgsToSelf
def __init__(a, b=1, c=2, d=4, e=5):
pass
>>> s=SomeClass(6,b=7,d=8)
>>> print s.a,s.b,s.c,s.d,s.e
6 7 2 8 5
while defining:
>>> import inspect
>>> def ArgsToSelf(f):
def act(self, *args, **kwargs):
arg_names,_,_,defaults = inspect.getargspec(f)
defaults=list(defaults)
for arg in args:
setattr(self, arg_names.pop(0),arg)
for arg_name,arg in kwargs.iteritems():
setattr(self, arg_name,arg)
defaults.pop(arg_names.index(arg_name))
arg_names.remove(arg_name)
for arg_name,arg in zip(arg_names,defaults):
setattr(self, arg_name,arg)
return f(*args, **kwargs)
return act
Of course you could define this decorator once and use it throughout your project.Also, This decorator works on any object function, not only __init__.
You can do it via setattr(), like:
[setattr(self, key, value) for key, value in kwargs.items()]
Is not very beautiful, but can save some space :)
So, you'll get:
kwargs = { 'd':1, 'e': 2, 'z': 3, }
class P():
def __init__(self, **kwargs):
[setattr(self, key, value) for key, value in kwargs.items()]
x = P(**kwargs)
dir(x)
['__doc__', '__init__', '__module__', 'd', 'e', 'z']
For that simple use-case I must say I like putting things explicitly (using the Ctrl+1 from PyDev), but sometimes I also end up using a bunch implementation, but with a class where the accepted attributes are created from attributes pre-declared in the class, so that I know what's expected (and I like it more than a namedtuple as I find it more readable -- and it won't confuse static code analysis or code-completion).
I've put on a recipe for it at: http://code.activestate.com/recipes/577999-bunch-class-created-from-attributes-in-class/
The basic idea is that you declare your class as a subclass of Bunch and it'll create those attributes in the instance (either from default or from values passed in the constructor):
class Point(Bunch):
x = 0
y = 0
p0 = Point()
assert p0.x == 0
assert p0.y == 0
p1 = Point(x=10, y=20)
assert p1.x == 10
assert p1.y == 20
Also, Alex Martelli also provided a bunch implementation: http://code.activestate.com/recipes/52308-the-simple-but-handy-collector-of-a-bunch-of-named/ with the idea of updating the instance from the arguments, but that'll confuse static code-analysis (and IMO can make things harder to follow) so, I'd only use that approach for an instance that's created locally and thrown away inside that same scope without passing it anywhere else).
I solved it for myself using locals() and __dict__:
>>> class Test:
... def __init__(self, a, b, c):
... l = locals()
... for key in l:
... self.__dict__[key] = l[key]
...
>>> t = Test(1, 2, 3)
>>> t.a
1
>>>
Disclaimer
Do not use this: I was simply trying to create the answer closest to OPs initial intentions. As pointed out in comments, this relies on entirely undefined behavior, and explicitly prohibited modifications of the symbol table.
It does work though, and has been tested under extremely basic circumstances.
Solution
class SomeClass():
def __init__(self, a, b, c):
vars(self).update(dict((k,v) for k,v in vars().iteritems() if (k != 'self')))
sc = SomeClass(1, 2, 3)
# sc.a == 1
# sc.b == 2
# sc.c == 3
Using the vars() built-in function, this snippet iterates through all of the variables available in the __init__ method (which should, at this point, just be self, a, b, and c) and set's self's variables equal to the same, obviously ignoring the argument-reference to self (because self.self seemed like a poor decision.)
One of the problems with #user3638162's answer is that locals() contain the 'self' variable. Hence, you end up with an extra self.self. If one doesn't mind the extra self, that solution can simply be
class X:
def __init__(self, a, b, c):
self.__dict__.update(locals())
x = X(1, 2, 3)
print(x.a, x.__dict__)
The self can be removed after construction by del self.__dict__['self']
Alternatively, one can remove the self during construction using dictionary comprehensions introduced in Python3
class X:
def __init__(self, a, b, c):
self.__dict__.update(l for l in locals().items() if l[0] != 'self')
x = X(1, 2, 3)
print(x.a, x.__dict__)
I have a method within which I need to pass an ever-increasing integer to another function.
I can do this like so:
def foo(i):
print i
def bar():
class Incrementer(object):
def __init__(self, start=0):
self.i = start
def __get__(self):
j = self.i
self.i += 1
return j
number = Incrementer()
foo(number)
foo(number)
foo(number)
which correctly outputs 0 1 2 ... but I feel like I'm overlooking a much easier (or built-in) way of doing this?
Try itertools.count() -- it does exactly what you need:
>>> c = itertools.count()
>>> next(c)
0
>>> next(c)
1
>>> next(c)
2
In general, if you need to retain state between one call to a function and the next, what you want is either an object (your solution) or a generator. In some cases one will be simpler than the other, but there's nothing wrong with how you've done it, in principle (though you seem to have some issues with the implementation).
Sven's suggestion, itertools.count(), is a generator. Its implementation is something like this:
def count():
i = 0
while True:
yield i
i += 1
Now, if you wanted it to be callable like a function, rather than having to do next(c), you could define a wrapper that made it so:
def count(c=itertools.count()):
return next(c)
Or the inevitable one-line lambda:
count = lambda c=itertools.count(): next(c)
Then count() returns the next integer each time you call it.
Of course, if you want to be able to create any number of callable functions, each with their own counter, you can write a factory for that:
def counter():
return lambda c=itertools.count(): next(c)
Then it's:
c = counter()
print c() # 0
print c() # 1
# etc
This still seems simpler to me than an object, but not by much. If your state or logic were any more complex, the encapsulation of the object might win out.
I don't think so, but I thought I'd ask just in case. For example, for use in a class that encapsulates an int:
i = IntContainer(3)
i + 5
And I'm not just interested in this int example, I was looking for something clean and general, not overriding every int and string method.
Thanks, sunqiang. That's just what I wanted. I didn't realize you could subclass these immutable types (coming from C++).
class IntContainer(int):
def __init__(self,i):
#do stuff here
self.f = 4
def MultiplyBy4(self):
#some member function
self *= self.f
return self
print 3+IntContainer(3).MultiplyBy4()
This should do what you need:
class IntContainer(object):
def __init__(self, x):
self.x = x
def __add__(self, other):
# do some type checking on other
return self.x + other
def __radd__(self, other):
# do some type checking on other
return self.x + other
Output:
In [6]: IntContainer(3) + 6
Out[6]: 9
In [7]: 6 + IntContainer(3)
Out[7]: 9
For more information search for "radd" in the following docs:
http://docs.python.org/reference/datamodel.html#special-method-names
You'll find other such methods for "right addition", "right subtraction", etc.
Here's another link covering the same operators:
http://www.siafoo.net/article/57#reversed-binary-operations
By the way, Python does have casting operators:
http://www.siafoo.net/article/57#casts
But, they won't accomplish what you need in your example (basically because methods don't have any type annotation for parameters, so there's no good way to cast implicitly). So you can do this:
class IntContainer2(object):
def __init__(self, x):
self.x = x
def __int__(self):
return self.x
ic = IntContainer2(3)
print int(ic) + 6
print 6 + int(ic)
But this will fail:
print ic + 6 # error: no implicit coercion
You won't get conversion operators like in C++ because Python does not have this kind of strong static type system. The only automatic conversion operators are those which handle default numeric values (int/float); they are predefined in the language and cannot be changed.
Type "conversion" is usually done by constructors/factories. You can then overload standard methods like __add__ to make it work more like other classes.
sometimes maybe just subclass from int directly is enough. then __add__ and __radd__ need not costuming.
class IntContainer(int):
pass
i = IntContainer(3)
print i + 5 # 8
print 4 + i # 7
class StrContainer(str):
pass
s = StrContainer(3)
print s + '5' # 35
print '4' + s # 43
Is this what you need?
In [1]: class IntContainer(object):
...: def __init__(self, val):
...: self.val = val
...: def __add__(self, val):
...: return self.val + val
...: def __radd__(self, val):
...: return self.val + val
...:
...:
In [2]: i = IntContainer(3)
In [3]: i + 5
Out[3]: 8
In [4]:
Sorry for coming to the party 8.5 years late.
You can derive from an immutable (ie. int). You cannot define __init__ because the immutable is already created and can't be modified (by definition). This is where __new__ comes in handy.
class IntContainer(int):
def __new__ (cls, val):
ival = int.__new__(cls, val)
ival._rval = 'IntContainer(%d)' % ival
return ival
def __repr__ (self):
return self._rval
In [1]: i = IntContainer(3)
In [2]: i
Out[2]: IntContainer(3)
In [3]: repr(i)
Out[3]: 'IntContainer(3)'
In [4]: str(i)
Out[4]: '3'
In [5]: i + 5
Out[5]: 8
In [6]: 4 + i
Out[6]: 7
In [7]: int(i)
Out[7]: 3
In [8]: float(i)
Out[8]: 3.0
Now, to answer your question about conversion operators. You can also define __int__, __long__, __float__, and obviously, __str__. To convert or cast to an arbitrary object, you will most likely need to modify the other object to get what you want. You can use the __new__ method of that other object. Or if the other object is already created, try using __call__.