I want to prevent mistakes when instanciating a complex class with lots of rules to instanciate it correctly.
For example I came up with the following complex class:
import math
sentinel = object()
class Foo( object ):
def __init__( self, a, c, d, g, b=sentinel, e=sentinel, f=sentinel, h=sentinel,
i=sentinel ):
# sentinel parameters are only needed in case other parameters have some value, and in
# some cases should be None, __init__ contains only simple logic.
...
def create_foo( a, c, d, g ):
# contains the difficult logic to create a Foo-instance correctly, e.g:
b = ( int( math.pi * 10**a ) / float(10**a) )
if c == "don't care":
e = None
f = None
elif c == 'single':
e = 3
f = None
else:
e = 6
f = 10
if g == "need I say more":
h = "ni"
i = "and now for something completely different"
elif g == "Brian":
h = "Always look at the bright side of life"
i = None
else:
h = None
i = "Always look at the bright side of death"
return Foo( a=a, b=b, c=c, d=d, e=e, f=f, g=g, h=h, i=i )
Since create_foo contains the logic to correctly create a Foo-instance I want to 'encourage'* users to use it.
What is the best pythonic way to do this.
*Yes I know I can't force people to use the factory-function, hence I want to 'encourage' them. ;-)
TL/DR
The key to solving this is marking the origin of the object in the factory and in the constructor (__new__), and checking the origin in the initializer (__init__). Do a warnings.warn() if the origin was the __new__ method by itself. You mark the origin in the factory by calling __new__ and __init__ separately in the factory and marking the factory origin in between.
Preamble
Since create_foo is the preferred method, it would likely be better to use that as the default. Implement the complex logic in the constructor, and refer to the lighter weight factory in your documentation as an alternative in those cases when the user doesn't want the complex logic. This avoids the problem of having to inform the user they are "doing it wrong".
However, assuming it is best to keep the complex logic out of the default constructor, your problem is twofold:
You need to implement - and detect - multiple methods of creation.
This first part is simple; lots of objects in the standard library have alternate constructors, so this is not unusual at all in the Python world. You could use a factory method as you have created already, but at the end I suggest reasons for including the factory in your class (this is what #classmethod was created for).
Detecting the origin is a little harder, but not too difficult. You simply mark the origin at instantiation and check the origin at initialization. Do this by calling __new__ and __init__ separately.
You want to inform the user what they should be doing, but still allow them to do what they want to do.
This is exactly what the warnings module was created for.
Use warnings
You can issue messages to the user- and still allow them to be in control of the messages- using the warnings module. Notify the user that they might want to use the factory by doing:
import warnings
warnings.warn("It is suggested to use the Foo factory.", UserWarning)
This way, the user can filter the warnings out if they wish.
A warnings warning
Quick sidebar: note that once the above warnings.warn message has been executed, then by default it will not come up again until you execute warnings.resetwarnings() or the Python session is restarted.
In other words, unless you change the settings the user will only see the message the first time they make a Foo. This may or may not be what you want.
Add origin attributes and check origin at initialization
Utilizing warnings.warn requires tracking the method of origin for your foos, and calling the warning if the factory isn't used. You could do it relatively simply as follows.
First, add a __new__ method and do an origin check at the end of the __init__ method:
class Foo( object ):
def __new__(cls, *args, **kwargs):
inst = super(Foo, cls).__new__(cls)
# mark the instance origin:
inst.origin == cls.__new__
return inst
def __init__( self, a, c, d, g, b=sentinel, e=sentinel, f=sentinel,
h=sentinel, i=sentinel ):
#### simple logic ####
if self.origin == type(self).__new__:
warnings.warn("It is suggested to use the {} factory for \
instance creation.".format(type(self).__name__), UserWarning)
Then in the factory, instantiate and initialize the new object separately, and setting the origin between the two:
def create_foo( a, c, d, g ):
#### complex logic #####
# instantiate using the constructor directly:
f = Foo.__new__(Foo, a, b, c, d, e, f, g, h, i )
# replace the origin with create_foo:
f.origin = create_foo
# NOW initialize manually:
f.__init__(a, b, c, d, e, f, g, h, i )
return f
Now you can detect where any Foo is coming from, and a warning will be issued to the user (once by default) if they did not use the factory function:
>>> Foo()
>>> __main__:8: UserWarning: It is suggested to use the Foo factory for instance creation.
Allow for Foo subclassing
One other suggested tweak: I would consider adding your factory function into the class as an alternate constructor, and then (in your user warning) suggesting the user use that constructor (rather than the factory function itself) for instantiation. This would allow Foo's child classes to utilize the factory method and still receive an instance of the child class from the factory, rather than a Foo.
Allowing for classes other than Foo to utilize the factory requires some minor changes to the factory:
def create_foo( foo_cls, a, c, d, g ):
'''Contains Foo creation logic.'''
#### complex logic #####
f = foo_cls.__new__(foo_cls, a, b, c, d, e, f, g, h, i )
# replace the marked origin with create_foo
f.origin = create_foo
# now initialize
f.__init__(a, b, c, d, e, f, g, h, i )
return f
Now we'll add the alternate constructor:
class Foo( object ):
#### other methods ####
def create(cls, *args, **kwargs):
inst = create_foo(cls, *args, **kwargs)
return inst
Now we can:
f = Foo.create( a, c, d, g )
But we can also:
class Bar(Foo):
pass
b = Bar.create( a, c, d, g )
And we still see this:
>>> Bar()
>>> __main__:8: UserWarning: It is suggested to use the Bar factory for instance creation.
You can strongly restrict the ordering of operations/properties.
Meaning preventing user from accessing some of class Foo method before calling create_foo method! The same idea is used in Serializer Module
Sample code:-
class Foo(object):
def __init__(self, a, b = sentinel):
self.a = a
self._create_foo_called = False
def create_foo(self):
self.b = "complex value"
self._create_foo_called = True
def do_something(self):
if(self._create_foo_called):
# WHATEVER
else:
raise AssertionError("Please call create_foo() method")
Related
I have a situation where I extend a class with several attributes:
class SuperClass:
def __init__(self, tediously, many, attributes):
# assign the attributes like "self.attr = attr"
class SubClass:
def __init__(self, id, **kwargs):
self.id = id
super().__init__(**kwargs)
And then I want to create instances, but I understand that this leads to a situation where a subclass can only be instantiated like this:
super_instance = SuperClass(tediously, many, attributes)
sub_instance = SubClass(id, tediously=super_instance.tediously, many=super_instance.many, attributes=super_instance.attributes)
My question is if anything prettier / cleaner can be done to instantiate a subclass by copying a superclass instance's attributes, without having to write a piece of sausage code to manually do it (either in the constructor call, or a constructor function's body)... Something like:
utopic_sub_instance = SubClass(id, **super_instance)
Maybe you want some concrete ideas of how to not write so much code?
So one way to do it would be like this:
class A:
def __init___(self, a, b, c):
self.a = a
self.b = b
self.c = c
class B:
def __init__(self, x, a, b, c):
self.x = x
super().__init__(a, b, c)
a = A(1, 2, 3)
b = B('x', 1, 2, 3)
# so your problem is that you want to avoid passing 1,2,3 manually, right?
# So as a comment suggests, you should use alternative constructors here.
# Alternative constructors are good because people not very familiar with
# Python could also understand them.
# Alternatively, you could use this syntax, but it is a little dangerous and prone to producing
# bugs in the future that are hard to spot
class BDangerous:
def __init__(self, x, a, b, c):
self.x = x
kwargs = dict(locals())
kwargs.pop('x')
kwargs.pop('self')
# This is dangerous because if in the future someone adds a variable in this
# scope, you need to remember to pop that also
# Also, if in the future, the super constructor acquires the same parameter that
# someone else adds as a variable here... maybe you will end up passing an argument
# unwillingly. That might cause a bug
# kwargs.pop(...pop all variable names you don't want to pass)
super().__init__(**kwargs)
class BSafe:
def __init__(self, x, a, b, c):
self.x = x
bad_kwargs = dict(locals())
# This is safer: you are explicit about which arguments you're passing
good_kwargs = {}
for name in 'a,b,c'.split(','):
good_kwargs[name] = bad_kwargs[name]
# but really, this solution is not that much better compared to simply passing all
# parameters explicitly
super().__init__(**good_kwargs)
Alternatively, let's go a little crazier. We'll use introspection to dynamically build the dict to pass as arguments. I have not included in my example the case where there are keyword-only arguments, defaults, *args or **kwargs
class A:
def __init__(self, a,b,c):
self.a = a
self.b = b
self.c = c
class B(A):
def __init__(self, x,y,z, super_instance):
import inspect
spec = inspect.getfullargspec(A.__init__)
positional_args = []
super_vars = vars(super_instance)
for arg_name in spec.args[1:]: # to exclude 'self'
positional_args.append(super_vars[arg_name])
# ...but of course, you must have the guarantee that constructor
# arguments will be set as instance attributes with the same names
super().__init__(*positional_args)
I managed to finally do it using a combination of an alt constructor and the __dict__ property of the super_instance.
class SuperClass:
def __init__(self, tediously, many, attributes):
self.tediously = tediously
self.many = many
self.attributes = attributes
class SubClass(SuperClass):
def __init__(self, additional_attribute, tediously, many, attributes):
self.additional_attribute = additional_attribute
super().__init__(tediously, many, attributes)
#classmethod
def from_super_instance(cls, additional_attribute, super_instance):
return cls(additional_attribute=additional_attribute, **super_instance.__dict__)
super_instance = SuperClass("tediously", "many", "attributes")
sub_instance = SubClass.from_super_instance("additional_attribute", super_instance)
NOTE: Bear in mind that python executes statements sequentially, so if you want to override the value of an inherited attribute, put super().__init__() before the other assignment statements in SubClass.__init__.
NOTE 2: pydantic has this very nice feature where their BaseModel class auto generates an .__init__() method, helps with attribute type validation and offers a .dict() method for such models (it's basically the same as .__dict__ though).
Kinda ran into the same question and just figured one could simply do:
class SubClass(SuperClass):
def __init__(self, additional_attribute, **args):
self.additional_attribute = additional_attribute
super().__init__(**args)
super_class = SuperClass("tediously", "many", "attributes")
sub_instance = SuperClass("additional_attribute", **super_class.__dict__)
I have several classes that are coupled to each other and I would like to have a design that minimizes redundancy in them.
Right now the design is as follows. The classes are
A
C, D
AC, AD: compositions of (A, C) and (A, D)
where all classes are initialized with the same data, each class performed some calculations and saves its own result. More specifically, users would create instances of A, AC, AD but not C and D. In other words, to the user, A, AC, AD are similar things and have uniform interface, whereas C and D are hidden. Here the logic is that, for example, the result of C is not useful unless we exclude the result of A from it. Some mock-up code is as follows
class A(object):
def __init__(self, data):
# some work
def postproc(self, *args):
# some real work
self.result = ..
class AC(object):
def __init__(self, data):
self.a = A(data)
self.c = C(data)
def postproc(self, *args):
result_a = self.a.postproc(*args[:3])
result_c = self.c.postproc(*args[3:])
self.result = exclude_c_from_a(result_a, result_c)
With the current design, if the user create instances of A and AC, (or AC and AD, or other combinations), the calculation of A is done multiple times. I would like to have no redundant calculations, how should the design be changed?
A little more details on the usage: there will be multiple instances of A with different initial data too. Thus I cannot use singleton to guarantee there is only one A instance.
I finally used memoization as the solution.
def memoize(func):
"""
Decorator for memoization. Note the func arguments need to be hashable.
#type func: a callable
"""
memo = func.memo = {}
#wraps(func)
def wrapper(*args):
if args not in memo:
memo[args] = func(*args)
return memo[args]
return wrapper
And it can decorate the class definition
#memoize
class A(object):
def __init__(self, arg1, arg2):
...
such that for the same input arguments, only one instance is created.
how should the design be changed?
Just pass A and C to the constructor of AC so that you don't have to instantiate A and C in the constructor.
Don't pass data to the constructor of A. Instead, pass a result to the constructor of A. You should prevent to do any work in the constructor other than setting up the dependency.
Create a class that is only responsible for calculating the result.
Imagine the following code (which is totally useless if taken alone):
# define a property with additional methods
class P(property):
def __init__(self, name):
property.__init__(self,
fget=lambda self: self._get(name),
fset=lambda self, v: self._set(name, v))
self._name = name
def some_fn(self):
print('name: ' + self._name)
# define a class with two 'enhanced' properties
class C:
p1 = P('p1')
p2 = P('p2')
def __init__(self):
self._values = {}
def _get(self, name):
return self._values[name]
def _set(self, name, v):
self._values[name] = v
c = C()
c.p1 = 5
c.p2 = c.p1
print(c.p1, c.p2)
I just create a class C with two properties which have an extra method some_fn().
The problem is now: you can't call some_fn() easily by just writing c.p1.some_fn() because you would evaluate c.p1 first, which results in some value which doesn't provide the method any more.
I've tried to find some workarounds / approaches for calling some_fn in the context of a certain property, not it's value but I'm not happy yet.
My goal is quite simple:
I want to be able read/assign properties without boilerplate:
c.p1 = c.p2 instead of c.p1.set(c.p2.get())
The way I call the extra method/function must be easy to read/write
I want to write code that can be statically verified by pylint, mypy etc. so some_fn('c.p1') is not an option because it can't be checked whether 'c.p1' is a valid attribute of an existing objectc`.
some_fn doesn't have to be a method. It can be a function or any other way to request functionality in context of a property
I don't even need real properties. Any other way to write s.th.
like c.p1 == c.p2 (e.g. using __getattr__/__setattr__) would be fine, too as long as the get/set operations are still trackable.
I collected some code to make clear, what I'm talking about:
# ==== What I want to do ==============
c.p1.some_fn() # <-- this is what I want to write but
# it's invalid since it evaluates to
# 5.some_fn()
some_fn(c.p1) # <-- something like this looks OK, too but
# it evalueates to some_fn(5) (useless)
# ==== These are options that came to mind but I'm not happy with ======
getattr(C, 'p1').some_fn() # <-- this works but it is ugly
some_fn("c.p1") # <-- this is possible, too but I can't
# check integrity statically (pylint/mypy)
c.p1.value = c.p2.value # <-- this is a valid approach but it
c.p1.some_fn() # increases
some_fn(c.p1) # (again) # <-- This can acutally work if you `inspect`
# the call stack inside `C._get()` but
# it's black magic and incredibly slow
with some_fn(): # <-- this can work when `some_fn` changes
c.p1 # some global state which get's evaluated
# inside `C._get()`
My goal is quite simple: I want to be able read/assign properties without boilerplate: c.p1 = c.p2
If that is the goal here, it sounds like you've misunderstood properties, because they already work like that.
class C(object):
#property
def p1(self):
# get value
#p1.setter
def p1(self, val):
# set value
#property
def p2(self):
# get value
#p2.setter
def p2(self, val):
# set value
Then if you have an object c = C(), you can do c.p1 = c.p2, and it'll just work. Sticking more methods onto a property object is the wrong way to go.
If you really want to stick methods onto properties, retrieve the property through the class:
C.p1.some_fn()
My code contains some objects which are used via Pythons "with" statement to ensure that they get savely closed.
Now i want to create a class where the methods can interact with these objects.
For example my code actually looks like this:
with ... as a, with ... as b:
# do something with a and b here
call_method(a, b) # pass a and b here
I'd like to put it into a class, so it Looks "like" this:
class Something(object):
def __init__(self):
with ... as a:
self.a = a
with ... as b:
self.b = b
def do_something(self):
# do something with self.a and self.b
self.call_method(self.a, self.b)
def call_method(self, a, b)
# do more with a, b
The objects need to stay "opened" all the time.
I don't know how to achieve this, so how can i do this?
You don't have a 'context' in your class to manage, don't use with in __init__. You'll have to close the files in some other manner.
You can always use try:, finally: if you want the file objects to be closed if there is an exception within the method:
def call_method(self, a, b)
try:
# do more with a, b
finally:
self.a.close()
self.b.close()
but it depends heavily on what you wanted to do with the files if you really wanted them to be closed at that point.
If your instances themselves should be used in a specific context (e.g. there is a block of code than starts and ends during which your instance should have the file open), then you can make the class a context manager by implementing the context manager special methods.
You alone as designer of the class API will know how long the files need to stay open for. It depends heavily on how the instance is used when it is time to close the file.
You could make your class itself a context manager:
class Something(object):
def __init__(self):
self.a = a
self.b = b
def __enter__(self):
self.a_e = a.__enter__(self)
self.b_e = b.__enter__(self)
def __exit__(self, *x):
xb = False
try:
xb = self.b_e(*x)
finally:
xa = self.a_e(*x)
return xa or xb # make sure both are called.
def do_something(self):
# do something with self.a and self.b
# - or, if present, with a_e, b_e
self.call_method(self.a, self.b)
def call_method(self, a, b)
# do more with a, b
This is just the raw idea. In order to make it work properly, you must do even more with try: except: finally:.
You can use it then with
with Something(x, y) as sth:
sth.do_something()
and it gets properly __enter__()ed and __exit__()ed.
class A:
def x ( self ):
print( self.__class__ )
class B ( A ):
pass
b = B()
b.x()
In the above situation, is there any way for the method x to get a reference to class A instead of B? Of course simply writing print( A ) is not allowed, as I want to add some functionality with a decorator that needs the class A (and as I can't pass A itself to the decorator directly, as the class doesn't exist yet at that point).
Unless you have super-calls involved (or more generally subclasses that override x and call directly to A.x(self) as part of their override's implementation), looking for the first item in type(self).mro() that has an x attribute would work --
next(c for c in type(self).mro() if hasattr(c, 'x'))
If you do need to cover for super-calls, it's harder -- unless you happen to know that no super-class of A defines an x attribute (and of course it's easier to know about your superclasses than your subclasses, though multiple inheritance does complicate it;-) in which case you only need the last appropriate item of the mro instead of the first one.
You can use name mangling to associate the class a method is defined in with a method:
def set_defining_class(cls):
setattr(cls, '_%s__defining_class' % (cls.__name__,), cls)
return cls
#set_defining_class
class A(object):
def x (self):
print(self.__defining_class)
#set_defining_class
class B ( A ):
pass
b = B()
b.x()
Done here with a class decorator, but you could do it by hand, or in a metaclass too.