What is the point of setting an attribute of class equal to output of a method of the same class? - python

This question should pertain to all OOP languages, though I'm only familiar with C++ and Python. I've seen this in several codebases and have seen this in Python and I think also in C++. I will illustrate in Python:
class sim:
def __init__(self, name="sim_today"):
self.name = name
self.properties = self.compute_properties()
def compute_properties(self):
# <insert logic to compute properties>
return properties
I don't understand this type of design. Why not just set properties directly within compute_properties like:
class sim:
def __init__(self, name="sim_today"):
self.name = name
self.compute_properties()
def compute_properties(self):
self.properties = <insert logic to compute properties>

Separation of concerns suggests that a method should generally return a value or mutate the object but not both. The constructor is the obvious place to initialize the object’s attributes, and this design makes it clear that what is conditional is what value is assigned and not anything more general like selecting an attribute or not assigning anything.
Of course, when the value can be computed from other attributes, that makes the one assigned a cache; recall that cache invalidation is one of the two hard things in computer science (along with naming things and off-by-one errors), so this would be justified only if the value were expensive to compute.

Related

advantage and disadvantage of #property in a Python Object [duplicate]

This question already has answers here:
What's the pythonic way to use getters and setters?
(8 answers)
Closed 4 months ago.
What advantages does the #property notation hold over the classic getter+setter? In which specific cases/situations should a programmer choose to use one over the other?
With properties:
class MyClass(object):
#property
def my_attr(self):
return self._my_attr
#my_attr.setter
def my_attr(self, value):
self._my_attr = value
Without properties:
class MyClass(object):
def get_my_attr(self):
return self._my_attr
def set_my_attr(self, value):
self._my_attr = value
Prefer properties. It's what they're there for.
The reason is that all attributes are public in Python. Starting names with an underscore or two is just a warning that the given attribute is an implementation detail that may not stay the same in future versions of the code. It doesn't prevent you from actually getting or setting that attribute. Therefore, standard attribute access is the normal, Pythonic way of, well, accessing attributes.
The advantage of properties is that they are syntactically identical to attribute access, so you can change from one to another without any changes to client code. You could even have one version of a class that uses properties (say, for code-by-contract or debugging) and one that doesn't for production, without changing the code that uses it. At the same time, you don't have to write getters and setters for everything just in case you might need to better control access later.
In Python you don't use getters or setters or properties just for the fun of it. You first just use attributes and then later, only if needed, eventually migrate to a property without having to change the code using your classes.
There is indeed a lot of code with extension .py that uses getters and setters and inheritance and pointless classes everywhere where e.g. a simple tuple would do, but it's code from people writing in C++ or Java using Python.
That's not Python code.
Using properties lets you begin with normal attribute accesses and then back them up with getters and setters afterwards as necessary.
The short answer is: properties wins hands down. Always.
There is sometimes a need for getters and setters, but even then, I would "hide" them to the outside world. There are plenty of ways to do this in Python (getattr, setattr, __getattribute__, etc..., but a very concise and clean one is:
def set_email(self, value):
if '#' not in value:
raise Exception("This doesn't look like an email address.")
self._email = value
def get_email(self):
return self._email
email = property(get_email, set_email)
Here's a brief article that introduces the topic of getters and setters in Python.
[TL;DR? You can skip to the end for a code example.]
I actually prefer to use a different idiom, which is a little involved for using as a one off, but is nice if you have a more complex use case.
A bit of background first.
Properties are useful in that they allow us to handle both setting and getting values in a programmatic way but still allow attributes to be accessed as attributes. We can turn 'gets' into 'computations' (essentially) and we can turn 'sets' into 'events'. So let's say we have the following class, which I've coded with Java-like getters and setters.
class Example(object):
def __init__(self, x=None, y=None):
self.x = x
self.y = y
def getX(self):
return self.x or self.defaultX()
def getY(self):
return self.y or self.defaultY()
def setX(self, x):
self.x = x
def setY(self, y):
self.y = y
def defaultX(self):
return someDefaultComputationForX()
def defaultY(self):
return someDefaultComputationForY()
You may be wondering why I didn't call defaultX and defaultY in the object's __init__ method. The reason is that for our case I want to assume that the someDefaultComputation methods return values that vary over time, say a timestamp, and whenever x (or y) is not set (where, for the purpose of this example, "not set" means "set to None") I want the value of x's (or y's) default computation.
So this is lame for a number of reasons describe above. I'll rewrite it using properties:
class Example(object):
def __init__(self, x=None, y=None):
self._x = x
self._y = y
#property
def x(self):
return self.x or self.defaultX()
#x.setter
def x(self, value):
self._x = value
#property
def y(self):
return self.y or self.defaultY()
#y.setter
def y(self, value):
self._y = value
# default{XY} as before.
What have we gained? We've gained the ability to refer to these attributes as attributes even though, behind the scenes, we end up running methods.
Of course the real power of properties is that we generally want these methods to do something in addition to just getting and setting values (otherwise there is no point in using properties). I did this in my getter example. We are basically running a function body to pick up a default whenever the value isn't set. This is a very common pattern.
But what are we losing, and what can't we do?
The main annoyance, in my view, is that if you define a getter (as we do here) you also have to define a setter.[1] That's extra noise that clutters the code.
Another annoyance is that we still have to initialize the x and y values in __init__. (Well, of course we could add them using setattr() but that is more extra code.)
Third, unlike in the Java-like example, getters cannot accept other parameters. Now I can hear you saying already, well, if it's taking parameters it's not a getter! In an official sense, that is true. But in a practical sense there is no reason we shouldn't be able to parameterize an named attribute -- like x -- and set its value for some specific parameters.
It'd be nice if we could do something like:
e.x[a,b,c] = 10
e.x[d,e,f] = 20
for example. The closest we can get is to override the assignment to imply some special semantics:
e.x = [a,b,c,10]
e.x = [d,e,f,30]
and of course ensure that our setter knows how to extract the first three values as a key to a dictionary and set its value to a number or something.
But even if we did that we still couldn't support it with properties because there is no way to get the value because we can't pass parameters at all to the getter. So we've had to return everything, introducing an asymmetry.
The Java-style getter/setter does let us handle this, but we're back to needing getter/setters.
In my mind what we really want is something that capture the following requirements:
Users define just one method for a given attribute and can indicate there
whether the attribute is read-only or read-write. Properties fail this test
if the attribute writable.
There is no need for the user to define an extra variable underlying the function, so we don't need the __init__ or setattr in the code. The variable just exists by the fact we've created this new-style attribute.
Any default code for the attribute executes in the method body itself.
We can set the attribute as an attribute and reference it as an attribute.
We can parameterize the attribute.
In terms of code, we want a way to write:
def x(self, *args):
return defaultX()
and be able to then do:
print e.x -> The default at time T0
e.x = 1
print e.x -> 1
e.x = None
print e.x -> The default at time T1
and so forth.
We also want a way to do this for the special case of a parameterizable attribute, but still allow the default assign case to work. You'll see how I tackled this below.
Now to the point (yay! the point!). The solution I came up for for this is as follows.
We create a new object to replace the notion of a property. The object is intended to store the value of a variable set to it, but also maintains a handle on code that knows how to calculate a default. Its job is to store the set value or to run the method if that value is not set.
Let's call it an UberProperty.
class UberProperty(object):
def __init__(self, method):
self.method = method
self.value = None
self.isSet = False
def setValue(self, value):
self.value = value
self.isSet = True
def clearValue(self):
self.value = None
self.isSet = False
I assume method here is a class method, value is the value of the UberProperty, and I have added isSet because None may be a real value and this allows us a clean way to declare there really is "no value". Another way is a sentinel of some sort.
This basically gives us an object that can do what we want, but how do we actually put it on our class? Well, properties use decorators; why can't we? Let's see how it might look (from here on I'm going to stick to using just a single 'attribute', x).
class Example(object):
#uberProperty
def x(self):
return defaultX()
This doesn't actually work yet, of course. We have to implement uberProperty and
make sure it handles both gets and sets.
Let's start with gets.
My first attempt was to simply create a new UberProperty object and return it:
def uberProperty(f):
return UberProperty(f)
I quickly discovered, of course, that this doens't work: Python never binds the callable to the object and I need the object in order to call the function. Even creating the decorator in the class doesn't work, as although now we have the class, we still don't have an object to work with.
So we're going to need to be able to do more here. We do know that a method need only be represented the one time, so let's go ahead and keep our decorator, but modify UberProperty to only store the method reference:
class UberProperty(object):
def __init__(self, method):
self.method = method
It is also not callable, so at the moment nothing is working.
How do we complete the picture? Well, what do we end up with when we create the example class using our new decorator:
class Example(object):
#uberProperty
def x(self):
return defaultX()
print Example.x <__main__.UberProperty object at 0x10e1fb8d0>
print Example().x <__main__.UberProperty object at 0x10e1fb8d0>
in both cases we get back the UberProperty which of course is not a callable, so this isn't of much use.
What we need is some way to dynamically bind the UberProperty instance created by the decorator after the class has been created to an object of the class before that object has been returned to that user for use. Um, yeah, that's an __init__ call, dude.
Let's write up what we want our find result to be first. We're binding an UberProperty to an instance, so an obvious thing to return would be a BoundUberProperty. This is where we'll actually maintain state for the x attribute.
class BoundUberProperty(object):
def __init__(self, obj, uberProperty):
self.obj = obj
self.uberProperty = uberProperty
self.isSet = False
def setValue(self, value):
self.value = value
self.isSet = True
def getValue(self):
return self.value if self.isSet else self.uberProperty.method(self.obj)
def clearValue(self):
del self.value
self.isSet = False
Now we the representation; how do get these on to an object? There are a few approaches, but the easiest one to explain just uses the __init__ method to do that mapping. By the time __init__ is called our decorators have run, so just need to look through the object's __dict__ and update any attributes where the value of the attribute is of type UberProperty.
Now, uber-properties are cool and we'll probably want to use them a lot, so it makes sense to just create a base class that does this for all subclasses. I think you know what the base class is going to be called.
class UberObject(object):
def __init__(self):
for k in dir(self):
v = getattr(self, k)
if isinstance(v, UberProperty):
v = BoundUberProperty(self, v)
setattr(self, k, v)
We add this, change our example to inherit from UberObject, and ...
e = Example()
print e.x -> <__main__.BoundUberProperty object at 0x104604c90>
After modifying x to be:
#uberProperty
def x(self):
return *datetime.datetime.now()*
We can run a simple test:
print e.x.getValue()
print e.x.getValue()
e.x.setValue(datetime.date(2013, 5, 31))
print e.x.getValue()
e.x.clearValue()
print e.x.getValue()
And we get the output we wanted:
2013-05-31 00:05:13.985813
2013-05-31 00:05:13.986290
2013-05-31
2013-05-31 00:05:13.986310
(Gee, I'm working late.)
Note that I have used getValue, setValue, and clearValue here. This is because I haven't yet linked in the means to have these automatically returned.
But I think this is a good place to stop for now, because I'm getting tired. You can also see that the core functionality we wanted is in place; the rest is window dressing. Important usability window dressing, but that can wait until I have a change to update the post.
I'll finish up the example in the next posting by addressing these things:
We need to make sure UberObject's __init__ is always called by subclasses.
So we either force it be called somewhere or we prevent it from being implemented.
We'll see how to do this with a metaclass.
We need to make sure we handle the common case where someone 'aliases'
a function to something else, such as:
class Example(object):
#uberProperty
def x(self):
...
y = x
We need e.x to return e.x.getValue() by default.
What we'll actually see is this is one area where the model fails.
It turns out we'll always need to use a function call to get the value.
But we can make it look like a regular function call and avoid having to use e.x.getValue(). (Doing this one is obvious, if you haven't already fixed it out.)
We need to support setting e.x directly, as in e.x = <newvalue>. We can do this in the parent class too, but we'll need to update our __init__ code to handle it.
Finally, we'll add parameterized attributes. It should be pretty obvious how we'll do this, too.
Here's the code as it exists up to now:
import datetime
class UberObject(object):
def uberSetter(self, value):
print 'setting'
def uberGetter(self):
return self
def __init__(self):
for k in dir(self):
v = getattr(self, k)
if isinstance(v, UberProperty):
v = BoundUberProperty(self, v)
setattr(self, k, v)
class UberProperty(object):
def __init__(self, method):
self.method = method
class BoundUberProperty(object):
def __init__(self, obj, uberProperty):
self.obj = obj
self.uberProperty = uberProperty
self.isSet = False
def setValue(self, value):
self.value = value
self.isSet = True
def getValue(self):
return self.value if self.isSet else self.uberProperty.method(self.obj)
def clearValue(self):
del self.value
self.isSet = False
def uberProperty(f):
return UberProperty(f)
class Example(UberObject):
#uberProperty
def x(self):
return datetime.datetime.now()
[1] I may be behind on whether this is still the case.
I think both have their place. One issue with using #property is that it is hard to extend the behaviour of getters or setters in subclasses using standard class mechanisms. The problem is that the actual getter/setter functions are hidden in the property.
You can actually get hold of the functions, e.g. with
class C(object):
_p = 1
#property
def p(self):
return self._p
#p.setter
def p(self, val):
self._p = val
you can access the getter and setter functions as C.p.fget and C.p.fset, but you can't easily use the normal method inheritance (e.g. super) facilities to extend them. After some digging into the intricacies of super, you can indeed use super in this way:
# Using super():
class D(C):
# Cannot use super(D,D) here to define the property
# since D is not yet defined in this scope.
#property
def p(self):
return super(D,D).p.fget(self)
#p.setter
def p(self, val):
print 'Implement extra functionality here for D'
super(D,D).p.fset(self, val)
# Using a direct reference to C
class E(C):
p = C.p
#p.setter
def p(self, val):
print 'Implement extra functionality here for E'
C.p.fset(self, val)
Using super() is, however, quite clunky, since the property has to be redefined, and you have to use the slightly counter-intuitive super(cls,cls) mechanism to get an unbound copy of p.
Using properties is to me more intuitive and fits better into most code.
Comparing
o.x = 5
ox = o.x
vs.
o.setX(5)
ox = o.getX()
is to me quite obvious which is easier to read. Also properties allows for private variables much easier.
I feel like properties are about letting you get the overhead of writing getters and setters only when you actually need them.
Java Programming culture strongly advise to never give access to properties, and instead, go through getters and setters, and only those which are actually needed.
It's a bit verbose to always write these obvious pieces of code, and notice that 70% of the time they are never replaced by some non-trivial logic.
In Python, people actually care for that kind of overhead, so that you can embrace the following practice :
Do not use getters and setters at first, when if they not needed
Use #property to implement them without changing the syntax of the rest of your code.
I would prefer to use neither in most cases. The problem with properties is that they make the class less transparent. Especially, this is an issue if you were to raise an exception from a setter. For example, if you have an Account.email property:
class Account(object):
#property
def email(self):
return self._email
#email.setter
def email(self, value):
if '#' not in value:
raise ValueError('Invalid email address.')
self._email = value
then the user of the class does not expect that assigning a value to the property could cause an exception:
a = Account()
a.email = 'badaddress'
--> ValueError: Invalid email address.
As a result, the exception may go unhandled, and either propagate too high in the call chain to be handled properly, or result in a very unhelpful traceback being presented to the program user (which is sadly too common in the world of python and java).
I would also avoid using getters and setters:
because defining them for all properties in advance is very time consuming,
makes the amount of code unnecessarily longer, which makes understanding and maintaining the code more difficult,
if you were define them for properties only as needed, the interface of the class would change, hurting all users of the class
Instead of properties and getters/setters I prefer doing the complex logic in well defined places such as in a validation method:
class Account(object):
...
def validate(self):
if '#' not in self.email:
raise ValueError('Invalid email address.')
or a similiar Account.save method.
Note that I am not trying to say that there are no cases when properties are useful, only that you may be better off if you can make your classes simple and transparent enough that you don't need them.
I am surprised that nobody has mentioned that properties are bound methods of a descriptor class, Adam Donohue and NeilenMarais get at exactly this idea in their posts -- that getters and setters are functions and can be used to:
validate
alter data
duck type (coerce type to another type)
This presents a smart way to hide implementation details and code cruft like regular expression, type casts, try .. except blocks, assertions or computed values.
In general doing CRUD on an object may often be fairly mundane but consider the example of data that will be persisted to a relational database. ORM's can hide implementation details of particular SQL vernaculars in the methods bound to fget, fset, fdel defined in a property class that will manage the awful if .. elif .. else ladders that are so ugly in OO code -- exposing the simple and elegant self.variable = something and obviate the details for the developer using the ORM.
If one thinks of properties only as some dreary vestige of a Bondage and Discipline language (i.e. Java) they are missing the point of descriptors.
In complex projects I prefer using read-only properties (or getters) with explicit setter function:
class MyClass(object):
...
#property
def my_attr(self):
...
def set_my_attr(self, value):
...
In long living projects debugging and refactoring takes more time than writing the code itself. There are several downsides for using #property.setter that makes debugging even harder:
1) python allows creating new attributes for an existing object. This makes a following misprint very hard to track:
my_object.my_atttr = 4.
If your object is a complicated algorithm then you will spend quite some time trying to find out why it doesn't converge (notice an extra 't' in the line above)
2) setter sometimes might evolve to a complicated and slow method (e.g. hitting a database). It would be quite hard for another developer to figure out why the following function is very slow. He might spend a lot of time on profiling do_something() method, while my_object.my_attr = 4. is actually the cause of slowdown:
def slow_function(my_object):
my_object.my_attr = 4.
my_object.do_something()
Both #property and traditional getters and setters have their advantages. It depends on your use case.
Advantages of #property
You don't have to change the interface while changing the implementation of data access. When your project is small, you probably want to use direct attribute access to access a class member. For example, let's say you have an object foo of type Foo, which has a member num. Then you can simply get this member with num = foo.num. As your project grows, you may feel like there needs to be some checks or debugs on the simple attribute access. Then you can do that with a #property within the class. The data access interface remains the same so that there is no need to modify client code.
Cited from PEP-8:
For simple public data attributes, it is best to expose just the attribute name, without complicated accessor/mutator methods. Keep in mind that Python provides an easy path to future enhancement, should you find that a simple data attribute needs to grow functional behavior. In that case, use properties to hide functional implementation behind simple data attribute access syntax.
Using #property for data access in Python is regarded as Pythonic:
It can strengthen your self-identification as a Python (not Java) programmer.
It can help your job interview if your interviewer thinks Java-style getters and setters are anti-patterns.
Advantages of traditional getters and setters
Traditional getters and setters allow for more complicated data access than simple attribute access. For example, when you are setting a class member, sometimes you need a flag indicating where you would like to force this operation even if something doesn't look perfect. While it is not obvious how to augment a direct member access like foo.num = num, You can easily augment your traditional setter with an additional force parameter:
def Foo:
def set_num(self, num, force=False):
...
Traditional getters and setters make it explicit that a class member access is through a method. This means:
What you get as the result may not be the same as what is exactly stored within that class.
Even if the access looks like a simple attribute access, the performance can vary greatly from that.
Unless your class users expect a #property hiding behind every attribute access statement, making such things explicit can help minimize your class users surprises.
As mentioned by #NeilenMarais and in this post, extending traditional getters and setters in subclasses is easier than extending properties.
Traditional getters and setters have been widely used for a long time in different languages. If you have people from different backgrounds in your team, they look more familiar than #property. Also, as your project grows, if you may need to migrate from Python to another language that doesn't have #property, using traditional getters and setters would make the migration smoother.
Caveats
Neither #property nor traditional getters and setters makes the class member private, even if you use double underscore before its name:
class Foo:
def __init__(self):
self.__num = 0
#property
def num(self):
return self.__num
#num.setter
def num(self, num):
self.__num = num
def get_num(self):
return self.__num
def set_num(self, num):
self.__num = num
foo = Foo()
print(foo.num) # output: 0
print(foo.get_num()) # output: 0
print(foo._Foo__num) # output: 0
Here is an excerpts from "Effective Python: 90 Specific Ways to Write Better Python" (Amazing book. I highly recommend it).
Things to Remember
✦ Define new class interfaces using simple public attributes and avoid
defining setter and getter methods.
✦ Use #property to define special behavior when attributes are
accessed on your objects, if necessary.
✦ Follow the rule of least surprise and avoid odd side effects in your
#property methods.
✦ Ensure that #property methods are fast; for slow or complex
work—especially involving I/O or causing side effects—use normal
methods instead.
One advanced but common use of #property is transitioning what was
once a simple numerical attribute into an on-the-fly calculation. This
is extremely helpful because it lets you migrate all existing usage of
a class to have new behaviors without requiring any of the call sites
to be rewritten (which is especially important if there’s calling code
that you don’t control). #property also provides an important stopgap
for improving interfaces over time.
I especially like #property because it lets you make incremental
progress toward a better data model over time.
#property is a tool to
help you address problems you’ll come across in real-world code. Don’t
overuse it. When you find yourself repeatedly extending #property
methods, it’s probably time to refactor your class instead of further
paving over your code’s poor design.
✦ Use #property to give existing instance attributes
new functionality.
✦ Make incremental progress toward better data
models by using #property.
✦ Consider refactoring a class and all call
sites when you find yourself using #property too heavily.

Should all member variables be initialized in __init__

Maybe this is more of a style question than a technical one but I have a class with several member variables and I want to have it work so that some of the member variables are initialized when the user first creates an instance of the class (i.e. in the __init__ function) and I want the other member variables to be defined from arguments of member functions that will be called later on. So my question is should I initialize all member variables in the __init__ function (and set the ones that will be defined later on to dummy values) or initialize some in the __init__ function and some in later functions. I realize this might be difficult to understand so here are a couple of examples.
This example has var3 set to 0 initially in the __init__ function, then set to the desired value later on in the my_funct function.
class myClass(object):
def __init__(self,var1,var2):
self.var1=var1
self.var2=var2
self.var3=0
def my_funct(self,var3):
self.var3=var3
and in this example, var3 is not defined at all in the __init__ function
class myClass(object):
def __init__(self,var1,var2):
self.var1=var1
self.var2=var2
def my_funct(self,var3):
self.var3=var3
I don't think either way would make a big difference (maybe a slight difference in memory usage). But I was wondering if one of these is preferred over the other for some reason.
In object-oriented programming it's up to the developer to ensure an object is always in a consistent state after instantiation and after a method finishes. Other than that you're free to develop the class as you wish (keeping in mind certain principles with subclassing / overriding and so on).
A tool such as Pylint will warn when you're setting instance variables outside __init__. It can be argued that setting all instance variables in the __init__ is cleaner but it's not a rule that must be abided by at all times.
I would actually discourage initializing variables you don't always need in __init__ to an arbitrary default value.
I do question your use of OO if this is the case, but I'm sure there is a valid and understandable case where __init__ will not do everything, and the class will want to further modify itself by adding additional attributes with other methods.
The proper way in my opinion to test if a variable was set while running a method that may want to use it would be to use hasattr. This is in the case that this is a valid way to use the method and the test just switches behavior in a sensible way.
Another way would be to try and use it and handle the exception and provide some user friendly information about what the user of your class is doing wrong. This is in the case the method needs the attribute to be set before running.
i.e. Hey man, you did initialize the class, but you need to make sure the z attribute exists by calling the z_init method before running the z_run method.
Another, arguably the more pythonic way, would be to just document how to use the method in the docstring and then let the exception fly when it is used improperly. This is good enough for the first implementation of something and you can then focus on the next task. This is in the same situation as above, the method needs the attribute to be set.
The reason I do not like the idea of initializing variables to arbitrary defaults is this can be confusing (because it is arbitrary) and is line noise.
If the value is not arbitrary and simply a default value that can be changed you should be using a default value in the __init__ method that can be overridden. It can also actually be a valid initial state, which is also not arbitrary and you should set it in the __init__ method.
So the real answer is it depends, and you should probably avoid it and question your use of OO if you are doing this either by adding attributes in other methods or initializing attributes to arbitrary values.
While Simeon Visser is saying to keep your object in a consistent state, he has no basis for what consistency is based on your abstract example. While Pylint warns on this kind of thing, warnings from lint programs are simply so a high level reviewer can be alerted of things that usually indicate code smell. I say high level reviewer because a real reviewer should be reading and understanding all of your code, and thus not really need Pylint.
An example that breaks the rule of thumb:
class Mutant(object):
"""A mutant!"""
def __init__(self):
"""A mutant is born with only 1 eye and 1 mouth"""
self.eyes = 1
self.mouth = 1
self.location = 'Montana'
def roll_to(self, location):
"""If they have limbs, running is less dangerous"""
if hasattr(self, 'limbs'):
print 'Your mutant broke its limbs off!!'
del self.limbs
self.location = location
def run_to(self, location):
"""If they don't have limbs, running is not effective"""
if not hasattr(self, 'limbs'):
print 'Your mutant tries to run but he has no limbs.'
else:
self.location = location
def grow_limbs(self, number_of_limbs):
"""Ah, evolution!"""
assert number_of_limbs > 0, 'Cannot grow 0 or less limbs...'
if hasattr(self, 'limbs'):
self.limbs += number_of_limbs
else:
self.limbs = number_of_limbs
Here is an excerpt from sololearn.com (a free site to learn python)
"Properties provide a way of customizing access to instance attributes.
They are created by putting the property decorator above a method, which means when the instance attribute with the same name as the method is accessed, the method will be called instead.
One common use of a property is to make an attribute read-only."
Example (also from sololearn.com):
class Pizza:
def __init__(self, toppings):
self.toppings = toppings
#property
def pineapple_allowed(self):
return False
pizza = Pizza(["cheese", "tomato"])
print(pizza.pineapple_allowed)
pizza.pineapple_allowed = True
Result:
>>>
False
AttributeError: can't set attribute
>>>
If var3 depends on var1 and var2 you could do
class myClass:
def __init__(self,var1,var2):
self.var1=var1
self.var2=var2
#property
def var3(self):
return(self.var1+self.var2) #var3 depends on var1 and var2
m1=myClass(1,2)
print(m1.var3) # var3 is 3
var3 can also be set to whatever you want using a setter function. Note that you can avoid setting var3 to an arbitrary value by using None.
class myClass2(object):
def __init__(self,var1,var2):
self.var1=var1
self.var2=var2
self._var3=None # None or an initial value that makes sense
#property
def var3(self):
return(self._var3)
#var3.setter
def var3(self,value):
self._var3=value
m2=myClass(1,2)
print(m2.var3) # var3 is none
print(m2.var3(10)) # var3 is set to 10

How dangerous is setting self.__class__ to something else?

Say I have a class, which has a number of subclasses.
I can instantiate the class. I can then set its __class__ attribute to one of the subclasses. I have effectively changed the class type to the type of its subclass, on a live object. I can call methods on it which invoke the subclass's version of those methods.
So, how dangerous is doing this? It seems weird, but is it wrong to do such a thing? Despite the ability to change type at run-time, is this a feature of the language that should completely be avoided? Why or why not?
(Depending on responses, I'll post a more-specific question about what I would like to do, and if there are better alternatives).
Here's a list of things I can think of that make this dangerous, in rough order from worst to least bad:
It's likely to be confusing to someone reading or debugging your code.
You won't have gotten the right __init__ method, so you probably won't have all of the instance variables initialized properly (or even at all).
The differences between 2.x and 3.x are significant enough that it may be painful to port.
There are some edge cases with classmethods, hand-coded descriptors, hooks to the method resolution order, etc., and they're different between classic and new-style classes (and, again, between 2.x and 3.x).
If you use __slots__, all of the classes must have identical slots. (And if you have the compatible but different slots, it may appear to work at first but do horrible things…)
Special method definitions in new-style classes may not change. (In fact, this will work in practice with all current Python implementations, but it's not documented to work, so…)
If you use __new__, things will not work the way you naively expected.
If the classes have different metaclasses, things will get even more confusing.
Meanwhile, in many cases where you'd think this is necessary, there are better options:
Use a factory to create an instance of the appropriate class dynamically, instead of creating a base instance and then munging it into a derived one.
Use __new__ or other mechanisms to hook the construction.
Redesign things so you have a single class with some data-driven behavior, instead of abusing inheritance.
As a very most common specific case of the last one, just put all of the "variable methods" into classes whose instances are kept as a data member of the "parent", rather than into subclasses. Instead of changing self.__class__ = OtherSubclass, just do self.member = OtherSubclass(self). If you really need methods to magically change, automatic forwarding (e.g., via __getattr__) is a much more common and pythonic idiom than changing classes on the fly.
Assigning the __class__ attribute is useful if you have a long time running application and you need to replace an old version of some object by a newer version of the same class without loss of data, e.g. after some reload(mymodule) and without reload of unchanged modules. Other example is if you implement persistency - something similar to pickle.load.
All other usage is discouraged, especially if you can write the complete code before starting the application.
On arbitrary classes, this is extremely unlikely to work, and is very fragile even if it does. It's basically the same thing as pulling the underlying function objects out of the methods of one class, and calling them on objects which are not instances of the original class. Whether or not that will work depends on internal implementation details, and is a form of very tight coupling.
That said, changing the __class__ of objects amongst a set of classes that were particularly designed to be used this way could be perfectly fine. I've been aware that you can do this for a long time, but I've never yet found a use for this technique where a better solution didn't spring to mind at the same time. So if you think you have a use case, go for it. Just be clear in your comments/documentation what is going on. In particular it means that the implementation of all the classes involved have to respect all of their invariants/assumptions/etc, rather than being able to consider each class in isolation, so you'd want to make sure that anyone who works on any of the code involved is aware of this!
Well, not discounting the problems cautioned about at the start. But it can be useful in certain cases.
First of all, the reason I am looking this post up is because I did just this and __slots__ doesn't like it. (yes, my code is a valid use case for slots, this is pure memory optimization) and I was trying to get around a slots issue.
I first saw this in Alex Martelli's Python Cookbook (1st ed). In the 3rd ed, it's recipe 8.19 "Implementing Stateful Objects or State Machine Problems". A fairly knowledgeable source, Python-wise.
Suppose you have an ActiveEnemy object that has different behavior from an InactiveEnemy and you need to switch back and forth quickly between them. Maybe even a DeadEnemy.
If InactiveEnemy was a subclass or a sibling, you could switch class attributes. More exactly, the exact ancestry matters less than the methods and attributes being consistent to code calling it. Think Java interface or, as several people have mentioned, your classes need to be designed with this use in mind.
Now, you still have to manage state transition rules and all sorts of other things. And, yes, if your client code is not expecting this behavior and your instances switch behavior, things will hit the fan.
But I've used this quite successfully on Python 2.x and never had any unusual problems with it. Best done with a common parent and small behavioral differences on subclasses with the same method signatures.
No problems, until my __slots__ issue that's blocking it just now. But slots are a pain in the neck in general.
I would not do this to patch live code. I would also privilege using a factory method to create instances.
But to manage very specific conditions known in advance? Like a state machine that the clients are expected to understand thoroughly? Then it is pretty darn close to magic, with all the risk that comes with it. It's quite elegant.
Python 3 concerns? Test it to see if it works but the Cookbook uses Python 3 print(x) syntax in its example, FWIW.
The other answers have done a good job of discussing the question of why just changing __class__ is likely not an optimal decision.
Below is one example of a way to avoid changing __class__ after instance creation, using __new__. I'm not recommending it, just showing how it could be done, for the sake of completeness. However it is probably best to do this using a boring old factory rather than shoe-horning inheritance into a job for which it was not intended.
class ChildDispatcher:
_subclasses = dict()
def __new__(cls, *args, dispatch_arg, **kwargs):
# dispatch to a registered child class
subcls = cls.getsubcls(dispatch_arg)
return super(ChildDispatcher, subcls).__new__(subcls)
def __init_subclass__(subcls, **kwargs):
super(ChildDispatcher, subcls).__init_subclass__(**kwargs)
# add __new__ contructor to child class based on default first dispatch argument
def __new__(cls, *args, dispatch_arg = subcls.__qualname__, **kwargs):
return super(ChildDispatcher,cls).__new__(cls, *args, **kwargs)
subcls.__new__ = __new__
ChildDispatcher.register_subclass(subcls)
#classmethod
def getsubcls(cls, key):
name = cls.__qualname__
if cls is not ChildDispatcher:
raise AttributeError(f"type object {name!r} has no attribute 'getsubcls'")
try:
return ChildDispatcher._subclasses[key]
except KeyError:
raise KeyError(f"No child class key {key!r} in the "
f"{cls.__qualname__} subclasses registry")
#classmethod
def register_subclass(cls, subcls):
name = subcls.__qualname__
if cls is not ChildDispatcher:
raise AttributeError(f"type object {name!r} has no attribute "
f"'register_subclass'")
if name not in ChildDispatcher._subclasses:
ChildDispatcher._subclasses[name] = subcls
else:
raise KeyError(f"{name} subclass already exists")
class Child(ChildDispatcher): pass
c1 = ChildDispatcher(dispatch_arg = "Child")
assert isinstance(c1, Child)
c2 = Child()
assert isinstance(c2, Child)
How "dangerous" it is depends primarily on what the subclass would have done when initializing the object. It's entirely possible that it would not be properly initialized, having only run the base class's __init__(), and something would fail later because of, say, an uninitialized instance attribute.
Even without that, it seems like bad practice for most use cases. Easier to just instantiate the desired class in the first place.
Here's an example of one way you could do the same thing without changing __class__. Quoting #unutbu in the comments to the question:
Suppose you were modeling cellular automata. Suppose each cell could be in one of say 5 Stages. You could define 5 classes Stage1, Stage2, etc. Suppose each Stage class has multiple methods.
class Stage1(object):
…
class Stage2(object):
…
…
class Cell(object):
def __init__(self):
self.current_stage = Stage1()
def goToStage2(self):
self.current_stage = Stage2()
def __getattr__(self, attr):
return getattr(self.current_stage, attr)
If you allow changing __class__ you could instantly give a cell all the methods of a new stage (same names, but different behavior).
Same for changing current_stage, but this is a perfectly normal and pythonic thing to do, that won't confuse anyone.
Plus, it allows you to not change certain special methods you don't want changed, just by overriding them in Cell.
Plus, it works for data members, class methods, static methods, etc., in ways every intermediate Python programmer already understands.
If you refuse to change __class__, then you might have to include a stage attribute, and use a lot of if statements, or reassign a lot of attributes pointing to different stage's functions
Yes, I've used a stage attribute, but that's not a downside—it's the obvious visible way to keep track of what the current stage is, better for debugging and for readability.
And there's not a single if statement or any attribute reassignment except for the stage attribute.
And this is just one of multiple different ways of doing this without changing __class__.
In the comments I proposed modeling cellular automata as a possible use case for dynamic __class__s. Let's try to flesh out the idea a bit:
Using dynamic __class__:
class Stage(object):
def __init__(self, x, y):
self.x = x
self.y = y
class Stage1(Stage):
def step(self):
if ...:
self.__class__ = Stage2
class Stage2(Stage):
def step(self):
if ...:
self.__class__ = Stage3
cells = [Stage1(x,y) for x in range(rows) for y in range(cols)]
def step(cells):
for cell in cells:
cell.step()
yield cells
For lack of a better term, I'm going to call this
The traditional way: (mainly abarnert's code)
class Stage1(object):
def step(self, cell):
...
if ...:
cell.goToStage2()
class Stage2(object):
def step(self, cell):
...
if ...:
cell.goToStage3()
class Cell(object):
def __init__(self, x, y):
self.x = x
self.y = y
self.current_stage = Stage1()
def goToStage2(self):
self.current_stage = Stage2()
def __getattr__(self, attr):
return getattr(self.current_stage, attr)
cells = [Cell(x,y) for x in range(rows) for y in range(cols)]
def step(cells):
for cell in cells:
cell.step(cell)
yield cells
Comparison:
The traditional way creates a list of Cell instances each with a
current stage attribute.
The dynamic __class__ way creates a list of instances which are
subclasses of Stage. There is no need for a current stage
attribute since __class__ already serves this purpose.
The traditional way uses goToStage2, goToStage3, ... methods to
switch stages.
The dynamic __class__ way requires no such methods. You just
reassign __class__.
The traditional way uses the special method __getattr__ to delegate
some method calls to the appropriate stage instance held in the
self.current_stage attribute.
The dynamic __class__ way does not require any such delegation. The
instances in cells are already the objects you want.
The traditional way needs to pass the cell as an argument to
Stage.step. This is so cell.goToStageN can be called.
The dynamic __class__ way does not need to pass anything. The
object we are dealing with has everything we need.
Conclusion:
Both ways can be made to work. To the extent that I can envision how these two implementations would pan-out, it seems to me the dynamic __class__ implementation will be
simpler (no Cell class),
more elegant (no ugly goToStage2 methods, no brain-teasers like why
you need to write cell.step(cell) instead of cell.step()),
and easier to understand (no __getattr__, no additional level of
indirection)

Pointing to another object's attributes and adding your own

Suppose I have a class:
class Car(object):
def __init__(self, name, tank_size=10, mpg=30):
self.name = name
self.tank_size = tank_size
self.mpg = mpg
I put together a list of the cars I'm looking at:
cars = []
cars.append(Car("Toyota", 11, 29))
cars.append(Car("Ford", 15, 12))
cars.append(Car("Honda", 12, 25))
If I assign a name to my current favorite (a "pointer" into the list, if you will):
my_current_fav = cars[1]
I can easily access the attributes of my current favorite:
my_current_fav.name # Returns "Ford"
my_current_fav.tank_size # Returns 15
my_current_fav.mpg # Returns 12
Here's where I start getting foggy. I would like to provide additional "computed" attributes only for my current favorite (let's assume these attributes are too "expensive" to store in the original list and are easier to just compute):
my_current_fav.range # Would return tank_size * mpg = 180
# (if pointing to "Ford")
In my case, I just capitulated and added 'range' as an attribute of Car(). But what if storing 'range' in each Car() instance was expensive but calculating it was cheap?
I considered making 'my_current_fav' a sub-class of Car(), but I couldn't figure out a way to do that and still maintain my ability to simply "point" 'my_current_favorite' to an entry in the 'cars' list.
I also considered using decorators to compute and return 'range', but couldn't figure out a way to also provide access to the attributes 'name', 'mpg', etc.
Is there an elegant way to point to any item in the list 'cars', provide access to the attributes of the instance being pointed to as well as provide additional attributes not found in the class Car?
Additional information:
After reading many of your answers, I see there is background information I should have put into the original question. Rather than comment on many answers individually, I'll put the additional info here.
This question is a simplification of a more complicated issue. The original problem involves modifications to an existing library. While making range a method call rather than an attribute is a good way to go, changing
some_var = my_current_favorite.range
to
some_var = my_current_favorite.range()
in many existing user scripts would be expensive. Heck, tracking down those user scripts would be expensive.
Likewise, my current approach of computing range for every car isn't "expensive" in Python terms, but is expensive in run-time because the real-world analog requires slow calls to the (embedded) OS. While Python itself isn't slow, those calls are, so I am seeking to minimize them.
This is easiest to do for your example, without changing Car, and changing as little else as possible, with __getattr__:
class Car(object):
def __init__(self, name, tank_size=10, mpg=30):
self.name = name
self.tank_size = tank_size
self.mpg = mpg
class Favorite(object):
def __init__(self, car):
self.car = car
def __getattr__(self, attr):
return getattr(self.car, attr)
#property
def range(self):
return self.mpg * self.tank_size
cars = []
cars.append(Car("Toyota", 11, 29))
cars.append(Car("Ford", 15, 12))
cars.append(Car("Honda", 12, 25))
my_current_fav = Favorite(cars[1])
print my_current_fav.range
Any attribute not found on an instance of Favorite will be looked up on the Favorite instances car attribute, which you set when you make the Favorite.
Your example of range isn't a particularly good one for something to add to Favorite, because it should just be a property of car, but I used it for simplicity.
Edit: Note that a benefit of this method is if you change your favorite car, and you've not stored anything car-specific on Favorite, you can change the existing favorite to a different car with:
my_current_fav.car = cars[0] # or Car('whatever')
If you have access to the class (and it sounds like you do), just create a function inside the class instead.
def range(self):
return self.tank_size * self.mpg
With regards to your example, you could make range a read-only property of class Car that would be computed on demand. No need for extra classes.
Why don't you just create a method:
class Car(object):
def __init__(self, name, tank_size=10, mpg=30):
self.name = name
self.tank_size = tank_size
self.mpg = mpg
def range(self):
return self.tank_size * self.mpg
Sounds like range() should be a method of the class. Methods are very cheap - the objects don't store them. The downside is it is computed each time you access the value of range.
class Car(object):
def __init__(self, name, tank_size=10, mpg=30):
[AS ABOVE]
def range(self):
return self.tank_size * self.mpg
If you prefer it to behave like a field, i.e. compute only once, you can store the value in the object during the range method:
def range(self):
if not hasattr(self,'_rangeval'):
self._rangeval = self.tank_size * self.mpg
return self._rangeval
This takes advantage of the fact that you can dynamically create fields in objects.
I don't understand why the default would be 180 when the default of the computed values is 300. If this strange behaviour is important, you will need to set another flag to see if the other parameters have been initialised to the default or not.
I'm not sure I understand what you're trying to do, so I'm going to cover a few different possible understandings of what you're thinking.
In my case, I just capitulated and added 'range' as an attribute of Car(). But what if storing 'range' in each Car() instance was expensive but calculating it was cheap?
First off, worrying about things being "expensive" is usually not that Pythonic. If it really mattered, you would be using a lower-level language most of the time. But in general, if you want something to be calculated rather than stored, the natural way is to use a method. Add it to the Car class. It does not cost per-object, unless of course you explicitly replace the method on a per-object basis.
Here's how it works: when you make a call to a_car.range(), the range attribute is looked up in a_car first. If it's not found there, then (skipping lots and lots of details here!) the class of a_car is identified as Car, and the attribute is looked up there. You define range as a Car method, so it gets found there, and is determined to be something that's actually callable, so it gets called. As a special syntax rule, a_car gets passed as the first parameter to the method (all of which partly explains why you need to have an explicit parameter - named self by convention - for methods in Python, unlike many other languages with an implicit this).
I considered making 'my_current_fav' a sub-class of Car(), but I couldn't figure out a way to do that and still maintain my ability to simply "point" 'my_current_favorite' to an entry in the 'cars' list.
You can definitely store a subclass of Car in the same list as a bunch of ordinary Cars. No problem there. Heck, you can store a Banana in the same list as a bunch of Cars if you like; Python is dynamically typed, and doesn't care. It will figure out what kind of object something is at the exact moment that it becomes relevant.
What you can't easily do is cause an existing Car to become a MyFavouriteCar. If you created a new MyFavouriteCar that my_favourite_car refers to (don't say "points at", please; object references are a higher-level abstraction, and unlike Java there is no "null pointer" in Python - None is an object), then you could replace an existing car in the list with it, but it's still a different object (even if it's somehow based on the original Car that it replaces). You can design in such a way that this doesn't matter; or you can resort to evil hackery (which I won't explain here); or you can (much better in your case) just offer the functionality to all Cars, because it's really free to do so.
From the Zen of Python: Special cases aren't special enough.
You talk about my_current_fav being a 'pointer' -- I just want to make sure you realize that, in fact,
my_current_fav = cars[1]
binds a name to cars[1] -- in other words, my_current_fav is cars[1] == True. There is no pointing going on. If you really want a pointer-stlye you can do something like this:
class Favorite(object):
def __init__(self, car_list, index):
self.car_list = car_list
self.index = index
def __getattr__(self, attr):
return getattr(self.car_list[self.index], attr)
def __index__(self):
return self.index
#property
def range(self):
return self.mpg * self.tank_size
my_current_fav = Favorite(cars, 1)
print my_current_fav.name
print my_current_fav.range
print cars[my_current_fav]

Accessing private variables when there's a getter/setter for them

I have a question about righteous way of programming in Python... Maybe there can be several different opinions, but here it goes:
Let's say I have a class with a couple of private attributes and that I have implemented two getters/setters (not overloading __getattr__ and __setattr__, but in a more “Java-tistic” style):
class MyClass:
def __init__(self):
self.__private1 = "Whatever1"
def setPrivate1(self, private1):
if isinstance(private1, str) and (private1.startswith("private")):
self.__private1 = private1
else:
raise AttributeError("Kaputt")
def getPrivate1(self):
return self.__private1
Now let's say a few lines below, in another method of the same class, I need to re-set the value of that “__private1”. Since it's the same class, I still have direct access to the private attribute self.__private1.
My question is: Should I use:
self.setPrivate1("privateBlaBlaBla")
or should I access directly as:
self.__private1 ="privateBlaBlaBla"
since I am the one setting the new value, I know that said value (“privateBlaBlaBla”) is correct (an str() that starts with “private”), so it is not going to leave the system inconsistent. On the other hand, if another programmer takes my code, and needs to change the functionality for the self.__private1 attribute, he will need to go through all the code, and see if the value of __private1 has been manually set somewhere else.
My guess is that the right thing to do is to always using the setPrivate1 method, and only access directly the __private1 variable in the get/set, but I'd like to know the opinion of more experienced Python programmers.
You can't present a classic example of bad Python and then expect people to have opinions on what do to about it. Use getters and setters.
class MyClass:
def __init__(self):
self._private1 = "Whatever1"
#property
def private1(self):
return self._private1
#private1.setter
def private1(self, value):
self._private1 = value
A side comment -- using double underscore names can be confusing, because Python actually mangles the name to stop you accessing them from outside the class. This provides no real security, but causes no end of headaches. The easiest way to avoid the headaches is to use single-underscore names, which is basically a universal convention for private. (Ish.)
If you want an opinion -- use properties =). If you want an opinion on your JavaPython monstrosity, I would use the setter -- after all, you've written it, that's what it's there for! There's no obvious benefit to setting the variable by hand, but there are several drawbacks.
Neither. In Python, use properties, not getters and setters.
class MyClass:
def __init__(self):
self._private1 = "Whatever1"
#property
def private1(self):
return self._private1
#private1.setter
def private1(self, private1):
if isinstance(private1, str) and (private1.startswith("private")):
self._private1 = private1
else:
raise AttributeError("Kaputt")
Then later on in your code, set the _private1 attribute with
self.private1="privateBlaBlaBla"

Categories