Note: I'm not talking about preventing the rebinding of a variable. I'm talking about preventing the modification of the memory that the variable refers to, and of any memory that can be reached from there by following the nested containers.
I have a large data structure, and I want to expose it to other modules, on a read-only basis. The only way to do that in Python is to deep-copy the particular pieces I'd like to expose - prohibitively expensive in my case.
I am sure this is a very common problem, and it seems like a constant reference would be the perfect solution. But I must be missing something. Perhaps constant references are hard to implement in Python. Perhaps they don't quite do what I think they do.
Any insights would be appreciated.
While the answers are helpful, I haven't seen a single reason why const would be either hard to implement or unworkable in Python. I guess "un-Pythonic" would also count as a valid reason, but is it really? Python does do scrambling of private instance variables (starting with __) to avoid accidental bugs, and const doesn't seem to be that different in spirit.
EDIT: I just offered a very modest bounty. I am looking for a bit more detail about why Python ended up without const. I suspect the reason is that it's really hard to implement to work perfectly; I would like to understand why it's so hard.
It's the same as with private methods: as consenting adults authors of code should agree on an interface without need of force. Because really really enforcing the contract is hard, and doing it the half-assed way leads to hackish code in abundance.
Use get-only descriptors, and state clearly in your documentation that these data is meant to be read only. After all, a determined coder could probably find a way to use your code in different ways you thought of anyways.
In PEP 351, Barry Warsaw proposed a protocol for "freezing" any mutable data structure, analogous to the way that frozenset makes an immutable set. Frozen data structures would be hashable and so capable being used as keys in dictionaries.
The proposal was discussed on python-dev, with Raymond Hettinger's criticism the most detailed.
It's not quite what you're after, but it's the closest I can find, and should give you some idea of the thinking of the Python developers on this subject.
There are many design questions about any language, the answer to most of which is "just because". It's pretty clear that constants like this would go against the ideology of Python.
You can make a read-only class attribute, though, using descriptors. It's not trivial, but it's not very hard. The way it works is that you can make properties (things that look like attributes but call a method on access) using the property decorator; if you make a getter but not a setter property then you will get a read-only attribute. The reason for the metaclass programming is that since __init__ receives a fully-formed instance of the class, you actually can't set the attributes to what you want at this stage! Instead, you have to set them on creation of the class, which means you need a metaclass.
Code from this recipe:
# simple read only attributes with meta-class programming
# method factory for an attribute get method
def getmethod(attrname):
def _getmethod(self):
return self.__readonly__[attrname]
return _getmethod
class metaClass(type):
def __new__(cls,classname,bases,classdict):
readonly = classdict.get('__readonly__',{})
for name,default in readonly.items():
classdict[name] = property(getmethod(name))
return type.__new__(cls,classname,bases,classdict)
class ROClass(object):
__metaclass__ = metaClass
__readonly__ = {'a':1,'b':'text'}
if __name__ == '__main__':
def test1():
t = ROClass()
print t.a
print t.b
def test2():
t = ROClass()
t.a = 2
test1()
While one programmer writing code is a consenting adult, two programmers working on the same code seldom are consenting adults. More so if they do not value the beauty of the code but them deadlines or research funds.
For such adults there is some type safety, provided by Enthought's Traits.
You could look into Constant and ReadOnly traits.
For some additional thoughts, there is a similar question posed about Java here:
Why is there no Constant feature in Java?
When asking why Python has decided against constant references, I think it's helpful to think of how they would be implemented in the language. Should Python have some sort of special declaration, const, to create variable references that can't be changed? Why not allow variables to be declared a float/int/whatever then...these would surely help prevent programming bugs as well. While we're at it, adding class and method modifiers like protected/private/public/etc. would help enforce compile-type checking against illegal uses of these classes. ...pretty soon, we've lost the beauty, simplicity, and elegance that is Python, and we're writing code in some sort of bastard child of C++/Java.
Python also currently passes everything by reference. This would be some sort of special pass-by-reference-but-flag-it-to-prevent-modification...a pretty special case (and as the Tao of Python indicates, just "un-Pythonic").
As mentioned before, without actually changing the language, this type of behaviour can be implemented via classes & descriptors. It may not prevent modification from a determined hacker, but we are consenting adults. Python didn't necessarily decide against providing this as an included module ("batteries included") - there was just never enough demand for it.
Related
This question already has answers here:
What's the pythonic way to use getters and setters?
(8 answers)
Closed 2 months ago.
Using get/set seems to be a common practice in Java (for various reasons), but I hardly see Python code that uses this.
Why do you use or avoid get/set methods in Python?
In python, you can just access the attribute directly because it is public:
class MyClass:
def __init__(self):
self.my_attribute = 0
my_object = MyClass()
my_object.my_attribute = 1 # etc.
If you want to do something on access or mutation of the attribute, you can use properties:
class MyClass:
def __init__(self):
self._my_attribute = 0
#property
def my_attribute(self):
# Do something if you want
return self._my_attribute
#my_attribute.setter
def my_attribute(self, value):
# Do something if you want
self._my_attribute = value
Crucially, the client code remains the same.
Cool link: Python is not Java :)
In Java, you have to use getters and setters because using public fields gives you no opportunity to go back and change your mind later to using getters and setters. So in Java, you might as well get the chore out of the way up front. In Python, this is silly, because you can start with a normal attribute and change your mind at any time, without affecting any clients of the class. So, don't write getters and setters.
Here is what Guido van Rossum says about that in Masterminds of Programming
What do you mean by "fighting the language"?
Guido: That usually means that they're
trying to continue their habits that
worked well with a different language.
[...] People will turn everything into
a class, and turn every access into an
accessor method,
where that is really not a wise thing to do in Python;
you'll have more verbose code that is
harder to debug and runs a lot slower.
You know the expression "You can write
FORTRAN in any language?" You can write Java in any language, too.
No, it's unpythonic. The generally accepted way is to use normal data attribute and replace the ones that need more complex get/set logic with properties.
The short answer to your question is no, you should use properties when needed. Ryan Tamyoko provides the long answer in his article Getters/Setters/Fuxors
The basic value to take away from all this is that you want to strive to make sure every single line of code has some value or meaning to the programmer. Programming languages are for humans, not machines. If you have code that looks like it doesn’t do anything useful, is hard to read, or seems tedious, then chances are good that Python has some language feature that will let you remove it.
Your observation is correct. This is not a normal style of Python programming. Attributes are all public, so you just access (get, set, delete) them as you would with attributes of any object that has them (not just classes or instances). It's easy to tell when Java programmers learn Python because their Python code looks like Java using Python syntax!
I definitely agree with all previous posters, especially #Maximiliano's link to Phillip's famous article and #Max's suggestion that anything more complex than the standard way of setting (and getting) class and instance attributes is to use Properties (or Descriptors to generalize even more) to customize the getting and setting of attributes! (This includes being able to add your own customized versions of private, protected, friend, or whatever policy you want if you desire something other than public.)
As an interesting demo, in Core Python Programming (chapter 13, section 13.16), I came up with an example of using descriptors to store attributes to disk instead of in memory!! Yes, it's an odd form of persistent storage, but it does show you an example of what is possible!
Here's another related post that you may find useful as well:
Python: multiple properties, one setter/getter
I had come here for that answer(unfortunately i couldn't) . But i found a work around else where . This below code could be alternative for get .
class get_var_lis:
def __init__(self):
pass
def __call__(self):
return [2,3,4]
def __iter__(self):
return iter([2,3,4])
some_other_var = get_var_lis
This is just a workaround . By using the above concept u could easily build get/set methodology in py too.
Our teacher showed one example on class explaining when we should use accessor functions.
class Woman(Human):
def getAge(self):
if self.age > 30:
return super().getAge() - 10
else:
return super().getAge()
Let's say we have the following classes:
class Duck(object):
pass
class OldFashionedDuck(Organism, Duck):
def look(self):
self.display_biological_appearance()
def walk(self):
self.keep_balance_on_two_feet()
def quack(self):
self.make_noise_with_lungs("Quack!")
class ArtificialDuck(Robot, Duck):
def look(self):
self.display_imitation_biological_appearance()
def walk(self):
self.engage_leg_clockwork()
def quack(self):
self.play_sound("quack.au")
In this example, OldFashionedDuck and ArtificialDuck have no common implementation, but by construction they will both return True for isinstance(..., Duck).
This is not perfect, but it is something that I thought might help respect both duck typing, and (via empty mixin inheritance) allow isinstance(). In essence it offers a contract to meet an interface, so it's not really calling isinstance() based on the class that does all the work, but based on an interface that anyone can opt-in to.
I've seen articles based on "isinstance() considered harmful," because it breaks duck typing. However, I at least as a programmer would like to know, if not necessarily where an object gets a function from, but whether it implements an interface.
Is this approach useful, and if so can it be improved upon?
I've seen articles based on "isinstance() considered harmful," because it breaks duck typing. However, I at least as a programmer would like to know, if not necessarily where an object gets a function from, but whether it implements an interface.
I think you're missing the point.
When we talk about "duck typing", what we really mean is not formalizing our interfaces. What you're trying to do, therefore, boils down to attempting to answer the question "how can I formalize my interface while still not formalizing my interface?".
We expect an object that was given to us to implement an interface - one that we described, not by making a base class, but by writing a bunch of documentation and describing behaviour (and if we're feeling especially frisky, setting up some kind of test suite) - because we said that this is what we expect (again in our documentation). We verify that the object implements the interface by attempting to use it as though it does, and treating any resulting errors as the caller's responsibility. Calling code that gives us the wrong object is naughty, and that's where the bug needs to be fixed. (Again, tests help us track these things down.)
In short, testing isinstance(this_fuzzball_that_was_handed_to_me, Duck) doesn't really help matters:
It could pass the isinstance check, but implement the methods in a way that violates our expectations (or, say, return NotImplemented). Only real testing will do here.
It could pass the check, but actually completely fail to implement one or more of the methods; after all, the base Duck doesn't contain any implementations, and Python has no reason to check for them in a derived class.
Perhaps more importantly, it could fail the check, even though it's a perfectly usable-as-a-duck fuzzball. Maybe it's some unrelated object that had quack, walk and look functions directly, manually attached to it as attributes (as opposed to them being attributes of its class, which become methods when looked up).
Okay, so, "don't do that", you say. But now you're making more work for everyone; if clients don't always opt in, then it's useless and dangerous for the duck-using code to make the check. And meanwhile, what are you gaining?
This is related to the EAFP principle: don't try to figure out whether something is a Duck by looking at it; figure out if it's a Duck by treating it as a Duck and dealing with the gory mess if it isn't.
But if you don't care about the duck typing philosophy, and absolutely must force some semblance of rigor onto things, you might be interested in the standard library abc module.
Even though I don't want to overestimate this term, but: Your approach is not pythonic. Do duck-typing, or don't do it.
If you want to be sure that your implementation of a "interface" implements everything it should: test it!
For smaller projects, it's easy to remember what you need. And you can simply try it.
I agree that for larger projects, or even cooperation in teams, it's better to make sure that your type has everything it needs. In such a scenario, you definitely should use unit-tests to make sure your type is complete. Even without duck-typing, you need tests, so probably you won't need any extra-tests.
Guido van Rossum has talked about some interesting thoughts about duck-typing in this talk. It's so inspiring, and definitely worth watching.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Python (and Python C API): new versus init
I'm at college just now and the lecturer was using the terms constructors and initializers interchangeably. I'm pretty sure that this is wrong though.
I've tried googling the answer but not found the answer I'm looking for.
In most OO languages, they are the same step, so he's not wrong for things like java, c++, etc. In python they are done in two steps: __new__ is the constructor; __init__ is the initializer.
Here is another answer that goes into more detail about the differences between them.
In almost all usual cases, Python does not have constructors in the same sense used by other OO languages because manually managing memory is generally discouraged. Instead, what you should usually do is define an __init__ method on the class. This method is called to initialize the new instance object automatically, first thing after it is constructed. Thus, it is not really a constructor, and talking about it as a constructor might confuse some people.
Of course some people want to call it a constructor because it is used a little bit like a constructor - fundamentally you can call it whatever you want as long as everyone understands what you are actually referring to. But in general, to be explicit and make yourself understood, call it an init method or something other than a constructor. Fundamentally, different languages just come with somewhat different terminology and speaking very clearly will always require adjustment to your subject matter and audience.
In Python it is possible to manage instance creation and destruction at a finer granularity, though you won't want to unless you know what you're doing. This is done by defining __new__ and __del__ methods to hook object instantiation and del statements. Whether these qualify as constructors and destructors precisely is a little more debatable (Python docs call the del method a destructor, but tend to be vaguer on what constitutes a constructor, e.g. including many functions which return object instances). I'd still encourage you to use the specific terminology for the language at hand, and in comparative discussions to define your terms up front. As always, your choice of terms while speaking involves tradeoffs between the audience being able to easily follow you and the audience potentially being led into confusion: if you are talking about memory management probably be as specific as possible, but if you are talking loosely then just use some word your audience understands and be ready to clarify.
Your instructor is being unclear at worst, I'm not aware of any one canonical definition of these terms but they might cause confusion for people who have learned very specific definitions from other languages.
http://docs.python.org/reference/datamodel.html#basic-customization
__new__ - constructor.
__init__ - initializer.
I am sorry all - I am not here to blame Python. This is just a reflection on whether what I believe is right. Being a Python devotee for two years, I have been writing only small apps and singing Python's praises wherever I go. I recently had the chance to read Django's code, and have started wondering if Python really follows its "readability counts" philosophy. For example,
class A:
a = 10
b = "Madhu"
def somemethod(self, arg1):
self.c = 20.22
d = "some local variable"
# do something
....
...
def somemethod2 (self, arg2):
self.c = "Changed the variable"
# do something 2
...
It's difficult to track the flow of code in situations where the instance variables are created upon use (i.e. self.c in the above snippet). It's not possible to see which instance variables are defined when reading a substantial amount of code written in this manner. It becomes very frustrating even when reading a class with just 6-8 methods and not more than 100-150 lines of code.
I am interested in knowing if my reading of this code is skewed by C++/Java style, since most other languages follow the same approach as them. Is there a Pythonic way of reading this code more fluently? What made Python developers adopt this strategy keeping "readability counts" in mind?
The code fragment you present is fairly atypical (which might also because you probably made it up):
you wouldn't normally have an instance variable (self.c) that is a floating point number at some point, and a string at a different point. It should be either a number or a string all the time.
you normally don't bring instance variables into life in an arbitrary method. Instead, you typically have a constructor (__init__) that initializes all variables.
you typically don't have instance variables named a, b, c. Instead, they have some speaking names.
With these fixed, your example would be much more readable.
A sufficiently talented miscreant can write unreadable code in any language. Python attempts to impose some rules on structure and naming to nudge coders in the right direction, but there's no way to force such a thing.
For what it's worth, I try to limit the scope of local variables to the area where they're used in every language that i use - for me, not having to maintain a huge mental dictionary makes re-familiarizing myself with a bit of code much, much easier.
I agree that what you have seen can be confusing and ought to be accompanied by documentation. But confusing things can happen in any language.
In your own code, you should apply whatever conventions make things easiest for you to maintain the code. With respect to this particular issue, there are a number of possible things that can help.
Using something like Epydoc, you can specify all the instance variables a class will have. Be scrupulous about documenting your code, and be equally scrupulous about ensuring that your code and your documentation remain in sync.
Adopt coding conventions that encourage the kind of code you find easiest to maintain. There's nothing better than setting a good example.
Keep your classes and functions small and well-defined. If they get too big, break them up. It's easier to figure out what's going on that way.
If you really want to insist that instance variables be declared before referenced, there are some metaclass tricks you can use. e.g., You can create a common base class that, using metaclass logic, enforces the convention that only variables that are declared when the subclass is declared can later be set.
This problem is easily solved by specifying coding standards such as declaring all instance variables in the init method of your object. This isn't really a problem with python as much as the programmer.
If what the code is doing becomes mysterious for some reason .. there should either be comments or the function names should make it obvious.
This is just my opinion though.
I personally think not having to declare variables is one of the dangerous things in Python, especially when doing classes. It is all too easy to accidentally create a variable by simple mistyping and then boggle at the code at length, unable to find the mistake.
Adding a property just before you need it will prevent you from using it before it's got a value. Personally, I always find classes hard to follow just from reading source - I read the documentation and find out what it's supposed to do, and then it usually makes sense when I read the source again.
The fact that such stuff is allowed is only useful in rare times for prototyping; while Javascript tends to allow anything and maybe such an example could be considered normal (I don't really know), in Python this is mostly a negative byproduct of omission of type declaration, which can help speeding up development - if you at some point change your mind on the type of a variable, fixing type declarations can take more time than the fixes to actual code, in some cases, including the renaming of a type, but also cases where you use a different type with some similar methods and no superclass/subclass relationship.
I'm a C++ programmer just starting to learn Python. I'd like to know how you keep track of instance variables in large Python classes. I'm used to having a .h file that gives me a neat list (complete with comments) of all the class' members. But since Python allows you to add new instance variables on the fly, how do you keep track of them all?
I'm picturing a scenario where I mistakenly add a new instance variable when I already had one - but it was 1000 lines away from where I was working. Are there standard practices for avoiding this?
Edit: It appears I created some confusion with the term "member variable." I really mean instance variable, and I've edited my question accordingly.
I would say, the standard practice to avoid this is to not write classes where you can be 1000 lines away from anything!
Seriously, that's way too much for just about any useful class, especially in a language that is as expressive as Python. Using more of what the Standard Library offers and abstracting away code into separate modules should help keeping your LOC count down.
The largest classes in the standard library have well below 100 lines!
First of all: class attributes, or instance attributes? Or both? =)
Usually you just add instance attributes in __init__, and class attributes in the class definition, often before method definitions... which should probably cover 90% of use cases.
If code adds attributes on the fly, it probably (hopefully :-) has good reasons for doing so... leveraging dynamic features, introspection, etc. Other than that, adding attributes this way is probably less common than you think.
pylint can statically detect attributes that aren't detected in __init__, along with many other potential bugs.
I'd also recommend writing unit tests and running your code often to detect these types of "whoopsie" programming mistakes.
Instance variables should be initialized in the class's __init__() method. (In general)
If that's not possible. You can use __dict__ to get a dictionary of all instance variables of an object during runtime. If you really need to track this in documentation add a list of instance variables you are using into the docstring of the class.
It sounds like you're talking about instance variables and not class variables. Note that in the following code a is a class variable and b is an instance variable.
class foo:
a = 0 #class variable
def __init__(self):
self.b = 0 #instance variable
Regarding the hypothetical where you create an unneeded instance variable because the other one was about one thousand lines away: The best solution is to not have classes that are one thousand lines long. If you can't avoid the length, then your class should have a well defined purpose and that will enable you to keep all of the complexities in your head at once.
A documentation generation system such as Epydoc can be used as a reference for what instance/class variables an object has, and if you're worried about accidentally creating new variables via typos you can use PyChecker to check your code for this.
This is a common concern I hear from many programmers who come from a C, C++, or other statically typed language where variables are pre-declared. In fact it was one of the biggest concerns we heard when we were persuading programmers at our organization to abandon C for high-level programs and use Python instead.
In theory, yes you can add instance variables to an object at any time. Yes it can happen from typos, etc. In practice, it rarely results in a bug. When it does, the bugs are generally not hard to find.
As long as your classes are not bloated (1000 lines is pretty huge!) and you have ample unit tests, you should rarely run in to a real problem. In case you do, it's easy to drop to a Python console at almost any time and inspect things as much as you wish.
It seems to me that the main issue here is that you're thinking in terms of C++ when you're working in python.
Having a 1000 line class is not a very wise thing anyway in python, (I know it happens alot in C++ though),
Learn to exploit the dynamism that python gives you, for instance you can combine lists and dictionaries in very creative ways and save your self hundreds of useless lines of code.
For example, if you're mapping strings to functions (for dispatching), you can exploit the fact that functions are first class objects and have a dictionary that goes like:
d = {'command1' : func1, 'command2': func2, 'command3' : func3}
#then somewhere else use this list to dispatch
#given a string `str`
func = d[str]
func() #call the function!
Something like this in C++ would take up sooo many lines of code!
The easiest is to use an IDE. PyDev is a plugin for eclipse.
I'm not a full on expert in all ways pythonic, but in general I define my class members right under the class definition in python, so if I add members, they're all relative.
My personal opinion is that class members should be declared in one section, for this specific reason.
Local scoped variables, otoh, should be defined closest to when they are used (except in C--which I believe still requires variables to be declared at the beginning of a method).
Consider using slots.
For example:
class Foo:
__slots__ = "a b c".split()
x = Foo()
x.a =1 # ok
x.b =1 # ok
x.c =1 # ok
x.bb = 1 # will raise "AttributeError: Foo instance has no attribute 'bb'"
It is generally a concern in any dynamic programming language -- any language that does not require variable declaration -- that a typo in a variable name will create a new variable instead of raise an exception or cause a compile-time error. Slots helps with instance variables, but doesn't help you with, module-scope variables, globals, local variables, etc. There's no silver bullet for this; it's part of the trade-off of not having to declare variables.