I see a lot of python code that does "loose" private variables / functions. They'll declare functions / variables with one underscore (eg. _foo) and then use it just in the class / file. It really annoys me that they don't use the double underscore because eventually, someone will call this "private" member from outside the class.
Is there some way to enforce the privacy on single underscores (without changing to doubles)?
Thanks!
No. And that's the philosophy of python: Do not make the compiler/parser enforce privacy since developers who want to access private members have ways to do so anyway (reflection etc.).
The idea is telling people Hey, this is not part of the public API. If you use it improperly, it could breaks things or kill your cat. I might also change the signature often since it's not part of the public API and I don't have to care about people using it
And fyi, you can actually access double-underscore variables (same applies for methods) from outside using obj._ClassName__variableName. Besides that it's not encouraged to use double underscores except for mixin objects - you can never know if someone wants to subclass your class.
In Python, there is no strict concept(like Java for example) of private methods. Even using double underscores is still accessible by doing _classname__method.
In short, don't go against the grain, rather go with the Python philosophy in which private is a convention, not a force.
Related
I was doing some research about the use of encapsulation in object oriented programming using Python and I have stumbled with this topic that has mixed opinions about how encapsulated attributes work and about the usage of them.
I have programmed this piece of code that only made matters more confuse to me:
class Dog:
def __init__(self,weight):
self.weight = weight
__color =''
def set_color(self,color):
self.__color = color
def get_color(self):
print(self.__color)
rex = Dog(59)
rex.set_color('Black')
rex.get_color()
rex.color = 'White'
rex.__color = rex.color
print(rex.__color)
rex.get_color()
The result is:
>Black
>White
>Black
I understand that the reason behind this is because when we do the assignment rex.__color = rex.color, a new attribute is created that does not point to the real __color of the instanced Dog.
My questions here are:
Is this a common scenario to occur?
Are private attributes a thing used really often?
In a language that does not have properties (eg. java) this is so common that it has become a standard, and all frameworks assume that getters/setters already exist.
However, in python you can have properties, which are essentially getters/setters that can be added later without altering the code that uses the variables. So, no reason to do it in python. Use the fields as public, and add properties if something changes later.
Note: use single instead of double underscore in your "private" variables. Not only it's the common convention, but also, double underscore is handled differently by the interpreter.
Encapsulation is not about data hidding but about keeping state and behaviour together. Data hidding is meant as a way to enforce encapsulation by preventing direct access to internal state, so the client code must use (public) methods instead. The main points here are 1/ to allow the object to maintain a coherent state (check the values, eventually update some other part of the state accordingly etc) and 2/ to allow implementation changes (internal state / private methods) without breaking the client code.
Languages like Java have no support for computed attributes, so the only way to maintain encapsulation in such languages is to make all attributes protected or private and to eventally provide accessors. Alas, some people never got the "eventually" part right and insist on providing read/write accessors for all attributes, which is a complete nonsense.
Python has a strong support for computed attributes thru the descriptor protocol (mostly known via the generic property type but you can of course write your own desciptors instead), so there's no need for explicit getters/setters - if your design dictates that some class should provide a publicly accessible attribute as part of it's API, you can always start with just a public attribute and if at some point you need to change implementation you can just replace it with a computed attribute.
This doesn't mean you should make all your attributes public !!! - most of the time, you will have "implementation attributes" (attributes that support the internal state but are in no way part of the class API), and you definitly want to keep those protected (by prefixing them with a single leading underscore).
Note that Python doesn't try to technically enforce privacy, it's only a naming convention and you can't prevent client code to access internal state. Nothing to be worried about here, very few peoples stupid enough to bypass the official API without good reasons, and then they know their code might break something and assume all consequences.
I know it is good style in other languages (C#, JAVA, ...) to make "private" methods (using two underscores) static if possible.
Is there any recommendation for Python?
I would rather not make them static, because they are only private (I don't want to move them to utility classes and they are not performance relevant) and I want to avoid cluttering my code with decorators. On the other hand, the static code analyzer tells me to do so (and I want to write code in the "Pythonic" way).
The short answer is you should almost never use static methods in Python. They were added by mistake.
I meant to implement class methods but at first I didn't understand
them and accidentally implemented static methods first.
- Guido van Rossum
You can almost definitely achieve what you want with a class method or an instance method.
More on why static methods in Python are mostly useless here.
In python there is no such thing like 'private method', all methods are public. The only thing that tells you that a method is not intended to use publicly is that it's name starts with an underscore.
If your methods are not referencing anything from self (an instance of the class) then you could make them static to optimize the memory usage of your application.
I was wondering, is it recommended/Pythonic to define and use custom double underscore variables/functions in a Python script? For example, __tablename__ as used in SQLAlchemy or __validateitem__() (a custom function that validates an item before applying __setitem__() to it).
If it does define that something magic happens, or that that specific variable/function is used indeed in a special way (like the two above examples), I feel it is a good idea using them.
I am interested in arguments on both best coding practices and potential risks in using this kind of naming.
From PEP8:
__double_leading_and_trailing_underscore__: "magic" objects or attributes that live in user-controlled namespaces. E.g. __init__, __import__ or __file__. Never invent such names; only use them as documented.
So, the advice is not to use the double underscore syntax for your own variables.
I would like to set up a class hierarchy in Python 3.2 with 'protected' access: Members of the base class would be in scope only for derived classes, but not 'public'.
A double underscore makes a member 'private', a single underscore indicates a warning but the member remains 'public'. What (if any...) is the correct syntax for designating a 'protected' member.
Member access allowance in Python works by "negotiation" and "treaties", not by force.
In other words, the user of your class is supposed to leave their hands off things which are not their business, but you cannot enforce that other than my using _xxx identifiers making absolutely clear that their access is (normally) not suitable.
Double underscores don't make a member 'private' in the C++ or Java sense - Python quite explicitly eschews that kind of language-enforced access rules. A single underscore, by convention, marks an attribute or a method as an "implementation detail" - that is, things outside can still get to it, but this isn't a supported part of the class' interface and, therefore, the guarantees that the class might make about invariants or back/forwards compatibility no longer apply. This solves the same conceptual problem as 'private' (separation of interface and implementation) in a different way.
Double underscores invoke name mangling which still isn't 'private' - it is just a slightly stronger formulation of the above, whereby:
- This function is an implementation detail of this class, but
- Subclasses might reasonably expect to have a method of the same name that isn't meant as an overridden version of the original
This takes a little bit of language support, whereby the __name is mangled to include the name of the class - so that subclass versions of it get different names instead of overriding. It is still quite possible for a subclass or outside code to call that method if it really wants to - and the goal of name mangling is explicitly not to prevent that.
But because of all this, 'protected' turns out not to make much sense in Python - if you really have a method that could break invariants unless called by a subclass (and, realistically, you probably don't even if you think you do), the Python Way is just to document that. Put a note in your docstring to the effect of "This is assumed to only be called by subclasses", and run with the assumption that clients will do the right thing - because if they don't, it becomes their own problem.
I recently posted a question on stackoverflow and I got a resolution.
Some one suggested to me about the coding style and I haven't received further input. I have the following question with reference to the prior query.
How can we declare private variables inside a class in python? I thought that by using a double underscore (__) the variable is treated as private. Please correct me.
As per the suggestion received before, we don't have to use a getter or setter method. Shouldn't we use a getter or setter or both? Please let me know your suggestion on this one.
Everything is public in Python, the __ is a suggestion by convention that you shouldn't use that function as it is an implementation detail.
This is not enforced by the language or runtime in any way, these names are decorated in a semi-obfuscated way, but they are still public and still visible to all code that tries to use them.
Idiomatic Python doesn't use get/set accessors, it is duplication of effort since there is no private scope.
You only use accessors when you want indirect access to a member variable to have code around it, and then you mark the member variable with __ as the start of its name and provide a function with the actual name.
You could go to great lengths with writing reams of code to try and protect the user from themselves using Descriptors and meta programming, but in the end you will end up with more code that is more to test and more to maintain, and still no guarantee that bad things won't happen. Don't worry about it - Python has survived 20 years this way so far, so it can't be that big of a deal.
PEP 8 (http://www.python.org/dev/peps/pep-0008/) has a section "Designing for inheritance" that should address most of these concerns.
To quote:
"We don't use the term "private" here, since no attribute is really
private in Python (without a generally unnecessary amount of work)."
Also:
"If your class is intended to be subclassed, and you have attributes
that you do not want subclasses to use, consider naming them with
double leading underscores and no trailing underscores."
If you've not read the entire section, I would encourage you to do so.
Update:
To answer the question (now that the title has changed). The pythonic way to use private variables, is to not use private variables. Trying to hide something in python is seldom seen as pythonic.
You can use Python properties instead of getters and setters. Just use an instance attribute and when you need something more complex, make this attribute a property without changing too much code.
http://adam.gomaa.us/blog/2008/aug/11/the-python-property-builtin/
Private variables:
If you use the double underscore at the beginning of your class members they are considered to be private, though not REALLY enforced by python. They simply get some naming tacked on to the front to prevent them from being easily accessed. Single underscore could be treated as "protected".
Getter/Setter:
You can use these if you want to do more to wrap the process and 'protect' your 'private' attributes. But its, again, not required. You could also use Properties, which has getter/setter features.
1) http://docs.python.org/tutorial/classes.html#private-variables
“Private” instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.
(continue reading for more details about class-private variables and name mangling)
2) http://docs.python.org/library/functions.html#property