I know it is good style in other languages (C#, JAVA, ...) to make "private" methods (using two underscores) static if possible.
Is there any recommendation for Python?
I would rather not make them static, because they are only private (I don't want to move them to utility classes and they are not performance relevant) and I want to avoid cluttering my code with decorators. On the other hand, the static code analyzer tells me to do so (and I want to write code in the "Pythonic" way).
The short answer is you should almost never use static methods in Python. They were added by mistake.
I meant to implement class methods but at first I didn't understand
them and accidentally implemented static methods first.
- Guido van Rossum
You can almost definitely achieve what you want with a class method or an instance method.
More on why static methods in Python are mostly useless here.
In python there is no such thing like 'private method', all methods are public. The only thing that tells you that a method is not intended to use publicly is that it's name starts with an underscore.
If your methods are not referencing anything from self (an instance of the class) then you could make them static to optimize the memory usage of your application.
Related
Suppose I have an instance method that contains a lot of nested conditionals. What would be a good way to encapsulate that code? Put in another instance method of the same class or a function? Could you say why a certain approach is preferred?
If the function is only used by one class, and especially if the module has more classes with potentially more utility functions (used only by one class), it might clarify things a bit if you kept the functions as static methods instead to make it obvious which class they belong to. Also, automated refactorings (using the e.g. the rope library, or PyCharm or PyDev etc) then automatically move the static method along with the class to wherever the class is moved.
P.S. #staticmethods, unlike module-level functions, can be overridden in subclasses, e.g. in case of a mathematical formula that doesn't depend on the object but does depend on the type of the object.
There are two different questions here. The first one is what to do with multiple nested conditionals. There's no single right answer: it depends on your coding style, how the conditions interact, the architecture of your program and so on. Have a look at this Programmers.SE question and Jeff Atwood's blog post for some ideas; personally, I like
if not check1: return
code1
if not check2: return
code 2
...
although some people object to the multiple exit points.
The second question is what to do with individual functions if you're writing object oriented Python. The usual answer is just to put them as functions inside the module containing the class, since there's no requirement that a function be attached to a particular class. If you want, though, you can include them in the class as static methods.
I recently posted a question on stackoverflow and I got a resolution.
Some one suggested to me about the coding style and I haven't received further input. I have the following question with reference to the prior query.
How can we declare private variables inside a class in python? I thought that by using a double underscore (__) the variable is treated as private. Please correct me.
As per the suggestion received before, we don't have to use a getter or setter method. Shouldn't we use a getter or setter or both? Please let me know your suggestion on this one.
Everything is public in Python, the __ is a suggestion by convention that you shouldn't use that function as it is an implementation detail.
This is not enforced by the language or runtime in any way, these names are decorated in a semi-obfuscated way, but they are still public and still visible to all code that tries to use them.
Idiomatic Python doesn't use get/set accessors, it is duplication of effort since there is no private scope.
You only use accessors when you want indirect access to a member variable to have code around it, and then you mark the member variable with __ as the start of its name and provide a function with the actual name.
You could go to great lengths with writing reams of code to try and protect the user from themselves using Descriptors and meta programming, but in the end you will end up with more code that is more to test and more to maintain, and still no guarantee that bad things won't happen. Don't worry about it - Python has survived 20 years this way so far, so it can't be that big of a deal.
PEP 8 (http://www.python.org/dev/peps/pep-0008/) has a section "Designing for inheritance" that should address most of these concerns.
To quote:
"We don't use the term "private" here, since no attribute is really
private in Python (without a generally unnecessary amount of work)."
Also:
"If your class is intended to be subclassed, and you have attributes
that you do not want subclasses to use, consider naming them with
double leading underscores and no trailing underscores."
If you've not read the entire section, I would encourage you to do so.
Update:
To answer the question (now that the title has changed). The pythonic way to use private variables, is to not use private variables. Trying to hide something in python is seldom seen as pythonic.
You can use Python properties instead of getters and setters. Just use an instance attribute and when you need something more complex, make this attribute a property without changing too much code.
http://adam.gomaa.us/blog/2008/aug/11/the-python-property-builtin/
Private variables:
If you use the double underscore at the beginning of your class members they are considered to be private, though not REALLY enforced by python. They simply get some naming tacked on to the front to prevent them from being easily accessed. Single underscore could be treated as "protected".
Getter/Setter:
You can use these if you want to do more to wrap the process and 'protect' your 'private' attributes. But its, again, not required. You could also use Properties, which has getter/setter features.
1) http://docs.python.org/tutorial/classes.html#private-variables
“Private” instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.
(continue reading for more details about class-private variables and name mangling)
2) http://docs.python.org/library/functions.html#property
I see a lot of python code that does "loose" private variables / functions. They'll declare functions / variables with one underscore (eg. _foo) and then use it just in the class / file. It really annoys me that they don't use the double underscore because eventually, someone will call this "private" member from outside the class.
Is there some way to enforce the privacy on single underscores (without changing to doubles)?
Thanks!
No. And that's the philosophy of python: Do not make the compiler/parser enforce privacy since developers who want to access private members have ways to do so anyway (reflection etc.).
The idea is telling people Hey, this is not part of the public API. If you use it improperly, it could breaks things or kill your cat. I might also change the signature often since it's not part of the public API and I don't have to care about people using it
And fyi, you can actually access double-underscore variables (same applies for methods) from outside using obj._ClassName__variableName. Besides that it's not encouraged to use double underscores except for mixin objects - you can never know if someone wants to subclass your class.
In Python, there is no strict concept(like Java for example) of private methods. Even using double underscores is still accessible by doing _classname__method.
In short, don't go against the grain, rather go with the Python philosophy in which private is a convention, not a force.
Note: I'm not talking about preventing the rebinding of a variable. I'm talking about preventing the modification of the memory that the variable refers to, and of any memory that can be reached from there by following the nested containers.
I have a large data structure, and I want to expose it to other modules, on a read-only basis. The only way to do that in Python is to deep-copy the particular pieces I'd like to expose - prohibitively expensive in my case.
I am sure this is a very common problem, and it seems like a constant reference would be the perfect solution. But I must be missing something. Perhaps constant references are hard to implement in Python. Perhaps they don't quite do what I think they do.
Any insights would be appreciated.
While the answers are helpful, I haven't seen a single reason why const would be either hard to implement or unworkable in Python. I guess "un-Pythonic" would also count as a valid reason, but is it really? Python does do scrambling of private instance variables (starting with __) to avoid accidental bugs, and const doesn't seem to be that different in spirit.
EDIT: I just offered a very modest bounty. I am looking for a bit more detail about why Python ended up without const. I suspect the reason is that it's really hard to implement to work perfectly; I would like to understand why it's so hard.
It's the same as with private methods: as consenting adults authors of code should agree on an interface without need of force. Because really really enforcing the contract is hard, and doing it the half-assed way leads to hackish code in abundance.
Use get-only descriptors, and state clearly in your documentation that these data is meant to be read only. After all, a determined coder could probably find a way to use your code in different ways you thought of anyways.
In PEP 351, Barry Warsaw proposed a protocol for "freezing" any mutable data structure, analogous to the way that frozenset makes an immutable set. Frozen data structures would be hashable and so capable being used as keys in dictionaries.
The proposal was discussed on python-dev, with Raymond Hettinger's criticism the most detailed.
It's not quite what you're after, but it's the closest I can find, and should give you some idea of the thinking of the Python developers on this subject.
There are many design questions about any language, the answer to most of which is "just because". It's pretty clear that constants like this would go against the ideology of Python.
You can make a read-only class attribute, though, using descriptors. It's not trivial, but it's not very hard. The way it works is that you can make properties (things that look like attributes but call a method on access) using the property decorator; if you make a getter but not a setter property then you will get a read-only attribute. The reason for the metaclass programming is that since __init__ receives a fully-formed instance of the class, you actually can't set the attributes to what you want at this stage! Instead, you have to set them on creation of the class, which means you need a metaclass.
Code from this recipe:
# simple read only attributes with meta-class programming
# method factory for an attribute get method
def getmethod(attrname):
def _getmethod(self):
return self.__readonly__[attrname]
return _getmethod
class metaClass(type):
def __new__(cls,classname,bases,classdict):
readonly = classdict.get('__readonly__',{})
for name,default in readonly.items():
classdict[name] = property(getmethod(name))
return type.__new__(cls,classname,bases,classdict)
class ROClass(object):
__metaclass__ = metaClass
__readonly__ = {'a':1,'b':'text'}
if __name__ == '__main__':
def test1():
t = ROClass()
print t.a
print t.b
def test2():
t = ROClass()
t.a = 2
test1()
While one programmer writing code is a consenting adult, two programmers working on the same code seldom are consenting adults. More so if they do not value the beauty of the code but them deadlines or research funds.
For such adults there is some type safety, provided by Enthought's Traits.
You could look into Constant and ReadOnly traits.
For some additional thoughts, there is a similar question posed about Java here:
Why is there no Constant feature in Java?
When asking why Python has decided against constant references, I think it's helpful to think of how they would be implemented in the language. Should Python have some sort of special declaration, const, to create variable references that can't be changed? Why not allow variables to be declared a float/int/whatever then...these would surely help prevent programming bugs as well. While we're at it, adding class and method modifiers like protected/private/public/etc. would help enforce compile-type checking against illegal uses of these classes. ...pretty soon, we've lost the beauty, simplicity, and elegance that is Python, and we're writing code in some sort of bastard child of C++/Java.
Python also currently passes everything by reference. This would be some sort of special pass-by-reference-but-flag-it-to-prevent-modification...a pretty special case (and as the Tao of Python indicates, just "un-Pythonic").
As mentioned before, without actually changing the language, this type of behaviour can be implemented via classes & descriptors. It may not prevent modification from a determined hacker, but we are consenting adults. Python didn't necessarily decide against providing this as an included module ("batteries included") - there was just never enough demand for it.
I have a class called Path for which there are defined about 10 methods, in a dedicated module Path.py. Recently I had a need to write 5 more methods for Path, however these new methods are quite obscure and technical and 90% of the time they are irrelevant.
Where would be a good place to put them so their context is clear? Of course I can just put them with the class definition, but I don't like that because I like to keep the important things separate from the obscure things.
Currently I have these methods as functions that are defined in a separate module, just to keep things separate, but it would be better to have them as bound methods. (Currently they take the Path instance as an explicit parameter.)
Does anyone have a suggestion?
If the method is relevant to the Path - no matter how obscure - I think it should reside within the class itself.
If you have multiple places where you have path-related functionality, it leads to problems. For example, if you want to check if some functionality already exists, how will a new programmer know to check the other, less obvious places?
I think a good practice might be to order functions by importance. As you may have heard, some suggest putting public members of classes first, and private/protected ones after. You could consider putting the common methods in your class higher than the obscure ones.
If you're keen to put those methods in a different source file at any cost, AND keen to have them at methods at any cost, you can achieve both goals by using the different source file to define a mixin class and having your Path class import that method and multiply-inherit from that mixin. So, technically, it's quite feasible.
However, I would not recommend this course of action: it's worth using "the big guns" (such as multiple inheritance) only to serve important goals (such as reuse and removing repetition), and separating methods out in this way is not really a particularly crucial goal.
If those "obscure methods" played no role you would not be implementing them, so they must have SOME importance, after all; so I'd just clarify in docstrings and comments what they're for, maybe explicitly mention that they're rarely needed, and leave it at that.
I would just prepend the names with an underscore _, to show that the reader shouldn't bother about them.
It's conventionally the same thing as private members in other languages.
Put them in the Path class, and document that they are "obscure" either with comments or docstrings. Separate them at the end if you like.
Oh wait, I thought of something -- I can just define them in the Path.py module, where every obscure method will be a one-liner that will call the function from the separate module that currently exists. With this compromise, the obscure methods will comprise of maybe 10 lines in the end of the file instead of 50% of its bulk.
I suggest making them accessible from a property of the Path class called something like "Utilties". For example: Path.Utilities.RazzleDazzle. This will help with auto-completion tools and general maintenance.