How can I make methods and data members private in Python? Or doesn't Python support private members?
9.6. Private Variables
“Private” instance variables that
cannot be accessed except from inside
an object, don’t exist in Python.
However, there is a convention that is
followed by most Python code: a name
prefixed with an underscore (e.g.
_spam) should be treated as a non-public part of the API (whether it
is a function, a method or a data
member). It should be considered an
implementation detail and subject to
change without notice.
Since there is a valid use-case for
class-private members (namely to avoid
name clashes of names with names
defined by subclasses), there is
limited support for such a mechanism,
called name mangling. Any identifier
of the form __spam (at least two
leading underscores, at most one
trailing underscore) is textually
replaced with _classname__spam, where
classname is the current class name
with leading underscore(s) stripped.
This mangling is done without regard
to the syntactic position of the
identifier, as long as it occurs
within the definition of a class.
So, for example,
class Test:
def __private_symbol(self):
pass
def normal_symbol(self):
pass
print dir(Test)
will output:
['_Test__private_symbol',
'__doc__',
'__module__',
'normal_symbol']
__private_symbol should be considered a private method, but it would still be accessible through _Test__private_symbol.
The other answers provide the technical details. I'd like to emphasise the difference in philosophy between Python on one hand and languages like C++/Java (which I presume you're familiar with based on your question).
The general attitude in Python (and Perl for that matter) is that the 'privacy' of an attribute is a request to the programmer rather than a barbed wire fence by the compiler/interpreter. The idea is summarised well in this mail and is often referred to as "We're all consenting adults" since it 'assumes' that the programmer is responsible enough to not meddle with the insides. The leading underscores serve as a polite message saying that the attribute is internal.
On the other hand, if you do want to access the internals for some applications (a notable example is documentation generators like pydoc), you're free to do so. Onus is on you as a programmer to know what you're doing and do it properly rather than on the language to force you do to things it's way.
There are no private of any other access protection mechanisms in Python. There is a convention documented in the Python style guide for indicating to the users of your your class that they should not be accessing certain attribute.
_single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.
single_trailing_underscore_: used by convention to avoid conflicts with Python keyword, e.g. Tkinter.Toplevel(master, class_='ClassName')
__double_leading_underscore: when naming a class attribute, invokes name mangling (inside class FooBar, __boo becomes _FooBar__boo; see below).
If the name of a Python function,
class method, or attribute starts with
(but doesn't end with) two
underscores, it's private; everything
else is public. Python has no concept
of protected class methods (accessible
only in their own class and descendant
classes). Class methods are either
private (accessible only in their own
class) or public (accessible from
anywhere).
Dive Into Python
Python does not support privacy directly . Programmer need to know when it is safe to modify attribute from outside but anyway with python you can achieve something like private with little tricks.
Now let's see a person can put anything private to it or not.
class Person(object):
def __priva(self):
print "I am Private"
def publ(self):
print " I am public"
def callpriva(self):
self.__priva()
Now When we will execute :
>>> p = Person()
>>> p.publ()
I am public
>>> p.__priva()
Traceback (most recent call last):
File "", line 1, in
p.__priva()
AttributeError: 'Person' object has no attribute '__priva'
#Explanation : You can see here we are not able to fetch that private method directly.
>>> p.callpriva()
I am Private
#Explanation : Here we can access private method inside class
Then how someone can access that variable ???
You can do like :
>>> p._Person__priva
I am Private
wow , actually if python is getting any variable starting with double underscore are “translated” by adding a single underscore and the class name to the beginning:
Note : If you do not want this name changing but you still want to send a signal for other objects to stay away, you can use a single initial underscore names with an initial underscore aren’t imported with starred imports (from module import *)
Example :
#test.py
def hello():
print "hello"
def _hello():
print "Hello private"
#----------------------
#test2.py
from test import *
print hello()
print _hello()
output-->
hello
Traceback (most recent call last):
File "", line 1, in
NameError: name '_hello' is not defined
Now if we will call _hello manually .
#test2.py
from test import _hello , hello
print hello()
print _hello()
output-->
hello
hello private
Finally : Python doesn’t really have an equivalent privacy support, although single
and double initial underscores do to some extent give you two levels of privacy
This might work:
import sys, functools
def private(member):
#functools.wraps(member)
def wrapper(*function_args):
myself = member.__name__
caller = sys._getframe(1).f_code.co_name
if (not caller in dir(function_args[0]) and not caller is myself):
raise Exception("%s called by %s is private"%(myself,caller))
return member(*function_args)
return wrapper
class test:
def public_method(self):
print('public method called')
#private
def private_method(self):
print('private method called')
t = test()
t.public_method()
t.private_method()
This is kinda a l-o-n-g answer but I think it gets to the root of the real problem here -- scope of visibility. Just hang in there while I slog through this!
Simply importing a module need not necessarily give the application developer access to all of its classes or methods; if I can't actually SEE the module source code how will I know what's available? Some one (or some THING) has to tell me what I can do and explain how to use those features I'm allowed to use, otherwise the whole thing is useless to me.
Those developing higher-level abstractions based on fundamental classes and methods via imported modules are presented with a specification DOCUMENT -- NOT the actual source code.
The module spec describes all the features intended to be visible to the client developer. When dealing with large projects and software project teams, the actual implementation of a module should ALWAYS remain hidden from those using it -- it's a blackbox with an interface to the outside world. For OOD purists, I believe the techie terms are "decoupling" and "coherence". The module user need only know the interface methods without being burden with the details of implementation.
A module should NEVER be changed without first changing its underlying spec document, which may require review / approval in some organizations prior to changing the code.
As hobby programmer (retired now), I start a new module with the spec doc actually written out as a giant comment block at the top of the module, this will be the part the user actually sees in the spec library. Since it's just me, I've yet to set up a library, but it would be easy enough to do.
Then I begin coding by writing the various classes and methods but without functional bodies -- just null print statements like "print()" -- just enough to allow the module to compile without syntax errors. When this step is complete I compile the completed null-module -- this is my spec. If I were working on a project team, I would present this spec/interface for review & commentary before proceeding with fleshing out the body.
I flesh out the bodies of each method one at a time and compile accordingly, ensuring syntax errors are fixed immediately on-the-fly. This is also a good time to start writing a temporary "main" execution section at the bottom to test each method as you code it. When the coding/testing are complete, all of the test code is commented out until you need it again should updates become necessary.
In a real-world development team, the spec comment block would also appear in a document control library, but that's another story. The point is: you, as the module client, see only this spec and NOT the source code.
PS: long before the beginning of time, I worked in the defense aerospace community and we did some pretty cool stuff, but things like proprietary algorithms and sensitive systems control logic were tightly vaulted and encrypted in super-duper secure software libraries. We had access to module / package interfaces but NOT the blackbox implementation bodies. There was a document management tool that handled all system-level designs, software specs, source code and test records -- it was all synched together. The government had strict requirements software quality assurance standards. Anyone remember a language called "Ada"? That's how old I am!
import inspect
class Number:
def __init__(self, value):
self.my_private = value
def set_private(self, value):
self.my_private = value
def __setattr__(self, my_private, value):
f = inspect.stack()[1][3]
if f not in ['__init__', 'set_private']:
raise Exception("can't access private member-my_private")
# the default behavior
self.__dict__[my_private] = value
def main():
n = Number(2)
print(n.my_private)
n.set_private(3)
print(n.my_private)
if __name__ == '__main__':
main()
I use Python 2.7 and 3.5. I wrote this code:
class MyOBject(object):
def __init__(self):
self.__private_field = 10
my_object = MyOBject()
print(my_object.__private_field)
ran it and got:
AttributeError: 'MyOBject' object has no attribute '__private_field'
Please see:
https://www.tutorialsteacher.com/python/private-and-protected-access-modifiers-in-python
Related
Should I use underscore before function name in single-file simple python script?
For example, in this script should I add underscore before definition of f and g? This script won't be imported from other files.
def f(x): # or _f(x)?
return x * 2
def g(x): # or _g(x)?
return x ** 2
def main():
x = f(100)
print(g(x))
if __name__ == "__main__":
main()
I read many documents about usage of underscores in python. Many of them says underscores are about OOP-style programming and how import statement works. However, in simple one-file script, I can't find good answer.
What is better pattern?
The most popular programming languages (such as Java and C/C++) have this kind of syntax to declare private attributes and methods of a class instance:
class MyClass {
private:
int _function() {
// Some code here
}
public:
bool _foo() {
// Some code here
}
}
Python doesn't have (and will probably never have) a syntax like this, so they simply create a conventional name to make developers understand that a method is private and should never be accessed from outside the class.
According to PEP-8:
We don't use the term "private" here, since no attribute is really private in Python (without a generally unnecessary amount of work).
According to Python 2 and Python 3 class documentation:
“Private” instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member).
Using _ may be useful for a tecnique explained below.
According to Python 3 class documentation:
Since there is a valid use-case for class-private members (namely to avoid name clashes of names with names defined by subclasses), there is limited support for such a mechanism, called name mangling.
NOTE: The Python documentation mostly talks about "internal use" referring to classes, but it can be referred to modules too, for example:
# main.py
import myModule
functionThatCanBeImported()
# myModule.py
def functionThatCanBeImported(): pass
def _functionThatShouldNeverBeImported(): pass
If somebody creates a package, they don't have to put private functions in the documentation, since they are for internal scope and explainations about them could be useful only to developers.
_ is used to let developers know that the variables and methods are private and should not be modified externally. So use _ if you want to keep the functionality within the class.
NOTE: using _ will not restrict the use of these methods or variables though, it's just a way of representation.
The Python PEP 8 style guide gives the following guidance for a single leading underscore in method names:
_single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose names start with an underscore.
What constitutes "internal use"?
Is this for methods only called within a given class?
MyClass:
def _internal_method(self):
# do_something
def public_method(self):
self._internal_method()
What about inherited methods - are they still considered "internal"?
BaseClass:
def _internal_method(self):
# do something
MyClass(BaseClass):
def public_method(self):
self._internal_method() # or super()._internal_method()
What about if the inheritance is from another module within a software package?
file1.py
BaseClass:
def _internal_method(self):
# do something
file2.py
from file1 import BaseClass
MyClass(BaseClass):
def public_method(self):
self._internal_method() # or super()._internal_method()
All these examples are fine technically, but are they all acceptable stylistically? At what point do you say the leading underscore is not necessary/helpful?
A single leading underscore is Python's convention for "private" and "protected" variables, available as hard-implementations in some other languages.
The "internal use" language is just to say that you are reserving that name, as developer, to be used by your code as you want, and other users of your module/code can't rely on the thing tied to that name to behave the same way in further versions, or even to exist. It is just the use case for "protected" attributes, but without a hard-implementation from the language runtime: users are supposed to know that attribute/function/method can be changed without any previous warning.
So, yes, as long as other classes using your _ prefixed methods are on the same code package - even if on other file, or folder (other completly distinct package), it is ok to use them.
If you have different Python packages, even if closely related, it would not be advisable to call directly on the internal stuff on the other package, style-wise.
And as for limits, sometimes there are entire modules and classes that are not supposed to be used by users of your class - and it would be somewhat impairing to prefix everything on those modules with an _ - I'd say that it is enough to document what public interfaces to your package users are supposed to call, and add on the docs that certain parts (modules/classes/functions) are designed for "internal use and may change without note" - no need to meddle with their names.
As an illustration, I am currently developing a set of tools/library for text-art on the terminal - I put everything users should call as public names in its __init__.py - the remaining names are meant to be "internal".
I'd like to provide documentation (within my program) on certain dynamically created objects, but still fall back to using their class documentation. Setting __doc__ seems a suitable way to do so. However, I can't find many details in the Python help in this regard, are there any technical problems with providing documentation on an instance? For example:
class MyClass:
"""
A description of the class goes here.
"""
a = MyClass()
a.__doc__ = "A description of the object"
print( MyClass.__doc__ )
print( a.__doc__ )
__doc__ is documented as a writable attribute for functions, but not for instances of user defined classes. pydoc.help(a), for example, will only consider the __doc__ defined on the type in Python versions < 3.9.
Other protocols (including future use-cases) may reasonably bypass the special attributes defined in the instance dict, too. See Special method lookup section of the datamodel documentation, specifically:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.
So, depending on the consumer of the attribute, what you intend to do may not be reliable. Avoid.
A safe and simple alternative is just to use a different attribute name of your own choosing for your own use-case, preferably not using the __dunder__ syntax convention which usually indicates a special name reserved for some specific use by the implementation and/or the stdlib.
There are some pretty obvious technical problems; the question is whether or not they matter for your use case.
Here are some major uses for docstrings that your idiom will not help with:
help(a): Type help(a) in an interactive terminal, and you get the docstring for MyClass, not the docstring for a.
Auto-generated documentation: Unless you write your own documentation generator, it's not going to understand that you've done anything special with your a value. Many doc generators do have some way to specify help for module and class constants, but I'm not aware of any that will recognize your idiom.
IDE help: Many IDEs will not only auto-complete an expression, but show the relevant docstring in a tooltip. They all do this statically, and without some special-case code designed around your idiom (which they're unlikely to have, given that it's an unusual idiom), they're almost certain to fetch the docstring for the class, not the object.
Here are some where it might help:
Source readability: As a human reading your source, I can tell the intent from the a.__doc__ = … right near the construction of a. Then again, I could tell the same intent just as easily from a Sphinx comment on the constant.
Debugging: pdb doesn't really do much with docstrings, but some GUI debuggers wrapped around it do, and most of them are probably going to show a.__doc__.
Custom dynamic use of docstrings: Obviously any code that you write that does something with a.__doc__ is going to get the instance docstring if you want it to, and therefore can do whatever it wants with it. However, keep in mind that if you want to define your own "protocol", you should use your own name, not one reserved for the implementation.
Notice that most of the same is true for using a descriptor for the docstring:
>>> class C:
... #property
... def __doc__(self):
... return('C doc')
>>> c = C()
If you type c.__doc__, you'll get 'C doc', but help(c) will treat it as an object with no docstring.
It's worth noting that making help work is one of the reasons some dynamic proxy libraries generate new classes on the fly—that is, a proxy to underlying type Spam has some new type like _SpamProxy, instead of the same GenericProxy type used for proxies to Hams and Eggseses. The former allows help(myspam) to show dynamically-generated information about Spam. But I don't know how important a reason it is; often you already need dynamic classes to, e.g., make special method lookup work, at which point adding dynamic docstrings comes for free.
I think it's preferred to keep it under the class via your doc string as it will also aid any developer that works on the code. However if you are doing something dynamic that requires this setup then I don't see any reason why not. Just understand that it adds a level of indirection that makes things less clear to others.
Remember to K.I.S.S. where applicable :)
I just stumbled over this and noticed that at least with python 3.9.5 the behavior seems to have changed.
E.g. using the above example, when I call:
help(a)
I get:
Help on MyClass in module __main__:
<__main__.MyClass object>
A description of the object
Also for reference, have a look at the pydoc implementation which shows:
def _getowndoc(obj):
"""Get the documentation string for an object if it is not
inherited from its class."""
try:
doc = object.__getattribute__(obj, '__doc__')
if doc is None:
return None
if obj is not type:
typedoc = type(obj).__doc__
if isinstance(typedoc, str) and typedoc == doc:
return None
return doc
except AttributeError:
return None
I found from the internet:
public="a" # is a public variable
_protected="b" # is a protected variable
__private="c" # is a private variable
Example of code:
class c1:
def __init__(self,a,b,c):
self.public=a
self._protected=b
self.__private=c
v=c1("A","B","C")
print(v.public)
v._protected="Be" # !??? I can access a protected variable
print(v._protected)
print(v.__private) # !??? AttributeError: 'c1' object has no attribute '__private'
I can access a protected variable!?
No, Python does not have access modifiers which outright prevent access. But then again, most languages don't. Even languages which sport protected and private keywords usually have some way through introspection or such to get at the value anyway in ways which "should not be allowed."
Access modifiers are, one way or another, just a hint as to how the property is supposed to be used.
Python's philosophy is to assume that everyone contributing code is a responsible adult, and that a hint in the form of one or two underscores is perfectly enough to prevent "unauthorised access" to a property. If it starts with an underscore, you probably shouldn't mess with it.
You could acess your protected variable as:
print(v._c1__private)
'C'
Python doesn't have modifiers like private, protected, public. You can emulate their behavior with __getattr__ and __getattribute__, but it's not a Pythonic way to write programs.
Using single underscore _ is a convention, so when you see an attribute or method starting with one underscore, consider that library developer didn't expect it to be part of public API. Also when executing from module import * Python interpreter doesn't import names starting with _, though there're ways to modify this behavior.
Using double underscore __ is not just a convention, it leads to "name-mangling" by interpreter - adding class name in front of attribute, so i.e.:
class Klass():
def __like_private():
print("Hey!")
will make _Klass__like_private.
You won't be able to access __like_private directly as defined, though you still will be able to get to it knowing how names are composed by using _Klass__like_private() in subclasses for example or in module:
Klass.__like_private() will give you an error.
Klass._Klass__like_private() will print Hey!.
we have a module of static methods in our python app. these methods use a lot of private (e.g: "__do_sub_task2(**args)") I would like to write unit tests for these private static methods within this module, but I am getting refernce errors.
is there a way to do this?
update: adding scenario
I have a module file named 'my_module.py'
contents of said file is as follows:
def public_method_foo(my_number):
return __sub_method_bar(my_number * 10)
def __sub_method_bar(other_number)
return other_number + 11
update #2
The reason I am asking this question is because I have a similar scenario as above, but when I add the following reference to my test.py module file:
from my_module import __sub_method_bar
and try to use it in my test, I get the following exception in my test
global name '_MyTests__sub_method_bar' is not defined
What you have are not methods, not private, and not static; they're just plain old public functions in the module. So you call them the same way as any other function. For your example:
>>> my_module.__sub_method_bar(5)
That's it; nothing tricky.*
* Well, there is one tricky thing, but it's probably not going to affect you here: If my_module doesn't have an __all__, and you do from my_module import *, you will not get any of the globals (including functions) whose names start with _. But normally your unit tests are going to import my_module, so this won't be relevant.
Methods are callables that are members of a class. And methods can be private ("private" in this sense means "visible only to this class, not even to super- or sub-classes", so it doesn't make sense for anything but methods). The tutorial chapter on Classes explains how private methods are implemented, with name-mangling. Methods (private or otherwise) can also be static ("static" in this context means "does not take the normal self", so again, it doesn't make sense for anything but methods). Either way, for a private method, you have to manually demangle the name to call it from outside:
>>> thingy = Thingy()
>>> thingy._Thingy__private_method(5)
>>> Thingy._Thingy__private_static_method(5)