I'm relatively new to Python.
When I did C/C++ programming, I used the internal classes quite often. For example, in some_file.cc, we may implement a class in the anonymous namespace to prevent it from being used outside. This is useful as a helper class specific to that file.
Then, how we can do a similar thing in Python?
class MyClassOuter:
def __init__(self,...):
class MyClassInner:
def __init__(self,...):
pass
self.my_class = MyClassInner(..)
would only have MyClassInner available inside the __init__ function of MyClassOuter
likewise you could put it inside a function
def my_class_factory(arg1,arg2,...):
class MyClass:
def __init__(self,arg1,arg2,...):
print "OK??"
return MyClass(arg1,arg2,...)
Python code doesn't have any such equivalent for an anonymous namespace, or static linkage for functions. There are a few ways you can get what you're looking for
Prefix with _. Names beginning with an underscore are understood
to be for internal use to that python file and are not exported by
from * imports. it's as simple as class _MyClass.
Use __all__: If a python file contains a list a list of strings
named __all__, the functions and classes named within are
understood to be local to that python file and are not exported by
from *.
Use local classes/functions. This would be done the same way you've
done so with C++ classes.
None these gets exactly what you want, but privacy and restricting in this way are just not part of the language (much like how there's no private data member equivalent). Pydoc is also well aware of these conventions and will provide informative documentation for the intended-to-be-public functions and classes.
Related
I am coding a small Python module composed of two parts:
some functions defining a public interface,
an implementation class used by the above functions, but which is not meaningful outside the module.
At first, I decided to "hide" this implementation class by defining it inside the function using it, but this hampers readability and cannot be used if multiple functions reuse the same class.
So, in addition to comments and docstrings, is there a mechanism to mark a class as "private" or "internal"? I am aware of the underscore mechanism, but as I understand it it only applies to variables, function and methods name.
Use a single underscore prefix:
class _Internal:
...
This is the official Python convention for 'internal' symbols; "from module import *" does not import underscore-prefixed objects.
Reference to the single underscore convention.
In short:
You cannot enforce privacy. There are no private classes/methods/functions in Python. At least, not strict privacy as in other languages, such as Java.
You can only indicate/suggest privacy. This follows a convention. The Python convention for marking a class/function/method as private is to preface it with an _ (underscore). For example, def _myfunc() or class _MyClass:. You can also create pseudo-privacy by prefacing the method with two underscores (for example, __foo). You cannot access the method directly, but you can still call it through a special prefix using the classname (for example, _classname__foo). So the best you can do is indicate/suggest privacy, not enforce it.
Python is like Perl in this respect. To paraphrase a famous line about privacy from the Perl book, the philosophy is that you should stay out of the living room because you weren't invited, not because it is defended with a shotgun.
For more information:
Private variables Python Documentation
Why are Python’s ‘private’ methods not actually private? Stack Overflow question 70528
Define __all__, a list of names that you want to be exported (see documentation).
__all__ = ['public_class'] # don't add here the 'implementation_class'
A pattern that I sometimes use is this:
Define a class:
class x(object):
def doThis(self):
...
def doThat(self):
...
Create an instance of the class, overwriting the class name:
x = x()
Define symbols that expose the functionality:
doThis = x.doThis
doThat = x.doThat
Delete the instance itself:
del x
Now you have a module that only exposes your public functions.
The convention is prepend "_" to internal classes, functions, and variables.
To address the issue of design conventions, and as chroder said, there's really no such thing as "private" in Python. This may sound twisted for someone coming from C/C++ background (like me a while back), but eventually, you'll probably realize following conventions is plenty enough.
Seeing something having an underscore in front should be a good enough hint not to use it directly. If you're concerned with cluttering help(MyClass) output (which is what everyone looks at when searching on how to use a class), the underscored attributes/classes are not included there, so you'll end up just having your "public" interface described.
Plus, having everything public has its own awesome perks, like for instance, you can unit test pretty much anything from outside (which you can't really do with C/C++ private constructs).
Use two underscores to prefix names of "private" identifiers. For classes in a module, use a single leading underscore and they will not be imported using "from module import *".
class _MyInternalClass:
def __my_private_method:
pass
(There is no such thing as true "private" in Python. For example, Python just automatically mangles the names of class members with double underscores to be __clssname_mymember. So really, if you know the mangled name you can use the "private" entity anyway. See here. And of course you can choose to manually import "internal" classes if you wanted to).
In fact you can achieve something similar to private members by taking advantage of scoping. We can create a module-level class that creates new locally-scoped variables during creation of the class, then use those variables elsewhere in that class.
class Foo:
def __new__(cls: "type[Foo]", i: int, o: object) -> "Foo":
_some_private_int: int = i
_some_private_obj: object = o
foo = super().__new__(cls)
def show_vars() -> None:
print(_some_private_int)
print(_some_private_obj)
foo.show_vars = show_vars
return foo
def show_vars(self: "Foo") -> None:
pass
We can then do, e.g.
foo = Foo(10, {"a":1})
foo.show_vars()
# 10
# {'a': 1}
Alternatively, here's a poor example that creates a class in a module that has access to variables scoped to the function in which the class is created. Do note that this state is shared between all instances (so be wary of this specific example). I'm sure there's a way to avoid this, but I'll leave that as an exercise for someone else.
def _foo_create():
_some_private_int: int
_some_private_obj: object
class Foo:
def __init__(self, i: int, o: object) -> None:
nonlocal _some_private_int
nonlocal _some_private_obj
_some_private_int = i
_some_private_obj = o
def show_vars(self):
print(_some_private_int)
print(_some_private_obj)
import sys
sys.modules[__name__].Foo = Foo
_foo_create()
As far as I am aware, there is not a way to gain access to these locally-scoped variables, though I'd be interested to know otherwise, if it is possible.
I'm new to Python but as I understand it, Python isn't like Java.
Here's how it happens in Python:
class Student:
__schoolName = 'XYZ School' # private attribute
def __nameprivamethod(self): # private function
print('two underscore')
class Student:
_schoolName = 'XYZ School' # protected attribute
Don't to check how to access the private and protected parts.
I have come across a python project that commonly calls external functions from class methods and passes the class instance and some other parameters to the external function.
The method used is shown in method_one below and I have never come across this implementation before. Using locals to get both the local method parameters and the self class instance seems strange to say the least. The code then relies upon the dictionary keys being named correctly i.e. the same as the parameters of the external function (some_function).
To me, the obvious, simpler direct alternative is method_two but even over that I would prefer either
making some_function a method of ExampleClass1 so it has direct access to self, or
passing only the required attributes of the ExampleClass1 instance to some_function.
Example code:
class ExampleClass1(object):
def __init__(self, something):
self.something = something
def method_one(self, param_1, param_2):
all_params = locals()
all_params['example_self'] = all_params.pop('self')
some_function(**all_params)
def method_two(self, param_1, param_2):
some_function(self, param_1, param_2)
def some_function(example_self, param_1, param_2):
print(example_self.something, param_1, param_2)
e = ExampleClass1("do")
e.method_one(1, "a")
e.method_two(2, "b")
So,
Is there any reason to be using method_one that I'm not aware of?
How would you offer advice on the best practice for this situation?
Passing self as a parameter to external functions is a totally standard practice. I'm a little unclear why the call to locals() is used and why keys are being shuffled around, but that's a separate matter. In general, I find that if you're using locals(), then 9 times out of 10 the code you're writing can be simpler. Notable exception being metaprogramming, which is another topic.
One example that I use this for is when you want to separate out code into several modules rather than have one large class with a bunch of methods. There's a lot of ways to organize code, but one approach that I use is to segregate functions to other modules based on their domain, and then pass self to those functions for their use.
Concrete example: a server object accepting requests can have the routes handling those requests live elsewhere, and then delegate the actual business logic to the external route functions. If those routes need the server object, though, then you may want to pass self (being the server) to them. You could make an argument they should just be methods then, but that's a matter of code style and depends a lot on exact use case.
In general, passing self around isn't a bad practice when used appropriately.
I would like to convert a singleton-object programmatically into a Python module so that I can use the methods of this singleton-object directly by importing them via the module instead of accessing them as object attributes. By "programmatically" I mean that I do not want to have to copy-paste the class methods explicitly into a module file. I need some sort of a workaround that allows me to import the object methods into to global scope of another module.
I would really appreciate if someone could help me on this one.
Here is a basic example that should illustrate my problem:
mymodule.py
class MyClass:
"""This is my custom class"""
def my_method(self):
return "myValue"
singleton = MyClass()
main_as_is.py
from mymodule import MyClass
myobject = MyClass()
print(myobject.my_method())
main_to_be.py
from mymodule import my_method # or from mymodule.singleton import my_method
print(my_method())
You can use the same strategy that the standard random module uses. All the functions in that module are actually methods of a "private" instance of the Random class. That's convenient for most common uses of the module, although sometimes it's useful to create your own instances of Random so that you can have multiple independent random streams.
I've adapted your code to illustrate that technique. I named the class and its instance with a single leading underscore, since that's the usual convention in Python to signify a private name, but bear in mind it's simply a convention, Python doesn't do anything to enforce this privacy.
mymodule.py
class _MyClass:
""" This is my custom class """
def my_method(self):
return "myValue"
_myclass = _MyClass()
my_method = _myclass.my_method
main_to_be.py
from mymodule import my_method
print(my_method())
output
myValue
BTW, the from mymodule import method1, method2 syntax is ok if you only import a small number of names, or it's clear from the name which module it's from (like math module functions and constants), and you don't import from many modules. Otherwise it's better to use this sort of syntax
import mymodule as mm
# Call a method from the module
mm.method1()
That way it's obvious which names are local, and which ones are imported and where they're imported from. Sure, it's a little more typing, but it makes the code a whole lot more readable. And it eliminates the possibility of name collisions.
FWIW, here's a way to automate adding all of the _myclass methods without explicitly listing them (but remember "explicit is better than implicit"). At the end of "mymodule.py", in place of my_method = _myclass.my_method, add this:
globals().update({k: getattr(_myclass, k) for k in _MyClass.__dict__
if not k.startswith('__')})
I'm not comfortable with recommending this, since it directly injects items into the globals() dict. Note that that code will add all class attributes, not just methods.
In your question you talk about singleton objects. We don't normally use singletons in Python, and many programmers in various OOP languages consider them to be an anti-pattern. See https://stackoverflow.com/questions/12755539/why-is-singleton-considered-an-anti-pattern for details. For this application there is absolutely no need at all to use a singleton. If you only want a single instance of _MyClass then simply don't create another instance of it, just use the instance that mymodule creates for you. But if your boss insists that you must use a singleton, please see the example code here.
I'm very new to python so please don't be displeased if I missed some thing.
It seems that class and instance variables in python simply sprint into existence when they are used, this combination of use and declaration of instance variables results in the declaration of a class no longer being the only place that a class's data structures can be defined. consequently given a class Cls one cannot tell what data structures been created for it for one to access simply by tracing up stream to the class's declaration, as one can do with languages such as C and Java. Instead, one is compel to search all modules that might have used/define variables for that class.
As a result when a class is used by multiple modules, although you know of some data structure belong to that class must have been defined, to know it's name and type, you have to search through all the imported files that might have used that class (in a way that create new variables) to see just where was it used/defined.
example
declaration:
class Cls:
pass
module1:
Cls.m = 2.3
module_n:
print Cls.m
And again the issue being for those who operate on module_n, before they write that print statement, they don't know whether the variable name will be defined in module1 or some other module in the import list that could have used Cls , but if they can't find where the variable's defined they can't use it, since they don't know the name of that variable, or even its existence.
As someone less than a day into Python who is reading a rather lengthy project, I want to know if there's any trick to solve this, both as a reader and as the writer, how can such headaches be avoided
I am coding a small Python module composed of two parts:
some functions defining a public interface,
an implementation class used by the above functions, but which is not meaningful outside the module.
At first, I decided to "hide" this implementation class by defining it inside the function using it, but this hampers readability and cannot be used if multiple functions reuse the same class.
So, in addition to comments and docstrings, is there a mechanism to mark a class as "private" or "internal"? I am aware of the underscore mechanism, but as I understand it it only applies to variables, function and methods name.
Use a single underscore prefix:
class _Internal:
...
This is the official Python convention for 'internal' symbols; "from module import *" does not import underscore-prefixed objects.
Reference to the single underscore convention.
In short:
You cannot enforce privacy. There are no private classes/methods/functions in Python. At least, not strict privacy as in other languages, such as Java.
You can only indicate/suggest privacy. This follows a convention. The Python convention for marking a class/function/method as private is to preface it with an _ (underscore). For example, def _myfunc() or class _MyClass:. You can also create pseudo-privacy by prefacing the method with two underscores (for example, __foo). You cannot access the method directly, but you can still call it through a special prefix using the classname (for example, _classname__foo). So the best you can do is indicate/suggest privacy, not enforce it.
Python is like Perl in this respect. To paraphrase a famous line about privacy from the Perl book, the philosophy is that you should stay out of the living room because you weren't invited, not because it is defended with a shotgun.
For more information:
Private variables Python Documentation
Why are Python’s ‘private’ methods not actually private? Stack Overflow question 70528
Define __all__, a list of names that you want to be exported (see documentation).
__all__ = ['public_class'] # don't add here the 'implementation_class'
A pattern that I sometimes use is this:
Define a class:
class x(object):
def doThis(self):
...
def doThat(self):
...
Create an instance of the class, overwriting the class name:
x = x()
Define symbols that expose the functionality:
doThis = x.doThis
doThat = x.doThat
Delete the instance itself:
del x
Now you have a module that only exposes your public functions.
The convention is prepend "_" to internal classes, functions, and variables.
To address the issue of design conventions, and as chroder said, there's really no such thing as "private" in Python. This may sound twisted for someone coming from C/C++ background (like me a while back), but eventually, you'll probably realize following conventions is plenty enough.
Seeing something having an underscore in front should be a good enough hint not to use it directly. If you're concerned with cluttering help(MyClass) output (which is what everyone looks at when searching on how to use a class), the underscored attributes/classes are not included there, so you'll end up just having your "public" interface described.
Plus, having everything public has its own awesome perks, like for instance, you can unit test pretty much anything from outside (which you can't really do with C/C++ private constructs).
Use two underscores to prefix names of "private" identifiers. For classes in a module, use a single leading underscore and they will not be imported using "from module import *".
class _MyInternalClass:
def __my_private_method:
pass
(There is no such thing as true "private" in Python. For example, Python just automatically mangles the names of class members with double underscores to be __clssname_mymember. So really, if you know the mangled name you can use the "private" entity anyway. See here. And of course you can choose to manually import "internal" classes if you wanted to).
In fact you can achieve something similar to private members by taking advantage of scoping. We can create a module-level class that creates new locally-scoped variables during creation of the class, then use those variables elsewhere in that class.
class Foo:
def __new__(cls: "type[Foo]", i: int, o: object) -> "Foo":
_some_private_int: int = i
_some_private_obj: object = o
foo = super().__new__(cls)
def show_vars() -> None:
print(_some_private_int)
print(_some_private_obj)
foo.show_vars = show_vars
return foo
def show_vars(self: "Foo") -> None:
pass
We can then do, e.g.
foo = Foo(10, {"a":1})
foo.show_vars()
# 10
# {'a': 1}
Alternatively, here's a poor example that creates a class in a module that has access to variables scoped to the function in which the class is created. Do note that this state is shared between all instances (so be wary of this specific example). I'm sure there's a way to avoid this, but I'll leave that as an exercise for someone else.
def _foo_create():
_some_private_int: int
_some_private_obj: object
class Foo:
def __init__(self, i: int, o: object) -> None:
nonlocal _some_private_int
nonlocal _some_private_obj
_some_private_int = i
_some_private_obj = o
def show_vars(self):
print(_some_private_int)
print(_some_private_obj)
import sys
sys.modules[__name__].Foo = Foo
_foo_create()
As far as I am aware, there is not a way to gain access to these locally-scoped variables, though I'd be interested to know otherwise, if it is possible.
I'm new to Python but as I understand it, Python isn't like Java.
Here's how it happens in Python:
class Student:
__schoolName = 'XYZ School' # private attribute
def __nameprivamethod(self): # private function
print('two underscore')
class Student:
_schoolName = 'XYZ School' # protected attribute
Don't to check how to access the private and protected parts.