So I have a class, which I'm using as a local namespace. I have some static functions in the class, but they can't access the class scope variables. Why is this?
class Foo:
foo_string = "I am a foo"
#staticmethod
def foo():
print foo_string
>>> Foo.foo()
[Stack Trace]
NameError: global name 'foo_string' is not defined
Any thoughts?
Python doesn't let class variables fall into scope this way, there are two ways to do this, the first is to use a class method:
#classmethod
def foo(cls):
print(cls.foo_string)
Which I would argue is the best solution.
The second is to access by name:
#staticmethod
def foo():
print(Foo.foo_string)
Do note that in general, using a class as a namespace isn't the best way to do it, simply use a module with top-level functions instead, as this acts more as you want to.
The reason for the lack of scoping like this is mainly due to Python's dynamic nature, how would it work when you insert a function into the class? It would have to have special behaviour added to it conditionally, which would be extremely awkward to implement and potentially fragile. It also helps keep things explicit rather than implicit - it's clear what is a class variable as opposed to a local variable.
Related
I am coding a small Python module composed of two parts:
some functions defining a public interface,
an implementation class used by the above functions, but which is not meaningful outside the module.
At first, I decided to "hide" this implementation class by defining it inside the function using it, but this hampers readability and cannot be used if multiple functions reuse the same class.
So, in addition to comments and docstrings, is there a mechanism to mark a class as "private" or "internal"? I am aware of the underscore mechanism, but as I understand it it only applies to variables, function and methods name.
Use a single underscore prefix:
class _Internal:
...
This is the official Python convention for 'internal' symbols; "from module import *" does not import underscore-prefixed objects.
Reference to the single underscore convention.
In short:
You cannot enforce privacy. There are no private classes/methods/functions in Python. At least, not strict privacy as in other languages, such as Java.
You can only indicate/suggest privacy. This follows a convention. The Python convention for marking a class/function/method as private is to preface it with an _ (underscore). For example, def _myfunc() or class _MyClass:. You can also create pseudo-privacy by prefacing the method with two underscores (for example, __foo). You cannot access the method directly, but you can still call it through a special prefix using the classname (for example, _classname__foo). So the best you can do is indicate/suggest privacy, not enforce it.
Python is like Perl in this respect. To paraphrase a famous line about privacy from the Perl book, the philosophy is that you should stay out of the living room because you weren't invited, not because it is defended with a shotgun.
For more information:
Private variables Python Documentation
Why are Python’s ‘private’ methods not actually private? Stack Overflow question 70528
Define __all__, a list of names that you want to be exported (see documentation).
__all__ = ['public_class'] # don't add here the 'implementation_class'
A pattern that I sometimes use is this:
Define a class:
class x(object):
def doThis(self):
...
def doThat(self):
...
Create an instance of the class, overwriting the class name:
x = x()
Define symbols that expose the functionality:
doThis = x.doThis
doThat = x.doThat
Delete the instance itself:
del x
Now you have a module that only exposes your public functions.
The convention is prepend "_" to internal classes, functions, and variables.
To address the issue of design conventions, and as chroder said, there's really no such thing as "private" in Python. This may sound twisted for someone coming from C/C++ background (like me a while back), but eventually, you'll probably realize following conventions is plenty enough.
Seeing something having an underscore in front should be a good enough hint not to use it directly. If you're concerned with cluttering help(MyClass) output (which is what everyone looks at when searching on how to use a class), the underscored attributes/classes are not included there, so you'll end up just having your "public" interface described.
Plus, having everything public has its own awesome perks, like for instance, you can unit test pretty much anything from outside (which you can't really do with C/C++ private constructs).
Use two underscores to prefix names of "private" identifiers. For classes in a module, use a single leading underscore and they will not be imported using "from module import *".
class _MyInternalClass:
def __my_private_method:
pass
(There is no such thing as true "private" in Python. For example, Python just automatically mangles the names of class members with double underscores to be __clssname_mymember. So really, if you know the mangled name you can use the "private" entity anyway. See here. And of course you can choose to manually import "internal" classes if you wanted to).
In fact you can achieve something similar to private members by taking advantage of scoping. We can create a module-level class that creates new locally-scoped variables during creation of the class, then use those variables elsewhere in that class.
class Foo:
def __new__(cls: "type[Foo]", i: int, o: object) -> "Foo":
_some_private_int: int = i
_some_private_obj: object = o
foo = super().__new__(cls)
def show_vars() -> None:
print(_some_private_int)
print(_some_private_obj)
foo.show_vars = show_vars
return foo
def show_vars(self: "Foo") -> None:
pass
We can then do, e.g.
foo = Foo(10, {"a":1})
foo.show_vars()
# 10
# {'a': 1}
Alternatively, here's a poor example that creates a class in a module that has access to variables scoped to the function in which the class is created. Do note that this state is shared between all instances (so be wary of this specific example). I'm sure there's a way to avoid this, but I'll leave that as an exercise for someone else.
def _foo_create():
_some_private_int: int
_some_private_obj: object
class Foo:
def __init__(self, i: int, o: object) -> None:
nonlocal _some_private_int
nonlocal _some_private_obj
_some_private_int = i
_some_private_obj = o
def show_vars(self):
print(_some_private_int)
print(_some_private_obj)
import sys
sys.modules[__name__].Foo = Foo
_foo_create()
As far as I am aware, there is not a way to gain access to these locally-scoped variables, though I'd be interested to know otherwise, if it is possible.
I'm new to Python but as I understand it, Python isn't like Java.
Here's how it happens in Python:
class Student:
__schoolName = 'XYZ School' # private attribute
def __nameprivamethod(self): # private function
print('two underscore')
class Student:
_schoolName = 'XYZ School' # protected attribute
Don't to check how to access the private and protected parts.
Why is this decorator strategy considered bad? (..or is it!?)
class User(object):
def __init__(self):
self.thing = 5
def __atomic_rate_change(fn):
def wrapper(self,*args,**kwargs):
print "start magic"
self.thing += 1
fn(self,*args,**kwargs)
print "end magic"
return wrapper
#__atomic_rate_change
def foo(self,add):
print self.__atomic_rate_change # <bound method User.__atomic_rate_change of <__main__.User object at 0x6ffffe1ef50>>
self.thing += add
print "normal call {0}".format(self.thing)
test = User()
test.foo(1)
This works. But, according to resource below, it's bad practice. Reasons would be that:
[...] there is major flaw in this approach: atomic_rating_change becomes an
instance method of the User class. That doesn’t make any sense. More
than this, it doesn’t even work as a method: if you call it, the
decorated parameter will be used as self.
https://medium.com/#vadimpushtaev/decorator-inside-python-class-1e74d23107f6
I don't understand why it's a problem/wrong/bad that atomic_rate_change is a instance method. I'm only intending the decorator to be used within the class. Perhaps in this case it's okay?
Stylistically, placing function definitions into the class definition which are not methods are kind of out of place (imho it can even be unpythonic). Flat is better than nested, so it is probably better to declare the function outside of the class. This way when the reader is looking at your class, there won't be the confusion of why there is a method which does not take self as an argument (because the function is declared like a method when it is merely a decorator, though this is slightly different if the function is a #staticmethod).
If you're worried about it being used outside of the class, prefix it with an _ and then from my_package import * won't import it. It can still be used in that module, but it won't be used outside unless explicitly imported.
Practically, the author is referring to the occasional odd behavior of scoping (similar to the debates in Javascript on whether to use function() { ... or () => { ... based on how things are scoped.) If you're not careful and accidentally have logic involving self in the wrong part of your decorator, you could have scoping issues.
The only advantages I can see to using functions inside of the classes are possibly because it is closer to the methods (but that introduces unnecessary nesting, potential scoping problems, and cognitive load of realizing that's a decorator and not a method), and better hiding of the function if it's name startswith _ or __.
TL;DR Stylistic/Pythonicity concerns, and potential scoping issues.
I have this:
class MyClass:
"""A simple example class"""
i = 12345
def f(self):
print i # self.i will work just fine
return 'hello world'
When I do:
>>> x = MyClass()
>>>
>>> x.f()
I get an error, as expected.
My question is:
Why do I get the error?
Why is there no namespace between the namespace of the function(or method) definition and the global namespace of the module containing the class?
Is there any other way to reference i inside f in this case other than using self?
You've got an error because print i is trying to print a global (for the module) variable i. If you want to use the member of the class you should write self.i.
Because Guido Van Rossum decided not to use namespaces. Actually, I don't know how to answer anymore.
Yes, but no. Yes, because you can use "reflection" (I can't remember how it is called in python) and access any member of the class. No, because using self is the only usable way.
Why do I get the error?
Answer: This is because the variable i & the symbol/level f which points to a function/method code are in MyClass namespace where as the value/callable code/body of the function f lies in global namespace.
Why is there no namespace between the namespace of the function(or
method) definition and the global namespace of the module containing
the class?
Answer: Not sure why the python designer decided to put function name/symbol in the class but its body in global namespace.
Is there any other way to reference i inside f in this case other than
using self?
Answer: I don't think so, but even if it is then it might not be easier logic/code
Reading the documentation I came across the following paragraph:
A scope defines the visibility of a name within a block. If a local
variable is defined in a block, its scope includes that block. If the
definition occurs in a function block, the scope extends to any blocks
contained within the defining one, unless a contained block introduces
a different binding for the name. The scope of names defined in a
class block is limited to the class block; it does not extend to the
code blocks of methods – this includes comprehensions and generator
expressions since they are implemented using a function scope.
I decided to try accessing class variable from a method myself:
>>> class A():
i = 1
def f(self):
print(i)
>>> a = A()
>>> a.i
1
>>> a.f()
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
a.f()
File "<pyshell#4>", line 4, in f
print(i)
NameError: global name 'i' is not defined
I know that the variable i may be accessed by explicitly pointing to the class name A.i:
>>> a = A()
>>> class A():
i = 1
def f(self):
print(A.i)
>>> a = A()
>>> a.f()
1
The question is why the developers of the language made class variables not visible from methods? What is the rationale behind it?
A class block is syntactic sugar for building a dictionary, which is then passed to the metaclass (usually type) to construct the class object.
class A:
i = 1
def f(self):
print(i)
Is roughly equivalent to:
def f(self):
print(i)
attributes = {"f": f, "i": 1}
A = type("A", (object,), attributes)
Seen that way, there is no outer scope the i name to come from. However there obviously is a temporary scope for you to execute the statements in the class block. It would be possible for that class block to desugar to something more like:
def attributes():
i = 1
def f(self):
print(i)
return locals()
A = type('A', (object,), attributes())
In this case the outer reference to i would work. However, this would be going "against the grain" of Python's object system philosophy.
Python has objects, which contain attributes. There's not really any concept of "variables" other than local variables in functions (which can be nested to create a scope chain). A bare name is looked up as a local variable, then in outer scopes (which come from functions). Attributes are looked up, using the dotted name syntax, on other objects, and you always specify which object to look in.
There is a protocol for resolving attribute references, which says that when attribute is not found on obj, obj.attribute can be resolved by looking in the class of obj (and its base classes, using the method resolution order). This is actually how methods are found; when in your example you executed a.f(), the a object contains no attribute f, so the class of a (which is A) is searched, and the method definition is found.
Having class attributes automatically available in an outer scope for all methods would be weird, because no other attribute works this way. It would also have the following drawbacks:
Functions defined outside the class and assigned to it later would have to use different syntax to refer to the class attribute than functions defined as part of a class.
Because it's shorter, it would encourage reference to class attributes including staticmethods and classmethods as bare names: thing rather than using Class.thing or self.thing. This makes them look like module globals when they're not (method definitions are usually short enough that you can easily see they're not defined locally).
Note that looking for the attributes on self allows them to play nicer with subclasses, as it allows subclasses to override the attribute. That probably isn't as big a deal for "class constants", but it's very important for staticmethods and classmethods.
Those are the main reasons I see, but ultimately it's just a choice the designers of Python made. You find it weird that you don't have this implicit ability to reference class variables, but I find implicit class and instance variable access in languages like C++ and Java to be weird. Different people have different opinions.
This seems to be related to the use of an explicit self parameter, and the requirement that all method calls and instance attribute accesses explicitly use self. It would be at least strange if the uncommon case of accessing a class scope function as a normal function would be much easier than the common case of accessing it as a method via self. Class variables are usually also accessed via the instance in Python.
In C++, in contrast, the class scope is visibile in all methods, but calling a method implicitly passes this. This seems to be the other sane choice.
Both of these blocks of code work. Is there a "right" way to do this?
class Stuff:
def __init__(self, x = 0):
global globx
globx = x
def inc(self):
return globx + 1
myStuff = Stuff(3)
print myStuff.inc()
Prints "4"
class Stuff:
def __init__(self, x = 0):
self.x = x
def inc(self):
return self.x + 1
myStuff = Stuff(3)
print myStuff.inc()
Also prints "4"
I'm a noob, and I'm working with a lot of variables in a class. Started wondering why I was putting "self." in front of everything in sight.
Thanks for your help!
You should use the second way, then every instance has a separate x
If you use a global variable then you may find you get surprising results when you have more than one instance of Stuff as changing the value of one will affect all the others.
It's normal to have explicit self's all over your Python code. If you try tricks to avoid that you will be making your code difficult to read for other Python programmers (and potentially introducing extra bugs)
There are 2 ways for "class scope variables". One is to use self, this is called instance variable, each instance of a class has its own copy of instance variables; another one is to define variables in the class definition, this could be achieved by:
class Stuff:
globx = 0
def __init__(self, x = 0):
Stuff.globx = x
...
This is called class attribute, which could be accessed directly by Stuff.globx, and owned by the class, not the instances of the class, just like the static variables in Java.
you should never use global statement for a "class scope variable", because it is not. A variable declared as global is in the global scope, e.g. the namespace of the module in which the class is defined.
namespace and related concept is introduced in the Python tutorial here.
Those are very different semantically. self. means it's an instance variable, i.e. each instance has its own. This is propably the most common kind, but not the only one. And then there are class variables, defined at class level (and therefore by the time the class definition is executed) and accessable in class methods. The equivalent to most uses of static methods, and most propably what you want when you need to share stuff between instances (this is perfectly valid, although not automatically teh one and only way for a given problem). You propably want one of those, depending on what you're doing. Really, we can't read your mind and tell you which one fits your problem.
Globals variables are a different story. They're, well, global - everyone has the same one. This is almost never a good idea (for reasons explained on many occasions), but if you're just writing a quick and dirty script and need share something between several places, they can be acceptable.