In Python, when I'm defining a function inside a class, I can include self as one of the arguments to access the member variables of that class, and I can also choose not include self as the argument if I don't need to access its member variables. But then I discovered that if the function does not have self as arguments, then it becomes invisible to other functions in that class. For example
class Test:
def __init__(self, val:int) -> None:
self.a = val
def f(a:int) -> int:
return a + 1
def g(self) -> int:
return f(self.a)
if __name__ == '__main__':
t = Test(2)
print(t.g())
The above codes will lead to the following error message:
Traceback (most recent call last):
File "/Users/louchenfei/Downloads/lc.py", line 11, in <module>
print(t.g())
File "/Users/louchenfei/Downloads/lc.py", line 7, in g
return f(self.a)
NameError: name 'f' is not defined
I wonder why that's the case and what are the rules for visibilities of functions defined in a class?
Its difficult to know what you want, but one solution is to make f a staticmethod. Still you need to have a reference to Test in some way:
class Test:
def __init__(self, val:int) -> None:
self.a = val
#staticmethod
def f(a:int) -> int:
return a + 1
def g(self) -> int:
return self.f(self.a)
if __name__ == '__main__':
t = Test(2)
print(t.g())
Here, the call is self.f(self.a).
The method f is not in scope inside another method.
I have a class with several methods. Outputs of a method are used in other methods. I don't want to pass these variables as input argument to other methods (to make code more simple).
I can add output of this method into self so I have access to these variables in other methods.
But, I want to be sure that it is a standard implementation. I am worried that it may cause unpredictable error. If you have experience in this regard, please let me know if the following example is a correct implementation or not.
class MyClass:
def method_1(self, A):
return A + 1
def method_2(self):
return self.B + 10
def method_3(self, C):
self.B = self.method_1(C)
result = self.method_2()
return result
z = MyClass()
z.method_3(1)
In the above example, I don't need to pass self.B into method_2. This code works but I want to be sure that it is a standard way.
The real program I working on is complicated, so I made a simple example for this question.
Yup it is more or less correct but the standard way of doing something like this is having a __init__() method and using function annotations.
class MyClass:
def __init__(self) -> None:
self.B = 0
def method_1(self, A: int) -> int:
return A + 1
def method_2(self) -> int:
return self.B + 10
def method_3(self, C: int) -> int:
self.B = self.method_1(C)
result = self.method_2()
return result
z = MyClass()
z.method_3(1)
Where method_2() relies on an attribute that may be unset, make it private so that people aren't tempted to use it. For example, what if I did this?
>>> z = MyClass()
>>> z.method_2()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tmp.py", line 9, in method_2
return self.B + 10
AttributeError: 'MyClass' object has no attribute 'B'
For that matter, it's probably best to make the attribute private too. So:
class MyClass:
def method_1(self, A):
return A + 1
def _method_2(self):
return self._B + 10
def method_3(self, C):
self._B = self.method_1(C)
result = self._method_2()
return result
By the way, where method_1() doesn't use self, consider making it a staticmethod.
Is there a magic method that can overload the assignment operator, like __assign__(self, new_value)?
I'd like to forbid a re-bind for an instance:
class Protect():
def __assign__(self, value):
raise Exception("This is an ex-parrot")
var = Protect() # once assigned...
var = 1 # this should raise Exception()
Is it possible? Is it insane? Should I be on medicine?
The way you describe it is absolutely not possible. Assignment to a name is a fundamental feature of Python and no hooks have been provided to change its behavior.
However, assignment to a member in a class instance can be controlled as you want, by overriding .__setattr__().
class MyClass(object):
def __init__(self, x):
self.x = x
self._locked = True
def __setattr__(self, name, value):
if self.__dict__.get("_locked", False) and name == "x":
raise AttributeError("MyClass does not allow assignment to .x member")
self.__dict__[name] = value
>>> m = MyClass(3)
>>> m.x
3
>>> m.x = 4
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __setattr__
AttributeError: MyClass does not allow assignment to .x member
Note that there is a member variable, _locked, that controls whether the assignment is permitted. You can unlock it to update the value.
No, as assignment is a language intrinsic which doesn't have a modification hook.
I don't think it's possible. The way I see it, assignment to a variable doesn't do anything to the object it previously referred to: it's just that the variable "points" to a different object now.
In [3]: class My():
...: def __init__(self, id):
...: self.id=id
...:
In [4]: a = My(1)
In [5]: b = a
In [6]: a = 1
In [7]: b
Out[7]: <__main__.My instance at 0xb689d14c>
In [8]: b.id
Out[8]: 1 # the object is unchanged!
However, you can mimic the desired behavior by creating a wrapper object with __setitem__() or __setattr__() methods that raise an exception, and keep the "unchangeable" stuff inside.
Inside a module, this is absolutely possible, via a bit of dark magic.
import sys
tst = sys.modules['tst']
class Protect():
def __assign__(self, value):
raise Exception("This is an ex-parrot")
var = Protect() # once assigned...
Module = type(tst)
class ProtectedModule(Module):
def __setattr__(self, attr, val):
exists = getattr(self, attr, None)
if exists is not None and hasattr(exists, '__assign__'):
exists.__assign__(val)
super().__setattr__(attr, val)
tst.__class__ = ProtectedModule
The above example assumes the code resides in a module named tst. You can do this in the repl by changing tst to __main__.
If you want to protect access through the local module, make all writes to it through tst.var = newval.
Using the top-level namespace, this is impossible. When you run
var = 1
It stores the key var and the value 1 in the global dictionary. It is roughly equivalent to calling globals().__setitem__('var', 1). The problem is that you cannot replace the global dictionary in a running script (you probably can by messing with the stack, but that is not a good idea). However you can execute code in a secondary namespace, and provide a custom dictionary for its globals.
class myglobals(dict):
def __setitem__(self, key, value):
if key=='val':
raise TypeError()
dict.__setitem__(self, key, value)
myg = myglobals()
dict.__setitem__(myg, 'val', 'protected')
import code
code.InteractiveConsole(locals=myg).interact()
That will fire up a REPL which almost operates normally, but refuses any attempts to set the variable val. You could also use execfile(filename, myg). Note this doesn't protect against malicious code.
I will burn in Python hell, but what's life without a little fun.
Important disclaimers:
I only provide this example for fun
I'm 100% sure I don't understand this well
It might not even be safe to do this, in any sense
I don't think this is practical
I don't think this is a good idea
I don't even want to seriously try to implement this
This doesn't work for jupyter (probably ipython too)*
Maybe you can't overload assignment, but you can (at least with Python ~3.9) achieve what you want even at the top-level namespace. It will be hard doing it "properly" for all cases, but here's a small example by hacking audithooks:
import sys
import ast
import inspect
import dis
import types
def hook(name, tup):
if name == "exec" and tup:
if tup and isinstance(tup[0], types.CodeType):
# Probably only works for my example
code = tup[0]
# We want to parse that code and find if it "stores" a variable.
# The ops for the example code would look something like this:
# ['LOAD_CONST', '<0>', 'STORE_NAME', '<0>',
# 'LOAD_CONST', 'POP_TOP', 'RETURN_VALUE', '<0>']
store_instruction_arg = None
instructions = [dis.opname[op] for op in code.co_code]
# Track the index so we can find the '<NUM>' index into the names
for i, instruction in enumerate(instructions):
# You might need to implement more logic here
# or catch more cases
if instruction == "STORE_NAME":
# store_instruction_arg in our case is 0.
# This might be the wrong way to parse get this value,
# but oh well.
store_instruction_arg = code.co_code[i + 1]
break
if store_instruction_arg is not None:
# code.co_names here is: ('a',)
var_name = code.co_names[store_instruction_arg]
# Check if the variable name has been previously defined.
# Will this work inside a function? a class? another
# module? Well... :D
if var_name in globals():
raise Exception("Cannot re-assign variable")
# Magic
sys.addaudithook(hook)
And here's the example:
>>> a = "123"
>>> a = 123
Traceback (most recent call last):
File "<stdin>", line 21, in hook
Exception: Cannot re-assign variable
>>> a
'123'
*For Jupyter I found another way that looked a tiny bit cleaner because I parsed the AST instead of the code object:
import sys
import ast
def hook(name, tup):
if name == "compile" and tup:
ast_mod = tup[0]
if isinstance(ast_mod, ast.Module):
assign_token = None
for token in ast_mod.body:
if isinstance(token, ast.Assign):
target, value = token.targets[0], token.value
var_name = target.id
if var_name in globals():
raise Exception("Can't re-assign variable")
sys.addaudithook(hook)
No there isn't
Think about it, in your example you are rebinding the name var to a new value.
You aren't actually touching the instance of Protect.
If the name you wish to rebind is in fact a property of some other entity i.e
myobj.var then you can prevent assigning a value to the property/attribute of the entity.
But I assume thats not what you want from your example.
Yes, It's possible, you can handle __assign__ via modify ast.
pip install assign
Test with:
class T():
def __assign__(self, v):
print('called with %s' % v)
b = T()
c = b
You will get
>>> import magic
>>> import test
called with c
The project is at https://github.com/RyanKung/assign
And the simpler gist: https://gist.github.com/RyanKung/4830d6c8474e6bcefa4edd13f122b4df
Generally, the best approach I found is overriding __ilshift__ as a setter and __rlshift__ as a getter, being duplicated by the property decorator.
It is almost the last operator being resolved just (| & ^) and logical are lower.
It is rarely used (__lrshift__ is less, but it can be taken to account).
Within using of PyPi assign package only forward assignment can be controlled, so actual 'strength' of the operator is lower.
PyPi assign package example:
class Test:
def __init__(self, val, name):
self._val = val
self._name = name
self.named = False
def __assign__(self, other):
if hasattr(other, 'val'):
other = other.val
self.set(other)
return self
def __rassign__(self, other):
return self.get()
def set(self, val):
self._val = val
def get(self):
if self.named:
return self._name
return self._val
#property
def val(self):
return self._val
x = Test(1, 'x')
y = Test(2, 'y')
print('x.val =', x.val)
print('y.val =', y.val)
x = y
print('x.val =', x.val)
z: int = None
z = x
print('z =', z)
x = 3
y = x
print('y.val =', y.val)
y.val = 4
output:
x.val = 1
y.val = 2
x.val = 2
z = <__main__.Test object at 0x0000029209DFD978>
Traceback (most recent call last):
File "E:\packages\pyksp\pyksp\compiler2\simple_test2.py", line 44, in <module>
print('y.val =', y.val)
AttributeError: 'int' object has no attribute 'val'
The same with shift:
class Test:
def __init__(self, val, name):
self._val = val
self._name = name
self.named = False
def __ilshift__(self, other):
if hasattr(other, 'val'):
other = other.val
self.set(other)
return self
def __rlshift__(self, other):
return self.get()
def set(self, val):
self._val = val
def get(self):
if self.named:
return self._name
return self._val
#property
def val(self):
return self._val
x = Test(1, 'x')
y = Test(2, 'y')
print('x.val =', x.val)
print('y.val =', y.val)
x <<= y
print('x.val =', x.val)
z: int = None
z <<= x
print('z =', z)
x <<= 3
y <<= x
print('y.val =', y.val)
y.val = 4
output:
x.val = 1
y.val = 2
x.val = 2
z = 2
y.val = 3
Traceback (most recent call last):
File "E:\packages\pyksp\pyksp\compiler2\simple_test.py", line 45, in <module>
y.val = 4
AttributeError: can't set attribute
So <<= operator within getting value at a property is the much more visually clean solution and it is not attempting user to make some reflective mistakes like:
var1.val = 1
var2.val = 2
# if we have to check type of input
var1.val = var2
# but it could be accendently typed worse,
# skipping the type-check:
var1.val = var2.val
# or much more worse:
somevar = var1 + var2
var1 += var2
# sic!
var1 = var2
In the global namespace this is not possible, but you could take advantage of more advanced Python metaprogramming to prevent multiple instances of a the Protect object from being created. The Singleton pattern is good example of this.
In the case of a Singleton you would ensure that once instantiated, even if the original variable referencing the instance is reassigned, that the object would persist. Any subsequent instances would just return a reference to the same object.
Despite this pattern, you would never be able to prevent a global variable name itself from being reassigned.
As mentioned by other people, there is no way to do it directly. It can be overridden for class members though, which is good for many cases.
As Ryan Kung mentioned, the AST of a package can be instrumented so that all assignments can have a side effect if the class assigned implements specific method(s). Building on his work to handle object creation and attribute assignment cases, the modified code and a full description is available here:
https://github.com/patgolez10/assignhooks
The package can be installed as: pip3 install assignhooks
Example <testmod.py>:
class SampleClass():
name = None
def __assignpre__(self, lhs_name, rhs_name, rhs):
print('PRE: assigning %s = %s' % (lhs_name, rhs_name))
# modify rhs if needed before assignment
if rhs.name is None:
rhs.name = lhs_name
return rhs
def __assignpost__(self, lhs_name, rhs_name):
print('POST: lhs', self)
print('POST: assigning %s = %s' % (lhs_name, rhs_name))
def myfunc():
b = SampleClass()
c = b
print('b.name', b.name)
to instrument it, e.g. <test.py>
import assignhooks
assignhooks.instrument.start() # instrument from now on
import testmod
assignhooks.instrument.stop() # stop instrumenting
# ... other imports and code bellow ...
testmod.myfunc()
Will produce:
$ python3 ./test.py
POST: lhs <testmod.SampleClass object at 0x1041dcc70>
POST: assigning b = SampleClass
PRE: assigning c = b
POST: lhs <testmod.SampleClass object at 0x1041dcc70>
POST: assigning c = b
b.name b
Beginning Python 3.8, it is possible to hint that a value is read-only using typing.Final. What this means is that nothing changes at runtime, allowing anyone to change the value, but if you're using any linter that can read type-hints then it's going to warn the user if they attempt to assign it.
from typing import Final
x: Final[int] = 3
x = 5 # Cannot assign to final name "x" (mypy)
This makes for way cleaner code, but it puts full trust in the user to respect it at runtime, making no attempt to stop users from changing values.
Another common pattern is to expose functions instead of module constants, like sys.getrecursionlimit and sys.setrecursionlimit.
def get_x() -> int:
return 3
Although users can do module.get_x = my_get_x, there's an obvious attempt on the user's part to break it, which can't be fixed. In this way we can prevent people from "accidentally" changing values in our module with minimal complexity.
A ugly solution is to reassign on destructor. But it's no real overload assignment.
import copy
global a
class MyClass():
def __init__(self):
a = 1000
# ...
def __del__(self):
a = copy.copy(self)
a = MyClass()
a = 1
I am using a object as key and number as value but getting the below error in line. Any help ?
dict[a] = 1
:
Traceback (most recent call last):
File "detect_hung_connections.py", line 24, in <module>
dict = {a:1}
TypeError: __hash__() takes exactly 3 arguments (1 given)
My Code is as follows:
class A:
def __init__(self,a,b):
self.a = a
self.b = b
def __hash__(self,a,b):
return hash(self.a,self.b)
def __eq__(self,other):
return (self.a,self.b) == (other.a,other.b)
dict ={}
a = A("aa","bb")
dict[a] = 1
b = A("aa","bb")
The signature of A.__hash__ should not take any extra arguments.
def __hash__(self):
return hash((self.a,self.b))
You're calling hash with the entire object and (redundantly) its two attributes. You can use only a single value to hash. Try this, perhaps:
def __hash__(self):
return hash(self.a + self.b)
This at least passes execution.
I'm struggling to subclass my own subclass of numpy.ndarray. I don't really understand what the problem is and would like someone to explain what goes wrong in the following cases and how to do what I'm trying to do.
What I'm trying to achieve:
I have a subclass of numpy.ndarry that behaves as I want (class A in the code below). I want to subclass A (class B in the code below) so that B contains additional information (name) and methods (the decorated .simple_data method).
Case 1:
import numpy as np
class A(np.ndarray):
def __new__(cls,data):
obj = np.asarray(data).view(cls)
return obj
def __array_finalize(self,obj):
if obj is None: return
class B(A):
def __init__(self,data,name):
super(B,self).__init__(data)
self.name = name
#property
def simple_data(self):
return [data[0,:],data[:,0]]
if __name__ == '__main__':
data = np.arange(20).reshape((4,5))
b = B(data,'B')
print type(b)
print b.simple_data
Running this code produces the output:
Traceback (most recent call last):
File "ndsubclass.py", line 24, in <module>
b = B(data,'B')
TypeError: __new__() takes exactly 2 arguments (3 given)
I assume that this is related to the 'name' variable in the construction of B and that due to A being a subclass of numpy.array, A's new method is being called before B's init method. Thus to fix this I assume that B also needs a new method that appropriately handles the additional argument.
My guess is something like:
def __new__(cls,data,name):
obj = A(data)
obj.name = name
return obj
should do it, but how do I change the class of obj?
Case 2:
import numpy as np
class A(np.ndarray):
def __new__(cls,data):
obj = np.asarray(data).view(cls)
return obj
def __array_finalize__(self,obj):
if obj is None: return
class B(A):
def __new__(cls,data):
obj = A(data)
obj.view(cls)
return obj
def __array_finalize__(self,obj):
if obj is None: return
#property
def simple_data(self):
return [self[0,:],self[:,0]]
if __name__ == '__main__':
data = np.arange(20).reshape((4,5))
b = B(data)
print type(b)
print b.simple_data()
When run the output is:
<class '__main__.A'>
Traceback (most recent call last):
File "ndsubclass.py", line 30, in <module>
print b.simple_data()
AttributeError: 'A' object has no attribute 'simple_data'
This surprises me as I was expecting:
<class '__main__.B'>
[array([0, 1, 2, 3, 4]), array([ 0, 5, 10, 15])]
I assume that the call to view() in B.new() is somehow not correctly setting the class of obj. Why?
I'm confused as to what is going on and would be very grateful if someone could explain it.
For Case 1, the simplest way is:
class B(A):
def __new__(cls,data,name):
obj = A.__new__(cls, data)
obj.name = name
return obj
__new__ is actually a static method that takes a class as the first argument, not a class method, so you can call it directly with the class of which you want to create an instance.
For Case 2, view doesn't work in-place, you need to assign the result to something, the simplest way is:
class B(A):
def __new__(cls,data):
obj = A(data)
return obj.view(cls)
Also, you've got __array_finalize__ defined the same in A and B there (probably just a typo) -- you don't need to do that.