We Keep Coding

Overwriting a class attribute after returning it in method

I have a class method which constructs a numpy array as a class attribute. I want this attribute to be set to None after it is returned. I'm using an attribute instead of a variable because the creating the array is done in a parallel way. These arrays can be very large so I want to avoid having it in memory twice. What would be the best way to do this?
import numpy as np
class foo:
def __init__(self):
self.values = None
def bar(self):
self.values = np.arange(1e9) # Large array
return self.values
self.values = None # To be set to None after returning

Python does not copy objects on assignment, it only passes a reference to the same object.
return np.arange(1e9) will simply return a reference to the created array. Creating any new variable holding it, like var = np.arange(1e9), will just assign a reference of the array to the variable.
You're welcome to visualize it if it helps.

what is proper way to update(add or remove elements) global numpy array between multiple class?

i have this example code.
np_arr = np.array([1,2,3,4,5,6,7]) # declare global numpy array
class A: # declare a class
def __init__(self, np_arr):
self.np_arr = np_arr
obj1 = A(np_arr=np_arr) # Create object1
obj2 = A(np_arr=np_arr) # Create object2
np_arr = np.append(np_arr,8) # update global numpy array
print(obj1.np_arr) # check reflect changes in object1
print(obj2.np_arr) # check reflect changes in object1
in above example code, i have created one global numpy array then create one class and creates its two object by passing same global numpy array. after that i update that numpy array by adding one more element and after that i print both array form class object. but in this example, value is not getting updated in both class instances. however, when i change existing element value and then print both array form class object. it works.
so my question is what is proper way to update(add or remove elements) numpy array in this situation to get reflation of changes in multiple class object shared numpy arrays?

np.append creates a copy of the array when called, so when you call it you are creating a new array and assigning it to the global reference np_arr, but not updating the reference in your objects.
You can create a static class variable, which will be the same for all objects of a given class like this:
class A: # declare a class
np_arr = np.array([1,2,3,4,5,6,7]) # declare class numpy array
def __init__(self):
pass
obj1 = A()
obj2 = A()
A.np_arr = np.append(A.np_arr, 8)
print(obj1.np_arr) # check reflect changes in object1
print(obj2.np_arr) # check reflect changes in object2

Create a numpy array with a user-defined method which assign a value

I need to create an empty numpy array (of unknown shape, at the creation time), and to be able to use specific methods (named "upload" and "download") in order to update its values and retrieve them.
Some more context: opencv has a class called cuda_GpuMat, which when you want to perform calculations on the GPU, you suppose to first create an instance of one, and then assign the np.array you wish to work on, using the upload() method, for example:
import numpy as np
import cv2
import cv2.cuda as cvc
x = np.arange(3)
x_gpu = cv2.cuda_GpuMat()
x_gpu.upload(x)
x_gpu = cvc.multiply(x_gpu, x_gpu)
x_gpu.download()
output:
array([[0],
[1],
[4]], dtype=int32)
I want a compatible class for regular cpu, so I can run the same code (only change: import cv2.cuda as cvc to import cv2 as cvc).
I read about ndarray subclassing but couldn't figure out how to do it properly.
what have I tried?
import cv2
import numpy as np
class CpuMat(np.ndarray):
def __new__(subtype, shape=0, **kwargs):
obj = super(CpuMat, subtype).__new__(subtype, shape, **kwargs)
obj.emptyflag = True
return obj
def __array_finalize__(self, obj):
if obj is None: return
def upload(self, img):
self.img = img
self.emptyflag = False
def download(self):
return self.img
def empty(self):
return self.emptyflag
Obviously this wouldn't work because, as in the example above, the cv2 functions operate on the array itself, but in my implementation the array is stored in the "img" attribute of the object.
I've tried several versions of the above (for example, I tried to define a repr() function of the class to return self.img, but it have to be a string so it didn't work), but I'm not sure I'm in the right direction to the solution.
So any help here would be great.

Python inheritance for lists of instances

I have a python object, with many attributes and functions (dummy example bellow):
class molecule:
def __init__(self, atoms, coords):
self.atoms=np.copy(atoms)
self.coords=np.copy(coords)
def shift(self,r):
self.coords=self.coords+r
I would like to generate preferably a numpy array (or a list) of these objects and to obtain its properties without always looping over the array. At the moment I create a list of molecule objects (mols) by a loop and check its attributes by loops eg:
atomList=[mol.atoms for mol in mols]
but I would prefer to obtain it as:
atomList=mols.atoms
Is there an automatic way to obtain such an array/list class without manually defining the molList class and manually add its attributes, functions etc?

You can use a class_variable. The difference between class variables and instance variables can be found here:
https://medium.com/python-features/class-vs-instance-variables-8d452e9abcbd#:~:text=Class%20variables%20are%20shared%20across,surprising%20behaviour%20in%20our%20code.
For your example, something like this ought to work:
class molecule:
atomList = [] # class variable
def __init__(self, atoms, coords):
self.atoms=np.copy(atoms) # instance variable
self.coords=np.copy(coords)
molecule.atomList.append(atoms) # update the class variable with each new instance of the class
def shift(self,r):
self.coords=self.coords+r
Then in your code, you can just do atomlist = molecule.atomList

Python ORM to NumPy arrays

I am building data simulation framework with numpy ORM, where it is much more convenient to work with classes and objects instead of numpy arrays directly. Nevertheless, output of the simulation should be numpy array. Also blockz is quite interesting as a backend here.
I would like to map all object attributes to numpy arrays. Thus, numpy arrays work like a column-oriented "persistent" storage for my classes. I also need to link "new" attributes to objects which I can calculate using numpy(pandas) framework. And then just link them to objects accordingly using the same back-end.
Is there any solution for such approach? Would you recommend any way to build it in a HPC way?
I have found only django-pandas. PyTables is quite slow on adding new columns-attributes.
Something like (working on pointers to np_array):
class Instance()
def __init__(self, np_array, np_position):
self.np_array = np_array
self.np_position = np_position
def get_test_property():
return(self.np_array[np_position])
def set_test_property(value):
self.np_array[np_position] = value

In fact there is a way to change NumPy or bcolz arrays by reference.
Simple example can be found in the following code.
a = np.arange(10)
class Case():
def __init__(self, gcv_pointer):
self.gcv = gcv_pointer
def gcv(self):
return(self.gcv)
def gcv_set(self, value):
self.gcv[:] = value
pass
#===============================================================================
# NumPy
#===============================================================================
caseList = []
for i in range(1, 10):
case = Case(a[i-1:i])
caseList.append(case)
gcvs = [case.GetGCV() for case in caseList]
caseList[1].SetGCV(5)
caseList[1].SetGCV(13)
caseList[1].gcv[:] = 6
setattr(caseList[1], 'dpd', a[5:6])
caseList[1].dpd
caseList[1].dpd[:] = 888