Is it possible to have serializable static class variables or methods in python?
As an example suppose, I have the following code snippet:
import pickle
class Sample:
count = 0 # class variable
def __init__(self, a1=0, a2=0):
self.a = a1
self.b = a2
Sample.count += 1
#MAIN
f = open("t1.dat", "wb")
d = dict()
for i in range(10):
s = Sample(i, i*i)
d[i] = s
pickle.dump(d,f)
print "Sample.count = " + str(Sample.count)
f.close()
The output is:
Sample.count = 10
Now, I have another reader program similar to above:
import pickle
class Sample:
count = 0 # class variable
def __init__(self, a1=0, a2=0):
self.a = a1
self.b = a2
Sample.count += 1
#MAIN
f = open("t1.dat", "rb")
d = pickle.load(f)
print "Sample.count = " + str(Sample.count)
The output is:
Sample.count = 0
My question is:
How do I load the class variable from my file? In other words, how do I serialize a class variable? If directly not possible, is there any alternative? Please suggest.
Since class variable cannot be picked, as an alternative, I have used the code snippet in main part when reading from the file as below:
#MAIN
f = open("t1.dat", "rb")
d = pickle.load(f)
Sample.count = len(d.values())
print "Sample.count = " + str(Sample.count)
The output is now:
Sample.count = 10
Is it acceptable solution? Any other alternative?
Quoting the section on "What can be pickled and unpickled?"
Similarly, classes are pickled by named reference, so the same restrictions in the unpickling environment apply. Note that none of the class’s code or data is pickled, so in the following example the class attribute attr is not restored in the unpickling environment:
class Foo:
attr = 'a class attr'
picklestring = pickle.dumps(Foo)
So because attr, or in your case count, is part of the class definition, it never gets pickled. In your 'write' example, you're printing Sample.count which does exist but is not pickled in the first place.
You could store Sample.count in each instance as _count and put Sample.count = self._count. But remember that since your d is a dict, they may unpickle in any order. So essentially this won't work.
You'll need to add __setstate__ to your class customize the way it pickles and put in some flag value (like _count) which you then manipulate (via whatever logic works consistently) in __getstate__. (Edit: doesn't help with the given problem unless you store count in a global variable and access that in getstate and manipulate further each time an object is unpickled.)
Another potential workaround but yuck: Add a variable to your dict d so that it also gets pickled. When you read it back, restore with Sample.count = d['_count']. So before pickle.dump(d,f) when you pickle, do d['_count'] = Sample.count.
Important caveat: This is not actually allowing you to pickle Sample.count since what you're actually pickling (d) is a dictionary of Samples.
Edit: The Sample.count = len(d.values()) which you've put as a workaround is very specific to your use case and not to class attr's in general.
Related
I have a class in Python that initializes the attributes of an environment. I am attempting to grab the topographyRegistry attribute list of my Environment class in a separate function, which when called, should take in the parameters of 'self' and the topography to be added. When this function is called, it should simply take an argument such as addTopographyToEnvironment(self, "Mountains") and append it to the topographyRegistry of the Environment class.
When implementing what I mentioned above, I ran into an error regarding the 'self' method not being defined. Hence, whenever I call the above line, it gives me:
print (Environment.addTopographyToEnvironment(self, "Mountains"))
^^^^
NameError: name 'self' is not defined
This leads me to believe that I am unaware of and missing a step in my implementation, but I am unsure of what that is exactly.
Here is the relevant code:
class EnvironmentInfo:
def __init__(self, perceivableFood, perceivableCreatures, regionTopography, lightVisibility):
self.perceivableFood = perceivableFood
self.perceivableCreatures = perceivableCreatures
self.regionTopography = regionTopography
self.lightVisibility = lightVisibility
class Environment:
def __init__(self, creatureRegistry, foodRegistry, topographyRegistery, lightVisibility):
logging.info("Creating new environment")
self.creatureRegistry = []
self.foodRegistry = []
self.topographyRegistery = []
self.lightVisibility = True
def displayEnvironment():
creatureRegistry = []
foodRegistry = []
topographyRegistery = ['Grasslands']
lightVisibility = True
print (f"Creatures: {creatureRegistry} Food Available: {foodRegistry} Topography: {topographyRegistery} Contains Light: {lightVisibility}")
def addTopographyToEnvironment(self, topographyRegistery):
logging.info(
f"Registering {topographyRegistery} as a region in the Environment")
self.topographyRegistery.append(topographyRegistery)
def getRegisteredEnvironment(self):
return self.topographyRegistry
if __name__ == "__main__":
print (Environment.displayEnvironment()) #Display hardcoded attributes
print (Environment.addTopographyToEnvironment(self, "Mountains"))#NameError
print (Environment.getRegisteredEnvironment(self)) #NameError
What am I doing wrong or not understanding when using 'self'?
Edit: In regard to omitting 'self' from the print statement, it still gives me an error indicating a TypeError:
print (Environment.addTopographyToEnvironment("Mountains"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Environment.addTopographyToEnvironment() missing 1 required positional argument: 'topographyRegistery'
Comments
Despite having def getRegisteredEnvironment(self): it wasn't indented, so it's not recognized as a class method.
self is a keyword used in conjunction with classes (class methods or attributes) - not functions. self is implied to be the instantiated object (eg a = Environment(...) -> self would refer to a) or the module's (I can't think of the proper term) class.
You didn't have your addTopographyToEnvironment class method defined.
In terms of your Environment class, you aren't using the variables you are passing to the class, so I made that change as well - I don't know if that was intentional or not.
As per your comment from the other answer, if you had def my_class_method(self) and you try to invoke it through an object with additional parameters, like so a = my_object(); a.my_class_method("Mountains"), you should get an error of the sorts, "2 positional arguments passed, expected 1.".
Your main problem is that you are doing Environment.class_method() and not creating an object from the class. Do a = Environment(whatever arguments here) to create an object from the class, then do a.addTopographyToEnvironment("Mountains") to do what you were going to do with "Mountains" and that object. What you have currently may be right, its just is missing the proper implementation, but the below article does a great job explaining the differences between all of them (Class Methods vs Static Methods vs Instance Methods), and is definitely worth the read.
class EnvironmentInfo:
def __init__(self, perceivableFood, perceivableCreatures, regionTopography, lightVisibility):
self.perceivableFood = perceivableFood
self.perceivableCreatures = perceivableCreatures
self.regionTopography = regionTopography
self.lightVisibility = lightVisibility
class Environment:
def __init__(self, creatureRegistry, foodRegistry, topographyRegistery, lightVisibility):
logging.info("Creating new environment")
self.creatureRegistry = creatureRegistry
self.foodRegistry = foodRegistry
self.topographyRegistery = topographyRegistery
self.lightVisibility = lightVisibility
def displayEnvironment(self):
creatureRegistry = []
foodRegistry = []
topographyRegistery = ['Grasslands']
lightVisibility = True
print (f"Creatures: {creatureRegistry} Food Available: {foodRegistry} Topography: {topographyRegistery} Contains Light: {lightVisibility}")
def addTopographyToEnvironment(self, environment):
return "Whatever this is supposed to return." + environment
def getRegisteredEnvironment(self):
return self.topographyRegistry
if __name__ == "__main__":
print (Environment.displayEnvironment()) #Display hardcoded attributes
print (Environment.addTopographyToEnvironment("Mountains"))#NameError
print (Environment.getRegisteredEnvironment()) #NameError
Object Instantiation In Python
With all that out of the way, I will answer the question as is posed, "Is there a way to grab list attributes that have been initialized using self and append data to them in Python?". I am assuming you mean the contents of the list and not the attributes of it, the attributes would be "got" or at least printed with dir()
As a simple example:
class MyClass:
def __init__(self, my_list):
self.my_list = my_list
if __name__ == "__main__":
a = MyClass([1, 2, 3, 4, 5])
print(a.my_list)
# will print [1, 2, 3, 4, 5]
a.my_list.append(6)
print(a.my_list)
# will print [1, 2, 3, 4, 5, 6]
print(dir(a.my_list))
# will print all object methods and object attributes for the list associated with object "a".
Sub Classing In Python
Given what you have above, it looks like you should be using method sub classing - this is done with the keyword super. From what I can guess, it would look like you'd implement that kind of like this:
class EnvironmentInfo:
def __init__(self, perceivableFood, perceivableCreatures, regionTopography, lightVisibility):
self.perceivableFood = perceivableFood
self.perceivableCreatures = perceivableCreatures
self.regionTopography = regionTopography
self.lightVisibility = lightVisibility
class Environment(EnvironmentInfo):
def __init__(self, creatureRegistry, foodRegistry, topographyRegistery, lightVisibility, someOtherThingAvailableToEnvironmentButNotEnvironmentInfo):
logging.info("Creating new environment")
super.__init__(foodRegistry, creatureRegistry, topographyRegistery, lightVisibility)
self.my_var1 = someOtherThingAvailableToEnvironmentButNotEnvironmentInfo
def displayEnvironment(self):
creatureRegistry = []
foodRegistry = []
topographyRegistery = ['Grasslands']
lightVisibility = True
print (f"Creatures: {creatureRegistry} Food Available: {foodRegistry} Topography: {topographyRegistery} Contains Light: {lightVisibility}")
def addTopographyToEnvironment(self, environment):
return "Whatever this is supposed to return." + environment
def getRegisteredEnvironment(self):
return self.topographyRegistry
def methodAvailableToSubClassButNotSuper(self)
return self.my_var1
if __name__ == "__main__":
a = Environment([], [], [], True, "Only accessible to the sub class")
print(a.methodAvailableToSubClassButNotSuper())
as the article describes when talking about super(), methods and attributes from the super class are available to the sub class.
Extra Resources
Class Methods vs Static Methods vs Instance Methods - "Difference #2: Method Defination" gives an example that would be helpful I think.
What is sub classing in Python? - Just glanced at it; probably an okay read.
Self represents the instance of the class and you don't have access to it outside of the class, by the way when you are calling object methods of a class you don't need to pass self cause it automatically be passed to the method you just need to pass the parameters after self so if you want to call an object method like addTopographyToEnvironment(self, newVal) you should do it like:
Environment.addTopographyToEnvironment("Mountains")
and it should work fine
I have a simple class that stores simple data. The class is as follows.
class DataFormater:
def __init__(self, N, P, K, price):
self.N = N
self.P = P
self.K = K
self.price = price
The code that calls this class is
from DataFormater import DataFormater
#global variables
ObjectList = [0,1,2,3,4,5,6,7,8,9,10,
11,12,13,14,15,16,17,18,19,20,
21,22,23,24,25,26,27,28,29,30,
31,32,33,34,35,36,37,38,39,40,
41,42,43,44,45,46,47,48,49,50]
ObjectListCounter = 0
# main
print "enter you N-P-K values, followed by a coma, then the price"
print "example ----> 5 5 5 %50 "
print "return as many values as you want to sort, then enter, 'done!' when done."
while True:
RawData = raw_input()
if RawData == 'done!':
break
else:
ObjectList[ObjectListCounter] = DataFormater
ObjectList[ObjectListCounter].N = int(RawData[0])
# very simple test way of putting first indice in ObjectList[ObjectListCounter].N
ObjectListCounter += 1
print ObjectList[0].N
print ObjectList[1].N
My idea is that ObjectList[0] would create that object '1' that I could call with 1.N
But, when I call these, it seems that I have overwritten the previous instances.
this is what prints...
return as many values as you want to sort, then enter, 'done!' when done.
12
1
done!
1
1
Thanks so much! And I know that my post is messy, I don't exactly know how to make it more "pretty"
So, it looks like you are assigning the actual class (instead of an instance of the class) in your loop. Where you do this:
ObjectList[ObjectListCounter] = DataFormater
I think what you actually want is this
ObjectList[ObjectListCounter] = DataFormater(...insert args here....)
EDIT to address the comments:
Your class init method looks like this:
def __init__(self, N, P, K, price):
That means that to create an instance of your class, it would look like this:
my_formater = DataFormater(1, 2, 3, 4)
You would then be able to access my_formater.N which would have a value of 1.
What you are trying to do instead is access a CLASS level attribute, DataFormater.N. This is generally used in situations where you have a constant variable that does not change between instances of the class. For example:
class DataFormater():
CONSTANT_THING = 'my thing that is always the same for every instance'
You would then be able to access that variable directly from the class, like this:
DataFormater.CONSTANT_THING
I hope that clears things up.
If I import a class and rename it by subclassing, it's fairly simple to find the new class name:
>>> from timeit import Timer
>>> class Test(Timer):
... pass
...
>>> test = Test()
>>> test.__class__.__name__
'Test'
However, if I alias the class as I import it, it retains the name from its host module:
>>> from timeit import Timer as Test2
>>> test2 = Test2()
>>> test2.__class__.__name__
'Timer'
Later, I want to provide user-facing output which is aware of the name they've given the class in their namespace. Consider:
def report_stats(timer):
print("Runtime statistics for %s:" % timer.__class__.__name__)
...
Is there a way to get a string reading "Test2", short of iterating over variables in the namespace to test for an exact match?
There's a really terrible answer to my own question; I won't be accepting this since it's probably pretty fragile (I only tested for a limited set of call circumstances). I mostly just hunted this down for the challenge; I will most likely be using something more durable for my actual use case.
This assumes we have access to the init function of the class we're trying to import as blah, and some sort of persistent external data store, at least for more complicated edge cases:
import inspect, dis
class Idiom(object):
description = None
alias = None
def __init__(self, desc):
global data_ob
self.description = desc
if self.__class__.__name__ == 'Idiom':
#cheat like hell to figure out who called us
self.alias = data_ob.name_idiom(inspect.currentframe().f_back)
else:
self.alias = self.__class__.__name__
class DataOb(object):
code = None
locations = {}
LOAD_NAME = 101
codelen = None
def name_idiom(self, frame):
if not self.code:
self.code = frame.f_code
self.codelen = len(self.code.co_code)
self.locations = {y:x for x, y in dis.findlinestarts(self.code)}
target_line = frame.f_lineno
addr_index = self.locations[target_line]+1
name_index = self.code.co_code[addr_index]
# there's a chance we'll get called again this line,
# so we want to seek to the next LOAD_NAME instance(101)
addr_index += 1
while addr_index < self.codelen:
if self.code.co_code[addr_index] == self.LOAD_NAME:
self.locations[target_line] = addr_index
break
addr_index += 1
return self.code.co_names[name_index]
The short explanation of how this works is:
we look up the previous frame from the init function
obtain the code object
find bytecode locations for the start of every line in the code
use the line-number from the frame to grab the bytecode location for the start of that line
locate a LOAD_NAME indicator in the bytecode for this line (I don't really follow this; my code assumes it'll be there)
look in the next bytecode position for an index which indicates which position in the code.co_names tuple contains the "name" of the LOAD_NAME call
From here we can do something like:
>>> from rabbit_hole import Idiom as timer_bob
>>> with timer_bob("down the rabbit hole"):
... waste_some_time = list(range(50000))
...
timer_bob: down the rabbit hole
runtime: 0:00:00.001909, children: 0:00:00, overhead: 0:00:00.001909
I'm stumped on a python problem. I'm writing a program that receives a command from Scratch (MIT) and then should create a new object, in this case named PiLight. The object only need to be created when the command is received so it doesn't have to loop, just be able to executed repeatedly and have the number increment each time it is executed.A list will not work for me due to the requirements of the program and talking between Scratch. I'm trying to figure out a way for the constructor, once initialized, to print out a statement something like
class Newpilight:
def __init__(self):
print "Pilight" + pilnumber + " created"
pilnumber should be 1 for 1st object, 2 for 2nd, etc
From there I need the creation of the object to change the number in the name of the object as well
PiLight(PiLnumber) = Newpilight()
I tried messing around with for loops but just ended up making more of a mess
Use number generator as class variable
from itertools import count
class NewPilight(object):
nums = count()
def __init__(self):
self.num = self.nums.next()
print "Pilight {self.num} created".format(self=self)
Then using in code:
>>> pl1 = NewPilight()
Pilight 0 created
>>> pl2 = NewPilight()
Pilight 1 created
>>> pl3 = NewPilight()
Pilight 2 created
>>> pl3.num
2
The trick is to have the nums (what is actually a generator of numbers, not list of numbers) as class property and not property of class instance. This way it is globally shared by all class instances.
class NewPilight:
def __init__(self, number):
self.number = number
print "Pilight" + number + " created"
for x in range(5):
NewPilight(x)
if you need to keep objects:
all_pilights = []
for x in range(5):
all_pilights.append( NewPilight(x) )
and now you have access to objects as
print all_pilights[0].number
print all_pilights[1].number
print all_pilights[2].number
class NewPiLight(object):
global_pilnumber = 0 # Since this is on the class definition, it is static
def __init__(self):
print "Pilight %s created" % NewPiLight.global_pilnumber
self.pilnumber = NewPiLight.global_pilnumber # Set the variable for this instance
NewPiLight.global_pilnumber += 1 # This increments the static variable
I will appreciate any effort to clarify the following: is there a way in Python to dynamically create one object per class, where several classes are declared? My silly guess can be described as following:
...
suppose we have some data from db
props = dict_cur.fetchall()
classes_names = []
data = []
for i in props:
classes_names.append(i['client_name'].title())
classes = []
data = []
for i in props:
data.append(dict(i))
for i, d in zip(classes_names, data):
classes.append(type(i, (object,), dict(**d)))
print classes
#printing list of classes
objects = []
for obj in classes:
objects.append(obj())
for obj in objects:
print obj.client_name, obj.client_id
This is very naive approach and it never lets inherit from created classes in a regular way, just like this:
class ClientProcess(Someclient): #Someclient is the name of the created class before
def __init__(self):
print "Someclient stuff"
The goal is pretty simple: create the objects of several classes, preferably with the properties that are stored in the tables, but at the same time have class declaration for every client which will have specific method implemented that will very from class to class. The initial script that works well and uses Python version of Factory method is not sufficient because it only can process one class(client) a time (based on command-line argument which is client id).
If I understand you correctly, you can use the following ways to subclass dynamically created classes:
In : classes = []
In : cls_name = 'BaseCls1'
In : classes.append(type(cls_name, (object, ), {'x': 1}))
In : classes[0].x
Out: 1
In : classes[0].__bases__
Out: (object,)
# two ways to create subclass out of BaseCls1
In : class SubCls1(classes[0]):
: x = 2
:
In : SubCls1.x
Out: 2
In : SubCls1.__bases__
Out: (__main__.BaseCls1,)
In : SubCls2 = type('SubCls2', (classes[0],), {'x': 2})
In : SubCls2.x
Out: 2
In : SubCls2.__bases__
Out: (__main__.BaseCls1,)
class GetConfig(object):
def __init__(self, client_id):
self.client_id = client_id
#construct the query here to get the clients data ...where client_id = self.client_id
d = {'logfile': 'some_long_path', 'contact_name': 'some_name'}
class FirstClient(object):
def __init__(self):
client_id = '111111111'
props = GetConfig(client_id)
#print props.d
def check_source(self):
print "Checking FirstClient source"
return "Something"
#print props.d
def check_downl(self):
print "Checking FirstClient downloaded"
class SecondClient(object):
def __init__(self):
client_id = "222222"
props = GetConfig(client_id)
def check_source(self):
print "Checking SecondClient source"
def check_downl(self):
print "Checking SecondClient downloaded"
myfactory = {
"firstclient" : FirstClient,
"secondclient" : SecondClient,
}
for i in myfactory.values():
i().check_source()
i().check_downl()
collections.namedtuple. done.
Edit: to elaborate,
from collections import namedtuple
rows = dict_cur.fetchall()
# creates the class Row which is a tuple, but each position argument
# corresponds to the column name in that position
# Row can be instantiated as a tuple and then its elements can be accessed
# by name class attributes
Row = namedtuple("Row", zip(*dict_cur.description)[0])
objects = [Row(row) for row in rows]
for o in objects:
print o.client_name, ' is ' , o