Related
I'm not quite sure I'm using the right wording in my researches -- if that's the case, please let me know, I may have missed obvious answers just because of that -- but I'd like to serialize (i.e. convert to a dictionary or JSON structure) both the main (outer) and inner class arguments of a class.
Here's an example:
class Outer(object):
def __init__(self, idx, array1, array2):
self.idx = idx
# flatten individual values:
## unpack first array
self.prop_a = array1[0]
self.prop_b = array1[1]
self.prop_c = array1[2]
## unpack second array
self.prop_d = array2[0]
self.prop_e = array2[1]
self.prop_f = array2[2]
# Nest elements to fit a wanted JSON schema
class inner1(object):
def __init__(self, outer):
self.prop_a = outer.prop_a
self.prop_b = outer.prop_b
self.prop_c = outer.prop_c
class inner2(object):
def __init__(self, outer):
self.prop_d = outer.prop_d
self.prop_e = outer.prop_e
self.prop_f = outer.prop_f
self.inner_first = inner1(self)
self.inner_second = inner2(self)
def serialize(self):
return vars(self)
Now I can call both:
import numpy as np
obj = Outer(10, np.array([1,2,3]), np.array([4,5,6]))
obj.prop_a # returns 1, or
obj.inner_first.prop_1 # also returns 1
But when I try to serialize it, it prints:
vars(obj) # prints:
{'idx': 10,
'prop_a': 1,
'prop_b': 2,
'prop_c': 3,
'prop_d': 4,
'prop_e': 5,
'prop_f': 6,
'inner_first': <__main__.Outer.__init__.<locals>.inner1 at 0x7f231a4fe3b0>,
'inner_second': <__main__.Outer.__init__.<locals>.inner2 at 0x7f231a4febc0>}
where I want it to print:
vars(obj) # prints:
{'idx': 10,
'prop_a': 1,
'prop_b': 2,
'prop_c': 3,
'prop_d': 4,
'prop_e': 5,
'prop_f': 6,
'inner_first': {'prop_a': 1, 'prop_b': 2, 'prop_c': 3},
'inner_second': {'prop_d': 4, 'prop_e': 5, 'prop_f': 6}}
with the 'inner_first' key being the actual results of vars(obj.inner_first), and same thing for the 'inner_second' key.
Ideally I'd like to call the serialize() method to convert my object to the desired output: obj.serialize()
I feel I'm close to the results but I can simply not see where I must go to solve this task.
At the really end, I wish I could simply:
obj = Outer(10, np.array([1,2,3]), np.array([4,5,6]))
obj.serialze()
{
'inner_first': {
'prop_a': 1,
'prop_b': 2,
'prop_c': 3
},
'inner_second': {
'prop_d': 4,
'prop_e': 5,
'prop_f': 6
}
}
in order to basically fit a given JSON structure that I have.
Info: this thread helped me to build the inner classes.
Also note that this question only embeds two "layers" or "levels" of the final structure, but I may have more than 2.
I'm taking an AI / ML online course, and some of the assignments include Python language phrases like:
input_img = tf.keras.Input(shape=input_shape)
Z1 = tfl.Conv2D(8, kernel_size=4,strides=(1,1), padding='SAME')(input_img)
It seems that the general syntax of the second line is:
obj1 = <class method>(<method parameters>)(obj2),
where obj1, obj2 are some class instances.
Could not find an explanation to this syntax.
Please direct me to a reference / example clarifying said syntax.
The syntax is that tfl.Conv2D(8, kernel_size=4,strides=(1,1), padding='SAME') returns something that is callable, like a callable object OR a method, then you call it with input_img params
A simple example
class A:
def __init__(self, *args):
self.values = list(args)
def run(self, *more_values):
print(f"A.run with {self.values} - {more_values}")
class B:
#staticmethod
def get_method(a, b, c):
return A(a, b, c).run
In the case B.get_method() returns a run method that is not called, so when retrieving it, you can call it, and split in 2 steps to understand better
B.get_method(1, 2, 3)(4, 5) # A.run with [1, 2, 3] - (4, 5)
my_method = B.get_method(1, 2, 3)
my_method(4, 5) # A.run with [1, 2, 3] - (4, 5)
What I want to do is to create a class that I will call a Cluster that I can use to store representations of multivariable data points. Like a bunch of people's heights, weights, and ages. I want to make it so when I create an instance of Cluster, it returns an empty dictionary. I then want to be able to add key value pairs. I want each value paired with each key to only be a list. I want each list to only contain only 1 data type.
However, they don't have to be the same data type for every list. What I mean is that one list can only consists of integers, and the other list is only full of strings. I even allow a list of tuples or lists or dictionaries. They don't even have to have the same data type for each value in itself. But, the overall list must then contain all dictionaries, tuples, etc.
Example of a Cluster:
{'Hobbies': [(3, 's', {'name': 'Sam'}), (4, 5), (), (True)], 'Weights': [1, 34, 3, 90]}
Notice how the above 'Hobbies' value pair is a list of only tuples and the 'Weights' value pair is a list of only integers. This is what I mean by what I am trying to say above.
I created a function that generates what a Cluster full of values should look like, but I want it to be a class. It gives you what I want to happen, but I want make a class, not a function that generates it. Other methods such as getting the length of the Cluster I can figure out, I just want everything that I said above to be true.
from random import randint
def create_cluster(a, b, c, *d):
cluster = {}
for key in d:
point = []
for integer in range(0, a):
point.append(randint(b, c))
cluster[key] = point
return cluster
print(create_cluster(4, 2, 9, 'Height', 'Weight', 'Age'))
Output:
{'Height': [5, 3, 3, 6], 'Weight': [7, 3, 5, 7], 'Age': [9, 5, 3, 6]}
As Code-Apprentice mentions, there is no mechanism by which python enforces type checking in the sense that it will stop you from compiling and running the code. As of version 3.5, however, Python allows you to implement a soft type checking - you can read about it here. What this means is that Python now offers you a way too check to see if your code is passing illegal values to functions/classes/variables etc.
Here's how it works:
from typing import Dict, List, Any, Tuple, Union
from random import randint
class Cluster:
def __init__(self):
self.data: Dict[str, List[Any]] = {}
def create_cluster(self, a, b, c, *d):
for key in d:
point = []
for integer in range(0, a):
point.append(randint(b, c))
self.data[key] = point
clust = Cluster()
clust.create_cluster(4, 2, 9, 'Height', 'Weight', 'Age')
print(clust.data)
Edit: for your example specifically, you'd want something like:
self.data: Dict[str, List[Union[Tuple, int]] = {}
Union is a type that indicates a wildcard of specified possibilities, so the dictionary value can be a list containing either tuples or ints. I guess if you wanted to further restrict the data types inside the tuples, you could do:
self.data: Dict[str, List[Union[Tuple[Union[int, str]], int]] = {}
Next, you need mypy installed. Mypy has to be called separately from a terminal on your script, but it will go through the code and perform a type check - that is, it will alert you to any instances where you have declared illegal variables in your class.
Alternatively, you could perform a series of in-code catches involving:
if type(data['Weight'][0]) != whatever:
raise("You're putting the wrong type of variable in")
But mypy seems like a better solution in the longer term. It'll save you from having to wait for your code to execute, run, and fail in a predictable spot.
Feedback and corrections welcome.
If I'm understanding what you need, we can actually make a dict object that will type check for you!
No imports needed.
We just have to subclass the dict class, and override the functions for adding items to the dictionary, so we can type check before allowing any additions.
Here is the code below.
class ClusterDict(dict):
key_type = str # we want all of our keys to be str type
value_type = list # same for our value as a list
# lets override the function for dictionary assignment so we can type check before we add to our dictionary
def __setitem__(self, key, value):
self.type_check_and_add(key, value)
# Lets override update as well
def update(self, **kwargs) -> None:
for key, value in kwargs.items():
self.type_check_and_add(key, value)
def type_check_and_add(self, key: str, value: list) -> None:
assert type(key) == self.key_type, f"Expected {self.key_type} for key, got {type(key)} instead."
if type(value) is not list:
value = [value] # Make it a list if its isn't, remove this if you know it will always come as a list, even if its a single entry, like [5], or [(2, 6)]
assert type(value) == self.value_type, f"Expected {self.value_type} for key, got {type(value)} instead."
val_type = type(value[0]) # type of first item in list, lets make sure they all match this one
for val in value:
assert type(val) == val_type, f"Items do not have matching types for key {key}."
super().__setitem__(key, value) # this then passes assignment as usual to the base class dict
Lets do some tests now!
if __name__ == '__main__':
clust = ClusterDict()
input_data = {
'Height': [4, 3, 5],
'Age': [(4, 9), (4, 6, 8)]
}
clust.update(**input_data)
print(clust)
{'Height': [4, 3, 5], 'Age': [(4, 9), (4, 6, 8)]}
Looks good so far, we can add whole dictionaries using update as usual
input_data = {'Weights': [1, 34, 3, 90]}
clust.update(**input_data)
print(clust)
{'Height': [4, 3, 5], 'Age': [(4, 9), (4, 6, 8)], 'Weights': [1, 34, 3, 90]}
Or just one entry at a time.
Lets try the usual way, dict[key] = value
clust['Hobbies'] = [(3, 's', {'name': 'Sam'}), (4, 5), (), (True,)]
print(clust)
{'Height': [4, 3, 5], 'Age': [(4, 9), (4, 6, 8)], 'Weights': [1, 34, 3, 90], 'Hobbies': [(3, 's', {'name': 'Sam'}), (4, 5), (), (True,)]}
Looking good. But now lets try adding a list with types that don't match.
clust['My Favorite Number'] = [7, '7', (1, 1, 1, 1, 1, 1, 1)]
print(clust)
assert type(val) == val_type, f"Items do not have matching types for key {key}."
AssertionError: Items do not have matching types for key My Favorite Number.
Which throws an AssertionError and prevents us from doing so, like we wanted.
Bonus though, because we subclassed dict, we get all of its methods for free!
>>>len(clust)
4
for k, v in clust.items():
... print(k, v)
...
Height [4, 3, 5]
Age [(4, 9), (4, 6, 8)]
Weights [1, 34, 3, 90]
Hobbies [(3, 's', {'name': 'Sam'}), (4, 5), (), (True,)]
Hopefully this is what you were looking for and it helps!
I am currently trying to code a text-based adventure in pure python. Therefore I have a Room class which looks kinda like this (shortened):
class Room1(Room):
def __init__(self):
Room.__init__(self)
self.init_objects()
self.description = """Looks like a living room. There's a lamp, a
cupboard, and
a door to your right. On the floor, there's a carpet."""
self.door_state = 1
self.carpet_state = 0
self.images = [(text-based images)]
def init_objects(self):
self.objects = {"door" : Door1(),
"frontdoor" : FrontDoor(),
"carpet" : Carpet(),
"lamp" : Lamp1(),
"clock" : Clock(),
"escritoire" : Escritoire(),
"booklet" : Booklet(),
"screws" : Screws(),
"trapdoor" : TrapDoor()}
def update_image(self):
IMG_MAPPER = {(0, 0, 0) : 0,
(0, 0, 1) : 1,
(1, 0, 0) : 2,
(1, 0, 1) : 3,
(1, 1, 1) : 4,
(1, 1, 0) : 5,
(0, 1, 1) : 6,
(0, 1, 0) : 7}
key = (self.objects["door"].state, self.objects["carpet"].state)
self.img = img_mapper[key]
My problem is with the Room's update_image() method. I need a mapper to be stored there to figure out the right image according to the object's states (opened / closed), and if I put this mapper at the start of the method, this dict is read and constructed by python everytime the method is called, is that right? So should I rather store this mapper dict as an instance variable like self.img_mapper_dict = {(0, 0, 0) : 0, ...}?
Any ideas on that anyone perhaps?
You are correct in assuming that the way your current code is structure, IMAGE_MAPPER would be assigned to every single time the update_image() method is called.
Now, since this is a static mapping that doesn't change over time, this is not a problem from a functionality standpoint - it's not like you're resetting a value that should be tracked between calls to update_image().
Performance wise, in this particular case, the drawbacks will also be absolutely negligible.
But it would probably make sense from a purely logical standpoint to make this mapping a class attribute:
class Room1(Room):
IMG_MAPPING = {(0, 0, 0) : 0,
(0, 0, 1) : 1,
(1, 0, 0) : 2,
(1, 0, 1) : 3,
(1, 1, 1) : 4,
(1, 1, 0) : 5,
(0, 1, 1) : 6,
(0, 1, 0) : 7}
def __init__(self):
# ...
def update_image(self):
# ...
self.img = Room1.IMG_MAPPING[key]
I'm recommending a class attribute instead of an instance attribute (or instance member) because that mapping is going to stay the same for all instances of Room1 (right? I'm assuming every instance of Room1 would have exactly the same layout, just different door states etc..). So it doesn't depend on any state of the instance, but it's a property of the class instead.
Also note that the class attribute is accessed differently: Room1.IMG_MAPPING instead of self.IMG_MAPPING. This also reflects the fact that it's not dependent on the instance.
The problem I think you're having is that img_mapper resets to the specified state every time update_image() is called. This is because the member function has its own scope, so when it is completed it deletes img_mapper and has to remake it next time the update function is called. Storing as a member variable would get rid of this problem.
First problem I see is that your IMG_MAPPER dict has three-tuple keys. But you are accessing it by providing a two-tuple.
I'm attempting to write a genetic algorithm framework in Python, and am running into issues with shallow/deep copying. My background is mainly C/C++, and I'm struggling to understand how these connections are persisting.
What I am seeing is an explosion in the length of an attribute list within a subclass. My code is below...I'll point out the problems.
This is the class for a single gene. Essentially, it should have a name, value, and boolean flag. Instances of Gene populate a list within my Individual class.
# gene class
class Gene():
# constructor
def __init__(self, name, is_float):
self.name_ = name
self.is_float_ = is_float
self.value_ = self.randomize_gene()
# create a random gene
def randomize_gene(self):
return random.random()
This is my Individual class. Each generation, a population of these are created (I'll show the creation code after the class declaration) and have the typical genetic algorithm operations applied. Of note is the print len(self.Genes_) call, which grows each time this class is instantiated.
# individual class
class Individual():
# genome definition
Genes_ = [] # genes list
evaluated_ = False # prevent re-evaluation
fitness_ = 0.0 # fitness value (from evaluation)
trace_ = "" # path to trace file
generation_ = 0 # generation to which this individual belonged
indiv_ = 0 # identify this individual by number
# constructor
def __init__(self, gen, indv):
# assign indices
self.generation_ = gen
self.indiv_ = indv
self.fitness_ = random.random()
# populate genome
for lp in cfg.params_:
g = Gene(lp[0], lp[1])
self.Genes_.append(g)
print len(self.Genes_)
> python ga.py
> 24
> 48
> 72
> 96
> 120
> 144
......
As you can see, each Individual should have 24 genes, however this population explodes quite rapidly.
I create an initial population of new Individuals like this:
# create a randomized initial population
def createPopulation(self, gen):
loc_population = []
for i in range(0, cfg.population_size_):
indv = Individual(gen, i)
loc_population.append(indv)
return loc_population
and later on my main loop (apologies for the whole dump, but felt it was necessary - if my secondary calls (mutation/crossover) are needed please let me know))
for i in range(0, cfg.generations_):
# evaluate current population
self.evaluate(i)
# sort population on fitness
loc_pop = sorted(self.population_, key=operator.attrgetter('fitness_'), reverse=True)
# create next population & preserve elite individual
next_population = []
elitist = copy.deepcopy(loc_pop[0])
elitist.generation_ = i
next_population.append(elitist)
# perform selection
selection_pool = []
selection_pool = self.selection(elitist)
# perform crossover on selection
new_children = []
new_children = self.crossover(selection_pool, i)
# perform mutation on selection
muties = []
muties = self.mutation(selection_pool, i)
# add members to next population
next_population = next_population + new_children + muties
# fill out the rest with random
for j in xrange(len(next_population)-1, cfg.population_size_ - 1):
next_population.append(Individual(i, j))
# copy next population over old population
self.population_ = copy.deepcopy(next_population)
# clear old lists
selection_pool[:] = []
new_children[:] = []
muties[:] = []
next_population[:] = []
I'm not not completely sure that I understand your question, but I suspect that your problem is that the Genes_ variable in your Individual() class is declared in the class namespace. This namespace is available to all members of the class. In other words, each instance of Individual() will share the same variable Genes_.
Consider the following two snippets:
class Individual():
# genome definition
genes = []
def __init__(self):
for i in xrange(10):
self.genes.append(i)
ind_1 = Individual()
print ind_1.genes
ind_2 = Individual()
print ind_1.genes
print ind_2.genes
and
class Individual():
# genome definition
def __init__(self):
self.genes = []
for i in xrange(10):
self.genes.append(i)
ind_1 = Individual()
print ind_1.genes
ind_2 = Individual()
print ind_1.genes
print ind_2.genes
The first snippet outputs
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
while the second snippet outputs
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In the first scenario, when the second Individual() is instantiated the genes list variable already exists, and the genes from the second individual are added to this existing list.
Rather than creating the Individual() class like this,
# individual class
class Individual():
# genome definition
Genes_ = [] # genes list
# constructor
def __init__(self, gen, indv):
# assign indices
self.generation_ = gen
self.indiv_ = indv
self.fitness_ = random.random()
you should consider declaring the Genes_ variable in init so that each Individual() instance gets its own gene set
# individual class
class Individual():
# constructor
def __init__(self, gen, indv):
# genome definition
self.Genes_ = [] # genes list
# assign indices
self.generation_ = gen
self.indiv_ = indv
self.fitness_ = random.random()
When you create a class, you are really creating exactly one 'class object'. These are objects just like any other object in Python; everything in Python is an object, and what those objects do is defined by their methods, not their class! That is the magic of duck typing. In Python you can even create new classes dynamically on the fly.
Anyway, you are adding exactly one list object to the "Genes_" attribute of the one and only "Individuals" class object. The upshot is that every instance object of the "Individual" class object is accessing the same "Genes_" list object.
Consider this
# In 2.2 <= Python < 3.0 you should ALWAYS inherit from 'object'.
class Foobar(object):
doodah = []
a = Foobar()
b = Foobar()
assert id(a.doodah) == id(b.doodah) # True
In this case, as you can see, "a.doodah" and "b.doodah" are the same object!
class Foobar(object):
def __init__(self):
self.doodah = []
a = Foobar()
b = Foobar()
assert id(a.doodah) != id(b.doodah) # True
In this case, they are different objects.
It's possible to have your cake and eat it too. Consider this
class Foobar(object):
doodah = []
a = Foobar()
b = Foobar()
a.doodah = 'hlaghalgh'
assert id(a.doodah) != id(b.doodah) # True
In this case a "doodah" attribute is added to the "a" object, which overrides the class attribute.
Hope this helps!