We Keep Coding

Unable to create an array of Python objects from YAML

I am trying to instantiate an array of python objects from YAML, inside a struct. Outside of a struct I am able to do this easily, but it seems that the YAML BaseLoader is failing to recursively search the sub nodes of my object.
import yaml
import ruamel.yaml
class Person:
def __init__(self, name: str = 'JohnDoe'):
self.name = name
#classmethod
def from_yaml(cls, constructor, node):
for m in constructor.construct_yaml_map(node):
pass
if 'Name' in m:
name = m['Name']
return cls(name=name)
def __repr__(self):
return f'Person(name={self.name})'
class Car:
def __init__(self):
self.passengers = []
def add_person(self, person: Person = None):
self.passengers.append(person)
#classmethod
def from_yaml(cls, constructor, node):
for m in constructor.construct_yaml_map(node):
pass
inst = cls()
if 'Driver' in m:
inst.passengers = [m['Driver']]+inst.passengers
if 'Passengers' in m:
foo = m['Passengers']
print(f'm[\'Passengers\'] = {foo}')
for person in m['Passengers']:
inst.add_person(person)
return inst
def __repr__(self):
return f'Car(passengers={self.passengers})'
if __name__ == "__main__":
yaml = ruamel.yaml.YAML(typ='safe')
yaml.register_class(Person)
yaml.register_class(Car)
data = yaml.load("""
- !Person &0
Name: 'Paul'
- !Person &1
Name: 'George'
- !Person &3
Name: 'John'
- !Car
Driver: *0
Passengers: [*1]
- !Car
Driver: *3
Passengers:
- !Person &4
Name: 'Ringo'
""")
print(f'data = {data}')
the above code prints the following to the console on execution:
m['Passengers'] = []
m['Passengers'] = []
data = [Person(name=Paul), Person(name=George), Person(name=John), Car(passengers=[Person(name=Paul)]), Car(passengers=[Person(name=John)])]
where as I would expect the output to be
m['Passengers'] = [Person(name=George)]
m['Passengers'] = [Person(name=Ringo)]
data = [Person(name=Paul), Person(name=George), Person(name=John), Car(passengers=[Person(name=Paul), Person(name=George)]), Car(passengers=[Person(name=John), Person(name=Ringo)])]
no matter what, even with an array of strings, the associated value of the key 'Passengers' is always [] in the dictionary m.
do I have to manually tell the constructor to travers the rest of the node first in the from_yaml function, or does YAML loader work recursively from the bottom up?

There is no need to import yaml in your example.
During the construction of the car, its passengers are not yet know. So
what you need to do is construct potentially recursive data such
as Person and Car in a two step process, first constructing and yielding the "empty" Car
then filling in the Driver and any Passangers on the already yielded instance. The loader knows
how to handle this, so you don't have to recurse into anything in your from_yaml.
Additionally you'll need to call constructor.construct_mapping(node, deep=True)
in from_yaml, instead of your iteration over constructor.construct_yaml_map(node):
import ruamel.yaml
class Person:
def __init__(self, name: str = 'JohnDoe'):
self.name = name
#classmethod
def from_yaml(cls, constructor, node):
inst = cls()
yield inst
m = constructor.construct_mapping(node, deep=True)
if 'Name' in m:
inst.name = m['Name']
def __repr__(self):
return f'Person(name={self.name})'
class Car:
def __init__(self):
self.passengers = []
def add_person(self, person: Person = None):
self.passengers.append(person)
#classmethod
def from_yaml(cls, constructor, node):
inst = cls()
yield inst
m = constructor.construct_mapping(node, deep=True)
if 'Driver' in m:
inst.passengers = [m['Driver']] + inst.passengers
if 'Passengers' in m:
foo = m['Passengers']
print(f'm[\'Passengers\'] = {foo}')
for person in m['Passengers']:
inst.add_person(person)
def __repr__(self):
return f'Car(passengers={self.passengers})'
if __name__ == "__main__":
yaml = ruamel.yaml.YAML(typ='safe')
yaml.register_class(Person)
yaml.register_class(Car)
data = yaml.load("""
- !Person &0
Name: 'Paul'
- !Person &1
Name: 'George'
- !Person &3
Name: 'John'
- !Car
Driver: *0
Passengers: [*1]
- !Car
Driver: *3
Passengers:
- !Person &4
Name: 'Ringo'
""")
print(f'data = {data}')
which gives:
m['Passengers'] = [Person(name=George)]
m['Passengers'] = [Person(name=Ringo)]
data = [Person(name=Paul), Person(name=George), Person(name=John), Car(passengers=[Person(name=Paul), Person(name=George)]), Car(passengers=[Person(name=John), Person(name=Ringo)])]
Although it is allowed to write the tag followed by the anchor, it is IMO more appropriate to write the anchor
followed by the tag, because you'll get an anchored instance of the tagged object.
So that leaves me wondering what the name is of &2 !Person (probably the same as for !Person &2), is it Pete?

I was able to find a partial answer in this post.
In the case of raumel.yaml, it looks like our constructor is the from_yaml function, which gets added when the class is registered. All we have to do is add a yield after the initialization of our class in from_yaml, and before we retrieve our recursive item Passengers.
class Car:
def __init__(self):
self.passengers = []
def add_person(self, person: Person = None):
self.passengers.append(person)
#classmethod
def from_yaml(cls, constructor, node):
for m in constructor.construct_yaml_map(node):
print(f'm{type(m)} = {m}')
pass
inst = cls()
yield inst # <-- This yield statement fixes our issue
if 'Driver' in m:
inst.passengers = [m['Driver']]+inst.passengers
if 'Passengers' in m:
foo = m['Passengers']
print(f'm[\'Passengers\'] = {foo}')
for person in m['Passengers']:
inst.add_person(person)
return inst
def __repr__(self):
return f'Car(passengers={self.passengers})'

Possible bug in python. Create decorator to automatically generate properties

I've created a repl.it to run the code online:
https://repl.it/repls/ShimmeringQuickwittedHarddrive#main.py
I am trying to create a decorator to "carry" properties from the object's attribute into the object.
simplistic example (pseudo code):
class carry:
''' Decorator. '''
class Prop:
''' Representation of '#property'. '''
# impl
# impl
class A:
def __init__(self):
self._val_0 = 'A: val_0'
self._val_1 = 'A: val_1'
#property
def val_0(self):
return self._val_0
#property
def val_1(self):
return self._val_1
# '_a' - the name of the attribute to get the #property'ies from.
# 'carry.Prop(...)' - a list of #property'ies to carry.
#carry(
'_a',
carry.Prop('val_0'),
carry.Prop('val_1')
)
class B:
def __init__(self):
self._a = A()
what I expect to get:
>>> b = B()
>>> print(b.val_0)
'A: val_0'
>>> print(b.val_1)
'A: val_1'
what I am getting:
(warning: the difference is quite subtle and easy to miss; b.val_0 returns the value of b.val_1)
>>> b = B()
>>> print(b.val_0)
'A: val_1'
>>> print(b.val_1)
'A: val_1'
Code below is an "executable" minimalistic sample to reproduces the error.
it's a lot of code so I split it into three sections:
Decorator:
Implementation of the decortor.
Error occurs somewhere inside carry.__call__().
Tests:
'print-debug' tests of the #carry decorator.
It's important to notice that when I'm printing b.val_0, instead of getting val_0 I am getting the last #property's representation that was passed into the #carry ctor.
To me it seems like writing setattr(<parrent>, <name>, property(<fget, fset, fdel>)) is bugged somewhere inside python. and instead of creating a property() i am wanting it creates, it overwrites them.
import pprint
from copy import copy
pp = pprint.PrettyPrinter(
indent = 2,
width = 80,
depth = None,
compact = False,
sort_dicts = False
)
##################################################
# Decorator:
##################################################
class carry:
class Prop:
''' Represents a 'property'. '''
def __init__(self, name:str, fget:bool=True, fset:bool=False, fdel:bool=False):
self._name = name
self._fget = fget
self._fset = fset
self._fdel = fdel
#property
def name(self) -> str:
return self._name
#property
def fget(self) -> bool:
return self._fget
#property
def fset(self) -> bool:
return self._fset
#property
def fdel(self) -> bool:
return self._fdel
def __init__(self, obj, *props):
self._obj = obj
self._props = props
def __call__(self, cls):
class Template(cls):
pass
Template.__name__ = cls.__name__
Template.__doc__ = cls.__doc__
for prop in self._props:
print('prop repr: ... {}'.format(prop.__dict__))
kwargs = {}
if prop.fget:
def fget(self):
print('fget id: ..... {}'.format(id(fget)))
print('fget addr: ... {}'.format(hex(id(fget))))
print('fget.obj: .... {}'.format(fget.obj))
print('fget.name: ... {}'.format(fget.name))
return getattr(getattr(self, fget.obj), fget.name)
fget.obj = copy(self._obj)
fget.name = copy(prop.name)
print('fget id: ..... {}'.format(id(fget)))
print('fget addr: ... {}'.format(hex(id(fget))))
print('fget.obj: .... {}'.format(fget.obj))
print('fget.name: ... {}'.format(fget.name))
kwargs['fget'] = fget
result = property(**kwargs)
print('kwargs: ...... {}'.format(kwargs))
print('prop: ........ {}'.format(result))
print('prop id: ..... {}'.format(id(result)))
print('prop addr: ... {}'.format(hex(id(result))))
setattr(Template, prop.name, result)
print()
# Return modified class:
return Template
##################################################
# Tests:
##################################################
class A:
def __init__(self):
self._val_0 = 'A: val_0: 0'
self._val_1 = 'A: val_1: 1'
self._val_2 = 'A: val_2: 2'
self._val_3 = 'A: val_3: 3'
#property
def val_0(self):
return self._val_0
#property
def val_1(self):
return self._val_1
#property
def val_2(self):
return self._val_2
#property
def val_3(self):
return self._val_3
print()
print('##################################################')
print()
props = [
carry.Prop('val_0'),
carry.Prop('val_1'),
carry.Prop('val_2'),
carry.Prop('val_3'),
]
#carry('_a', *props)
class B:
def __init__(self):
self._a = A()
print('##################################################')
print()
print(B)
print(B.__name__)
pp.pprint(dir(B))
print()
print('##################################################')
print()
b = B()
print(b.val_0)
print()
print(b.val_1)
print()
print(b.val_2)
print()
print(b.val_3)
print()

First, let me reiterate my comment: You appear to be implementing a "delegate" pattern. Here is some code that does that with fewer lines and more flexibility:
https://www.michaelcho.me/article/method-delegation-in-python
The name 'carry' does not tell me what to expect. Unless you have some problem domain where that name is already well-established, I would urge you to consider a different name. With that said ...
I believe your problem is that you are trying to use a closure without creating a closure. The loop that you have for iterating over all the properties is not a proper closure, and so there is leakage-- all references end up pointing to the last value of the iteration variables.
I modified your example code like this:
if prop.fget:
fget = self.make_fget(self._obj, prop.name)
#def fget(self):
# print('fget id: ..... {}'.format(id(fget)))
# print('fget addr: ... {}'.format(hex(id(fget))))
# print('fget.obj: .... {}'.format(fget.obj))
# print('fget.name: ... {}'.format(fget.name))
# return getattr(getattr(self, fget.obj), fget.name)
#fget.obj = copy(self._obj)
#fget.name = copy(prop.name)
And added this method:
def make_fget(self, attribute_name, property_name):
""" Construct and return an fget closure. """
def fget(self):
print('fget id: ..... {}'.format(id(fget)))
print('fget addr: ... {}'.format(hex(id(fget))))
print('fget.obj: .... {}'.format(fget.obj))
print('fget.name: ... {}'.format(fget.name))
return getattr(getattr(self, fget.obj), fget.name)
fget.obj = attribute_name
fget.name = property_name
return fget
The result was more what I think you expect:
# ... many, many lines elided ...
fget id: ..... 139848106927168
fget addr: ... 0x7f30ecbc6440
fget.obj: .... _a
fget.name: ... val_0
A: val_0: 0
fget id: ..... 139848106463104
fget addr: ... 0x7f30ecb54f80
fget.obj: .... _a
fget.name: ... val_1
A: val_1: 1
fget id: ..... 139848106491984
fget addr: ... 0x7f30ecb5c050
fget.obj: .... _a
fget.name: ... val_2
A: val_2: 2
fget id: ..... 139848106492128
fget addr: ... 0x7f30ecb5c0e0
fget.obj: .... _a
fget.name: ... val_3
A: val_3: 3

Python class doesn't see second object in list(attribute)

I want to code movie classification class, which can find (based on some criterias), add and print movies.
Here is my code:
class Movie:
def __init__(self,name,director,year,location):
self.name = name
self.director = director
self.year = year
self.location = location
self.information = {'name':self.name,'director':self.director,'year':self.year,'location':self.location}
def get_name(self):
return self.name
def get_director(self):
return self.director
def get_year(self):
return self.year
def get_location(self):
return self.location
def get_information(self):
return self.information
def __str__(self):
return f"Name = {self.name},director = {self.director},year = {self.year},location = {self.location}"
class Classification:
def __init__(self):
self.movie_list = []
def length(self):
return len(self.movie_list)
def __getitem__(self,key):
if isinstance(self.movie_list,slice):
return self.movie_list[key]
def add_movie(self,movie):
self.movie_list.append(movie)
def print_movie(self):
for movie in self.movie_list:
print(movie)
def find_movie(self,**kwargs):
check_list = []
for movie in self.movie_list:
for name,value in kwargs.items():
if movie.get_information()[name] == value:
check_list.append(True)
else:
check_list.append(False)
if all(item == True for item in check_list):
print(movie)
check_list.clear()
Here i have a class Movie and Classification; Classification has only 1 attribute, which is a list of movies. But i have two problems:
a = Movie('Matrix','Dan Yefimov','1999','New York')
b = Movie('Legend','Mak Markus','2005','Kiev')
clasif = Classification()
clasif.add_movie(a)
clasif.add_movie(b)
clasif.find_movie(location = 'New York')
find_movie() works for the first movie (a in our case) in the list. But for the second one it prints nothing, even when I enter correct parameters for a search.
Slicing doesn't work. There is no error message, it just prints nothing.
Can you help me with my problems?
P.S I would like also to hear some general advices about improving of my code.

By removing the unnecessary methods, we can shorten the code considerably. We can also just pass __getitem__ through to the list. I would also use __len__ over defining a length method.
class Movie:
def __init__(self,name,director,year,location):
self.name = name
self.director = director
self.year = year
self.location = location
def __str__(self):
return f"Name = {self.name},director = {self.director},year = {self.year},location = {self.location}"
class Classification:
def __init__(self):
self.movie_list = []
def __len__(self): # __len__ lets you do len(classif)
return len(self.movie_list)
def __getitem__(self,key):
return self.movie_list[key]
def add_movie(self,movie):
self.movie_list.append(movie)
def print_movie(self):
for movie in self.movie_list:
print(movie)
def find_movie(self,**kwargs):
for movie in self.movie_list:
if all(hasattr(movie, k) and getattr(movie, k) == v for k, v in kwargs.items()):
print(movie)

Here is a version of your code will a little less code:
class Movie:
ATTRIBUTES = ('name', 'director', 'year', 'location')
def __init__(self, name, director, year, location):
self.name = name
self.director = director
self.year = year
self.location = location
def __str__(self):
return ', '.join(
'{} = {}'.format(attr_name, getattr(self, attr_name))
for attr_name in self.ATTRIBUTES)
class Classification:
def __init__(self):
self.movie_list = []
def __len__(self):
return len(self.movie_list)
def __getitem__(self, key):
return self.movie_list[key]
def add_movie(self, movie):
self.movie_list.append(movie)
def print_movies(self):
for movie in self.movie_list:
print(movie)
def find_movies(self, **kwargs):
for movie in self.movie_list:
do_print = True
for attr_name, attr_value in kwargs.items():
if attr_name in Movie.ATTRIBUTES:
if getattr(movie, attr_name) != attr_value:
do_print = False
if do_print:
print(movie)
I added a class attribute called ATTRIBUTES to Movie; this is used in Movie.__str__() and also in Classification.find_movies(). It is a suggestion to avoid repeating the attributes a lot in the code.
In the method Classification.find_movies() I check that it is a valid attribute before I compare it to the movie instance. Invalid parameters are ignored, but you could change the code so that invalid parameters automatically cause the movies to not be printed (all will be excluded).

python prototype structure, not working

whenever i print the c1 object, it prints:
<main.Car object at 0x7fde8b29a240>
however i added the str method, to format it to a proper string, why is it not printing a readable string?
import copy
class Prototype:
def __init__(self):
# constructor method to create the object
self._objects = {}
def register_object(self, name, obj):
# this method is used to register an object
self._objects[name] = obj
def unregister_object(self, name):
# this method is used to unregister an object
del self._objects[name]
def clone(self, name, **attr):
obj = copy.deepcopy(self._objects.get(name))
obj.__dict__.update(attr)
return obj
class Car:
def __init__(self):
self.name = "Skylark"
self.color = "blue"
self.options = "extra horsepower in engine"
def __str__(self):
return '{} | {} | {}'.format(self.name, self.color, self.options)
c = Car()
prototype = Prototype()
prototype.register_object('skylark',c)
c1 = prototype.clone('skylark')
print(c1)

There is a problem with the indentation in your code. I've corrected this and can get the desired answer too. The indentation is a bit off for the function defs. in both the classes.
I've called this file as test.py
import copy
class Prototype:
def __init__(self):
# constructor method to create the object
self._objects = {}
def register_object(self, name, obj):
# this method is used to register an object
self._objects[name] = obj
def unregister_object(self, name):
# this method is used to unregister an object
del self._objects[name]
def clone(self, name, **attr):
obj = copy.deepcopy(self._objects.get(name))
obj.__dict__.update(attr)
return obj
class Car:
def __init__(self):
self.name = "Skylark"
self.color = "blue"
self.options = "extra horsepower in engine"
def __str__(self):
return '{} | {} | {}'.format(self.name, self.color, self.options)
c = Car()
prototype = Prototype()
prototype.register_object('skylark',c)
c1 = prototype.clone('skylark')
print(c1)
When I run the file
$ python test.py
The output is:
#Output: Skylark | blue | extra horsepower in engine

unsupported operand type(s) for +: 'method' and 'float'

class Coffee:
def __init__(self):
self._price=4.0
def price(self):
return self._price
def __str__(self):
return "Coffee with price "+ str(self._price)
class CoffeeWithMilk:
def __init__(self, coffee):
self.price+=coffee.price+0.5
def price(self):
return self.price
coffee=Coffee()
x=CoffeeWithMilk(coffee)
coffeeWithMilk=CoffeeWithMilk(x)
print(coffeeWithMilk)
How to fix this? Thanks

coffee.price is a method, so coffee.price + 0.5 gives you that error.
If you wanted to get the result of that method instead, call the method:
self._price = coffee.price() + 0.5
Note, I replaced += with = here, you are setting a new attribute after all. I also renamed the attribute, because otherwise your CoffeeWithMilk.price method will get really confused too, causing a second error that looks a lot the same, as self.price is still a method too. This necessitates fixing the def price(self) method to:
def price(self):
return self._price
So the completed code looks like this:
class Coffee:
def __init__(self):
self._price = 4.0
def price(self):
return self._price
def __str__(self):
return "Coffee with price " + str(self._price)
class CoffeeWithMilk:
def __init__(self, coffee):
self._price = coffee.price() + 0.5
def price(self):
return self._price
You can avoid re-defining the price method altogether by using class inheritance; make CoffeeWithMilk a specialised version of Coffee:
class Coffee:
name = 'Coffee'
def __init__(self):
self._price = 4.0
def price(self):
return self._price
def __str__(self):
return "{} with price {}".format(self.name, self._price)
class CoffeeWithMilk(Coffee):
name = 'Coffee with milk'
def __init__(self, coffee):
self._price = coffee.price() + 0.5
You get the __str__ implementation along too, so your final print(coffeeWithMilk) will output something a little more interesting.
You could also make Coffee.price a property; properties are methods that are automatically called for you each time you access the attribute:
class Coffee:
name = 'Coffee'
def __init__(self):
self._price = 4.0
#property
def price(self):
return self._price
def __str__(self):
return "{} with price {}".format(self.name, self._price)
class CoffeeWithMilk(Coffee):
name = 'Coffee with milk'
def __init__(self, coffee):
self._price = coffee.price + 0.5
In this case, I'd not use either a method or a property however. There is no need to make _price hidden here. Just replace it with a direct attribute:
class Coffee:
name = 'Coffee'
def __init__(self):
self.price = 4.0
def __str__(self):
return "{} with price {}".format(self.name, self.price)
class CoffeeWithMilk(Coffee):
name = 'Coffee with milk'
def __init__(self, coffee):
self.price = coffee.price + 0.5
That's because neither the method nor the property do anything more than pass along the _price attribute. You may as well just directly access it.
Last but not least, you create a CoffeeWithMilk instance from a Coffee instance, then another CoffeeWithMilk instance from the first CoffeeWithMilk instance, so your final instance has added 0.5 to 4 twice:
>>> coffee = Coffee()
>>> x = CoffeeWithMilk(coffee) # first instance, from coffee
>>> print(x)
Coffee with milk with price 4.5
>>> coffeeWithMilk = CoffeeWithMilk(x) # second instance, from x
>>> print(coffeeWithMilk)
Coffee with milk with price 5.0