We Keep Coding

Promote instantiated class/object to a class in python?

Is there are a way in Python to store instantiated class as a class 'template' (aka promote object to a class) to create new objects of same type with same fields values, without relying on using data that was used to create original object again or on copy.deepcopy?
Like, for example I have the dictionary:
valid_date = {"date":"30 february"} # dict could have multiple items
and I have the class:
class AwesomeDate:
def __init__(self, dates_dict):
for key, val in dates_dict.items():
setattr(self, key, val);
I create the instance of the class like:
totally_valid_date = AwesomeDate(valid_date)
print(totally_valid_date.date) # output: 30 february
and now I want to use it to create new instances of the AwesomeDate class using the totally_valid_date instance as a template, i.e. like:
how_make_it_work = totally_valid_date()
print(how_make_it_work.date) # should print: 30 february
Is there are way to do so or no? I need a generic solution, not a solution for this specific example.

I don't really see the benefit of having a class act both as a template to instances, and as the instance itself, both conceptually and coding-wise. In my opinion, you're better off using two different classes - one for the template, one for the objects it is able to create.
You can think about awesome_date as a template class that stores the valid_date attributes upon initialization. Once called, the template returns an instance of a different class that has the expected attributes.
Here's a simple implementation (names have been changed to generalize the idea):
class Thing:
pass
class Template:
def __init__(self, template_attrs):
self.template_attrs = template_attrs
def __call__(self):
instance = Thing()
for key, val in self.template_attrs.items():
setattr(instance, key, val)
return instance
attrs = {'date': '30 february'}
template = Template(template_attrs=attrs)
# Gets instance of Thing
print(template()) # output: <__main__.Thing object at 0x7ffa656f8668>
# Gets another instance of Thing and accesses the date attribute
print(template().date) # output: 30 february

Yes, there are ways to do it -
there could even be some tweaking of inheriting from type and meddling with __call__ to make all instances automatically become derived classes. But I don't think that would be very sane. Python's own enum.Enum does something along this, because it has some use for the enum values - but the price is it became hard to understand beyond the basic usage, even for seasoned Pythonistas.
However, having a custom __init_subclass__ method that can inject some code to run prior to __init__ on the derived class, and then a method that will return a new class bound with the data that the new classes should have, can suffice:
import copy
from functools import wraps
def wrap_init(init):
#wraps(init)
def wrapper(self, *args, **kwargs):
if not getattr(self, "_initalized", False):
self.__dict__.update(self._template_data or {})
self._initialized = True
return init(self, *args, **kwargs)
wrapper._template_wrapper = True
return wrapper
class TemplateBase:
_template_data = None
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
if getattr(cls.__init__, "_template_wraper", False):
return
init = cls.__init__
cls.__init__ = wrap_init(init)
def as_class(self):
cls= self.__class__
new_cls = type(cls.__name__ + "_templated", (cls,), {})
new_cls._template_data = copy.copy(self.__dict__)
return new_cls
And using it:
class AwesomeDate(TemplateBase):
def __init__(self, dates_dict):
for key, val in dates_dict.items():
setattr(self, key, val)
On the REPL we have:
In [34]: x = AwesomeDate({"x":1, "y":2})
In [35]: Y = x.as_class()
In [36]: y = Y({})
In [37]: y.x
Out[37]: 1
Actually, __init_subclass__ itself could be supressed, and decorating __init__ could be done in one shot on the as_class method. This code takes some care so that mixin classes can be used, and it will still work.

It seems like you are going for something along the lines of the prototype design pattern.
What is the prototype design pattern?
From Wikipedia: Prototype pattern
The prototype pattern is a creational design pattern in software development. It is used when the type of objects to create is determined by a prototypical instance, which is cloned to produce new objects. This pattern is used to avoid subclasses of an object creator in the client application, like the factory method pattern does and to avoid the inherent cost of creating a new object in the standard way (e.g., using the 'new' keyword) when it is prohibitively expensive for a given application.
From Refactoring.guru: Prototype
Prototype is a creational design pattern that lets you copy existing objects without making your code dependent on their classes. The Prototype pattern delegates the cloning process to the actual objects that are being cloned. The pattern declares a common interface for all objects that support cloning. This interface lets you clone an object without coupling your code to the class of that object. Usually, such an interface contains just a single clone method.
The implementation of the clone method is very similar in all classes. The method creates an object of the current class and carries over all of the field values of the old object into the new one. You can even copy private fields because most programming languages let objects access private fields of other objects that belong to the same class. An object that supports cloning is called a prototype. When your objects have dozens of fields and hundreds of possible configurations, cloning them might serve as an alternative to subclassing. Here’s how it works: you create a set of objects, configured in various ways. When you need an object like the one you’ve configured, you just clone a prototype instead of constructing a new object from scratch.
Implementing this for your problem, along with your other ideas
From your explanation, it seems like you want to:
Provide a variable containing a dictionary, which will be passed to the __init__ of some class Foo
Instantiate class Foo and pass the variable containing the dictionary as an argument.
Implement __call__ onto class Foo, allowing us to use the function call syntax on an object of class Foo.
The implementation of __call__ will COPY/CLONE the “template” object. We can then do whatever we want with this copied/cloned instance.
The Code (edited)
import copy
class Foo:
def __init__(self, *, template_attrs):
if not isinstance(template_attrs, dict):
raise TypeError("You must pass a dict to instantiate this class.")
self.template_attrs = template_attrs
def __call__(self):
return copy.copy(self)
def __repr__(self):
return f"{self.template_attrs}"
def __setitem__(self, key, value):
self.template_attrs[key] = value
def __getitem__(self, key):
if key not in self.template_attrs:
raise KeyError(f"Key {key} does not exist in '{self.template_attrs=}'.")
return self.template_attrs[key]
err = Foo(template_attrs=1) # Output: TypeError: You must pass a dict to instantiate this class.
# remove err's assignment to have code under it run
base = Foo(template_attrs={1: 2})
print(f"{base=}") # Output: base={1: 2}
base_copy = base()
base_copy["hello"] = "bye"
print(f"{base_copy=}") # Output: base_copy={1: 2, 'hello': 'bye'}
print(f"{base_copy[1]=}") # Output: base_copy[1]=2
print(f"{base_copy[10]=}") # Output: KeyError: "Key 10 does not exist in 'self.template_attrs={1: 2, 'hello': 'bye'}'."
I also added support for subscripting and item assignment through __getitem__ and __setitem__ respectively. I hope that this helped a bit with your problem! Feel free to comment on this if I missed what you were asking.
Reasons for edits (May 16th, 2022 at 8:49 PM CST | Approx. 9 hours after original answer)
Fix code based on suggestions by comment from user jsbueno
Handle, in __getitem__, if an instance of class Foo is subscripted with a key that doesn't exist in the dict.
Handle, in __init__, if the type of template_attrs isn't dict (did this based on the fact that you used a dictionary in the body of your question)

How to extend the list data structure in Python without violating Liskov substitution - supply an attribute instead of an instance?

I’m building a class that extends the list data structure in Python, called a Partitional. I’m adding a few methods that I find myself using frequently when dividing a list into partitions.
The class is initialized with a (nullable) list, which exists as an attribute on the class.
class Partitional(list):
"""Extends the list data type. Adds methods for dividing a list into partition sets
and returning data about those partition sets"""
def __init__(self, source_list: list=[]):
super().__init__()
self.source_list: list = source_list
self.n: int = len(source_list)
...
I want to be able to reliably replace list instances with Partitional instances without violating Liskov substitution. So for list’s methods, I wrote methods on the Partitional class that operate on self.source_list, e.g.
...
def remove(self, matched_item):
self.source_list.remove(matched_item)
self.__init__(self.source_list)
def pop(self, *args):
popped_item = self.source_list.pop(*args)
self.__init__(self.source_list)
return popped_item
def clear(self):
self.source_list.clear()
self.__init__(self.source_list)
...
(the __init__ call is there because the Partitional class builds some internal attributes based on self.source_list when it’s initialized, so these need to be rebuilt if source_list changes.)
And I also want Python’s built-in methods that take a list as an argument to work with a Partitional instance, so I set to work writing method overrides for those as well, e.g.
...
def __len__(self):
return len(self.source_list)
def __enumerate__(self):
return enumerate(self.source_list)
...
The relevant built-in methods are a finite set for any given Python version, but... is there not a simpler way to do this?
My question:
Is there a way to write a class such that, if an instance of that class is used as the argument for a function, the class provides an attribute to the function instead, by default?
That way I’d only need to override this default behaviour for a subset of built-in methods.
So for example, if a use case involving a list instance looks like this:
example_list: list = [1,2,3,4,5]
length = len(example_list)
we substitute a Partitional instance built from the same list:
example_list: list = [1,2,3,4,5]
example_partitional = Partitional(example_list)
length = len(example_partitional)
and what’s “actually” happening is this:
length = len(example_partitional.source_list)
i.e.
length = len([1,2,3,4,5])
Other notes:
In working on this, I’ve realized that there are two broad categories of Liskov substitution violation possible:
Inherent violation, where the structure of the child class will make it incompatible with any use case where the child class is used in place of the parent class, e.g. if you override some fundamental property or structure of the parent.
Context-dependent violation, where, for any given piece of software, so long as you never use the child class in a way that would violate Liskov substitution, you’re fine. E.g. You override a method on the parent class that would change how a built-in function acts when it takes an instance of the class as an argument, but you never use that built-in method with the class instance in your system. Or any system that depends on your system. Or... (you see how relying on this caveat is not foolproof)
What I’m looking to do is come up with a technique that will protect against both categories of violation, without having to worry about use cases and context.

Is this sound software engineering practice for class construction?

Is this a plausible and sound way to write a class where there is a syntactic sugar #staticmethod that is used for the outside to interact with? Thanks.
###scrip1.py###
import SampleClass.method1 as method1
output = method1(input_var)
###script2.py###
class SampleClass(object):
def __init__(self):
self.var1 = 'var1'
self.var2 = 'var2'
#staticmethod
def method1(input_var):
# Syntactic Sugar method that outside uses
sample_class = SampleClass()
result = sample_class._method2(input_var)
return result
def _method2(self, input_var):
# Main method executes the various steps.
self.var4 = self._method3(input_var)
return self._method4(self.var4)
def _method3(self):
pass
def _method4(self):
pass

Answering to both your question and your comment, yes it is possible to write such a code but I see no point in doing it:
class A:
def __new__(cls, value):
return cls.meth1(value)
def meth1(value):
return value + 1
result = A(100)
print(result)
# output:
101
You can't store a reference to a class A instance because you get your method result instead of an A instance. And because of this, an existing __init__will not be called.
So if the instance just calculates something and gets discarded right away, what you want is to write a simple function, not a class. You are not storing state anywhere.
And if you look at it:
result = some_func(value)
looks exactly to what people expect when reading it, a function call.
So no, it is not a good practice unless you come up with a good use case for it (I can't remember one right now)
Also relevant for this question is the documentation here to understand __new__ and __init__ behaviour.
Regarding your other comment below my answer:
defining __init__ in a class to set the initial state (attribute values) of the (already) created instance happens all the time. But __new__ has the different goal of customizing the object creation. The instance object does not exist yet when __new__is run (it is a constructor function). __new__ is rarely needed in Python unless you need things like a singleton, say a class A that always returns the very same object instance (of A) when called with A(). Normal user-defined classes usually return a new object on instantiation. You can check this with the id() builtin function. Another use case is when you create your own version (by subclassing) of an immutable type. Because it's immutable the value was already set and there is no way of changing the value inside __init__ or later. Hence the need to act before that, adding code inside __new__. Using __new__ without returning an object of the same class type (this is the uncommon case) has the addtional problem of not running __init__.
If you are just grouping lots of methods inside a class but there is still no state to store/manage in each instance (you notice this also by the absence of self use in the methods body), consider not using a class at all and organize these methods now turned into selfless functions in a module or package for import. Because it looks you are grouping just to organize related code.
If you stick to classes because there is state involved, consider breaking the class into smaller classes with no more than five to 7 methods. Think also of giving them some more structure by grouping some of the small classes in various modules/submodules and using subclasses, because a long plain list of small classes (or functions anyway) can be mentally difficult to follow.
This has nothing to do with __new__ usage.
In summary, use the syntax of a call for a function call that returns a result (or None) or for an object instantiation by calling the class name. In this case the usual is to return an object of the intended type (the class called). Returning the result of a method usually involves returning a different type and that can look unexpected to the class user. There is a close use case to this where some coders return self from their methods to allow for train-like syntax:
my_font = SomeFont().italic().bold()
Finally if you don't like result = A().method(value), consider an alias:
func = A().method
...
result = func(value)
Note how you are left with no reference to the A() instance in your code.
If you need the reference split further the assignment:
a = A()
func = a.method
...
result = func(value)
If the reference to A() is not needed then you probably don't need the instance too, and the class is just grouping the methods. You can just write
func = A.method
result = func(value)
where selfless methods should be decorated with #staticmethod because there is no instance involved. Note also how static methods could be turned into simple functions outside classes.
Edit:
I have setup an example similar to what you are trying to acomplish. It is also difficult to judge if having methods injecting results into the next method is the best choice for a multistep procedure. Because they share some state, they are coupled to each other and so can also inject errors to each other more easily. I assume you want to share some data between them that way (and that's why you are setting them up in a class):
So this an example class where a public method builds the result by calling a chain of internal methods. All methods depend on object state, self.offset in this case, despite getting an input value for calculations.
Because of this it makes sense that every method uses self to access the state. It also makes sense that you are able to instantiate different objects holding different configurations, so I see no use here for #staticmethod or #classmethod.
Initial instance configuration is done in __init__ as usual.
# file: multistepinc.py
def __init__(self, offset):
self.offset = offset
def result(self, value):
return self._step1(value)
def _step1(self, x):
x = self._step2(x)
return self.offset + 1 + x
def _step2(self, x):
x = self._step3(x)
return self.offset + 2 + x
def _step3(self, x):
return self.offset + 3 + x
def get_multi_step_inc(offset):
return MultiStepInc(offset).result
--------
# file: multistepinc_example.py
from multistepinc import get_multi_step_inc
# get the result method of a configured
# MultiStepInc instance
# with offset = 10.
# Much like an object factory, but you
# mentioned to prefer to have the result
# method of the instance
# instead of the instance itself.
inc10 = get_multi_step_inc(10)
# invoke the inc10 method
result = inc10(1)
print(result)
# creating another instance with offset=2
inc2 = get_multi_step_inc(2)
result = inc2(1)
print(result)
# if you need to manipulate the object
# instance
# you have to (on file top)
from multistepinc import MultiStepInc
# and then
inc_obj = MultiStepInc(5)
# ...
# ... do something with your obj, then
result = inc_obj.result(1)
print(result)
Outputs:
37
13
22

Why does this mypy, slots, and abstract class hack work?

I've got a relatively big Python project and in an effort to minimise debugging time I'm trying to emulate a few aspects of a lower-level language. Specifically
Ability to type cast (Static Typing)
Prevent dynamic attribute addition to classes.
I've been using mypy to catch type casting errors and I've been defining __slots__ in my class instances to prevent dynamic addition.
At one point I need a List filled with two different children class (they have the same parent) that have slightly different attributes. mypy didn't like the fact that there were calls to attributes for a list item that weren't present in ALL the list items. But then making the parent object too general meant that dynamic addition of variables present in the other child wasn't prevented.
To fix this I debugged/brute-forced myself to the following code example which seems to work:
from abc import ABCMeta
from typing import List
class parentclass(metaclass=ABCMeta):
__slots__:List[str] = []
name: None
class withb(parentclass):
__slots__ = ['b','name']
def __init__(self):
self.b: int = 0
self.name: str = "john"
class withf(parentclass):
__slots__ = ['f','name']
def __init__(self):
self.name: str = 'harry'
self.f: int = 123
bar = withb()
foo = withf()
ls: List[parentclass] = [bar, foo]
ls[0].f = 12 ## Needs to fail either in Python or mypy
for i in range(1):
print(ls[i].name)
print(ls[i].b) ## This should NOT fail in mypy
This works. But I'm not sure why. If I don't initialise the variables in the parent (i.e. only set them to None or int) then they don't seem to be carried into the children. However if I give them a placeholder value e.g. f:int = 0 in the parent then they make it into the children and my checks don't work again.
Can anyone explain this behaviour to an idiot like me? I'd like to know just so that I don't mess up implementing something and introduce even more errors!
As an aside: I did try List[Union[withb, withf]] but that didn't work either!

Setting a name to a value in the parent creates a class attribute. Even though the instances are limited by __slots__, the class itself can have non-slotted names, and when an instance lacks an attribute, its class is always checked for a class-level attribute (this is how you can call methods on instances at all).
Attempting to assign to a class attribute via an instance doesn't replace the class attribute though. instance.attr = someval will always try to create the attribute on the instance if it doesn't exist (shadowing the class attribute). When all classes in the hierarchy use __slots__ (without a __dict__ slot), this will fail (because the slot doesn't exist).
When you just for f: None, you've annotated the name f, but not actually created a class attribute; it's the assignment of a default that actually creates it. Of course, in your example, it makes no sense to assign a default in the parent class, because not all children have f or b attributes. If all children must have a name though, that should be part of the parent class, e.g.:
class parentclass(metaclass=ABCMeta):
# Slot for common attribute on parent
__slots__:List[str] = ['name']
def __init__(self, name: str):
# And initializer for parent sets it (annotation on argument covers attribute type)
self.name = name
class withb(parentclass):
# Slot for unique attributes on child
__slots__ = ['b']
def __init__(self):
super().__init__("john") # Parent attribute initialized with super call
self.b: int = 0 # Child attribute set directly
class withf(parentclass):
__slots__ = ['f']
def __init__(self):
super().__init__('harry')
self.f: int = 123
If the goal is to dynamically choose whether to use f or b based on the type of the child class, mypy understands isinstance checks, so you can change the code using it to:
if isinstance(ls[0], withf): # Added to ensure `ls[0]` is withf before using it
ls[0].f = 12 ## Needs to fail either in Python or mypy
for x in ls:
print(x.name)
if isinstance(x, withb): # Added to only print b for withb instances in ls
print(x.b) ## This should NOT fail in mypy
In cases where isinstance isn't necessary (you know the type, because certain indices are guaranteed to be withf or withb), you can explicitly cast the type, but be aware that this throws away mypy's ability to check; lists are intended as a homogeneous data structure, and making position important (a la tuple, intended as a heterogeneous container) is misusing them.

Pythonic accessors / mutators for "internal" lists

I'm aware that attribute getters and setters are considered "unpythonic", and the pythonic way to do things is to simply use an normal attribute and use the property decorator if you later need to trigger some functionality when an attribute is accessed or set.
e.g. What's the pythonic way to use getters and setters?
But how does this apply when the value of an attribute is a list, for example?
class AnimalShelter(object):
def __init__(self):
dogs = []
cats = []
class Cat(object):
pass
class Dog(object):
pass
Say that initially, the interface works like this:
# Create a new animal shelter
woodgreen = AnimalShelter()
# Add some animals to the shelter
dog1 = Dog()
woodgreen.dogs.append(dog1)
This would seem to be in line with the "pythonic" idea of just using straightforward attributes rather than creating getters, setters, mutators etc. I could have created an addDog method instead. But while not strictly speaking a setter (since it mutates the value of an attribute rather than setting an attribute), it still seems setter-like compared to my above solution.
But then, say that later on you need to trigger some functionality when dogs are added. You can't fall back on the using the property decorator, since adding a dog is not setting a property on the object, but retrieving a list which is the value of that attribute, and mutating that list.
What would be the "pythonic" way of dealing with such a situation?

What's unpythonic are useless getters and setters - since Python have a strong support for computed attributes. This doesn't mean you shouldn't properly encapsulate your implementation.
In your above exemple, the way your AnimalShelter class handles it's "owned" animals is an implementation detail and should not be exposed, so it's totally pythonic to use protected attribute and expose a relevant set of public methods / properties:
class AnimalShelter(object):
def __init__(self):
self._dogs = []
self._cats = []
def add_dog(self, dog):
if dog not in self._dogs:
self._dogs.append(dog)
def get_dogs(self):
return self._dogs[:] # return a shallow copy
# etc