Recently I have started to use python3 and found out that there is built-in Enum type and it is possible to use it like it is used in Java, say for singleton creation. I can load a config of my telegram bot like this:
class Config(Enum):
DATA = 'data'
def __init__(self, dummy):
with open('./app/config/data.yml', 'r') as f:
self.__data = yaml.load(f)
def __getitem__(self, item):
"""
May be too expensive, but config should not be mutable
so it returns a defensive copy for mutable sequences.
:param item: requested item
:return: value of the item
"""
value = self.__data[item]
return deepcopy(self.__data[item]) if isinstance(value, (MutableMapping, MutableSequence)) else value
#property
def authorized_url(self):
return '{}/bot{}'.format(self['base_url'], self['token'])
#property
def updates_url(self):
return self.authorized_url + self['endpoints']['get_updates']
#property
def send_message_url(self):
return self.authorized_url + self['endpoints']['send_message']
I assume it appropriate as config is a kind of a singleton, and I don't have to bother with instance creation. It is created automatically during module import and encapsulated inside of the class. So I get everything just out of the box, plus I can define handy methods for aggregation of values.
So I am curious whether it is OK to use enums like this in python?
If not, I would like to hear why not.
This is not the purpose of enums. This is using a side effect of enums to do something else entirely.
.. and I don't have to bother with instance creation.
Instance creation in this case is one line of code:
my_config = Config()
And the inclusion of this one line would allow the elimination of one line, as this:
from enum import Enum
class Config(Enum):
becomes simply:
class Config():
So this construct actually requires more code to express, and seems to be rather overdone. If I ran across this in a code review I would reject it.
Related
Is there are a way in Python to store instantiated class as a class 'template' (aka promote object to a class) to create new objects of same type with same fields values, without relying on using data that was used to create original object again or on copy.deepcopy?
Like, for example I have the dictionary:
valid_date = {"date":"30 february"} # dict could have multiple items
and I have the class:
class AwesomeDate:
def __init__(self, dates_dict):
for key, val in dates_dict.items():
setattr(self, key, val);
I create the instance of the class like:
totally_valid_date = AwesomeDate(valid_date)
print(totally_valid_date.date) # output: 30 february
and now I want to use it to create new instances of the AwesomeDate class using the totally_valid_date instance as a template, i.e. like:
how_make_it_work = totally_valid_date()
print(how_make_it_work.date) # should print: 30 february
Is there are way to do so or no? I need a generic solution, not a solution for this specific example.
I don't really see the benefit of having a class act both as a template to instances, and as the instance itself, both conceptually and coding-wise. In my opinion, you're better off using two different classes - one for the template, one for the objects it is able to create.
You can think about awesome_date as a template class that stores the valid_date attributes upon initialization. Once called, the template returns an instance of a different class that has the expected attributes.
Here's a simple implementation (names have been changed to generalize the idea):
class Thing:
pass
class Template:
def __init__(self, template_attrs):
self.template_attrs = template_attrs
def __call__(self):
instance = Thing()
for key, val in self.template_attrs.items():
setattr(instance, key, val)
return instance
attrs = {'date': '30 february'}
template = Template(template_attrs=attrs)
# Gets instance of Thing
print(template()) # output: <__main__.Thing object at 0x7ffa656f8668>
# Gets another instance of Thing and accesses the date attribute
print(template().date) # output: 30 february
Yes, there are ways to do it -
there could even be some tweaking of inheriting from type and meddling with __call__ to make all instances automatically become derived classes. But I don't think that would be very sane. Python's own enum.Enum does something along this, because it has some use for the enum values - but the price is it became hard to understand beyond the basic usage, even for seasoned Pythonistas.
However, having a custom __init_subclass__ method that can inject some code to run prior to __init__ on the derived class, and then a method that will return a new class bound with the data that the new classes should have, can suffice:
import copy
from functools import wraps
def wrap_init(init):
#wraps(init)
def wrapper(self, *args, **kwargs):
if not getattr(self, "_initalized", False):
self.__dict__.update(self._template_data or {})
self._initialized = True
return init(self, *args, **kwargs)
wrapper._template_wrapper = True
return wrapper
class TemplateBase:
_template_data = None
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
if getattr(cls.__init__, "_template_wraper", False):
return
init = cls.__init__
cls.__init__ = wrap_init(init)
def as_class(self):
cls= self.__class__
new_cls = type(cls.__name__ + "_templated", (cls,), {})
new_cls._template_data = copy.copy(self.__dict__)
return new_cls
And using it:
class AwesomeDate(TemplateBase):
def __init__(self, dates_dict):
for key, val in dates_dict.items():
setattr(self, key, val)
On the REPL we have:
In [34]: x = AwesomeDate({"x":1, "y":2})
In [35]: Y = x.as_class()
In [36]: y = Y({})
In [37]: y.x
Out[37]: 1
Actually, __init_subclass__ itself could be supressed, and decorating __init__ could be done in one shot on the as_class method. This code takes some care so that mixin classes can be used, and it will still work.
It seems like you are going for something along the lines of the prototype design pattern.
What is the prototype design pattern?
From Wikipedia: Prototype pattern
The prototype pattern is a creational design pattern in software development. It is used when the type of objects to create is determined by a prototypical instance, which is cloned to produce new objects. This pattern is used to avoid subclasses of an object creator in the client application, like the factory method pattern does and to avoid the inherent cost of creating a new object in the standard way (e.g., using the 'new' keyword) when it is prohibitively expensive for a given application.
From Refactoring.guru: Prototype
Prototype is a creational design pattern that lets you copy existing objects without making your code dependent on their classes. The Prototype pattern delegates the cloning process to the actual objects that are being cloned. The pattern declares a common interface for all objects that support cloning. This interface lets you clone an object without coupling your code to the class of that object. Usually, such an interface contains just a single clone method.
The implementation of the clone method is very similar in all classes. The method creates an object of the current class and carries over all of the field values of the old object into the new one. You can even copy private fields because most programming languages let objects access private fields of other objects that belong to the same class. An object that supports cloning is called a prototype. When your objects have dozens of fields and hundreds of possible configurations, cloning them might serve as an alternative to subclassing. Here’s how it works: you create a set of objects, configured in various ways. When you need an object like the one you’ve configured, you just clone a prototype instead of constructing a new object from scratch.
Implementing this for your problem, along with your other ideas
From your explanation, it seems like you want to:
Provide a variable containing a dictionary, which will be passed to the __init__ of some class Foo
Instantiate class Foo and pass the variable containing the dictionary as an argument.
Implement __call__ onto class Foo, allowing us to use the function call syntax on an object of class Foo.
The implementation of __call__ will COPY/CLONE the “template” object. We can then do whatever we want with this copied/cloned instance.
The Code (edited)
import copy
class Foo:
def __init__(self, *, template_attrs):
if not isinstance(template_attrs, dict):
raise TypeError("You must pass a dict to instantiate this class.")
self.template_attrs = template_attrs
def __call__(self):
return copy.copy(self)
def __repr__(self):
return f"{self.template_attrs}"
def __setitem__(self, key, value):
self.template_attrs[key] = value
def __getitem__(self, key):
if key not in self.template_attrs:
raise KeyError(f"Key {key} does not exist in '{self.template_attrs=}'.")
return self.template_attrs[key]
err = Foo(template_attrs=1) # Output: TypeError: You must pass a dict to instantiate this class.
# remove err's assignment to have code under it run
base = Foo(template_attrs={1: 2})
print(f"{base=}") # Output: base={1: 2}
base_copy = base()
base_copy["hello"] = "bye"
print(f"{base_copy=}") # Output: base_copy={1: 2, 'hello': 'bye'}
print(f"{base_copy[1]=}") # Output: base_copy[1]=2
print(f"{base_copy[10]=}") # Output: KeyError: "Key 10 does not exist in 'self.template_attrs={1: 2, 'hello': 'bye'}'."
I also added support for subscripting and item assignment through __getitem__ and __setitem__ respectively. I hope that this helped a bit with your problem! Feel free to comment on this if I missed what you were asking.
Reasons for edits (May 16th, 2022 at 8:49 PM CST | Approx. 9 hours after original answer)
Fix code based on suggestions by comment from user jsbueno
Handle, in __getitem__, if an instance of class Foo is subscripted with a key that doesn't exist in the dict.
Handle, in __init__, if the type of template_attrs isn't dict (did this based on the fact that you used a dictionary in the body of your question)
Let's assume I am using a library which gives me instances of classes defined in that library when calling its functions:
>>> from library import find_objects
>>> result = find_objects("name = any")
[SomeObject(name="foo"), SomeObject(name="bar")]
Let's further assume that I want to attach new attributes to these instances. For example a classifier to avoid running this code every time I want to classify the instance:
>>> from library import find_objects
>>> result = find_objects("name = any")
>>> for row in result:
... row.item_class= my_classifier(row)
Note that this is contrived but illustrates the problem: I now have instances of the class SomeObject but the attribute item_class is not defined in that class and trips up the type-checker.
So when I now write:
print(result[0].item_class)
I get a typing error. It also trips up auto-completion in editors as the editor does not know that this attribute exists.
And, not to mention that this way of implementing this is quite ugly and hacky.
One thing I could do is create a subclass of SomeObject:
class ExtendedObject(SomeObject):
item_class = None
def classify(self):
cls = do_something_with(self)
self.item_class = cls
This now makes everything explicit, I get a chance to properly document the new attributes and give it proper type-hints. Everything is clean. However, as mentioned before, the actual instances are created inside library and I don't have control over the instantiation.
Side note: I ran into this issue in flask for the Response class. I noticed that flask actually offers a way to customise the instantiation using Flask.response_class. But I am still interested how this could be achieved in libraries that don't offer this injection seam.
One thing I could do is write a wrapper that does something like this:
class WrappedObject(SomeObject):
item_class = None
wrapped = None
#staticmethod
def from_original(wrapped):
self.wrapped = wrapped
self.item_class = do_something_with(wrapped)
def __getattribute__(self, key):
return getattr(self.wrapped, key)
But this seems rather hacky and will not work in other programming languages.
Or try to copy the data:
from copy import deepcopy
class CopiedObject(SomeObject):
item_class = None
#staticmethod
def from_original(wrapped):
for key, value in vars(wrapped):
setattr(self, key, deepcopy(value))
self.item_class = do_something_with(wrapped)
but this feels equally hacky, and is risky when the objects sue properties and/or descriptors.
Are there any known "clean" patterns for something like this?
I would go with a variant of your WrappedObject approach, with the following adjustments:
I would not extend SomeObject: this is a case where composition feels more appropriate than inheritance
With that in mind, from_original is unnecessary: you can have a proper __init__ method
item_class should be an instance variable and not a class variable. It should be initialized in your WrappedObject class constructor
Think twice before implementing __getattribute__ and forwarding everything to the wrapped object. If you need only a few method and attributes of the original SomeObject class, it might be better to implement them explicitly as methods and properties
class WrappedObject:
def __init__(self, wrapped):
self.wrapped = wrapped
self.item_class = do_something_with(wrapped)
def a_method(self):
return self.wrapped.a_method()
#property
def a_property(self):
return self.wrapped.a_property
I want to create a subclass of a class of an existing package (whose source code I don't want to/cannot change). The objects of the class are initialized just using a string and then populated later on using all kind of add functions. A minimal example could look like this (without any add functions):
import copy
class Origin(object):
def __init__(self, name):
self.name = name
self.dummy_list = [1, 2, 'a']
self.dummy_stuff = {'a': [12, 'yt']}
def make_copy(self):
return copy.deepcopy(self)
def dummy_function(self):
return len(self.dummy_list)
I want to create a subclass in such a way that I can initialize its instances using an instance of Origin. A straightforward way would be
class BasedOnOrigin(Origin):
def __init__(self, origin_instance, new_prop):
Origin.__init__(self, origin_instance.name)
self.dummy_list = copy.deepcopy(origin_instance.dummy_list)
self.dummy_stuff = copy.deepcopy(origin_instance.dummy_stuff)
self.new_prop = new_prop
The annoying thing there is, that I need to copy all kind of things which I need to know about in advance.
Another option would be
class BasedOnOrigin2(Origin):
def __init__(self, origin_instance, new_prop):
Origin.__init__(self, origin_instance.name)
self = origin_instance.make_copy()
self.new_prop = new_prop
but the self = part looks rather non-standard and new_prop is not set, so I would need an extra function for this.
Is there a standard way of doing this?
An alternative to the above would be to add the additional functions to existing instances using e.g.
from functools import partial
def add_function(obj, func):
setattr(obj, func.__name__, partial(func, obj))
but this can be annoying if there are (i) a lot of functions to add and (ii) a lot of instances to which one wants to add functions.
but the self = part looks rather non-standard and new_prop is not set
self is just a plain local variable, so rebinding it only effects the local scope indeed.
Is there a standard way of doing this?
From what you describe it looks like your real problem is that you have instances of class created by another lib that you don't want / cannot modify and what you really want is to add new methods (and eventually override some methods) to those objects, but cannot since you can tell this lib to use your own class instead.
If the point is purely and simply "replace" the original class with your own version of it (so all instances of the original class are impacted by the change), the canonical solution is to monkeypatch the original class:
from otherlib import TheClass
def patch_the_class():
# we do this in a function to avoid
# polluting the global namespace
# add a new method
def newmethod(self):
# code here
TheClass.newmethod = newmethod
# override an existing method
# keep a reference to the original so
# we can still use it:
_original = TheClass.some_method
def mymethod(self, arg):
something = _original(self, arg)
# additional stuff here
return something
TheClass.some_method = mymethod
patch_the_class()
Just make sure this is executed before any use of the patched class and you're done.
The pro of this solution (wrt/ patching each instance individually) is a lesser cost and the assurance that no one will ever forget to patch an instance.
Now note that monkeypatches are to be considered as either a temporary workaround or a last-resort hack. If the lib you are patching is OSS, you can modify it to either improve the original class or implement some way to make the concrete class to use configurable and contribute it back.
I think the best approach is defining a function that will extend original origin instance without copying it e.g.
def exdend(*origin_instances):
def my_function_one(self):
pass
def my_function_two(self):
pass
for origin_instance in origin_instances:
setattr(origin_instance, my_function_one.__name__, partial(my_function_one, origin_instance))
setattr(origin_instance, my_function_two.__name__, partial(my_function_two, origin_instance))
return origin_instances
I'm trying to use the heappq module in the python 3.5 standard library to make a priority queue of objects of the same type. I'd like to be able to heapify based on an attribute of the objects, then change the value of some of those attributes, then re-heapify based on the new values. I'm wondering how I go about doing this.
import heappq
class multiNode:
def __init__(self, keyValue):
self.__key = keyValue
def setKey(self, keyValue):
self.__key = keyValue
def getKey(self):
return self.__key
queue = [multiNode(1), multiNode(2), multiNode(3)]
heapq.heapify(queue) #want to heapify by whatever getKey returns for each node
queue[0].setKey(1000)
heapq.heapify(queue) #re heapify with those new values
There are a variety of ways of making your code work. For instance, you could make your items orderable by implementing some of the rich comparison operator methods (and perhaps use functools.total_ordering to implement the rest):
#functools.total_ordering
class multiNode:
def __init__(self, keyValue):
self.__key = keyValue
def setKey(self, keyValue):
self.__key = keyValue
def getKey(self):
return self.__key
def __eq__(self, other):
if not isinstance(other, multiNode):
return NotImplemented
return self.__key == other.__key
def __lt__(self, other):
if not isinstance(other, multiNode):
return NotImplemented
return self.__key < other.__key
This will make your code work, but it may not be very efficient to reheapify your queue every time you make a change to a node within it, especially if there are a lot of nodes in the queue. A better approach might be to write some extra logic around the queue so that you can invalidate a queue entry without removing it or violating the heap property. Then when you have an item you need to update, you just invalidate it's old entry and add in a new one with the new priority.
Here's a quick and dirty implementation that uses a dictionary to map from a node instance to a [pritority, node] list. If the node is getting its priority updated, the dictionary is checked and the node part of the list gets set to None. Invalidated entries are ignored when popping nodes off the front of the queue.
queue = []
queue_register = {}
def add_to_queue(node)
item = [node.getKey(), node]
queue.heappush(queue, item)
queue_register[node] = item
def update_key_in_queue(node, new_key):
queue_register[node][1] = None # invalidate old item
node.setKey(new_key)
add_to_queue(node)
def pop_from_queue():
node = None
while node is None:
_, node = heapq.heappop(queue) # keep popping items until we find one that's valid
del queue_register[node] # clean up our bookkeeping record
return node
You may want to test this against reheapifying to see which is faster for your program's actual usage of the queue.
A few final notes about your multiNode class (unrelated to what you were asking about in your question):
There are a number of things you're doing in the class that are not very Pythonic. To start with, the most common naming convention for Python uses CapitalizedNames for classes, and lower_case_names_with_underscores for almost everything else (variables of all kinds, functions, modules).
Another issue using double leading underscores for __key. Double leading (and not trailing) undescrores invokes Python's name mangling system. This may seem like its intended as a way to make variables private, but it is not really. It's more intended to help prevent accidental name collisions, such as when you're setting an attribute in a proxy object (that otherwise mimics the attributes of some other object) or in a mixin class (which may be inherited by other types with unknown attributes). If code outside your class really wants to access the mangled attribute __key in your multiNode class, they can still do so by using _multiNode__key. To hint that something is intended to be a private attribute, you should just use a single underscore _key.
And that brings me right to my final issue, that key probably shouldn't be private at all. It is not very Pythonic to use getX and setX methods to modify a private instance variable. It's much more common to document that the attribute is part of the class's public API and let other code access it directly. If you later decide you need to do something fancy whenever the attribute is looked up or modified, you can use a property descriptor to automatically transform attribute access into calls to a getter and setter function. Other programming languages usually start with getters and setters rather than public attributes because there is no such way of changing implementation of an attribute API later on. So anyway, I'd make your class's __init__ just set self.key = keyValue and get rid of setKey and getKey completely!
A crude way of doing what you're looking for would be to use dicts and Python's built in id() method. This method would basically allow you keep your heap as a heap of the id's of the objects that you create and then update those objects by accessing them in the dict where their id's are the keys. I tried this on my local machine and it seems to do what you're looking for:
import heapq
class multiNode:
def __init__(self, keyValue):
self.__key = keyValue
def setKey(self, keyValue):
self.__key = keyValue
def getKey(self):
return self.__key
first_node = multiNode(1)
second_node = multiNode(2)
thrid_node = multiNode(3)
# add more nodes here
q = [id(first_node), id(second_node), id(third_node)]
mutilNode_dict = {
id(first_node): first_node,
id(second_node): second_node,
id(third_node): third_node
}
heapq.heapify(q)
multiNode_dict[q[0]].setKey(1000)
heapq.heapify(q)
heapify() won't really do too much here because the id of the object is going to be the same until it's deleted. It is more useful if you're adding new objects to the heap and taking objects out.
This is a snippet for registers for an emulator I'm working on:
class registers(object):
def __init__(self):
self._AF = registerpair()
def _get_AF(self):
return self._AF.getval()
def _set_AF(self, val):
self._AF.setval(val)
AF = property(_get_AF, _set_AF)
The registerpair() class has an increment() method. I would like to know if there is any way I could do the following:
r = registers()
r.AF.increment()
rather than having to do:
r._AF.increment()
As is, no. You have set the fget method to return a getval() for your registerpair() class.
Since the property is for the _AF attribute which is a registerpair() instance, I believe it would be more reasonable to change your fget (and fset for that matter) to actually return it, and maybe create an auxiliary function to actually get the value with getval() or access it directly.
So if your _get_AF looked something like:
def _get_AF(self):
return self._AF
you can then call r.AF.increment() just fine. Then you could move the getval() call to another function in your class:
def getAFval(self):
self._AF.getval()
Or just make direct calls like r.AF.getval() which seems like the most clear way to do things.
You are effectively modifying the interface to the registerpair class using this wrapper class, and in doing so hiding the original interface. As such in your new interface the property() in Python refers to the values stored in the registerpair, not to the registerpair itself, as it reimplements the getval() and setval() interface of the registerpair.
So a couple of suggestions, firstly if this wrapper class is just reimplementing the interface to the registerpair, should you not just inherit from the registerpair, that way the original interface would be available?
Alternatively you could implement the remainder of the registerpair interface, using for example a method such as registers.increment_AF():
class registers(object):
def __init__(self):
self._AF = registerpair()
def _get_AF(self):
return self._AF.getval()
def _set_AF(self, val):
self._AF.setval(val)
AF = property(_get_AF, _set_AF)
def increment_AF(self):
self._AF.increment()
If I understand you correctly
You can call r._AF.increment() which references registerpair() object but since self._AF is a private method you cannot use
r.AF.increment()
for further information..check this
https://www.python.org/dev/peps/pep-0008/
an extract from this site
_single_leading_underscore : weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.
single_trailing_underscore_ : used by convention to avoid conflicts with Python keyword, e.g.