I have an API that receives a serialized representation of an object that I expect to comply with a particular interface (known at development time). The serialized data I receive includes details that are used to create implementations of the methods/properties of this interface, so the actual object gets constructed at runtime; however, method names, signatures, property types, etc. are expected to match those known from the interface at development time. I would like to be able to construct this object at runtime, and then verify interface compliance, preferably failing immediately once an invalid object is constructed, not just when I try to invoke a method that's not there.
I am new to Python, so I am not sure if there is an idiomatic way of doing such a check. I have investigated using Abstract Base Classes, and annotating my constructed object with such a class. Using annotations is convenient during development time because I can get intellisense in VSCode, but they are not used to verify that my constructed object implements the ABC correctly at runtime - for example, when it is passed into a method as a parameter, like this:
def my_method(self, generated_object: MyABC):
Is there another approach to doing what I have described (casting/coercing to the ABC, or perhaps using a different language feature)? Or is my best bet to implement my own validator that will compare the methods/properties on the constructed object vs those on the ABC?
import abc
class Base(abc.ABC):
#abc.abstractmethod
def i_require_this(self):
pass
class Concrete(Base):
def __init__(self):
return
concrete = Concrete()
TypeError: Can't instantiate abstract class Concrete with abstract methods i_require_this
Related
I am trying to create a custom trait which represents a unipath.Path object. It seems advantageous to re-use the machinery provided by the File trait, so my thought was to use multiple inheritance.
from unipath import Path
from traits import File
class PathTrait(Path,File):
pass
class A(HasTraits):
p = PathTrait()
However, when i used this via A(p='/tmp/'), A.p does not have any methods associated with the Path object, as i would expect. Should i be implementing get and set methods?
What do you expect A(p='/tmp') should do?
I can tell what you are trying to do but this statement should fail with TypeError if your code was correct. Instead of type error, you are replacing the variable P on the A object, which was previously an instance of PathTrait, with a string.
What you're trying to do is conceptually mixed up. File is a class which represents a trait object. Technically python allows you to extend this object, because python has very little type safety, but it doesn't mean your python class will now suddenly act like a trait.
To define custom traits you will need to use tools designed to operate with traits such as the Trait constructor.
I have to serialise a dynamically created class hierarchy. And a bunch of objects - instances of the latter classes.
Python pickle is not of big help, its wiki says "Classes ... cannot be pickled". O there may be some trick that I cannot figure.
Performance requirement:
Deserialization should be pretty fast, because the serialised staff serves for cache and should save me the work of creating the same class hierarchy.
Details:
classes are created dynamically using type and sometimes meta-classes.
If you provide a custom object.__reduce__() method I believe you can still use pickling.
Normally, when pickling, the class import path is stored, plus instance state. On unpickling, the class is imported, and a new instance is created using the stored state. This is why pickling cannot work with dynamic classes, there is nothing to import.
The object.__reduce__() method lets you store a different instance factory. The callable returned by this function is stored (again by import path), and called with specified arguments to produce an instance. This instance is then used to apply state to, in the same way a regular instance would be unpickled:
def class_factory(name):
return globals()[name]()
class SomeDynamicClass(object):
def __reduce__(self):
return (class_factory, (type(self).__name__,), self.__dict__)
Here __reduce__ returns a function, the arguments for the function, and the instance state.
All you need to do then, is provide the right arguments to the factory function to recreate the class, and return an instance of that class. It'll be used instead of importing the class directly.
Classes are normal python objects, so, in theory, should be picklable, if you provide __reduce__ (or implement other pickle protocol methods) for them. Try to define __reduce__ on their metaclass.
I've been reading lots of previous SO discussions of factory functions, etc. and still don't know what the best (pythonic) approach is to this particular situation. I'll admit up front that i am imposing a somewhat artificial constraint on the problem in that i want my solution to work without modifying the module i am trying to extend: i could make modifications to it, but let's assume that it must remain as-is because i'm trying to understand best practice in this situation.
I'm working with the http://pypi.python.org/pypi/icalendar module, which handles parsing from and serializing to the Icalendar spec (hereafter ical). It parses the text into a hierarchy of dictionary-like "component" objects, where every "component" is an instance of a trivial derived class implementing the different valid ical types (VCALENDAR, VEVENT, etc.) and they are all spit out by a recursive factory from the common parent class:
class Component(...):
#classmethod
def from_ical(cls, ...)
I have created a 'CalendarFile' class that extends the ical 'Calendar' class, including in it generator function of its own:
class CalendarFile(Calendar):
#classmethod
def from_file(cls, ics):
which opens a file (ics) and passes it on:
instance = cls.from_ical(f.read())
It initializes and modifies some other things in instance and then returns it. The problem is that instance ends up being a Calendar object instead of a CalendarFile object, in spite of cls being CalendarFile. Short of going into the factory function of the ical module and fiddling around in there, is there any way to essentially "recast" that object as a 'CalendarFile'?
The alternatives (again without modifying the original module) that I have considered are:make the CalendarFile class a has-a Calendar class (each instance creates its own internal instance of a Calendar object), but that seems methodically stilted.
fiddle with the returned object to give it the methods it needs (i know there's a term for creating a customized object but it escapes me).
make the additional methods into functions and just have them work with instances of Calendar.
or perhaps the answer is that i shouldn't be trying to subclass from a module in the first place, and this type of code belongs in the module itself.
Again i'm trying to understand what the "best" approach is and also learn if i'm missing any alternatives. Thanks.
Normally, I would expect an alternative constructor defined as a classmethod to simply call the class's standard constructor, transforming the arguments that it receives into valid arguments to the standard constructor.
>>> class Toy(object):
... def __init__(self, x):
... self.x = abs(x)
... def __repr__(self):
... return 'Toy({})'.format(self.x)
... #classmethod
... def from_string(cls, s):
... return cls(int(s))
...
>>> Toy.from_string('5')
Toy(5)
In most cases, I would strongly recommend something like this approach; this is the gold standard for alternative constructors.
But this is a special case.
I've now looked over the source, and I think the best way to add a new class is to edit the module directly; otherwise, scrap inheritance and take option one (your "has-a" option). The different classes are all slightly differentiated versions of the same container class -- they shouldn't really even be separate classes. But if you want to add a new class in the idiom of the code as it it is written, you have to add a new class to the module itself. Furthermore, from_iter is deceptively named; it's not really a constructor at all. I think it should be a standalone function. It builds a whole tree of components linked together, and the code that builds the individual components is buried in a chain of calls to various factory functions that also should be standalone functions but aren't. IMO much of that code ought to live in __init__ where it would be useful to you for subclassing, but it doesn't.
Indeed, none of the subclasses of Component even add any methods. By adding methods to your subclass of Calendar, you're completely disregarding the actual idiom of the code. I don't like its idiom very much but by disregarding that idiom, you're making it even worse. If you don't want to modify the original module, then forget about inheritance here and give your object a has-a relationship to Calendar objects. Don't modify __class__; establish your own OO structure that follows standard OO practices.
In C# I would go:
myObj.base
I have a Date class which inherits from date.datetime. The Date class overrides __gt__() and __lt__() so when using the < and > operators they are called. I do not want to use these overrides - I want to use the date.datetime methods on an instance of Date.
Use super() to get the superclass object. Type help(super) in the Python command prompt.
From the manual:
class super(object)
| super(type) -> unbound super object
| super(type, obj) -> bound super object; requires isinstance(obj, type)
| super(type, type2) -> bound super object; requires issubclass(type2, type)
| Typical use to call a cooperative superclass method:
| class C(B):
| def meth(self, arg):
| super(C, self).meth(arg)
If I understand your question correctly, this doesn't make sense in Python. There's no "base instance" inside the instance of a subclass.
A Python instance is just one thing, containing a collection of attributes (any of which may have been set/modified by any of its base classes, or indeed from code outside any of its classes at all). Indeed it's possible to change the class of an instance at runtime, even transplanting it into an entirely different inheritance heirarchy (this is not frequently a good idea, but it's well-defined). No matter what, the instance remains a single unitary object, which only knows what attributes it has and which class it's an instance of (and in fact that's just an attribute: __class__).
Edit: If what you want is to be able to invoke overridden methods on an instance, then you do that by using super, as hocl answered. However it seems from your comments that you do not fully grok what super is doing (which is natural, as it's quite complex).
super(Date, myObj) doesn't return "the underlying datetime.date" instance, because there's no such thing, only the myObj object. Although for your purposes it sounds like this will fill your needs (and you can probably stop at this sentence).
What it does is return is a magical wrapper around myObj that looks up methods starting just "behind" Date; i.e. it finds the method that would be called if Date didn't override it. So in this case it will find all methods from datetime.date, because you only have single inheritance going on.
A key difference is that this supports multiple inheritance. If someone makes another class that inherits from Date and also inherits from datetime.date by another path, then super(Date, instanceOfThatClass) may not actually hit datetime.date's methods. It depends on the details of that inheritance heirarchy. This is actually the situation super was designed for though; it enabled classes in complex multiple inheritance hierarchies to cooperatively call each other's implementations, ensuring that each is called only once, and in an order that is sensible (though it may not be the same order for each ultimate leaf class, so classes in the middle of the hierarchy actually don't know which super-class implementation they're calling). My understanding is this complex situation cannot arise in C#, hence it can provide a simple .base syntax.
My understanding is also (and this is a bit of a guess), that because C# is statically typed and supports inheriting from classes defined in pre-compiled libraries, that when you have class Sub inheriting from class Base, inside an instance of Sub there really is a complete instance of Base which you can get at and then call methods on. This will affect shadowed fields; I would expect (again, as a non-C# programmer guessing a bit) that after getting the Base instance from a Sub instance, any direct reference to a field overridden by Sub would hit the field from Base, not the one from Sub.
In Python, OTOH, there is no base instance, and classes can't override fields. If the constructor and methods of Date and datetime.date both refer to the same field, it's just the one field in the instance that they're both sharing. So using super won't change what field you'll access, as you might expect if you think of it as getting the base instance.
Given that you're not using a complex multiple inheritance situation, if you wanted a simple syntax you could actually call Date.__lt__(myObj, otherObj) directly, though it looks ugly because you don't get to use the infix operator syntax when you do it that way. It's less horrible if you're considering ordinary methods; in that case it's possibly simpler than using super.
Take home message: I'm pretty sure super(Date, myObj) is what you want in this case. But if you get into more complicated situations, you don't want to think of super as the way to get "the base instance", like you would in C#. That understanding will trip you up when in multiple inheritance (which can be bloody confusing anyway), but also when you have multiple layers in the inheritance hierarchy using the same field.
I want to define a base class so that when derived class instances are pickled, they are pickled as if they are instances of the base class. This is because the derived classes may exist on the client side of the pickling but not on the server side, but this is not important to the server since it only needs information from the base class. I don't want to have to dynamically create classes for every client.
The base class is simply an "object handle" which contains an ID, with methods defined on the server, but I would like the client to be able to subclass the server classes and define new methods (which would only be seen by the client, but that doesn't matter).
I believe you can do it by giving the object a __reduce__ method, returning a tuple, the first part of which should be BaseClass.__new__ (this will be called when loading the object in unpickling). See the pickle documentation (Python 2, Python 3) for the full details. I haven't attempted this.
Depending on what you're doing, it might be easier to use a simpler serialisation format like JSON, and have code on each side to reconstruct the relevant objects.
You can change an object's class dynamically in Python:
import cPickle as pickle
class Foo(object):
def __init__(self):
self.id=1
class Bar(Foo):
def derived_class_method(self): pass
bar=Bar()
bar.id=2
bar.__class__=Foo # changes `bar`'s class to Foo
bar_pickled=pickle.dumps(bar)
bar2=pickle.loads(bar_pickled)
bar.__class__=Bar # reset `bar`'s class to Bar
print(repr(bar2))
# <__main__.Foo object at 0xb76b08ec>
print(bar2.id)
# 2
I'm not sure using this is the best design decision, however. I like Thomas K's idea of using JSON.