We Keep Coding

How can I specialise instances of objects when I don't have access to the instantiation code?

Let's assume I am using a library which gives me instances of classes defined in that library when calling its functions:
>>> from library import find_objects
>>> result = find_objects("name = any")
[SomeObject(name="foo"), SomeObject(name="bar")]
Let's further assume that I want to attach new attributes to these instances. For example a classifier to avoid running this code every time I want to classify the instance:
>>> from library import find_objects
>>> result = find_objects("name = any")
>>> for row in result:
... row.item_class= my_classifier(row)
Note that this is contrived but illustrates the problem: I now have instances of the class SomeObject but the attribute item_class is not defined in that class and trips up the type-checker.
So when I now write:
print(result[0].item_class)
I get a typing error. It also trips up auto-completion in editors as the editor does not know that this attribute exists.
And, not to mention that this way of implementing this is quite ugly and hacky.
One thing I could do is create a subclass of SomeObject:
class ExtendedObject(SomeObject):
item_class = None
def classify(self):
cls = do_something_with(self)
self.item_class = cls
This now makes everything explicit, I get a chance to properly document the new attributes and give it proper type-hints. Everything is clean. However, as mentioned before, the actual instances are created inside library and I don't have control over the instantiation.
Side note: I ran into this issue in flask for the Response class. I noticed that flask actually offers a way to customise the instantiation using Flask.response_class. But I am still interested how this could be achieved in libraries that don't offer this injection seam.
One thing I could do is write a wrapper that does something like this:
class WrappedObject(SomeObject):
item_class = None
wrapped = None
#staticmethod
def from_original(wrapped):
self.wrapped = wrapped
self.item_class = do_something_with(wrapped)
def __getattribute__(self, key):
return getattr(self.wrapped, key)
But this seems rather hacky and will not work in other programming languages.
Or try to copy the data:
from copy import deepcopy
class CopiedObject(SomeObject):
item_class = None
#staticmethod
def from_original(wrapped):
for key, value in vars(wrapped):
setattr(self, key, deepcopy(value))
self.item_class = do_something_with(wrapped)
but this feels equally hacky, and is risky when the objects sue properties and/or descriptors.
Are there any known "clean" patterns for something like this?

I would go with a variant of your WrappedObject approach, with the following adjustments:
I would not extend SomeObject: this is a case where composition feels more appropriate than inheritance
With that in mind, from_original is unnecessary: you can have a proper __init__ method
item_class should be an instance variable and not a class variable. It should be initialized in your WrappedObject class constructor
Think twice before implementing __getattribute__ and forwarding everything to the wrapped object. If you need only a few method and attributes of the original SomeObject class, it might be better to implement them explicitly as methods and properties
class WrappedObject:
def __init__(self, wrapped):
self.wrapped = wrapped
self.item_class = do_something_with(wrapped)
def a_method(self):
return self.wrapped.a_method()
#property
def a_property(self):
return self.wrapped.a_property

How can I resolve a circular import of two Python classes in the same module?

I have two tightly-coupled Python classes that need references to each other (at the class, not instance, level). How can I resolve the circular imports? Ideally I'd like to be able to make it work either within the same module or between two distinct modules, but I'll settle for one or the other.
# yin_yang.py
class MyYin(Yin):
__yang__ = MyYang
class MyYang(Yang):
__yin__ = MyYin

You could set the class attributes for one or both classes after they have been declared.
class MyYin(Yin):
pass
class MyYang(Yang):
__yin__ = MyYin
MyYin.__yang__ = MyYang

While #phillip-martin's response is the most pythonic one, there is an alternative way to accomplish the task:
from werkzeug import LocalProxy
class MyYin:
__yang__ = LocalProxy(lambda: MyYang)
foo = 42
class MyYang:
__yin__ = LocalProxy(lambda: MyYin)
bar = 9002
print(MyYin.__yang__.bar)
print(MyYang.__yin__.foo)
And the magic behind LocalProxy trick comes from overriding all the __getattr__, __setattr__, __etc__ methods. Check it out in the werkzeug repo.

When the Python interpreter finds a class declaration, it creates a new scope and executes the code inside the class in this code block, i.e., all class variables are instantiated when the class declaration is executed.
You can avoid this in a really simple manner:
class MyYin(Yin):
pass
class MyYang(Yang):
__yin__ = MyYin
MyYin.__yang__ = MyYang
or
class MyYang(Yang):
pass
class MyYin(Yin):
__yang__ = MyYang
MyYang.__yin__ = MyYin

Set class variable value, the value returned by a class method

I'm trying to create a class which maps to a mongoDB collection.
My code looks like this:
class Collection:
_collection = get_collection() # This seems not working
#classmethod
def get_collection(cls):
collection_name = cls.Meta.collection_name if cls.Meta.collection_name \
else cls.__name__.lower()
collection = get_collection_by_name(collection_name) # Pseudo code, please ignore
return collection
class Meta:
collection_name = 'my_collection'
I came across a situation where I need to assign the class variable _collection with the return value of get_collection.
I also tried _collection = Collection.get_collection() which also seems not to be working
As a work-around, I subclassed Collection and set value of _collection in the child class.
Would like to know any simple solution for this.
Thanks in advance

As DeepSpace mentions, here:
class Collection:
_collection = get_collection() # This seems not working
#classmethod
def get_collection(cls):
# code that depends on `cls`
the get_collection method is not yet defined when you call it. But moving this line after the method definition won't work either, since the method depends on the Collection class (passed as cls to the method), which itself won't be defined before the end of the class Collection: statement's body.
The solution here is to wait until the class is defined to set this attribute. Since it looks like a base class meant to be subclassed, the better solution would be to use a metaclass:
class CollectionType(type):
def __init__(cls, name, bases, attrs):
super(CollectionType, cls).__init__(name, bases, attrs)
cls._collection = cls.get_collection()
# py3
class Collection(metaclass=CollectionType):
# your code here
# py2.7
class Collection(object):
__metaclass__ = CollectionType
# your code here
Note however that if Collection actually inherit from a another class already having a custom metaclass (ie Django Model class or equivalent) you will need to make CollectionType a subclass of this metaclass instead of a subclass of type.

There are some design/syntax errors in your code.
When the line _collection = get_collection() executes, get_collection is not yet defined. As a matter of fact, the whole Collection class is not yet defined.
get_collection_by_name is not defined anywhere.
EDIT OP updated the question so the below points may not be relevant anymore
collection = get_collection(collection_name) should be collection = cls.get_collection(collection_name)
Sometimes you are passing a parameter to get_collection and sometimes you don't, however get_collection's signature never accepts a parameter.
Calling get_collection will lead to an infinite recursion.
You have to take a step back and reconsider the design of your class.

python - register all subclasses

I have a class Step, which I want to derive by many sub-classes. I want every class deriving from Step to be "registered" by a name I choose for it (not the class's name), so I can later call Step.getStepTypeByName().
Something like this, only working :):
class Step(object):
_STEPS_BY_NAME = {}
#staticmethod
def REGISTER(cls, name):
_STEPS_BY_NAME[name] = cls
class Derive1(Step):
REGISTER(Derive1, "CustomDerive1Name")
...
class Derive2(Step):
REGISTER(Derive2, "CustomDerive2Name")
...

Your solution do not work for three reasons.
The first one is that _STEPS_BY_NAME only exists as an attribute of the Step class, so Step.REGISTER cannot access _STEPS_BY_NAME without a reference to the Step class. IOW you have to make it a classmethod (cf below)
The second one is that you need to explicitely use Step.REGISTER(cls) - the name REGISTER does not exist outside the Step class.
The third reason is that within a class statement's body, the class object has not yet been created not bound to it's name, so you cannot not reference the class itself at this point.
IOW, you'd want this instead:
class Step(object):
_STEPS_BY_NAME = {}
# NB : by convention, "ALL_UPPER" names denote pseudo-constants
#classmethod
def register(cls, name):
# here `cls` is the current class
cls._STEPS_BY_NAME[name] = stepclass
class Derive1(Step):
...
Step.register(Derive1, "CustomDerive1Name")
class Derive2(Step):
...
Step.register(Derive2, "CustomDerive2Name")
Now with a minor modification to Step.register you could use it as a class decorator, making things much clearer:
class Step(object):
_STEPS_BY_NAME = {}
#classmethod
def register(cls, name):
def _register(stepclass):
cls._STEPS_BY_NAME[name] = stepclass
return stepclass
return _register
#Step.register("CustomDerive1Name")
class Derive1(Step):
...
#Step.register("CustomDerive2Name")
class Derive2(Step):
...
As a last note: unless you have a compelling reason to register your subclasses in the base class itself, it might be better to use module-level variables and functions (a Python module is actually a kind of singleton):
# steps.py
class Step(object):
#....
_STEPS_BY_NAME = {}
def register(name):
def _register(cls):
_STEPS_BY_NAME[name] = cls
return cls
return _register
def get_step_class(name):
return _STEPS_BY_NAME[name]
And in your other modules
import steps
#steps.register("CustomDerive1Name")
class Derive1(steps.Step):
# ...
The point here is to avoid giving too many responsabilies to your Step class. I don't know your concrete use case so I can't tell which design best fits your need, but I've been using this last one on quite a few projects and it always worked fine so far.

You are close. Use this
class Step(object):
pass
class Derive1(Step):
pass
class Derive2(Step):
pass
_STEPS_BY_NAME = {
'foo': Step,
'bar': Derive1,
'bar': Derive2
}
def get_step_by_name(name):
return _STEPS_BY_NAME[name]
Warning: there might be better approaches depending on what you are trying to achieve. Such a mapping from strings to methods is a maintenance nightmare. If you want to change the name of a method, you would have to remember to change it in multiple place. You won't get any autocomplete help from your IDE either.

Why do classes with no constructor arguments need parenthesis

I started off learning programming/OOP in PHP. To the best of my knowledge of best practices in PHP, you can instantiate a class without parenthesis if it does not take any arguments.
Such as
$class = new Class;
As opposed to:
$class = new Class();
I am starting to expand my skills into python and wasted about 5 hours yesterday trying to figure out why a function wouldn't pass an argument even though it was ridiculously simple. My Code:
class MainViewWidgets(MainViewContainer):
def __init__(self):
# instantiating like this prevents MainViewController.getHeaderItems from returning the arg passed to it, however the code still "works" in some sense
self.controller = MainViewController
#this works
self.controller = MainViewController()
def createHeaderOptionCheckbox(self, pane):
self.header_string = StringVar()
header_checkbox = ttk.Checkbutton(pane, text='Data Contains Headers', variable=self.header_string, onvalue='headers', offvalue='keys')
self.header_string.trace('w', self.headerOptionCheckboxChanged)
return header_checkbox
def headerOptionCheckboxChanged(self, *args):
print(self.header_string.get())
#will print "headers" or "keys" on checkbox toggle
print(self.controller.getHeaderItems(self.header_string.get()))
#prints "default"
class MainViewController:
def __init__(self):
self.CheckFile = CheckFile()
get_config = GetConfiguration('config.ini')
self.config_file = get_config.getProperty('directory', 'input_file')
self.csv = CSVReader(self.config_file)
self.chosen_index = None
def getHeaderItems(self, header='default'):
return header
Can someone please help me understand why in Python you need to instantiate a class with parenthesis even if there are no constructor arguments other than self. Also, why did the MainViewController still kind of work, but it did not behave as I wanted it to? As in it was loaded, and the functions "did things", but it would not seem to accept arguments. Is there any advantages of instantiating a class without its parenthesis?
Please note, I do not need help getting this code to work, I just want to understand why this happens.

Can someone please help me understand why in Python you need to instantiate a class with parenthesis even if there are no constructor arguments other than self.
The reason is simple: when you instantiate an object, you are actually calling its class (which is itself an object), and you call objects using ().
In python, everything is a first-class object, even classes (and functions!) themselves. In order for a class to be a first class object, it follows that the class needs its own class (metaclass) to define its behavior. We call the class of a class "metaclass" so as to avoid confusion when talking about classes and classes of classes.
To answer the second part of your question: "things" were happening when you used MainViewController instead of MainViewController() because MainViewController is a full-fledged object, just like any other object.
So you might ask: what is the class - actually the metaclass - of the MainViewController object?
As you know, you can create a class like this:
class MyClass:
pass
When you do this, you are in actuality creating a new instance of the metaclass known as type.
Note that you can create the same class this way; there is literally no difference between the below and the above:
MyClass = type('MyClass', (object,), {})
The type metaclass is the base metaclass of all classes. All python "new style classes" (not so "new" anymore since they were implemented in python 2.1, I believe) are of the class type:
print(type(MyClass)) # type
print(type(list)) # type
print(type(int)) # type
# Note that above, type is being used as a "function" (it's really just a callable)
Interestingly enough, type is even its own metaclass:
print(type(type)) # type
So to reiterate: the class MyClass is actually an instantiation of type. It follows, then, that calling the class results in running the __call__ method of its metaclass.
When you do:
obj = MyClass()
...you are calling MyClass, which results (in the background) in running the method type.__call__().
This is the case with all user defined classes, btw; if you include the __call__ method in your class, your class is callable, and the __call__ method is executed when you call class instances:
class MyCallable():
def __call__(self):
print("You rang?")
my_instance = MyCallable()
my_instance() # You rang?
You can see this in action. If you create your own metaclass by subclassing type, you can cause things to happen when an instance of the class based on your custom metaclass is created. For example:
class MyMeta(type):
def __call__(self, *args, **kwargs):
print "call: {} {} {}".format(self, args, kwargs)
return super().__call__(*args, **kwargs)
# Python 3:
class MyClass(metaclass = MyMeta):
pass
# Python 2:
class MyClass():
__metaclass__ = MyMeta
pass
Now when you do MyClass(), you can see that the __call__ method of MyMeta happens before anything else (including before __new__ AND before __init__).

Because function calls require (). When you do MyClass(), you are calling MyClass. The expression MyClass evaluates to the class itself, which is an object.