We Keep Coding

Need to combine classes from two python modules into a single class

I need to combine Classes from two separate Python modules (which are similar in purpose but with different Methods) into a single Class so that the Methods can be accessed from the same object in a natural way both in code and for automatic documentation generation.
I am currently accomplishing the former but not the latter with the following code (this is not verbatim, as I can't share my actual source, but there's nothing different here that would impact the conversation).
Basically, I am creating the new class via a function which combines the __dict__ attributes of the two child Classes and returns a new Class.
def combine(argone, argtwo):
"""
Combine Classes
"""
_combined_arg = "some_string_%s_%s" % argone, argtwo
_temp = type('Temp', (ModuleOne, ModuleTwo), dict())
self = _temp(_combined_arg) # Calling the constructor with our combined arg
# The two classes have an identical constructor method within their __init__() methods
# Return the object we've instantiated off of the combined class
return self
This method works fine for producing an object that lets me call Methods from either of the original Classes, but my IDE can't auto-complete Method names nor can documentation generators (like pdoc) produce any documentation beyond our combine() function.
This process is necessary because we are generating code off of other code (descriptive, I know, sorry!) and it isn't practical to combine them upstream (ie, by hand).
Any ideas?
Thank you in advance!!!
ADDENDUM:
What I can say about what we are doing here is that we're just combining client Methods generated off of REST API endpoints that happen to be split into two, non-overlapping, namespaces for practical reasons that aren't important to this discussion. So that's why simply dropping the methods from ModuleTwo into ModuleOne would be all that needs doing.
If there are suggestions on an automatable and clean way to do this before shipping either module, I am definitely open to hearing them. Not having to do this work would be far preferable. Thanks!

There is no need for combine to define a new class every time it is called.
class CombinedAPI(APIOne, APITwo):
#classmethod
def combine(cls, arg_one, arg_two):
arg = "some_string_%s_%s" % (argone, argtwo)
return cls(arg)
obj = CombinedAPI.combine(foo, bar)

Instance variables in methods outside the constructor (Python) -- why and how?

My questions concern instance variables that are initialized in methods outside the class constructor. This is for Python.
I'll first state what I understand:
Classes may define a constructor, and it may also define other methods.
Instance variables are generally defined/initialized within the constructor.
But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
An example of (2) and (3) -- see self.meow and self.roar in the Cat class below:
class Cat():
def __init__(self):
self.meow = "Meow!"
def meow_bigger(self):
self.roar = "Roar!"
My questions:
Why is it best practice to initialize the instance variable within the constructor?
What general/specific mess could arise if instance variables are regularly initialized in methods other than the constructor? (E.g. Having read Mark Lutz's Tkinter guide in his Programming Python, which I thought was excellent, I noticed that the instance variable used to hold the PhotoImage objects/references were initialized in the further methods, not in the constructor. It seemed to work without issue there, but could that practice cause issues in the long run?)
In what scenarios would it be better to initialize instance variables in the other methods, rather than in the constructor?
To my knowledge, instance variables exist not when the class object is created, but after the class object is instantiated. Proceeding upon my code above, I demonstrate this:
>> c = Cat()
>> c.meow
'Meow!'
>> c.roar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Cat' object has no attribute 'roar'
>>> c.meow_bigger()
>>> c.roar
'Roar!'
As it were:
I cannot access the instance variable (c.roar) at first.
However, after I have called the instance method c.meow_bigger() once, I am suddenly able to access the instance variable c.roar.
Why is the above behaviour so?
Thank you for helping out with my understanding.

Why is it best practice to initialize the instance variable within the
constructor?
Clarity.
Because it makes it easy to see at a glance all of the attributes of the class. If you initialize the variables in multiple methods, it becomes difficult to understand the complete data structure without reading every line of code.
Initializing within the __init__ also makes documentation easier. With your example, you can't write "an instance of Cat has a roar attribute". Instead, you have to add a paragraph explaining that an instance of Cat might have a "roar" attribute, but only after calling the "meow_louder" method.
Clarity is king. One of the smartest programmers I ever met once told me "show me your data structures, and I can tell you how your code works without seeing any of your code". While that's a tiny bit hyperbolic, there's definitely a ring of truth to it. One of the biggest hurdles to learning a code base is understanding the data that it manipulates.
What general/specific mess could arise if instance variables are
regularly initialized in methods other than the constructor?
The most obvious one is that an object may not have an attribute available during all parts of the program, leading to having to add a lot of extra code to handle the case where the attribute is undefined.
In what scenarios would it be better to initialize instance variables
in the other methods, rather than in the constructor?
I don't think there are any.
Note: you don't necessarily have to initialize an attribute with it's final value. In your case it's acceptable to initialize roar to None. The mere fact that it has been initialized to something shows that it's a piece of data that the class maintains. It's fine if the value changes later.

Remember that class members in "pure" Python are just a dictionary. Members aren't added to an instance's dictionary until you run the function in which they are defined. Ideally this is the constructor, because that then guarantees that your members will all exist regardless of the order that your functions are called.
I believe your example above could be translated to:
class Cat():
def __init__(self):
self.__dict__['meow'] = "Meow!"
def meow_bigger(self):
self.__dict__['roar'] = "Roar!"
>>> c = Cat() # c.__dict__ = { 'meow': "Meow!" }
>>> c.meow_bigger() # c.__dict__ = { 'meow': "Meow!", 'roar': "Roar!" }

To initialize instance variables within the constructor, is - as you already pointed out - only recommended in python.
First of all, defining all instance variables within the constructor is a good way to document a class. Everybody, seeing the code, knows what kind of internal state an instance has.
Secondly, order matters. if one defines an instance variable V in a function A and there is another function B also accessing V, it is important to call A before B. Otherwise B will fail since V was never defined. Maybe, A has to be invoked before B, but then it should be ensured by an internal state, which would be an instance variable.
There are many more examples. Generally it is just a good idea to define everything in the __init__ method, and set it to None if it can not / should not be initialized at initialization.
Of course, one could use hasattr method to derive some information of the state. But, also one could check if some instance variable V is for example None, which can imply the same then.
So in my opinion, it is never a good idea to define an instance variable anywhere else as in the constructor.
Your examples state some basic properties of python. An object in Python is basically just a dictionary.
Lets use a dictionary: One can add functions and values to that dictionary and construct some kind of OOP. Using the class statement just brings everything into a clean syntax and provides extra stuff like magic methods.
In other languages all information about instance variables and functions are present before the object was initialized. Python does that at runtime. You can also add new methods to any object outside the class definition: Adding a Method to an Existing Object Instance

3.) But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
I'd recommend providing a default state in initialization, just so its clear what the class should expect. In statically typed languages, you'd have to do this, and it's good practice in python.
Let's convey this by replacing the variable roar with a more meaningful variable like has_roared.
In this case, your meow_bigger() method now has a reason to set has_roar. You'd initialize it to false in __init__, as the cat has not roared yet upon instantiation.
class Cat():
def __init__(self):
self.meow = "Meow!"
self.has_roared = False
def meow_bigger(self):
print self.meow + "!!!"
self.has_roared = True
Now do you see why it often makes sense to initialize attributes with default values?
All that being said, why does python not enforce that we HAVE to define our variables in the __init__ method? Well, being a dynamic language, we can now do things like this.
>>> cat1 = Cat()
>>> cat2 = Cat()
>>> cat1.name = "steve"
>>> cat2.name = "sarah"
>>> print cat1.name
... "steve"
The name attribute was not defined in the __init__ method, but we're able to add it anyway. This is a more realistic use case of setting variables that aren't defaulted in __init__.

I try to provide a case where you would do so for:
3.) But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
I agree it would be clear and organized to include instance field in the constructor, but sometimes you are inherit other class, which is created by some other people and has many instance fields and api.
But if you inherit it only for certain apis and you want to have your own instance field for your own apis, in this case, it is easier for you to just declare extra instance field in the method instead override the other's constructor without bothering to deep into the source code. This also support Adam Hughes's answer, because in this case, you will always have your defined instance because you will guarantee to call you own api first.
For instance, suppose you inherit a package's handler class for web development, you want to include a new instance field called user for handler, you would probability just declare it directly in the method--initialize without override the constructor, I saw it is more common to do so.
class BlogHandler(webapp2.RequestHandler):
def initialize(self, *a, **kw):
webapp2.RequestHandler.initialize(self, *a, **kw)
uid = self.read_cookie('user_id') #get user_id by read cookie in the browser
self.user = User.by_id(int(uid)) #run query in data base find the user and return user

These are very open questions.
Python is a very "free" language in the sense that it tries to never restrict you from doing anything, even if it looks silly. This is why you can do completely useless things such as replacing a class with a boolean (Yes you can).
The behaviour that you mention follows that same logic: if you wish to add an attribute to an object (or to a function - yes you can, too) dynamically, anywhere, not necessarily in the constructor, well... you can.
But it is not because you can that you should. The main reason for initializing attributes in the constructor is readability, which is a prerequisite for maintenance. As Bryan Oakley explains in his answer, class fields are key to understand the code as their names and types often reveal the intent better than the methods.
That being said, there is now a way to separate attribute definition from constructor initialization: pyfields. I wrote this library to be able to define the "contract" of a class in terms of attributes, while not requiring initialization in the constructor. This allows you in particular to create "mix-in classes" where attributes and methods relying on these attributes are defined, but no constructor is provided.
See this other answer for an example and details.

i think to keep it simple and understandable, better to initialize the class variables in the class constructor, so they can be directly called without the necessity of compiling of a specific class method.
class Cat():
def __init__(self,Meow,Roar):
self.meow = Meow
self.roar = Roar
def meow_bigger(self):
return self.roar
def mix(self):
return self.meow+self.roar
c=Cat("Meow!","Roar!")
print(c.meow_bigger())
print(c.mix())
Output
Roar!
Roar!
Meow!Roar!

Why use a classmethod over an Instance method in python

I have been trying to get my head around classmethods for a while now. I know how they work but I don't understand why use them or not use them.
For example.
I know i can use an instance method like this:
class MyClass():
def __init__(self):
self.name = 'Chris'
self.age = 27
def who_are_you(self):
print('Hello {}, you are {} years old'.format(self.name, self.age)
c = MyClass()
c.who_are_you()
I also know that by using the classmethod I can call the who_are_you() without creating an instance of my class:
class MyClass():
name = 'Chris'
age = 27
#classmethod
def who_are_you(cls):
print('Hello {}, you are {} years old'.format(cls.name, cls.age)
MyClass.who_are_you()
I dont get why you would pick one method over the other

In your second example, you've hard-coded the name and age into the class. If name and age are indeed properties of the class and not a specific instance of the class, than using a class method makes sense. However, if your class was something like Human of which there are many instances with different names and ages, then it wouldn't be possible to create a class method to access the unique names and ages of the specific instance. In that case, you would want to use an instance method.
In general:
If you want to access a property of a class as a whole, and not the property of a specific instance of that class, use a class method.
If you want to access/modify a property associated with a specific instance of the class, then you will want to use an instance method.

Class methods are called when you don't have, or don't need, or can't have, an instance. Sometimes, a class can serve as a singleton when used this way. But probably the most common use of class methods is as a non-standard constructor.
For example, the Python dict class has a non-standard constructor called dict.fromkeys(seq, [value]). Clearly, there can be no instance involved - the whole point is to create an instance. But it's not the standard __init__() constructor, because it takes data in a slightly different format.
There are similar methods in the standard library: int.from_bytes, bytes.fromhex and bytearray.fromhex() and float.fromhex().
If you think about the Unix standard library, the fdopen function is a similar idea - it constructs a file from a descriptor, instead of a string path. Python's open() will accept file handles instead of paths, so it doesn't need a separate constructor. But the concept is more common than you might suspect.

#classmethod declares that method is static, therefore you could use it without creating new instance of class. One the other hand, in first example you have to create instance before youcould use method.
Static methods are very useful for controllers in MVC pattern, etc, while nonstatic methods are used in models.
More about #classmethod and #staticmethod here
https://stackoverflow.com/a/12179752/5564059

Python : serialise class hierarchy

I have to serialise a dynamically created class hierarchy. And a bunch of objects - instances of the latter classes.
Python pickle is not of big help, its wiki says "Classes ... cannot be pickled". O there may be some trick that I cannot figure.
Performance requirement:
Deserialization should be pretty fast, because the serialised staff serves for cache and should save me the work of creating the same class hierarchy.
Details:
classes are created dynamically using type and sometimes meta-classes.

If you provide a custom object.__reduce__() method I believe you can still use pickling.
Normally, when pickling, the class import path is stored, plus instance state. On unpickling, the class is imported, and a new instance is created using the stored state. This is why pickling cannot work with dynamic classes, there is nothing to import.
The object.__reduce__() method lets you store a different instance factory. The callable returned by this function is stored (again by import path), and called with specified arguments to produce an instance. This instance is then used to apply state to, in the same way a regular instance would be unpickled:
def class_factory(name):
return globals()[name]()
class SomeDynamicClass(object):
def __reduce__(self):
return (class_factory, (type(self).__name__,), self.__dict__)
Here __reduce__ returns a function, the arguments for the function, and the instance state.
All you need to do then, is provide the right arguments to the factory function to recreate the class, and return an instance of that class. It'll be used instead of importing the class directly.

Classes are normal python objects, so, in theory, should be picklable, if you provide __reduce__ (or implement other pickle protocol methods) for them. Try to define __reduce__ on their metaclass.

What's the best way to extend the functionality of factory-produced classes outside of the module in python?

I've been reading lots of previous SO discussions of factory functions, etc. and still don't know what the best (pythonic) approach is to this particular situation. I'll admit up front that i am imposing a somewhat artificial constraint on the problem in that i want my solution to work without modifying the module i am trying to extend: i could make modifications to it, but let's assume that it must remain as-is because i'm trying to understand best practice in this situation.
I'm working with the http://pypi.python.org/pypi/icalendar module, which handles parsing from and serializing to the Icalendar spec (hereafter ical). It parses the text into a hierarchy of dictionary-like "component" objects, where every "component" is an instance of a trivial derived class implementing the different valid ical types (VCALENDAR, VEVENT, etc.) and they are all spit out by a recursive factory from the common parent class:
class Component(...):
#classmethod
def from_ical(cls, ...)
I have created a 'CalendarFile' class that extends the ical 'Calendar' class, including in it generator function of its own:
class CalendarFile(Calendar):
#classmethod
def from_file(cls, ics):
which opens a file (ics) and passes it on:
instance = cls.from_ical(f.read())
It initializes and modifies some other things in instance and then returns it. The problem is that instance ends up being a Calendar object instead of a CalendarFile object, in spite of cls being CalendarFile. Short of going into the factory function of the ical module and fiddling around in there, is there any way to essentially "recast" that object as a 'CalendarFile'?
The alternatives (again without modifying the original module) that I have considered are:make the CalendarFile class a has-a Calendar class (each instance creates its own internal instance of a Calendar object), but that seems methodically stilted.
fiddle with the returned object to give it the methods it needs (i know there's a term for creating a customized object but it escapes me).
make the additional methods into functions and just have them work with instances of Calendar.
or perhaps the answer is that i shouldn't be trying to subclass from a module in the first place, and this type of code belongs in the module itself.
Again i'm trying to understand what the "best" approach is and also learn if i'm missing any alternatives. Thanks.

Normally, I would expect an alternative constructor defined as a classmethod to simply call the class's standard constructor, transforming the arguments that it receives into valid arguments to the standard constructor.
>>> class Toy(object):
... def __init__(self, x):
... self.x = abs(x)
... def __repr__(self):
... return 'Toy({})'.format(self.x)
... #classmethod
... def from_string(cls, s):
... return cls(int(s))
...
>>> Toy.from_string('5')
Toy(5)
In most cases, I would strongly recommend something like this approach; this is the gold standard for alternative constructors.
But this is a special case.
I've now looked over the source, and I think the best way to add a new class is to edit the module directly; otherwise, scrap inheritance and take option one (your "has-a" option). The different classes are all slightly differentiated versions of the same container class -- they shouldn't really even be separate classes. But if you want to add a new class in the idiom of the code as it it is written, you have to add a new class to the module itself. Furthermore, from_iter is deceptively named; it's not really a constructor at all. I think it should be a standalone function. It builds a whole tree of components linked together, and the code that builds the individual components is buried in a chain of calls to various factory functions that also should be standalone functions but aren't. IMO much of that code ought to live in __init__ where it would be useful to you for subclassing, but it doesn't.
Indeed, none of the subclasses of Component even add any methods. By adding methods to your subclass of Calendar, you're completely disregarding the actual idiom of the code. I don't like its idiom very much but by disregarding that idiom, you're making it even worse. If you don't want to modify the original module, then forget about inheritance here and give your object a has-a relationship to Calendar objects. Don't modify __class__; establish your own OO structure that follows standard OO practices.