We Keep Coding

Mypy and Inheriting from a class that is an attribute on an instance

EDIT
The issue arises when trying to inherit from a class that is an attribute of an instance. This mcve repros it, I'll leave the rest of the question below for posterity:
class A:
class SubA:
pass
a = A()
class B(a.SubA):
pass
mypy output:
Name 'a.SubA' is not defined
This passes:
class A:
class SubA:
pass
class B(A.SubA):
pass
The example in this Related Issue is pretty much exactly what Flask-SQLAlchemy does to provide the declarative base class under the db namespace. In the issue, mypy maintainer asserts that they wouldn't support the pattern.
My question is, what is incorrect about the above pattern such that mypy would not support it? Especially in the context that it is used by a large project such as Flask-SQLAlchemy.
Further, what is the best way for users of Flask-SQLAlchemy and mypy to manage this in their projects?
ORIGINAL QUESTION
This question isn't about the lack of Flask-SQLAlchemy stubs. Accepting that, I came across this question.
Please help me to understand why the following does not work as I expect.
In my environment I have only Flask-SQLAlchemy and mypy installed.
I have mypy configured with ignore_missing_imports = True.
A mypy pass over the following:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Widget(db.Model):
id = db.Column(db.Integer, primary_key=True)
reveals:
error: Name 'db.Model' is not defined
So, I've attempted to subclass SQLAlchemy to provide an annotation for Model, which shows in __annotations__ on the object, yet the mypy analysis of the module doesn't change:
from flask_sqlalchemy import SQLAlchemy
from flask_sqlalchemy.model import DefaultMeta
class TypedSQLAlchemy(SQLAlchemy):
Model: DefaultMeta
db = TypedSQLAlchemy()
print(db.__annotations__) # {'Model': <class 'flask_sqlalchemy.model.DefaultMeta'>}
class Widget(db.Model):
id = db.Column(db.Integer, primary_key=True)
When I execute the file, the print(db.__annotations__) command displays {'Model': <class 'flask_sqlalchemy.model.DefaultMeta'>}, yet still mypy has the same error:
error: Name 'db.Model' is not defined
I would expect that providing an annotation for db.Model should make that error go away.
Earlier Edit
I've initially misinterpreted the error as it doesn't suggest the attribute Model doesn't exist on db, it suggests that the name db.Model doesn't exist in the namespace. But why would it treat db.Model as a full name, and not db as the name defined locally and Model as it's attribute? Is this something to do with trying to inheriting from a class variable?
Also, my annotation was incorrect, should be:
class TypedSQLAlchemy(SQLAlchemy):
Model: Type[DefaultMeta]

You should use A.SubA.
As I know access to nested class via instance variable is not allowed from mypy point of view. Because derived classes from A could override nested class, and mypy cannot recognize this case, something like this:
class A:
class SubA:
pass
class C(A):
class SubA:
pass
def foo(a: A):
class B(a.SubA): # What SubA here ?
pass
foo(C())
Update:
As for Flask-SQLAlchemy, the following workaround was suggested in this discussion:
from app import db
from sqlalchemy.ext.declarative import DeclarativeMeta
BaseModel: DeclarativeMeta = db.Model
class MyModel(BaseModel): ...
If you are using flask_sqlalchemy then you can use from flask_sqlalchemy.model import DefaultMeta instead of DeclarativeMeta.

python object instance variable looks like a class variable?

I'm using SQLAlchemy in Python 3 and am confused why the following code works - it looks to me like what should be an object level variable is acting as a class variable. I have seen Why is instance variable behaving like a class variable in Python? which doesn't look related to my question.
I have the following declarations in a db module in which I create an instance of Base and set a query variable on it that points to the SessionFactory query_property()
import sqlalchemy as sa
import sqlalchemy.ext.declarative as sa_ed
Base = sa_ed.declarative_base()
engine = sa.create_engine('connection string')
session_factory = session or sa_orm.scoped_session(sa_orm.sessionmaker(autocommit=False, autoflush=False, bind=engine))
Base.query = session_factory.query_property() # make the query_property available in models for querying
My model classes are then declared as follows:
import db
class MyModel(db.Base):
id = Column(Integer)
# more model stuff
I can then run queries by accessing the query variable as follows:
return MyModel.query.filter(MyModel.id == 22).first()
This does work, but it looks as though the query variable exists at the class level and not at the object instance level as I am able to access it directly through the class.
What am I not understanding?

You put the query property on the parent class earlier:
Base.query = session_factory.query_property() # make the query_property available in models for querying
So query is most definitely a member of a class (Base). And since MyModel inherits from Base, MyModel should also have a query member (due to the magic of inheritance).

Django, Python, and Class Variables

I'm simultaneously learning Python while picking up Django. I'm familiar with many other languages.
In the following code snippet, x is a class variable of class Foo.
class Foo(object):
x = 9000
Given the previous declaration, the following works fine.
print Foo.x
The Django framework lets you create your model by defining Python classes. It makes fields out of the different class variables in your Python classes.
class Question(models.Model):
question_text = models.CharField(max_length=200)
Why does the following code snippet:
#!/usr/bin/env
import os, django
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
django.setup()
from polls.models import Question, Choice
print Question.question_text
throw the following error:
AttributeError: type object 'Question' has no attribute 'question_text'
As far as I'm understanding everything my Question class has a single static member defined: Question.question_text.

Django models use a metaclass to alter what is normal class behaviour.
Use dir(Question) and you'll see there are different attributes on that class now. This is custom behaviour just for Django models however.
If you are curious you can study the metaclass __new__ method, but it does a lot of work specific to Object Relational Mapping tasks.

Magic.
No, really.
Python classes aren't set-in-stone structure, like they are in C++. They are, themselves, just objects — instances of another type:
class Foo(object):
pass
print(type(Foo)) # <class 'type'>
You can even make a class like you'd make any other object, by calling type. This:
class Bar(object):
a = 1
b = 2
Is really (more or less) syntactic sugar for this:
Bar = type('Bar', (object,), {'a': 1, 'b': 2})
type takes the name of your new class, a list of its superclasses, and a dict of all the attributes of the class, and spits out a new class.
But, because type is just a class like any other, it's possible to subclass it and give it different behavior. And this is what Django has done: it's created a subclass of type that does something different with the dict of attributes you pass to it.
You don't see this happening directly in your own code, but if you check type(models.Model), you'll find out its type is not type, but something specific to Django. It probably has "meta" in the name, because it's called a metaclass: the class of a class.
This is a fairly common pattern for making "declarative" libraries in Python, where the attributes of a class actually define some kind of structure. You can see the same thing in form validation (wtforms), schema validation (colander), other ORMs (sqlalchemy), and even the stdlib enum module.

Question is an object of type type. You want an instance of Question:
>>> q= Question(text = "Does a dog have the buddha nature?")
Then you should get
q.text
"Does a dog have the buddha nature?"
Note that this object will not persist unless you save() it:
>>> q.save()

Python: Pickle derived classes as if they were an instance of the base class

I want to define a base class so that when derived class instances are pickled, they are pickled as if they are instances of the base class. This is because the derived classes may exist on the client side of the pickling but not on the server side, but this is not important to the server since it only needs information from the base class. I don't want to have to dynamically create classes for every client.
The base class is simply an "object handle" which contains an ID, with methods defined on the server, but I would like the client to be able to subclass the server classes and define new methods (which would only be seen by the client, but that doesn't matter).

I believe you can do it by giving the object a __reduce__ method, returning a tuple, the first part of which should be BaseClass.__new__ (this will be called when loading the object in unpickling). See the pickle documentation (Python 2, Python 3) for the full details. I haven't attempted this.
Depending on what you're doing, it might be easier to use a simpler serialisation format like JSON, and have code on each side to reconstruct the relevant objects.

You can change an object's class dynamically in Python:
import cPickle as pickle
class Foo(object):
def __init__(self):
self.id=1
class Bar(Foo):
def derived_class_method(self): pass
bar=Bar()
bar.id=2
bar.__class__=Foo # changes `bar`'s class to Foo
bar_pickled=pickle.dumps(bar)
bar2=pickle.loads(bar_pickled)
bar.__class__=Bar # reset `bar`'s class to Bar
print(repr(bar2))
# <__main__.Foo object at 0xb76b08ec>
print(bar2.id)
# 2
I'm not sure using this is the best design decision, however. I like Thomas K's idea of using JSON.

Dynamic Class Creation in SQLAlchemy

We have a need to create SQLAlchemy classes to access multiple external data sources that will increase in number over time. We use the declarative base for our core ORM models and I know we can manually specify new ORM classes using the autoload=True to auto generate the mapping.
The problem is that we need to be able generate them dynamically taking something like this:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
stored={}
stored['tablename']='my_internal_table_name'
stored['objectname']='MyObject'
and turning it into something like this dynamically:
class MyObject(Base):
__tablename__ = 'my_internal_table_name'
__table_args__ = {'autoload':True}
We don't want the classes to persist longer than necessary to open a connection, perform the queries, and then closing the connection. Therefore, ideally, we can put the items in the "stored" variable above into a database and pull them as needed. The other challenge is that the object name (e.g. "MyObject") may be used on different connections so we cannot define it once and keep it around.
Any suggestions on how this might be accomplished would be greatly appreciated.
Thanks...

You can dynamically create MyObject using the 3-argument call to type:
type(name, bases, dict)
Return a new type object. This is essentially a dynamic form of the
class statement...
For example:
mydict={'__tablename__':stored['tablename'],
'__table_args__':{'autoload':True},}
MyObj=type(stored['objectname'],(Base,),mydict)
print(MyObj)
# <class '__main__.MyObject'>
print(MyObj.__base__)
# <class '__main__.Base'>
print(MyObj.__tablename__)
# my_internal_table_name
print(MyObj.__table_args__)
# {'autoload': True}