We Keep Coding

Mypy and Inheriting from a class that is an attribute on an instance

EDIT
The issue arises when trying to inherit from a class that is an attribute of an instance. This mcve repros it, I'll leave the rest of the question below for posterity:
class A:
class SubA:
pass
a = A()
class B(a.SubA):
pass
mypy output:
Name 'a.SubA' is not defined
This passes:
class A:
class SubA:
pass
class B(A.SubA):
pass
The example in this Related Issue is pretty much exactly what Flask-SQLAlchemy does to provide the declarative base class under the db namespace. In the issue, mypy maintainer asserts that they wouldn't support the pattern.
My question is, what is incorrect about the above pattern such that mypy would not support it? Especially in the context that it is used by a large project such as Flask-SQLAlchemy.
Further, what is the best way for users of Flask-SQLAlchemy and mypy to manage this in their projects?
ORIGINAL QUESTION
This question isn't about the lack of Flask-SQLAlchemy stubs. Accepting that, I came across this question.
Please help me to understand why the following does not work as I expect.
In my environment I have only Flask-SQLAlchemy and mypy installed.
I have mypy configured with ignore_missing_imports = True.
A mypy pass over the following:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Widget(db.Model):
id = db.Column(db.Integer, primary_key=True)
reveals:
error: Name 'db.Model' is not defined
So, I've attempted to subclass SQLAlchemy to provide an annotation for Model, which shows in __annotations__ on the object, yet the mypy analysis of the module doesn't change:
from flask_sqlalchemy import SQLAlchemy
from flask_sqlalchemy.model import DefaultMeta
class TypedSQLAlchemy(SQLAlchemy):
Model: DefaultMeta
db = TypedSQLAlchemy()
print(db.__annotations__) # {'Model': <class 'flask_sqlalchemy.model.DefaultMeta'>}
class Widget(db.Model):
id = db.Column(db.Integer, primary_key=True)
When I execute the file, the print(db.__annotations__) command displays {'Model': <class 'flask_sqlalchemy.model.DefaultMeta'>}, yet still mypy has the same error:
error: Name 'db.Model' is not defined
I would expect that providing an annotation for db.Model should make that error go away.
Earlier Edit
I've initially misinterpreted the error as it doesn't suggest the attribute Model doesn't exist on db, it suggests that the name db.Model doesn't exist in the namespace. But why would it treat db.Model as a full name, and not db as the name defined locally and Model as it's attribute? Is this something to do with trying to inheriting from a class variable?
Also, my annotation was incorrect, should be:
class TypedSQLAlchemy(SQLAlchemy):
Model: Type[DefaultMeta]

You should use A.SubA.
As I know access to nested class via instance variable is not allowed from mypy point of view. Because derived classes from A could override nested class, and mypy cannot recognize this case, something like this:
class A:
class SubA:
pass
class C(A):
class SubA:
pass
def foo(a: A):
class B(a.SubA): # What SubA here ?
pass
foo(C())
Update:
As for Flask-SQLAlchemy, the following workaround was suggested in this discussion:
from app import db
from sqlalchemy.ext.declarative import DeclarativeMeta
BaseModel: DeclarativeMeta = db.Model
class MyModel(BaseModel): ...
If you are using flask_sqlalchemy then you can use from flask_sqlalchemy.model import DefaultMeta instead of DeclarativeMeta.

How does Base.metadata work in Sqlalchemy?

Code/Example
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
If I run print(Base) then I get result:
<class 'sqlalchemy.ext.declarative.api.Base'>
(I.e Base is a class created by the function declarative_base).
If I run print(Base.metadata) then I get
Metadata(bind=engine(postgres//:user:password#host/database))
But exactly how does print(Base.metadata) even run if Base is a class and metadata is an object?
How are the two connected?
I could understand if metadata was a class attribute but from what I understand it is not so how does this work?
Is there something in python syntax that I have missed?
Looking in metadata code:
from sqlalchemy import MetaData
metadata = MetaData()
What is here the definition of metadata and MetaData?
Are they both objects?
Is MetaData() a function or something else?
According to SQLAlchemy documentation MetaData is:
a container object that keeps together many different features of a database (or multiple databases) being described.
But if it is an object so then how can it be run as something similar to a function: Metadata()?
Summary:
How are Base and metadata connected if one is a class and the other is an object?
How can the command (Base.metadata) be run?
What is the difference between metadata and MetaData and what is the MetaData() definition?

You are overthinking this. In Python everything is an object. Classes, functions, instances, numbers, strings, lists, dictionaries, modules, ..., everything.
The SQLAlchemy objects you are looking at are no different, there really is nothing special about these.
Yes, the declarative_base() function returns a new class object, one you assigned to the name Base. That class has attributes, and one of those attributes is named metadata. All Python classes have attributes, they can point to any object you like:
class Foo:
def __init__(self, name):
self.name = name
def print(self):
print(f"Hello, {self.name}!")
class Bar:
spam = Foo("World")
The above class Bar has an attribute spam that is an instance of the class Foo. You can call methods on it:
>>> Bar.spam.print()
Hello, World!
Note that Bar is still a class, and Bar.spam is an instance of the Foo class:
>>> Bar
<class '__main__.Bar'>
>>> Bar.spam
<__main__.Foo object at 0x10d43c650>
>>> type(Bar.spam)
<class '__main__.Foo'>
Because Bar is just another object, you can also pass it around, put it in a list, or return it from a function. declarative_base() constructed a class object for you, with attributes, and returned it to the caller.
Similarly, Base.metadata is just an instance of the MetaData class:
>>> type(Base.metadata)
<class 'sqlalchemy.sql.schema.MetaData'>
and so is your metadata = MetaData() object.
Note that if you only run Base = declarative_base() and nothing else, the MetaData instance is not actually showing any database connection information:
>>> Base.metadata
MetaData(bind=None)
That's because you didn't actually connect the metadata to a database session. Once you start connecting a database with an engine and start using the models you define by subclassing Base, the metadata object at some point will need to know more about the specifics of the database (such as what kind of SQL dialect is supported) and it'll be bound by SQLAlchemy.

How does Model Class work, Django?

Before posting this question, I have read through the Official Django Documentation, scouring it for a comprehensive explanation for beginners. I have read the code of the actual Model Class, and searched around on StackOverflow.
When working with databases in Django, you work with classes inheriting from the Model class in the models module. This helps programmers avoid double-typing everything, jumping between database specific syntax and python. As I have read, 'the model class that each model inherits from automatically takes care of translation'.
How does this work? How Does the Model Class convert model attributes to database columns? I suppose some methods inherited from the parent Model Class are able to use the variables specified in each new model, but would like a better explanation if possible!
Also, why write 'models.Model' if the Model class is within models.base?
LINK TO MODEL CLASS: https://docs.djangoproject.com/en/1.11/_modules/django/db/models/base/#Model
EDIT:
Figured out the reason behind why models.Model work.

How Does the Model Class convert model attributes to database columns?
The Model class doesn't really do any conversion itself. You create a subclass of Model that has some column information,
which Django's ORM uses when building the database query corresponding to your Django ORM query. The conversion is done by a database driver when it actually communicates with your specific database.
Here's a toy ORM that behaves a little like Django's Model. You can implement QuerySet for fun if you want:
class Column:
'''
Represents a database column.
This is used to create the underlying table in the database
and to translate database types to Python types.
'''
def __init__(self, type):
self.type = type
class Manager:
'''
Accessed via `YourModel.objects`. This is what constructs
a `QuerySet` object in Django.
'''
def __init__(self, model):
self.model = model
def get(self, id):
'''
Pretend `YourModel.objects.get(id=123)` queries the database directly.
'''
# Create an instance of the model. We only keep track of the model class.
instance = self.model()
# Populate the instance's attributes with the result of the database query
for name in self.model._columns:
# Pretend we load the values from the database
value = 123
setattr(instance, name, value)
# This would be done above if we actually queried the database
instance.id = id
# Finally return the instance of `self.model`
return instance
class ModelBase(type):
def __new__(cls, name, bases, attrs):
new_cls = super().__new__(cls, name, bases, attrs)
# The `Manager` instance is made a class attribute
new_cls.objects = Manager(new_cls)
# Keep track of the columns for conveniece
new_cls._columns = {}
for name, attr in attrs.items():
if isinstance(attr, Column):
new_cls._columns[name] = attr
# The class is now ready
return new_cls
class Model(metaclass=ModelBase):
'''
Django's `Model` is more complex.
This one only uses `ModelBase` as its metaclass so you can just inherit from it
'''
pass
class MyModel(Model):
id = Column(int)
column2 = Column(float)
column3 = Column(str)
if __name__ == '__main__':
print(MyModel._columns)
instance = MyModel.objects.get(id=5)
print(instance.id)
The main functionality is provided by Model having ModelBase as a metaclass. The metaclass's __new__ method is called
when Model or any subclass is created (not an instance, the class itself), which allows the metaclass to modify the class arbitrarily.
Each Model subclass contains information about its own columns and gets a objects class attribute that queries the database for it.
Also, why write 'models.Model' if the Model class is within models.base?
models/__init__.py imports Model from models/base.py so you don't have to write models.base.Model.

When you create a model class and run
python manage.py makemigrations
It creates the corresponding scripts to create a table in your database.
You can find this script in your apps "migrations" folder.
And when you run
python manage.py migrate
These scripts are mapped to the correct commands and are executed on the database by Django.

How to subclass the default `db.Model` to use as a custom `Base` with SQLAlchemy

I have a flask application spread across modules with blueprints.
Each module/blueprint has its own models.py file where models are defined.
With my desktop applications, using SQLAlchemy API directly, I would subclass object to define a Base class with some columns (ex: id, date_created ..), which then would serve as my declarative base (ex: Base = declarative_base(cls=Base)).
How can I subclass flask-SQLALchemy's db.Model so that I can use it as a Base with default columns which I want all tables to have?

Simply set __abstract__ to True:
class BaseModel(db.Model):
__abstract__ = True

Dynamic Class Creation in SQLAlchemy

We have a need to create SQLAlchemy classes to access multiple external data sources that will increase in number over time. We use the declarative base for our core ORM models and I know we can manually specify new ORM classes using the autoload=True to auto generate the mapping.
The problem is that we need to be able generate them dynamically taking something like this:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
stored={}
stored['tablename']='my_internal_table_name'
stored['objectname']='MyObject'
and turning it into something like this dynamically:
class MyObject(Base):
__tablename__ = 'my_internal_table_name'
__table_args__ = {'autoload':True}
We don't want the classes to persist longer than necessary to open a connection, perform the queries, and then closing the connection. Therefore, ideally, we can put the items in the "stored" variable above into a database and pull them as needed. The other challenge is that the object name (e.g. "MyObject") may be used on different connections so we cannot define it once and keep it around.
Any suggestions on how this might be accomplished would be greatly appreciated.
Thanks...

You can dynamically create MyObject using the 3-argument call to type:
type(name, bases, dict)
Return a new type object. This is essentially a dynamic form of the
class statement...
For example:
mydict={'__tablename__':stored['tablename'],
'__table_args__':{'autoload':True},}
MyObj=type(stored['objectname'],(Base,),mydict)
print(MyObj)
# <class '__main__.MyObject'>
print(MyObj.__base__)
# <class '__main__.Base'>
print(MyObj.__tablename__)
# my_internal_table_name
print(MyObj.__table_args__)
# {'autoload': True}