I'm using cython for code obfuscation, so performance is not an issue at the moment. The problem is with using dataclasses.
There is no error during compilation time when cythonize code that contains dataclass definitions. But when running the code, I get a TypeError: <field> is a field but has no type annotation.
Here is the code I'm trying to cythonize:
from dataclasses import dataclass, field
from typing import Dict, Any, List
#dataclass
class dataclass_test:
ddict: Dict[str, Any]
sstr: str
bbool: bool
llist: List[str]
ffloat: float
llist1: List[str] = field(default_factory=list)
Running the code without cythonization works fine. But after cythonization I get the following error message:
File "dataclass_.py", line 4, in init dataclass_
#dataclass File "/home/aryskin/miniconda3/envs/tf113_gpu_conda/lib/python3.7/dataclasses.py", line 991, in dataclass
return wrap(_cls) File "/home/aryskin/miniconda3/envs/tf113_gpu_conda/lib/python3.7/dataclasses.py", line 983, in wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen) File "/home/aryskin/miniconda3/envs/tf113_gpu_conda/lib/python3.7/dataclasses.py", line 857, in _process_class
raise TypeError(f'{name!r} is a field but has no type annotation') TypeError: 'llist1' is a field but has no type annotation
Is there any way to avoid this problem without or with minimal rewriting of the source code?
Until support for this is merged into Cython, a workaround is to manually add the __annotations__ attribute to your class and repeat your type annotations:
#dataclass
class Fieldset:
label: str
fields: List[Field] = []
__annotations__ = {
'label': str,
'fields': List[Field],
}
Related
When working with modular imports with FastAPI and SQLModel, I am getting the following error if I open /docs:
TypeError: issubclass() arg 1 must be a class
Python 3.10.6
pydantic 1.10.2
fastapi 0.85.2
sqlmodel 0.0.8
macOS 12.6
Here is a reproducible example.
user.py
from typing import List, TYPE_CHECKING, Optional
from sqlmodel import SQLModel, Field
if TYPE_CHECKING:
from item import Item
class User(SQLModel):
id: int = Field(default=None, primary_key=True)
age: Optional[int]
bought_items: List["Item"] = []
item.py
from sqlmodel import SQLModel, Field
class Item(SQLModel):
id: int = Field(default=None, primary_key=True)
price: float
name: str
main.py
from fastapi import FastAPI
from user import User
app = FastAPI()
#app.get("/", response_model=User)
def main():
return {"message": "working just fine"}
I followed along the tutorial from sqlmodel https://sqlmodel.tiangolo.com/tutorial/code-structure/#make-circular-imports-work.
If I would put the models in the same file, it all works fine. As my actual models are quite complex, I need to rely on the modular imports though.
Traceback:
Traceback (most recent call last):
File "/Users/felix/opt/anaconda3/envs/fastapi_test/lib/python3.10/site-packages/fastapi/utils.py", line 45, in get_model_definitions
m_schema, m_definitions, m_nested_models = model_process_schema(
File "pydantic/schema.py", line 580, in pydantic.schema.model_process_schema
File "pydantic/schema.py", line 621, in pydantic.schema.model_type_schema
File "pydantic/schema.py", line 254, in pydantic.schema.field_schema
File "pydantic/schema.py", line 461, in pydantic.schema.field_type_schema
File "pydantic/schema.py", line 847, in pydantic.schema.field_singleton_schema
File "pydantic/schema.py", line 698, in pydantic.schema.field_singleton_sub_fields_schema
File "pydantic/schema.py", line 526, in pydantic.schema.field_type_schema
File "pydantic/schema.py", line 921, in pydantic.schema.field_singleton_schema
File "/Users/felix/opt/anaconda3/envs/fastapi_test/lib/python3.10/abc.py", line 123, in __subclasscheck__
return _abc_subclasscheck(cls, subclass)
TypeError: issubclass() arg 1 must be a class
TL;DR
You need to call User.update_forward_refs(Item=Item) before the OpenAPI setup.
Explanation
So, this is actually quite a bit trickier and I am not quite sure yet, why this is not mentioned in the docs. Maybe I am missing something. Anyway...
If you follow the traceback, you'll see that the error occurs because in line 921 of pydantic.schema in the field_singleton_schema function a check is performed to see if issubclass(field_type, BaseModel) and at that point field_type is not in fact a type instance.
A bit of debugging reveals that this occurs, when the schema for the User model is being generated and the bought_items field is being processed. At that point the annotation is processed and the type argument for List is still a forward reference to Item. Meaning it is not the actual Item class itself. And that is what is passed to issubclass and causes the error.
This is a fairly common problem, when dealing with recursive or circular relationships between Pydantic models, which is why they were so kind to provide a special method just for that. It is explained in the Postponed annotations section of the documentation. The method is update_forward_refs and as the name suggests, it is there to resolve forward references.
What is tricky in this case, is that you need to provide it with an updated namespace to resolve the Item reference. To do that you need to actually have the real Item class in scope because that is what needs to be in that namespace. Where you do it does not really matter. You could for example import User model into your item module and call it there (obviously below the definition of Item):
from sqlmodel import SQLModel, Field
from .user import User
class Item(SQLModel):
id: int = Field(default=None, primary_key=True)
price: float
name: str
User.update_forward_refs(Item=Item)
But that call needs to happen before an attempt is made to set up that schema. Thus you'll at least need to import the item module in your main module:
from fastapi import FastAPI
from .user import User
from . import item
api = FastAPI()
#api.get("/", response_model=User)
def main():
return {"message": "working just fine"}
At that point it is probably simpler to have a sub-package with just the model modules and import all of them in the __init__.py of that sub-package.
The reason I gave the example of putting the User.update_forward_refs call in below your Item definition is that these situations typically occur, when you actually have a circular relationship, i.e. if your Item class had a users field for example, which was typed as list[User]. Then you'd have to import User there anyway and might as well just update the references there.
In your specific example, you don't actually have any circular dependencies, so there is strictly speaking no need for the TYPE_CHECKING escape. You can simply do from .item import Item inside user.py and put the actual class in your annotation as bought_items: list[Item]. But I assume you simplified the actual use case and simply forgot to include the circular dependency.
Maybe I am missing something and someone else here can find a way to call update_forward_refs without the need to provide Item explicitly, but this way should definitely work.
For anyone ending up here who (just like me) got the same error but couldn't resolve it using the solution above, my script looked like this. It seems that SQLModel relies on the pydantic.BaseModel so this solution also applies here.
from pydantic import BaseModel
class Model(BaseModel):
values: list[int, ...]
class SubModel(Model):
values = list[int, int, int]
It took me a long time to realize what my mistake was, but in SubModel I used = (assignment) whereas I should have used : (type hint).
The strangest thing was that it did work in a docker container (Linux) but not locally (Windows). Also, mypy did not pick up on this.
I want to have deafult values for Base class variables while not having default values for derived class variables.
I have read about 'kw_only' at:
https://medium.com/#aniscampos/python-dataclass-inheritance-finally-686eaf60fbb5
And tried it on my code:
from dataclasses import dataclass, field
#dataclass(kw_only=True)
class Base:
c:int = field(default=8, compare=False)
def printBVars(self):
print("Base:", self.c)
#dataclass(kw_only=True)
class Derive(Base):
cc:int = field()
def printDVars(self):
super().printBVars()
print("Derive:", self.cc)
a = Derive()
a.printDVars()
But Python is givving this error:
File "<string>", line 8, in <module> TypeError: dataclass() got an unexpected keyword argument 'kw_only'
What am I doing wrong?
Im using python 3.9
I have read about some solution suggestions from google, tried some but did not help. such as running:
pip install attrs --upgrade
As my MRE, I've got the following file:
blah.py
'''Blah module'''
import pydantic
class Foo:
'''Foo class'''
class Bar(pydantic.BaseModel):
'''Bar class'''
x: str = pydantic.Field(description='The x.')
#pydantic.validator('x')
def do_nothing(cls, value: str) -> str:
return value
I'm attempting to use Sphinx to generate documentation for this module. In my conf.py, I have
extensions = [
'sphinx.ext.autodoc',
'sphinxcontrib.autodoc_pydantic',
]
My blah.rst is
Blah
====
.. automodule:: blah.blah
:members:
I've pip installed pydantic and autodoc_pydantic.
However, when I make html, I get
Exception occurred:
File "/home/user/Projects/Workspace/env/lib/python3.10/site-packages/sphinxcontrib/autodoc_pydantic/inspection.py", line 311, in __init__
self.attribute: Dict = self.model.Config
AttributeError: type object 'Foo' has no attribute 'Config'
It appears that autodoc_pydantic thinks that Foo inherits from pydantic.BaseModel when it's really Bar that does. If I remove 'sphinxcontrib.autodoc_pydantic' from extensions, the error goes away.
More interestingly, if I delete the validator, the error goes away as well.
autodoc_pydantic is version 1.6.1.
This issue was fixed in version 1.7.2.
I'm creating a dataclass in 3.8.8 to store result from REST API call, and one of its attributes is used to store HTTP response status:
from dataclasses import dataclass
from http import HTTPStatus
#dataclass
class APICallResult:
response_text: str
http_status: HTTPStatus
When I try to create an object of this class as below:
result = APICallResult('foo', HTTPStatus.OK)
PyCharm 2020.3.3 just flags HTTPStatus.OK with error:
Expected type 'HTTPStatus', got 'int' instead`.
Now I wonder what's the proper type hint for IntEnum in this case? I think Literal might be an awkward option here.
HTTPStatus.OK is a member of the HTTPStatus IntEnum subclass:
class http.HTTPStatus
A subclass of enum.IntEnum that defines a set of HTTP status codes, reason phrases and long descriptions written in English.
The error you are getting is bug in the PyCharm static type checker, if you write the attribute type hint like in the example below and test it with mypy you'll see there is no error:
from dataclasses import dataclass
from http import HTTPStatus
from typing import Literal
#dataclass
class APICallResult:
response_text: str
http_status: Literal[HTTPStatus.OK]
result = APICallResult('foo', HTTPStatus.OK)
mypy issues no warning
Success: no issues found in 1 source file
If you try to annotate the dataclass attribute like this:
from dataclasses import dataclass
from http import HTTPStatus
#dataclass
class APICallResult:
response_text: str
http_status: HTTPStatus.OK
result = APICallResult('foo', HTTPStatus.OK)
mypy gives the following warning:
main.py:9: error: Invalid type: try using Literal[HTTPStatus.OK] instead?
Found 1 error in 1 file (checked 1 source file)
The rules for annotating a Enum member are given in PEP 586 -- Literal Types:
Legal parameters for Literal at type check time
Literal[Color.RED] # Assuming Color is some enum
In one application I have code which generates dynamic classes which reduces the amount of duplicated code considerably. But adding type-hints for mypy checking resulted in an error. Consider the following example code (simplified to focus on the relevant bits):
class Mapper:
#staticmethod
def action() -> None:
raise NotImplementedError('Not yet implemnented')
def magic(new_name: str) -> type:
cls = type('%sMapper' % new_name.capitalize(), (Mapper,), {})
def action() -> None:
print('Hello')
cls.action = staticmethod(action)
return cls
MyCls = magic('My')
MyCls.action()
Checking this with mypy will result in the following error:
dynamic_type.py:15: error: "type" has no attribute "action"
dynamic_type.py:21: error: "type" has no attribute "action"
mypy is obviously unable to tell that the return-value from the type call is a subclass of Mapper, so it complains that "type" has not attribute "action" when I assign to it.
Note that the code functions perfectly and does what it is supposed to but mypy still complains.
Is there a way to flag cls as being a type of Mapper? I tried to simply append # type: Mapper to the line which creates the class:
cls = type('%sMapper' % new_name.capitalize(), (Mapper,), {}) # type: Mapper
But then I get the following errors:
dynamic_type.py:10: error: Incompatible types in assignment (expression has type "type", variable has type "Mapper")
dynamic_type.py:15: error: Cannot assign to a method
dynamic_type.py:15: error: Incompatible types in assignment (expression has type "staticmethod", variable has type "Callable[[], None]")
dynamic_type.py:16: error: Incompatible return value type (got "Mapper", expected "type")
dynamic_type.py:21: error: "type" has no attribute "action"
One possible solution is basically to:
Type your magic function with the expected input and output types
Leave the contents of your magic function dynamically typed with judicious use of Any and # type: ignore
For example, something like this would work:
class Mapper:
#staticmethod
def action() -> None:
raise NotImplementedError('Not yet implemnented')
def magic(new_name: str) -> Mapper:
cls = type('%sMapper' % new_name.capitalize(), (Mapper,), {})
def action() -> None:
print('Hello')
cls.action = staticmethod(action) # type: ignore
return cls # type: ignore
MyCls = magic('My')
MyCls.action()
It may seem slightly distasteful to leave a part of your codebase dynamically typed, but in this case, I don't think there's any avoiding it: mypy (and the PEP 484 typing ecosystem) deliberately does not try and handle super-dynamic code like this.
Instead, the best you can do is to cleanly document the "static" interface, add unit tests, and keep the dynamic portions of your code confined to as small of region as possible.