We Keep Coding

Issue while trying to set enum data type in MySQL database

What am I trying to do?
Django does not support setting enum data type in mysql database. Using below code, I tried to set enum data type.
Error Details
_mysql.connection.query(self, query) django.db.utils.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use
near 'NOT NULL, created_at datetime(6) NOT NULL, user_id bigint
NOT NULL)' at line 1")
Am I missing anything?
Enumeration class with all choices
class enumTokenTypes(models.TextChoices):
Registration = "Registration"
ForgotPassword = "Forgot Password"
User Token class in model
class tblusertokens(models.Model):
token_id = AutoField(primary_key=True)
token_type = EnumField(max_length=20, choices=enumTokenTypes.choices)
created_at = DateTimeField(auto_now_add=True, blank=True)
user = ForeignKey(tblusers, on_delete = models.CASCADE)
User token create model in migration
class EnumField(CharField):
def db_type(self, connection):
return "enum"
migrations.CreateModel(
name='tblusertokens',
fields=[
('token_id', models.AutoField(primary_key=True, serialize=False)),
('token_type', clientauth.models.EnumField(choices=[('Registration', 'Registration'), ('Forgot Password', 'Forgotpassword')], max_length=20)),
('created_at', models.DateTimeField(auto_now_add=True)),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='clientauth.tblusers')),
],
)
Bounty Question
set 2 parameters to the function to pass comma separated values and a default value.

The data type should be enum('Registration', 'Forgot Password') instead of just enum.
class EnumField(CharField):
def db_type(self, connection):
if connection.vendor == 'mysql':
return 'enum({0})'.format(','.join("'%s'" % value for value, label in self.choices))
return super().db_type(connection)
Reference: https://dev.mysql.com/doc/refman/8.0/en/enum.html
DB default
While not explicitly mentioned in the MySQL 8.0 docs above, you can also specify a DB default.
class EnumField(CharField):
def __init__(self, *args, **kwargs):
self.db_default = kwargs.pop('db_default', None)
super().__init__(*args, **kwargs)
def db_type(self, connection):
if connection.vendor == 'mysql':
if self.db_default is not None:
return "enum({0}) DEFAULT '{1}'".format(','.join("'%s'" % value for value, label in self.choices), self.db_default)
return 'enum({0})'.format(','.join("'%s'" % value for value, label in self.choices))
return super().db_type(connection)
def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
if self.db_default:
kwargs['db_default'] = self.db_default
return name, path, args, kwargs
Usage:
token_type = EnumField(max_length=20, choices=enumTokenTypes.choices, db_default=enumTokenTypes.ForgotPassword)
About deconstruct() method
From https://docs.djangoproject.com/en/3.2/howto/custom-model-fields/#field-deconstruction:
The counterpoint to writing your __init__() method is writing the deconstruct() method. It’s used during model migrations to tell Django how to take an instance of your new field and reduce it to a serialized form - in particular, what arguments to pass to __init__() to re-create it.
If you add a new keyword argument, you need to write code in deconstruct() that puts its value into kwargs yourself.

You can print out the sql for that migration to see specifically whats wrong, but defining db_type to return "enum" is definitely not the right way to approach it.
('token_type', CharField(choices=enumTokenTypes.choices, max_length=22)),
Does the recommended syntax from the docs on Enumeration types not work for you for some reason?

Response to Bounty Question
For setting default value: add the default parameter in EnumField. Below example I have set up enumTokenTypes Registration to be its default value. Look into Django documentation for enum implementation with examples
class tblusertokens(models.Model):
token_id = AutoField(primary_key=True)
token_type = EnumField(max_length=20, choices=enumTokenTypes.choices, default=enumTokenTypes.Registration )
created_at = DateTimeField(auto_now_add=True, blank=True)
user = ForeignKey(tblusers, on_delete = models.CASCADE)

Google Cloud Datastore 'NoneType' object has no attribute 'email'

When I try getting a kind from my datastore it returns NoneType, as if the query is empty. I know the datastore is working properly while saving, but pulling a kind from the a query is not.
Also using the GQL Query in the Google cloud Console website and using SELECT * FROM User does return all the kinds. User kind has no parents, it is at the root. I made sure all the properties are indexed as well.
I am not sure what I am doing wrong on GET.
MyApp.py
import webapp2
from google.appengine.ext import ndb
from google.appengine.ext.db import GqlQuery
class MainHandler(webapp2.RequestHandler):
def post(self):
message = self.request.body
message = message.splitlines()
if message[0] == "register":
user = User.create_user(message[1], message[2], message[3])
user_key = User.save_user(user)
if user_key is not None:
self.response.write(user_key)
else:
user = User.get_by_id(User.email == message[0])
if User.token == message[1]:
self.response.write("CURRENT")
else:
User.token = message[1]
User.save_user(user)
self.response.write("UPDATED")
def get(self):
self.response.write("CONNECTED")
user= User.query().get()
self.response.write("\n" + query.email)
class User(ndb.Model):
email = ndb.StringProperty()
token = ndb.StringProperty()
name = ndb.StringProperty()
#classmethod
def create_user(cls, email, token, name):
user = User(email=email, token=token, name=name, id=email)
return user
#classmethod
def save_user(cls, user):
user_key = user.put()
return user_key
#classmethod
def get_user(cls, email):
return User.get_by_id(User.email == email)
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)

You seem to be confusing .get_by_id() with a query.
The get_by_id method is actually mapped to ndb.Model._get_by_id which invokes ndb.Model._get_by_id_async, which requires an entity key identifier to determine the entity's key used to do a direct entity lookup (not a query!). From appengine.ext.ndb.model.py:
#classmethod
#utils.positional(3)
def _get_by_id_async(cls, id, parent=None, app=None, namespace=None,
**ctx_options):
"""Returns an instance of Model class by ID (and app, namespace).
This is the asynchronous version of Model._get_by_id().
"""
key = Key(cls._get_kind(), id, parent=parent, app=app, namespace=namespace)
return key.get_async(**ctx_options)
But in your code you're passing as id a bool: User.email == message[0], which most likely won't match any existing entity key identifiers, hence the None result causing the error you see.
Since the info you have available is the value of an entity's property (the email value) you probably want to perform a query, something along these lines:
results = User.query(User.email == message[0]).fetch(limit=1)
if results:
user = results[0]

So I figured out what was wrong. So there seems to be an issue with the way Google Cloud SDK is set up on my computer. When running the same code on the google servers rather than on my network everything seems to work properly.

Adding to sqlalchemy mapping class non db attributes

The app has such logic: list of people stored in the database, each man has a rating calculated in realtime and this value is never stored in database. And I want to use one class to work with dababase fields: name, age etc. and non database field: rating.
Is it possible in sqlalchemy? Now I'm using inheritance Man -> ManMapping:
class Man:
rating = None
def get_rating(self):
return self.rating
...
class ManMapping(Base, Man):
__tablename__ = 'man'
id = Column('man_id', Integer, primary_key=True)
name = Column(Unicode)
...
It works but it looks terrible for me. Is it right approach or I have to do something else?

This is the correct solution: https://docs.sqlalchemy.org/en/13/orm/constructors.html
Hybrid properties are somewhat less flexible that this. The accepted answer is not an actual answer to the problem.
The SQLAlchemy ORM does not call init when recreating objects from database rows. The ORM’s process is somewhat akin to the Python standard library’s pickle module, invoking the low level new method and then quietly restoring attributes directly on the instance rather than calling init.
If you need to do some setup on database-loaded instances before they’re ready to use, there is an event hook known as InstanceEvents.load() which can achieve this; it is also available via a class-specific decorator called reconstructor(). When using reconstructor(), the mapper will invoke the decorated method with no arguments every time it loads or reconstructs an instance of the class. This is useful for recreating transient properties that are normally assigned in init:
from sqlalchemy import orm
class MyMappedClass(object):
def __init__(self, data):
self.data = data
# we need stuff on all instances, but not in the database.
self.stuff = []
#orm.reconstructor
def init_on_load(self):
self.stuff = []

If you are using any data from the DB to calculate rating I would recommend looking at hybrid property. Otherwise I would add self.rating to init and have your function inside the ManMapping class. Something like:
class ManMapping(Base):
__tablename__ = 'man'
id = Column('man_id', Integer, primary_key=True)
name = Column(Unicode)
def __init__(self)
self.rating = None
def get_rating(self):
return self.rating

In my point of view, you should have two distincts classes.
One for the logic in your code and one to communicate with your DB.
class Man(object):
"""This class is for your application"""
def __init__(self, name, rating):
# If the identifier is only used by the DB it should not be in this class
self.name = name
self.rating = rating
class ManModel(Base):
"""This model is only to communicate with the DB"""
__tablename__ = 'man'
id = Column('man_id', Integer, primary_key=True)
name = Column(Unicode)
You should have a provider that does queries to DB with ManModel objects, then maps results to Man objects and return your mapped data to the caller.
Your application will only use Man objects and your provider will do the mapping.
Something like below :
class DbProvider(object):
def get_man(self, id):
man_model = session.query(ManModel).filter(ManModel.id == id).one_or_none()
return self.man_mapper(man_model) if man_model else None
def get_men(self):
men_model = session.query(ManModel).all()
return [self.man_mapper(man_model) for man_model in men_model]
def man_mapper(self, man_model):
return Man(man_model.name, self.calculate_rating(man_model))
class Test(object):
def display_man(self):
man = db_provider.get_man(15)
if man:
print man.name, man.rating

Attaching extra information to model instance - django

I have a django model that I want to attach an extra piece of information to, depending on the environment the instance is in (which user is logged in). For this reason, I don't want to do it at the database level.
Is this okay to do? Or are there problems that I don't foresee?
in models.py
class FooOrBar(models.Model):
"""Type is 'foo' or 'bar'
"""
def __init__(self, type):
self.type = type
in views.py
class FooCheck(FooOrBar):
"""Never saved to the database
"""
def __init__(self, foo_or_bar):
self.__dict__ = foo_or_bar.__dict__.copy()
def check_type(self, external_type):
if external_type == 'foo':
self.is_foo = True
else:
self.is_foo = False
foos_or_bars = FooOrBar.objects.all()
foochecks = map(FooCheck, foos_or_bars)
for foocheck in foochecks:
foocheck.check_type('foo')
extra credit question: Is there a more efficient way of calling a method on multiple objects i.e. replacing the last forloop with something clever?

Okay, this does not work. Trying to delete a FooOrBar objects throws a complaint about
OperationalError at /
no such table: test_FooCheck
To get around this I'm just not going to inherit from FooOrBar, but if anyone has a suggestion on a better way to do it I'd be interested in hearing it

I had a similar issue, I did something like:
class Foo(models.Model):
# specific info goes here
class Bar(models.Model):
# specific info goes here
class FooBar(models.Model):
CLASS_TYPES = {
"foo":Foo,
"bar":Bar
}
type = models.CharField(choices=CLASS_TYPES)
id = models.IntegerField()
#field to identify FooBar
then you can get the object back using
object = FooBar.CLASS_TYPES[instance.type].objects.get(id=instance.id)
where instance is the FooBar instance

Creating fields on the fly in django

I have two tables (say Parent and Child). I want parent and Child to link to each other using a foreign key. The child table has two fields, one among them being 'country_sig'. This country_sig is actually made from joining two fields in parent table named 'country' and 'code'.
Eg: If country is 'IN' and code is '15' then country_sig is IN.15.
Now I want that during creation of django model for parent table, it should automatically create this field country_sig derived from 'country' and 'code' so that I can reference it to the child table.
PS : Since the parent table is very large, I don't want to create another field in the db derived from those two fields, but I can tweak the other table (child table) to divide the country_sig field into two columns with 'country' and 'code' but that will not work since django does'nt support composite primary key.
EDIT:
Actually I am implementing it in tastypie, I want tastypie to think as if it is a real field. This would solve my real problem.

Perhaps don't use the standard tasypie ModelResources, but write your own resource:
class DictWrapper(object):
'''
to be used as generic tastypie resource object
'''
def __init__(self, initial=None):
self.__dict__['_data'] = {}
if hasattr(initial, 'items'):
self.__dict__['_data'] = initial
def __getattr__(self, name):
return self._data.get(name, None)
def __setattr__(self, name, value):
self.__dict__['_data'][name] = value
def to_dict(self):
return self._data
class ParentResource(Resource):
def get_object_list(self, request):
ret = []
for parent in Parent.objects.all():
ret.append(DictWrapper({"country_sig":"{0}.{1}".format(parent.country, parent.code})))
return return
def obj_get_list(self, request=None, **kwargs):
return self.get_object_list(request)