I'm building an application in Flask and I have several SQLAlchemy models defined. I have a dictionary with key/value pairs for each of the model types.
I want a generalised insert using a dictionary... would this require a mapper? I know that wtforms.ext.sqlalchemy.orm.model_form() generates an object with populate_obj(model) so it is possible. I've combed through the documentation but can't find it. I can perform the commit later, but need a shortcut to populate the object for now. Please, does anyone have expertise?
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy()
db.init_app(app)
employee_data = {'firstname':'John','lastname':'Smith'}
project_data = {'name':'project1'}
dict_generalised_insert(model=Employee,dictionary=employee_data)
dict_generalised_insert(model=Project,dictionary=project_data)
def dict_generalised_insert(model=None,dictionary={})
obj = model.model()
obj.populate_obj(dictionary) # ???
return obj
class Employee(db.Model):
id = db.Column(db.Integer, primary_key=True)
firstname = db.Column(db.String(80))
lastname = db.Column(db.String(80))
class Project(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80))
The idiomatic way to unpack a dictionary is to use the double star operator **.
To use it with flask-sqlalchemy:
class Employee(db.Model)
id = db.Column(...)
firstname = db.Column(...)
lastname = db.Column(...)
employee_data = {'firstname':'John','lastname':'Smith'}
employee = Employee(**employee_data)
db.session.add(employee)
db.session.commit()
Be aware that the keys in the dictionary have to match the attribute names of the class. Unpacking in this manner is the same as:
employee = Employee(firstname='John', lastname='Smith')
You can also do this with a list if you define an __init__ (or other method) with positional arguments however you only use a single star:
def __init__(self, firstname, lastname):
self.firstname = firstname
self.lastname = lastname
employee_data = ['John', 'Smith']
employee = Employee(*employee_data)
...
Note here the order of the values is what's important.
Related
I'm following a tutorial and using the below code. I'm also using Postman to view the status of the server for http://localhost:5000/planets , but I'm getting 500 INTERNAL SERVER ERROR, when I should see my JSON data of the planets I created.
In the command line I also see: AttributeError: 'list' object has no attribute 'data'
I feel it might have to do with the line that has: return jsonify(result.data) but I'm not sure.
from flask import Flask, jsonify, request
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import Column, Integer, String, Float
import os
from flask_marshmallow import Marshmallow
from marshmallow import Schema
app = Flask(__name__)
basedir = os.path.abspath(os.path.dirname(__file__))
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///C:/Users/krist/Py3/flask2/planets.db'
db = SQLAlchemy(app)
ma = Marshmallow(app)
#app.cli.command('db_create')
def db_create():
db.create_all()
print("DB Created")
#app.cli.command('db_seed')
def deb_seed():
mercury = Planet(planet_name='Mercury',
planet_type='Class D',
home_star='Sol',
mass=3.25e23,
radius=1516,
distance=35.98e6)
venus = Planet(planet_name='Venus',
planet_type='Class K',
home_star='Sol',
mass=8.95e24,
radius=3516,
distance=67.98e6)
earth = Planet(planet_name='Earth',
planet_type='Class M',
home_star='Sol',
mass=5.97e24,
radius=3916,
distance=92.96e6)
db.session.add(mercury)
db.session.add(venus)
db.session.add(earth)
test_user = User(first_name='William',
last_name='Hershel',
email='test#test.com',
password='p#ssw0rd')
db.session.add(test_user)
db.session.commit()
print("DB Seeded")
#app.route('/planets', methods=['GET'])
def planets():
planets_list = Planet.query.all()
result = planets_schema.dump(planets_list)
return jsonify(result.data)
class User(db.Model):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
email = Column(String, unique=True)
password = Column(String)
class Planet(db.Model):
__tablename__ = 'planets'
planet_id = Column(Integer, primary_key=True)
planet_name = Column(String)
planet_type = Column(String)
home_star = Column(String)
mass = Column(Float)
radius = Column(Float)
distance = Column(Float)
class UserSchema(ma.Schema):
class Meta:
fields = ('id', 'first_name', 'last_name', 'email', 'password')
class PlanetSchema(ma.Schema):
class Meta:
fields = ('planet_id', 'planet_name', 'planet_type', 'home_star', 'mass', 'radius', 'distance')
user_schema = UserSchema()
users_schema = UserSchema(many=True)
planet_schema = PlanetSchema()
planets_schema = PlanetSchema(many=True)
if __name__ == '__main__':
app.run()
Instead of
result = planets_schema.dump(planets_list)
return jsonify(result.data)
Try
result = planets_schema.dump(planets_list)
return jsonify(result)
Why this works:
Here you are querying the Planet Mapper to return a list of Planet ORM objects
planets_list = Planet.query.all()
Then the Marshmallow schema is used to marshal, or transform the ORM object into a python dictionary object. This is the basic principle of marshaling - transforming data from one format into another when the data is about to be transmitted or stored. So in this case you transform you data from a list of SQLAlchemy ORM objects into a list of Python dictionary objects.
result = planets_schema.dump(planets_list)
Now you have result (which could more aptly be names results that contains a list of dictionary objects.
Then you are attempting to access the data variable on this list object. However Python lists have no data variable, so you get an error.
return jsonify(result.data)
The jsonify method from flask accepts a list of dictionaries as input, so simply modifying this line to the below should work:
return jsonify(result)
I have a models like this:
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
first_name = db.Column(db.String(64), index=True)
last_name = db.Column(db.String(64), index=True)
def full_name(self):
return '%s %s' % (self.first_name, self.last_name)
I want to get the full_name method in a query, I try it like it:
user = db.session.query(User.full_name()).all()
But I get this error message:
TypeError: full_name() missing 1 required positional argument: 'self'
Then I try to call the function without brackets:
user = db.session.query(User.full_name).all()
Then I got this error message:
sqlalchemy.exc.InvalidRequestError: SQL expression, column, or mapped entity expected - got '<function User.full_name at 0x7f265960aae8>'
So, what is the correct way to query full_name() method in the User model..?
There are a number of ways to accomplish this. Firstly, a hybrid attribute, a property that defines full_name on an instance and on the class (for querying).
This example is pure SQLAlchemy, but the hybrid attributes should be the same in Flask-SQLAlchemy.
import sqlalchemy as sa
from sqlalchemy.ext import hybrid
class User(Base):
__tablename__ = 'users'
id = sa.Column(sa.Integer, primary_key=True)
first_name = sa.Column(sa.String)
last_name = sa.Column(sa.String)
#hybrid.hybrid_property
def full_name(self):
return self.first_name + ' ' + self.last_name
users = session.query(User).filter_by(full_name='Joan Doe').all()
Credit to Ilja for pointing out that this can be done using a single method in this case. This works because SQLAlchemy maps the + operator to the database's CONCAT function. If this weren't the case, an additional property decorated with hybrid.expression would be necessary to implement the behaviour at the class level.
SQLAlchemy inserts the necessary expressions into SQL statements as required, for example:
SELECT users.id, users.first_name, users.last_name
FROM users
WHERE users.first_name || ? || users.last_name = ?
Another way to achieve this is by defining a column_property:
from sqlalchemy import orm
class User(Base):
...
full_name = orm.column_property(first_name + " " + last_name)
This inserts an expression into all queries on the model, so
query = sa.select(User)
will generate this SQL:
SELECT users.first_name || ? || users.last_name AS anon_1,
users.id,
users.first_name,
users.last_name
FROM users
Finally, there are computed columns. These are conceptually similar to column properties, but implemented on the database side (provided the database supports the required syntax). The database may or may not physically store the computed value; it may be possible to condition this behaviour by passing the persisted boolean keyword argument to Computed.
class User(Base):
...
full_name = sa.Column(sa.String, sa.Computed(first_name + ' ' + last_name))
You can use #classmethod.
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
first_name = db.Column(db.String(64), index=True)
last_name = db.Column(db.String(64), index=True)
#classmethod
def full_name_filter(cls, fname, lname):
return (cls.first_name == fname, cls.last_name == lname)
Then
user = db.session.query(User).filter(*User.full_name_filter("first", "last")).all()
How would one best implement a many-to-many, self-referential, non symmetrical relationship (think Twitter) in SqlAlchemy? I want to use an association object (let's call this class "Follow") so that I can have additional attributes associated with the relationship.
I've seen plenty of examples which use an association tables, but none like I've describe above. Here's what I have so far:
class UserProfile(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
full_name = Column(Unicode(80))
gender = Column(Enum(u'M',u'F','D', name='gender'), nullable=False)
description = Column(Unicode(280))
followed = relationship(Follow, backref="followers")
class Follow(Base):
__tablename__ = 'follow'
follower_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
followee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
status = Column(Enum(u'A',u'B', name=u'status'), default=u'A')
created = Column(DateTime, default=func.now())
followee = relationship(UserProfile, backref="follower")
Thoughts?
This is already almost answered in here. Here this is improved by having the advantages of a many-to-many made with a bare link table.
I'm not good in SQL and neither in SqlAlchemy but since I had this problem in mind for some longer time, I tried to find a solution that has both advantages: an association object with additional attributes and a direct association like with a bare link table (which doesn't provide an object on its own for the association). Stimulated by additional suggestions of the op the following seems quiet nice to me:
#!/usr/bin/env python3
# coding: utf-8
import sqlalchemy as sqAl
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, relationship, backref
from sqlalchemy.ext.associationproxy import association_proxy
engine = sqAl.create_engine('sqlite:///m2m-w-a2.sqlite') #, echo=True)
metadata = sqAl.schema.MetaData(bind=engine)
Base = declarative_base(metadata)
class UserProfile(Base):
__tablename__ = 'user'
id = sqAl.Column(sqAl.Integer, primary_key=True)
full_name = sqAl.Column(sqAl.Unicode(80))
gender = sqAl.Column(sqAl.Enum('M','F','D', name='gender'), default='D', nullable=False)
description = sqAl.Column(sqAl.Unicode(280))
following = association_proxy('followeds', 'followee')
followed_by = association_proxy('followers', 'follower')
def follow(self, user, **kwargs):
Follow(follower=self, followee=user, **kwargs)
def __repr__(self):
return 'UserProfile({})'.format(self.full_name)
class Follow(Base):
__tablename__ = 'follow'
followee_id = sqAl.Column(sqAl.Integer, sqAl.ForeignKey('user.id'), primary_key=True)
follower_id = sqAl.Column(sqAl.Integer, sqAl.ForeignKey('user.id'), primary_key=True)
status = sqAl.Column(sqAl.Enum('A','B', name=u'status'), default=u'A')
created = sqAl.Column(sqAl.DateTime, default=sqAl.func.now())
followee = relationship(UserProfile, foreign_keys=followee_id, backref='followers')
follower = relationship(UserProfile, foreign_keys=follower_id, backref='followeds')
def __init__(self, followee=None, follower=None, **kwargs):
"""necessary for creation by append()ing to the association proxy 'following'"""
self.followee = followee
self.follower = follower
for kw,arg in kwargs.items():
setattr(self, kw, arg)
Base.metadata.create_all(engine, checkfirst=True)
session = sessionmaker(bind=engine)()
def create_sample_data(sess):
import random
usernames, fstates, genders = ['User {}'.format(n) for n in range(4)], ('A', 'B'), ('M','F','D')
profs = []
for u in usernames:
user = UserProfile(full_name=u, gender=random.choice(genders))
profs.append(user)
sess.add(user)
for u in [profs[0], profs[3]]:
for fu in profs:
if u != fu:
u.follow(fu, status=random.choice(fstates))
profs[1].following.append(profs[3]) # doesn't work with followed_by
sess.commit()
# uncomment the next line and run script once to create some sample data
# create_sample_data(session)
profs = session.query(UserProfile).all()
print( '{} follows {}: {}'.format(profs[0], profs[3], profs[3] in profs[0].following))
print('{} is followed by {}: {}'.format(profs[0], profs[1], profs[1] in profs[0].followed_by))
for p in profs:
print("User: {0}, following: {1}".format(
p.full_name, ", ".join([f.full_name for f in p.following])))
for f in p.followeds:
print(" " * 25 + "{0} follow.status: '{1}'"
.format(f.followee.full_name, f.status))
print(" followed_by: {1}".format(
p.full_name, ", ".join([f.full_name for f in p.followed_by])))
for f in p.followers:
print(" " * 25 + "{0} follow.status: '{1}'"
.format(f.follower.full_name, f.status))
It seems indispensible to define two relations for the Association Object. The association_proxy method seems to be not ideally tailored for self-referential relations. The argument oder of the Follow constructor doesn't seem logical to me but works only this way (this is explained here).
In the book Rick Copeland - Essential Sqlalchemy on page 117 you find the following note regarding the secondary-parameter to relationship():
Note that, if you are using SQLAlchemy’s ability to do M:N
relationships, the join table should only be used to join the two
tables together, not to store auxilliary properties. If you need to
use the intermediate join table to store addi- tional properties of
the relation, you should use two 1:N relations instead.
Sorry for that this is a little bit verbose but I like code that I can copy, paste, and execute directly. This works with Python 3.4 and SqlAlchemy 0.9 but likely also with other versions.
I am creating a Flask application and accessing the MySQL database using Flask-Alchemy.
I have following Class to access a table:
class price_table(db.Model):
id = db.Column(db.Integer, primary_key = True)
trans_id = db.Column(db.Integer)
timestamp = db.Column(db.Integer)
order_type = db.Column(db.String(25))
price = db.Column(db.Numeric(15,8))
quantity = db.Column(db.Numeric(25,8))
def __repr__(self):
return 'id'
For the table 'price_table' this works brilliantly, but problem is I have a few tables with the same columns as 'price_table' from which I only know the name at runtime.
I want to reuse the class above so I thought I could change tablename to the name of the table I need to read, but that does not work, the program keeps reading the 'price-table'
How do I override the tablename at runtime?
You should use: __tablename__ :
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50), unique=True)
email = Column(String(120), unique=True)
http://flask.pocoo.org/docs/0.12/patterns/sqlalchemy/
Based on the comment left by jbub I found the following solution that does the trick just as needed.
from app import db
def ClassFactory(name):
tabledict={'id':db.Column(db.Integer, primary_key = True),
'trans_id':db.Column(db.Integer),
'timestamp':db.Column(db.Integer),
'order_type':db.Column(db.String(25)),
'price':db.Column(db.Numeric(25,8)),
'quantity':db.Column(db.Numeric(25,8)),}
newclass = type(name, (db.Model,), tabledict)
return newclass
You can overwrite price_table.table.name attribute, yet keep in mind that it will affect your price_table model so, unless you want to use it to create a new specialized version of this table in the db and you are not interacting with price_table model - I wouldn't recommend that.
I am creating a Point of Sales application, with the typical data hierarchy :
Company->branches->Sales->SaleData, this is the model definition (Note that, the User model already prepare and working as a flask-login compatible model) :
from flask_sqlalchemy import SQLAlchemy
from main import db
from collections import OrderedDict
class Users(db.Model,object):
'''
Adding object to trun sqlalchemy into json object
'''
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(60), unique=True)
firstname = db.Column(db.String(20))
lastname = db.Column(db.String(20))
password = db.Column(db.String)
email = db.Column(db.String(100), unique=True)
role = db.Column(db.String(20))
active = db.Column(db.Boolean)
company_id = db.Column(db.Integer, db.ForeignKey('companies.id'))
def __init__(self, username=None, password=None, email=None, firstname=None, lastname=None):
self.username = username
self.email = email
self.firstname = firstname
self.lastname = lastname
self.password = password
self.active = True
self.role = 'Admin'
def is_authenticated(self):
return True
def is_active(self):
return self.active
def is_anonymous(self):
return False
def get_id(self):
return unicode(self.id)
def _asdict(self):
'''
Thanks to http://stackoverflow.com/questions/7102754/jsonify-a-sqlalchemy-result-set-in-flask
'''
result = OrderedDict()
for key in self.__mapper__.c.keys():
result[key] = getattr(self, key)
return result
class Companies(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), unique=True)
address = db.Column(db.String)
users = db.relation('Users', backref=db.backref('users'))
token = db.Column(db.String) #for identification of client
branches = db.relationship("Branches")
def __init__(self, name=None, address=None, token=None):
self.name = name
self.address = address
self.token = token
class Branches(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(255), unique=True)
address = db.Column(db.String)
user_id = db.Column(db.Integer, db.ForeignKey('users.id'))
token = db.Column(db.String) #for identification of client
company_id = db.Column(db.Integer, db.ForeignKey('companies.id'))
sales = db.relation('Sales',
backref=db.backref('sales', lazy='dynamic'),
cascade="all, delete-orphan",
lazy='dynamic',
passive_deletes=True)
def __init__(self, name=None, address=None, token=None, user_id=None):
self.name = name
self.address = address
self.token = token
self.user_id = user_id
class Sales(db.Model):
id = db.Column(db.Integer, primary_key=True)
day = db.Column(db.Date)
branch_id = db.Column(db.Integer, db.ForeignKey('branches.id'))
data = db.relationship("SaleData")
def __init__(self, day=None):
self.day = day
class SaleData(db.Model):
id = db.Column(db.Integer, primary_key=True)
sale_id = db.Column(db.Integer, db.ForeignKey('sales.id'))
cash_start_of_day = db.Column(db.Integer)
cash_end_of_day = db.Column(db.Integer)
income = db.Column(db.Integer) # which is end - start
def __init__(self, cash_start_of_day = None, cash_end_of_day = None, income = None):
self.cash_start_of_day = cash_start_of_day
self.cash_end_of_day = cash_end_of_day
self.income = income
Now, if I try to add a Sales data to the branches, it didn't happen if I do this :
branch1 = company1.branches.filter().all()
I can olny do this :
branch1 = company1.branches[0]
If not using that [] operator, I got error message : AttributeError: 'InstrumentedList' object has no attribute'. I have already browse another answer here in SO, it got to do with that lazy things in backref definition, so I already modify my current model
But it seems like I am missing something here.. any clue?
Thanks!
EDIT 1 : Unit test added & User Model added too
I already got a concise answer from Mark Hildreth, and it saves me a lot! Because of that, I am going to put here the complete unit test of this model. I am sure it will help newbies out there in their very first step in SQLAlchemy. So, here goes :
import unittest
from main import db
import models
import md5
import helper
class DbTest(unittest.TestCase):
def setUp(self):
db.drop_all()
db.create_all()
def test_user_and_company(self):
"""admin of a company"""
user1 = models.Users('eko', helper.hash_pass('rahasia'), 'swdev.bali#gmail.com')
db.session.add(user1)
db.session.commit()
"""the company"""
company1 = models.Companies('CDI','Glagah Kidul', 'empty')
db.session.add(company1)
company1.users.append(user1)
db.session.commit()
assert company1.users[0].id == user1.id
"""branches"""
company1.branches.append(models.Branches(name='Kopjar',address='Penjara Malaysia', token='empty token', user_id=user1.id))
company1.branches.append(models.Branches(name='Selangor',address='Koperasi Selangor', token='empty token', user_id=user1.id))
db.session.commit()
'''sales'''
branch1 = company1.branches.filter(models.Branches.name=='Kopjar').first()
assert branch1.name=='Kopjar' and branch1.company_id == company1.id
sale = models.Sales(day='2013-02-02')
sale.data.append(models.SaleData(cash_start_of_day = 0, cash_end_of_day = 500000, income = 500000))
branch1.sales.append(sale)
db.session.commit()
assert sale.id is not None
if __name__ == '__main__':
unittest.main()
There may be bad practice in this model or unit test, and I will be delighted if you point that out :)
Thanks!
You may wish to review the "Collection Configuration" section of the documentation. There are a few main, built-in ways to deal with how relationships are handled in SQLAlchemy, and that section of the documentation shows the various ways.
By default, when you have...
class Companies(db.Model):
...
branches = db.relationship("Branches")
...
Loading a Company will load all of the branches in (and by default, it loads them into a list). Therefore, after retrieving a company, company.branches returns to you a list of branches. Because it is a list, it doesn't have functions such as filter() or all(). If you are not expecting a large list of branches, this might be preferred since it might make more sense for you to use branches as a list rather than as a query object. Having this as a list allows you to do things such as...
company = session.query(Companies).first()
my_branch = Branches(...)
company.branches.append(my_branch)
session.commit()
This will properly create the new Branches object without needing to add it specifically to the session (which I think is pretty nifty).
As a side note, if you were to do type(company.branches), you would not get <type 'list'>, because in order to pull off this magic, SQLAlchemy will actually set branches to an object type that works LIKE a list, but actually has additional SQLAlchemy-specific info. This object type, if you haven't guessed, is the "InstrumentedList" that you are getting the error message about.
However, you might not want to do this; specifically, this requires you to load in all of the branches at once, and you might only want to load a few in at a time because you have thousands of them (thousands of branches in a company, just imagine the bureaucracy...)
So, you change the relation, as the docs say...
A key feature to enable management of a large collection is the
so-called “dynamic” relationship. This is an optional form of
relationship() which returns a Query object in place of a collection
when accessed. filter() criterion may be applied as well as limits and
offsets, either explicitly or via array slices:
It appears that this is what you want to do if you want to be able to do things like company.branches.filter(...).all(). To do this, you would do as the docs show, by making the lazy attribute of the relationship "dynamic"...
class Companies(db.Model):
...
branches = db.relationship("Branches", lazy='dynamic')
...
It looks like you've done this already for the branches -> sales relationship, but you haven't for the company -> branches relationship, which is what is giving you the error.