I am using sqlalchemy ORM with declarative extensions in a Flask project for a web API.
The Flask documentation shows how to execute query properly inside an endpoint here
Now this is all good, but what if i want to execute a group by query. For example given the user domain object in the flask documentation:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50), unique=True)
email = Column(String(120), unique=True)
I want to retrieve the sum of ids for all the users with the same name (doesn't make sense, take it as toy example for sake of simplicity!).
In pure SQL i could do it easily with:
select name, sum(id) from user group by name
However inside an endpoint the reccomanded way to execute a query (again see here) is to start from a ref to the domain object User, for instance:
User.query.filter(User.name == 'admin').first()
To execute a group-by query I would rather use something like this (which indeed works!):
#see database module http://flask.pocoo.org/docs/0.12/patterns/sqlalchemy/#declarative)
from database import db_session
#app.route('/')
def index():
db_session.query(func.sum(User.id).label('num_id'), User.name).group_by(User.name).all()
In this case the query starts from db_session.
Is this correct and safe? Remeamber we are in a concurrent application inside Flask.
Any help?
It's fine.
But I usually use .with_entities(*entities) for such cases:
User.query.filter(filters).with_entities(User.name.label('name'), func.count(User.id).label('count')).group_by(User.name)
Related
I've written things in SQLAlchemy that work fine but the same workflow/commands seem to be askew when doing them in Flask. For example, these types of lines are getting flagged as "undefined":
metadata = MetaData(db)
abc = Table('abc', metadata, autoload=True)
s = select([abc.name])
I do have from flask.ext.sqlalchemy import SQLAlchemy at the top.
Is there something else I need to be doing additionally or instead?
(For reference, I'm running it at pythonanywhere)
Importing SQLAlchemy will not give you direct access to the names inside that module. You should also be aware that using the flask_sqlalchemy (formerly flask.ext.sqlalchemy) module uses a somewhat different mechanism to access SQLAlchemy features. This means that any attempt to transfer your current knowledge of SQLAlchemy shouold be informed by a study of the flask_sqlalchemy documentation.
Typically you will create a Flask application and then pass that to a call to SQLAlchemy as in this example. The relevant code is shown below.
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
The db object now has the Model, Column and the various datatypes as attributes, so you can define a table/model in the following way.
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
def __init__(self, username, email):
self.username = username
self.email = email
def __repr__(self):
return '<User %r>' % self.username
I personally am not fond of this particular access model, since it forces me to qualify the various names inside the db namespace rather than simply importing them from a module and using them unqualified, but it seems to work (at least for relatively uncomplicated databases).
If you are an experienced SQLAlchemy user you might want to consider using the standard access mechanisms, though this may render you vulnerable to subtle bugs due to unanticipated thread/web session interactions. I have also heard that it can be tricky to deploy multiple databases. I have no direct evidence of this, so please regard it as anecdotal.
First time on the site, so hi to all and thanks in advance. Longtime lurker and newb.
I'm working on a web app in flask, using Flask-SqlAlchemy and SqlAlchemy-Searchable (docs-> https://sqlalchemy-searchable.readthedocs.org/en/latest/index.html). For a reason I can't figure out, when I try a similar example to the code shown on the docs page:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy, BaseQuery
from sqlalchemy_searchable import SearchQueryMixin
from sqlalchemy_utils.types import TSVectorType
from sqlalchemy_searchable import make_searchable
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+psycopg2://usr:admin#localhost/dev'
app.config['SECRET_KEY'] = 'notreallyasecret'
db = SQLAlchemy(app)
make_searchable()
class ArticleQuery(BaseQuery, SearchQueryMixin):
pass
class Article(db.Model):
query_class = ArticleQuery
__tablename__ = 'article'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(255))
content = db.Column(db.UnicodeText)
search_vector = db.Column(TSVectorType('name', 'content'))
My search queries don't work properly. I opened a python shell and created the db, and inserted five identical articles
a= Article(name='finland',content='finland')
db.session.add(a)
db.session.commit() #a-e
with 'finland' both as name and content. According to the example:
Article.query.search(u'finland').limit(5).all()
There should be articles returned that have finland somewhere in them. In my case, I get an empty list. I get an object back if I modify the example query to:
Article.query.search(' ').first()
But it's rather useless searching for empty spaces. Any ideas?
Adding a bit more to it: I noticed in the article table, the 'search_vector tsvector' column is completely empty despite data being in the content and name columns; I'm not sure if that has anything to do with it.
I ran into this exact issue once, too, when using Flask-Script to add a manage.py management tool to my application.
The fact that the search_vector column is empty despite you having added the appropriate TSVectorType parameters means that the SQLAlchemy-Searchable trigger isn't present in the postgres DB. You can verify its absence by doing a \df+ in psql command line tool -- you will not see a trigger named article_search_vector_update. SQLAlchemy-Searchable sets up this trigger to update the content of the search_vector column when the columns named in TSVectorType(...) change.
In the case of manage.py, I had to first call:
db.configure_mappers()
Essentially, you have to configure SQLAlchemy's mappers before calling create_all(). Without doing this, SQLAlchemy-Searchable will not be given the opportunity to add its search_vector trigger to populate the TSVectorType column in the model.The SQLAlchemy-Searchable docs have more on this.
In total, a minimal manage.py that properly configures SQLAlchemy-Searchable as you require might look like:
#!/usr/bin/env python
from flask.ext.script import Manager
from app import app, db
manager = Manager(app)
#manager.command
def init_db():
"""
Drops and re-creates the SQL schema
"""
db.drop_all()
db.configure_mappers()
db.create_all()
db.session.commit()
On Collin Allen's answer: actually, the flask-sqlalchemy ''db'' exposes the configure_mappers function.
Replace:
from sqlalchemy.orm.mapper import configure_mappers
...
configure_mappers()
with:
...
db.configure_mappers()
I'm following the Flask-SQLAlchemy tutorial.
I have Flask 0.9, sqlalchemy 0.7.8 and flask-sqlalchemy 0.16 on python 2.6.
I'm trying to create a "one to many" relationship, like in their tutorial.
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
addresses = db.relationship('Address', backref='person',
lazy='dynamic')
class Address(db.Model):
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(50))
person_id = db.Column(db.Integer, db.ForeignKey('person.id'))
After that, I can create the database :
from DataBase.Tables.MyClass import db
db.create_all()
It works well when both classes are created on the same file.
It does not work anymore when I want to create this through 2 different files (2 different modules).
This is just an example (I'm trying to do something much more complicated with plenty of classes and I need the relationship to exist between 2 different modules but I'll simplify it so my question can be easier to understand.)
I have 2 modules : Person and Address, both of them have :
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///C:\\MyBase\\Base.sqlite'
db = SQLAlchemy(app)
and each of them has the declaration of the class written before.
My main function is in a 3rd module:
from DataBase.Tables.Person import db as person_db
from DataBase.Tables.Address import db as address_db
if __name__ == "__main__":
import DataBase.Tables.Person
import DataBase.Tables.Address
person_db.create_all()
address_db.create_all()
I still get an error in Eclipse:
*sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'Address.person_id' could not find table 'person' with which to generate a foreign key to target column 'sid'*
I could find another post with someone suggesting the use of "metadata" but I couldn't find a proper way to use that.
Does anyone have an idea to solve this ?
Thanks !
You need to have only one set of the below, and not a separate copy for each model:
app = Flask(my_app_name)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///C:\\MyBase\\Base.sqlite'
db = SQLAlchemy(app)
This can be defined in a separate module (lets call it shared), and imported into each model definition file.
In this case the main module will look more like:
from DataBase.Tables.shared import db
if __name__ == "__main__":
import DataBase.Tables.Person # will load Person model into the db
import DataBase.Tables.Address # will load Address model into the db
db.create_all() # will create all models
I have recently made a decision to start using the Pyramid (python web framework) for my projects from now on.
I have also decided to use SQLalchemy, and I want to use raw MySQL (personal reasons) but still keep the ORM features.
The first part of the code in models.py reads:
DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
Base = declarative_base()
Now from here how do I exectue a query for CREATE TABLE using raw MySQL.
the traditional SQLalchemy way would be:
class Page(Base):
__tablename__ = 'pages'
id = Column(Integer, primary_key=True)
name = Column(Text, unique=True)
data = Column(Text)
def __init__(self, name, data):
self.name = name
self.data = data
DBSession.execute('CREATE TABLE ....')
Have a look at sqlalchemy.text() for parametrized queries.
My own biased suggestion would be to use http://pypi.python.org/pypi/khufu_sqlalchemy to setup the sqlalchemy engine.
Then inside a pyramid view you can do something like:
from khufu_sqlalchemy import dbsession
db = dbsession(request)
db.execute("select * from table where id=:id", {'id':7})
Inside the views.py if you are adding form elements, first create an object of the database.
In your snippet, do it as
pg = Page()
and add it with
DBSession.add(pg)
for all the form elements you want to add e.g name and data from your snippet.
the final code would be similar to:
pg = Page()
name = request.params['name']
data = request.params['data']
DBSession.add(pg)
I have problem with setting up database connection. I want to set connection, where I can see this connection in all my controllers.
Now I use something like this in my controller:
db = create_engine('mysql://root:password#localhost/python')
metadata = MetaData(db)
email_list = Table('email',metadata,autoload=True)
In development.ini I have:
sqlalchemy.url = mysql://root#password#localhost/python
sqlalchemy.pool_recycle = 3600
How do I set _____init_____.py?
I hope you got pylons working; for anyone else that may later read question I'll present some pointers in the right direction.
First of all, you are only creating a engine and a metadata object. While you can use the engine to create connections directly you would almost always use a Session to manage querying and updating your database.
Pylons automatically setups this for you by creating a engine from your configuration file, then passing it to yourproject.model.__init__.py:init_model() which binds it to a scoped_session object.
This scoped_session object is available from yourproject.model.meta and is the object you would use to query your database. For example:
record = meta.Session.query(model.MyTable).filter(id=42)
Because it is a scoped_session it automatically creates a Session object and associates it with the current thread if it doesn't already exists. Scoped_session passes all action (.query(), .add(), .delete()) down into the real Session object and thus allows you a simple way to interact the database with having to manage the non-thread-safe Session object explicitly.
The scoped_session, Session, object from yourproject.model.meta is automatically associated with a metadata object created as either yourproject.model.meta:metadata (in pylons 0.9.7 and below) or yourproject.model.meta:Base.metadata (in pylons 1.0). Use this metadata object to define your tables. As you can see in newer versions of pylons a metadata is associated with a declarative_base() object named Base, which allows you to use SqlAlchemy's declarative style.
Using this from the controller
from yourproject import model
from yourproject.model import Session
class MyController(..):
def resource(self):
result = Session.query(model.email_list).\
filter(model.email_list.c.id=42).one()
return str(result)
Use real connections
If you really want to get a connection object simply use
from yourproject.model import Session
connection = Session.connection()
result = connection.execute("select 3+4;")
// more connection executions
Session.commit()
However this is all good, but what you should be doing is...
This leaves out that you are not really using SqlAlchemy much. The power of SqlAlchemy really shines when you start mapping your database tables to python classes. So anyone looking into using pylons with a database should take a serious look at what you can do with SqlAlchemy. If SqlAlchemy starts out intimidating simply start out with using its declarative approach, which should be enough for almost all pylons apps.
In your model instead of defining Table constructs, do this:
from sqlalchemy import Column, Integer, Unicode, ForeignKey
from sqlalchemy.orm import relation
from yourproject.model.meta import Base
class User(Base):
__tablename__ = 'users'
# primary_key implies nullable=False
id = Column(Integer, primary_key=True, index=True)
# nullable defaults to True
name = Column(Unicode, nullable=False)
notes = relation("UserNote", backref="user")
query = Session.query_property()
class UserNote(Base):
__tablename__ = 'usernotess'
# primary_key implies nullable=False
id = Column(Integer, primary_key=True, index=True)
userid = Column(Integer, index=True, ForeignKey("User.id"))
# nullable defaults to True
text = Column(Unicode, nullable=False)
query = Session.query_property()
Note the query objects. These are smart object that live on the class and associates your classes with the scoped_session(), Session. This allows you to event more easily extract data from your database.
from sqlalchemy.orm import eagerload
def resource(self):
user = User.query.filter(User.id==42).options(eagerload("notes")).one()
return "\n".join([ x.text for x in user.notes ])
1.0 version of Pylons use declarative syntax. More about this, you can see here .
In mode/init.py you can write somthing like this:
from your_programm.model.meta import Session, Base
from sqlalchemy import *
from sqlalchemy.types import *
def init_model(engine):
Session.configure(bind=engine)
class Foo(Base) :
__tablename__ = "foo"
id = Column(Integer, primary_key=True)
name = Column(String)
...
What you want to do is modify the Globals class in your app_globals.py file to include a .engine (or whatever) attribute. Then, in your controllers, you use from pylons import app_globals and app_globals.engine to access the engine (or metadata, session, scoped_session, etc...).