I've updated SQLAlchemy to 0.6 but it broke everything. I've noticed it returns tuple not a dictionary anymore. Here's a sample query:
query = session.query(User.id, User.username, User.email).filter(and_(User.id == id, User.username == username)).limit(1)
result = session.execute(query).fetchone()
This piece of code used to return a dictionary in 0.5.
My question is how can I return a dictionary?
session.execute has never returned a dict, it returns a RowProxy object, that can be indexed like a dict using either integer keys for positional lookup, string keys for label based lookup up or Column objects to lookup the value of that column. The problem here is that session.execute(query) doesn't do what you seem to expect it to do. It converts the Query object to a Select statement, executes that and returns the result directly. The resultset doesn't know anything about ORM level features. What changed between 0.5 ad 0.6 is that ORM uses a different algorithm to label the columns in queries, it now prepends the table name to the label. So when previously row['id'] happened to work, now row['users_id'] works. In both cases row[User.__table__.columns['id']] works.
To execute ORM queries you should actually use the .all(), .first() and .one() methods or iterate over it or using numeric indexing. Query returns named tuple objects. Zip the tuple with its keys if you want a dict:
row = session.query(User.id, User.username, User.email)\
.filter(and_(User.id == id, User.username == username)).first()
print("id=%s username=%s email=%s" % row) # positional
print(row.id, row.username) # by label
print(dict(zip(row.keys(), row))) # as a dict
Are you sure it isn't a ResultProxy which pretends to be a tuple when you print it? Many objects in the ORM are not what their __str__ function returns.
This should work:
dict(zip(['id','username','email'],result)) (or you could use a dictionary comprehension if you're on Python 3.x).
Also, you don't need to call session.execute on a session.query object. You'll want to use the .one() method on it instead. This also obviates the need for the .limit(1) call hanging off the end of your query.
I solved this using:
# Get a list os user_ids
data_from_db = session.execute(select(User.id)).all()
# Parsing each Row object to a dict by "_mapping" attribute
return [dict(data._mapping) for data in data_from_db]
Related
I am trying to retrieve different .values() from query sets but am having an issue with it returning the proper values. How to write my model so that I can retrieve attributes using the .values() method?
I have tried to change the model's __str__ method to return a dictionary but that does not work or I am doing it wrong.
class Settings(models.Model):
bb_bonus_qualify = models.CharField(max_length=16, default=38.00)
service_breakpoint = models.CharField(max_length=16, default=1700.00)
def __str__(self):
return '%s: %s, %s: %s' % (
'bb_bonus', self.bb_bonus_qualify, 'service_breakpoint', self.service_breakpoint)
I would like to say Settings.objects.last().values('bb_bonus') and be returned the value which is self.bb_bonus_qualify. The common error I seem to get is: AttributeError: 'Settings' object has no attribute 'values'
The problem here is that your .last() will retrieve the last Settings object. You thus will call .values('bb_bonus') on the Settings object. Since a model has no .values(..) method, it will thus not return anything.
You can however retrieve the value of a certain column from a queryset, with:
Settings.objects.values_list('bb_bonus_qualify', flat=True).last()
We here thus use .values_list(..) [Django-doc], this accepts the names of the columns as parameters. It will then usually return a QuerySet of lists with these values. But if you specify one column; then, as the documentation says:
If you only pass in a single field, you can also pass in the flat parameter. If True, this will mean the returned results are single values, rather than one-tuples.
So that means we create a QuerySet of singular values, and we then will retrieve the last entry of that queryset. Note that we do not fetch all the elements from the queryset, the .last() is "injected" in the query we perform on the database, so the result is the scalar value of that column for the last record.
The .values_list(..) thus needs to be performed before the .last(), since otherwise you are talking to a Settings object, not to a QuerySet.
AFAIK __str__ has nothing to do with .values() - the problem here is that you need to specify the values before getting a specific item, rather than the other way round:
Settings.objects.values('bb_bonus').last()
So I have a dict passed from a web page. I want to build the query dynamically based on the dict. I know I can do:
session.query(myClass).filter_by(**web_dict)
However, that only works when the values are an exact match. I need to do 'like' filtering. My best attempt using the __dict__ attribute:
for k,v in web_dict.items():
q = session.query(myClass).filter(myClass.__dict__[k].like('%%%s%%' % v))
Not sure how to build the query from there. Any help would be awesome.
You're on the right track!
First thing you want to do different is access attributes using getattr, not __dict__; getattr will always do the right thing, even when (as may be the case for more convoluted models) a mapped attribute isn't a column property.
The other missing piece is that you can specify filter() more than once, and just replace the old query object with the result of that method call. So basically:
q = session.query(myClass)
for attr, value in web_dict.items():
q = q.filter(getattr(myClass, attr).like("%%%s%%" % value))
I'm trying to return a totals/averages row from my dataset which contains the SUM of certain fields and the AVG of others.
I could do this in SQL via:
SELECT SUM(field1) as SumFld, AVG(field2) as AvgFld
FROM Rating WHERE url=[url_string]
My attempt to translate this into SQLAlchemy is as follows:
totals = Rating.query(func.avg(Rating.field2)).filter(Rating.url==url_string.netloc)
But this is erroring out with:
TypeError: 'BaseQuery' object is not callable
You should use something like:
from sqlalchemy.sql import func
session.query(func.avg(Rating.field2).label('average')).filter(Rating.url==url_string.netloc)
You cannot use MyObject.query here, because SqlAlchemy tries to find a field to put result of avg function to, and it fails.
You cannot use MyObject.query here, because SqlAlchemy tries to find a field to put result of avg function to, and it fails.
This isn't exactly true. func.avg(Rating.field2).label('average') returns a Column object (the same type object that it was given to be precise). So you can use it with the with_entities method of the query object.
This is how you would do it for your example:
Rating.query.with_entities(func.avg(Rating.field2).label('average')).filter(Rating.url == url_string.netloc)
attention = Attention_scores.query
.with_entities(func.avg(Attention_scores.score))
.filter(classroom_number == classroom_number)
.all()
I tried it like this and it gave the correct average.
Is it possible to control which columns are queried in the query method of SQLAlchemy, while still returning instances of the object you are querying (albeit partially populated)?
Or is it necessary for SQLAlchemy to perform a SELECT * to map to an object?
(I do know that querying individual columns is available, but it does not map the result to an object, only to a component of a named tuple).
For example, if the User object has the attributes userid, name, password, and bio, but you want the query to only fill in userid and name for the objects it returns:
# hypothetical syntax, of course:
for u in session.query(User.columns[userid, name]).all():
print u
would print:
<User(1, 'bob', None, None)>
<User(2, 'joe', None, None)>
...
Is this possible; if so, how?
A simple solution that worked for me was:
users = session.query(User.userid, User.name)
for user in users:
print user
would print:
<User(1, 'bob')>
<User(2, 'joe')>
...
you can query for individual columns, which returns named tuples that do in fact act pretty much like your mapped object if you're just passing off to a template or something:
http://www.sqlalchemy.org/docs/orm/tutorial.html#querying
or you can establish various columns on the mapped class as "deferred", either configurationally or using options:
http://docs.sqlalchemy.org/en/latest/orm/loading_columns.html#deferred-column-loading
there's an old ticket in trac for something called "defer_everything_but()", if someone felt like providing tests and such there's no reason that couldn't be a feature add, here's a quick version:
from sqlalchemy.orm import class_mapper, defer
def defer_everything_but(entity, cols):
m = class_mapper(entity)
return [defer(k) for k in
set(p.key for p
in m.iterate_properties
if hasattr(p, 'columns')).difference(cols)]
s = Session()
print s.query(A).options(*defer_everything_but(A, ["q", "p"]))
defer() should really accept multiples, added ticket #2250 for that (edit: as noted in the comment this is in 0.9 as load_only())
Latest doc for load_only is here
http://docs.sqlalchemy.org/en/latest/orm/loading_columns.html#load-only-cols
If you're looking at a way to control that at model definition level, use deferred
http://docs.sqlalchemy.org/en/latest/orm/loading_columns.html#deferred
I have a method in my Python code that returns a tuple - a row from a SQL query. Let's say it has three fields: (jobId, label, username)
For ease of passing it around between functions, I've been passing the entire tuple as a variable called 'job'. Eventually, however, I want to get at the bits, so I've been using code like this:
(jobId, label, username) = job
I've realised, however, that this is a maintenance nightmare, because now I can never add new fields to the result set without breaking all of my existing code. How should I have written this?
Here are my two best guesses:
(jobId, label, username) = (job[0], job[1], job[2])
...but that doesn't scale nicely when you have 15...20 fields
or to convert the results from the SQL query to a dictionary straight away and pass that around (I don't have control over the fact that it starts life as a tuple, that's fixed for me)
#Staale
There is a better way:
job = dict(zip(keys, values))
I'd say that a dictionary is definitely the best way to do it. It's easily extensible, allows you to give each value a sensible name, and Python has a lot of built-in language features for using and manipulating dictionaries. If you need to add more fields later, all you need to change is the code that converts the tuple to a dictionary and the code that actually makes use of the new values.
For example:
job={}
job['jobid'], job['label'], job['username']=<querycode>
This is an old question, but...
I'd suggest using a named tuple in this situation: collections.namedtuple
This is the part, in particular, that you'd find useful:
Subclassing is not useful for adding new, stored fields. Instead, simply create a new named tuple type from the _fields attribute.
Perhaps this is overkill for your case, but I would be tempted to create a "Job" class that takes the tuple as its constructor argument and has respective properties on it. I'd then pass instances of this class around instead.
I would use a dictionary. You can convert the tuple to a dictionary this way:
values = <querycode>
keys = ["jobid", "label", "username"]
job = dict([[keys[i], values [i]] for i in xrange(len(values ))])
This will first create an array [["jobid", val1], ["label", val2], ["username", val3]] and then convert that to a dictionary. If the result order or count changes, you just need to change the list of keys to match the new result.
PS still fresh on Python myself, so there might be better ways off doing this.
An old question, but since no one mentioned it I'll add this from the Python Cookbook:
Recipe 81252: Using dtuple for Flexible Query Result Access
This recipe is specifically designed for dealing with database results, and the dtuple solution allows you to access the results by name OR index number. This avoids having to access everything by subscript which is very difficult to maintain, as noted in your question.
With a tuple it will always be a hassle to add or change fields. You're right that a dictionary will be much better.
If you want something with slightly friendlier syntax you might want to take a look at the answers this question about a simple 'struct-like' object. That way you can pass around an object, say job, and access its fields even more easily than a tuple or dict:
job.jobId, job.username = jobId, username
If you're using the MySQLdb package, you can set up your cursor objects to return dicts instead of tuples.
import MySQLdb, MySQLdb.cursors
conn = MySQLdb.connect(..., cursorclass=MySQLdb.cursors.DictCursor)
cur = conn.cursor() # a DictCursor
cur2 = conn.cursor(cursorclass=MySQLdb.cursors.Cursor) # a "normal" tuple cursor
How about this:
class TypedTuple:
def __init__(self, fieldlist, items):
self.fieldlist = fieldlist
self.items = items
def __getattr__(self, field):
return self.items[self.fieldlist.index(field)]
You could then do:
j = TypedTuple(["jobid", "label", "username"], job)
print j.jobid
It should be easy to swap self.fieldlist.index(field) with a dictionary lookup later on... just edit your __init__ method! Something like Staale does.