I've been using peewee with SQLite for some time and now I'm switching to SQLAlchemy with Postgres and I can't find equivalent of DoesNotExist (see example)
try:
return models.User.get(models.User.id == userid)
except models.DoesNotExist:
return None
Do you know how to achieve the same with SQLAlchemy? I've checked stuff which I can import from sqlalchemy.ext but nothing seemed right.
The closest could be this one: - http://docs.sqlalchemy.org/en/latest/orm/exceptions.html#sqlalchemy.orm.exc.NoResultFound
Code Sample:
from sqlalchemy.orm.exc import NoResultFound
try:
user = session.query(User).one()
except NoResultFound, e:
print "No users found"
Peewee does work with Postgresql, you know. ;)
Related
I've got a Python 2.7 script I'm working on that retrieves rows from a MySQL table, loops through the data to process it, then is supposed to do the following things in this order:
UPDATE the table rows we just got previously to set a locked value
in each row to TRUE
After the UPDATE query executes and commits via MySQLdb, a ThreadPool of processes should run on the data from the original loop.
What's actually happening is that the UPDATE query seems to be committing somehow after the ThreadPool is done. I tried refactoring it into a try/finally statement to make sure, but now it either still does it afterwards, or just doesn't commit the UPDATE and runs the ThreadPool anyways.
It's a head scratcher, to be sure. I assume I'm just doing something really wrong and obvious but not catching it after looking at this for so long. Any input is greatly appreciated!
Here's the gist:
from multiprocessing.pool import ThreadPool, IMapIterator
import MySQLdb as mdb
import os, sys, time
import re
from boto.s3.connection import S3Connection
from boto.s3.bucket import Bucket
...
con = mdb.connect('localhost', 'user', 'pass', 'db')
with con:
cur = con.cursor()
cur.execute("SELECT preview_queue.filename, preview_queue.product_id, preview_queue.track, products.name, preview_queue.id FROM preview_queue join `catalog_module-products` AS products on products.id = preview_queue.product_id where locked != 1")
rows = cur.fetchall()
mp3s_to_download = []
lock_ids = []
last_directory = ""
if len(rows) > 0:
for row in rows:
base_dir = str(get_base_dir(row[1], row[3]))
mp3s_to_download.append([base_dir, str(row[0])])
if last_directory != "preview_temp/"+base_dir:
if not os.path.exists("preview_temp/"+base_dir):
try:
os.makedirs("preview_temp/"+base_dir)
except OSError, e:
pass
last_directory = "preview_temp/"+base_dir
lock_ids.append(str(row[4]))
if len(lock_ids) > 0:
action_ids = ','.join(lock_ids)
try:
cur.execute("UPDATE preview_queue SET locked = 1 WHERE id IN ({})".format(action_ids))
con.commit()
finally:
pool = ThreadPool(processes=20)
pool.map(download_file, mp3s_to_download)
cur.close()
A finally clause is guaranteed to execute, even if the try clause raises an exception. What is probably happening here is that an exception is being raised, preventing the update from being committed, but the threads are triggered anyway.
This doesn't really seen to be an appropriate use of try/finally. Rather, use a normal try/except to catch and log any exceptions, and then perhaps use an else clause to start the threads only if no exception was raised.
I wrote this code mentioned below to get a basic select statement output from mysql database using python.
The code is :
import warnings
warnings.filterwarnings(action="ignore", message='the sets module is deprecated')
import MySQLdb
import sys
try:
db =MySQLdb.connect(host='********',user='Admin',passwd='******',db='data')
except Exception as e:
sys.exit('Not working')
cursor = db.cursor()
cursor.execute('select * from user')
results = cursor.fetchall()
print results
But the output that is get is :
((1L,), (2L,))
Please assist
Thanks in advance :)
I have the following in my django model, which I am using with PostgresSql
class Business(models.Model):
location = models.CharField(max_length=200,default="")
name = models.CharField(max_length=200,default="",unique=True)
In my view I have:
for b in bs:
try:
p = Business(**b)
p.save()
except IntegrityError:
pass
When the app is run and an IntegrityError is triggered I would like to grab the already inserted record and also the object (I assume 'p') that triggered the error and update the location field.
In pseudocode:
for b in bs:
try:
p = Business(**b)
p.save()
except IntegrityError:
EXISTING_RECORD.location = EXISTING_RECORD.location + p.location
EXISTING_RECORD.save()
How is this done in django?
This is the way I got the existing record that you are asking for.
In this case, I had MyModel with
unique_together = (("owner", "hsh"),)
I used regex to get the owner and hsh of the existing record that was causing the issue.
import re
from django.db import IntegrityError
try:
// do something that might raise Integrity error
except IntegrityError as e:
#example error message (e.message): 'duplicate key value violates unique constraint "thingi_userfile_owner_id_7031f4ac5e4595e3_uniq"\nDETAIL: Key (owner_id, hsh)=(66819, 4252d2eba0e567e471cb08a8da4611e2) already exists.\n'
import re
match = re.search( r'Key \(owner_id, hsh\)=\((?P<owner_id>\d+), (?P<hsh>\w+)\) already', e.message)
existing_record = MyModel.objects.get(owner_id=match.group('owner_id'), hsh=match.group('hsh'))
I tried get_or_create, but that doesn't quite work the way you want (if you do get_or_create with both the name and the location, you still get an integrity error; if you do what Joran suggested, unless you overload update, it will overwrite location as opposed to append.
This should work the way you want:
for b in bs:
bobj, new_flag = Business.objects.get_or_create(name=b['name'])
if new_flag:
bobj.location = b['location']
else:
bobj.location += b['location'] # or possibly something like += ',' + b['location'] if you wanted to separate them
bobj.save()
It would be nice (and may be possible but I haven't tried), in the case where you can have multiple unique constraints, to be able to inspect the IntegrityException (similar to the accepted answer in IntegrityError: distinguish between unique constraint and not null violations, which also has the downside of appearing to be postgres only) to determine which field(s) violated. Note that if you wanted to follow your original framework, you can do collidedObject = Business.objects.get(name=b['name']) in your exception but that only works in the case where you know for sure that it was a name collision.
for b in bs:
p = Business.objects.get_or_create(name=b['name'])
p.update(**b)
p.save()
I think anyway
I have this code:
try:
principal = cls.objects.create(
user_id=user.id,
email=user.email,
path='something'
)
except IntegrityError:
principal = cls.objects.get(
user_id=user.id,
email=user.email
)
It tries to create a user with the given id and email, and if there already exists one - tries to get the existing record.
I know this is a bad construction and it will be refactored anyway. But my question is this:
How do i determine what kind of IntegrityError has happened: the one related to unique constraint violation (there is unique key on (user_id, email)) or the one related to not null constraint (path cannot be null)?
psycopg2 provides the SQLSTATE with the exception as the pgcode member, which gives you quite fine-grained error information to match on.
python3
>>> import psycopg2
>>> conn = psycopg2.connect("dbname=regress")
>>> curs = conn.cursor()
>>> try:
... curs.execute("INVALID;")
... except Exception as ex:
... xx = ex
>>> xx.pgcode
'42601'
See Appendix A: Error Codes in the PostgreSQL manual for code meanings. Note that you can match coarsely on the first two chars for broad categories. In this case I can see that SQLSTATE 42601 is syntax_error in the Syntax Error or Access Rule Violation category.
The codes you want are:
23505 unique_violation
23502 not_null_violation
so you could write:
try:
principal = cls.objects.create(
user_id=user.id,
email=user.email,
path='something'
)
except IntegrityError as ex:
if ex.pgcode == '23505':
principal = cls.objects.get(
user_id=user.id,
email=user.email
)
else:
raise
That said, this is a bad way to do an upsert or merge. #pr0gg3d is presumably right in suggesting the right way to do it with Django; I don't do Django so I can't comment on that bit. For general info on upsert/merge see depesz's article on the topic.
Update as of 9-6-2017:
A pretty elegant way to do this is to try/except IntegrityError as exc, and then use some useful attributes on exc.__cause__ and exc.__cause__.diag (a diagnostic class that gives you some other super relevant information on the error at hand - you can explore it yourself with dir(exc.__cause__.diag)).
The first one you can use was described above. To make your code more future proof you can reference the psycopg2 codes directly, and you can even check the constraint that was violated using the diagnostic class I mentioned above:
except IntegrityError as exc:
from psycopg2 import errorcodes as pg_errorcodes
assert exc.__cause__.pgcode == pg_errorcodes.UNIQUE_VIOLATION
assert exc.__cause__.diag.constraint_name == 'tablename_colA_colB_unique_constraint'
edit for clarification: I have to use the __cause__ accessor because I'm using Django, so to get to the psycopg2 IntegrityError class I have to call exc.__cause__
It could be better to use:
try:
obj, created = cls.objects.get_or_create(user_id=user.id, email=user.email)
except IntegrityError:
....
as in https://docs.djangoproject.com/en/dev/ref/models/querysets/#get-or-create
The IntegrityError should be raised only in the case there's a NOT NULL constraint violation.
Furthermore you can use created flag to know if the object already existed.
How can you fix the SQL-statement in Python?
The db connection works. However, cur.execute returns none which is false.
My code
import os, pg, sys, re, psycopg2
try:
conn = psycopg2.connect("dbname='tk' host='localhost' port='5432' user='naa' password='123'")
except: print "unable to connect to db"
cur = conn.cursor()
print cur.execute("SELECT * FROM courses") # problem here
The SQL-command in Psql returns me the correct output.
I can similarly run INSERT in Psql, but not by Python's scripts.
I get no warning/error to /var/log.
Possible bugs are
cursor(), seems to be right however
the syntax of the method connect(), seems to be ok however
You have to call one of the fetch methods on cur (fetchone, fetchmany, fetchall) to actually get the results of the query.
You should probably have a read through the a tutorial for DB-API.
You have to call cur.fetchall() method (or one of other fetch*() methods) to get results from query.