I'm inserting/updating objects into a MySQL database using the peewee ORM for Python. I have a model like this:
class Person(Model):
person_id = CharField(primary_key=True)
name = CharField()
I create the objects/rows with a loop, and each time through the loop have a dictionary like:
pd = {"name":"Alice","person_id":"A123456"}
Then I try creating an object and saving it.
po = Person()
for key,value in pd.items():
setattr(po,key,value)
po.save()
This takes a while to execute, and runs without errors, but it doesn't save anything to the database -- no records are created.
This works:
Person.create(**pd)
But also throws an error (and terminates the script) when the primary key already exists. From reading the manual, I thought save() was the function I needed -- that peewee would perform the update or insert as required.
Not sure what I need to do here -- try getting each record first? Catch errors and try updating a record if it can't be created? I'm new to peewee, and would normally just write INSERT ... ON DUPLICATE KEY UPDATE or even REPLACE.
Person.save(force_insert=True)
It's documented: http://docs.peewee-orm.com/en/latest/peewee/models.html#non-integer-primary-keys-composite-keys-and-other-tricks
I've had a chance to re-test my answer, and I think it should be replaced. Here's the pattern I can now recommend; first, use get_or_create() on the model, which will create the database row if it doesn't exist. Then, if it is not created (object is retrieved from db instead), set all the attributes from the data dictionary and save the object.
po, created = Person.get_or_create(person_id=pd["person_id"],defaults=pd)
if created is False:
for key in pd:
setattr(fa,key,pd[key])
po.save()
As before, I should mention that these are two distinct transactions, so this should not be used with multi-user databases requiring a true upsert in one transaction.
I think you might try get_or_create()? http://peewee.readthedocs.org/en/latest/peewee/querying.html#get-or-create
You may do something like:
po = Person()
for key,value in pd.items():
setattr(po,key,value)
updated = po.save()
if not updated:
po.save(force_insert=True)
Related
I'm using Pony ORM version 0.7 with a Sqlite3 database on disk, and running into this issue: I am performing a select, then an update, then a select, then another update, and getting an error message of
pony.orm.core.UnrepeatableReadError: Value of Task.order_id for
Task[23654] was updated outside of current transaction (was: 1, now: 2)
I've reduced the problem to the minimum set of commands that causes the problem (i.e. removing anything causes the problem not to occur):
#db_session
def test_method():
tasks = list(map(Task.to_dict, Task.select()))
db.execute("UPDATE Task SET order_id=order_id*2")
task_to_move = select(task for task in Task if task.order_id == 2).first()
task_to_move.order_id = 1
test_method()
For completeness's sake, here is the definition of Task:
class Task(db.Entity):
text = Required(unicode)
heading = Required(int)
create_timestamp = Required(datetime)
done_timestamp = Optional(datetime)
order_id = Required(int)
Also, if I remove the constraint that task.order_id == 2 from my select, the problem no longer occurs, so I assume the problem has something to do with querying based on a field that has been changed since the transaction has started, but I don't know why the error message is telling me that it was changed by a different transaction (unless maybe db.execute is executing in a separate transaction because it is raw SQL?)
I've already looked at this similar question, but the problem was different (Pony ORM reports record "was updated outside of current transaction" while there is not other transaction) and at this documentation (https://docs.ponyorm.com/transactions.html) but neither solved my problem.
Any ideas what might be going on here?
Pony uses optimistic concurrency control by default. For each attribute Pony remembers its current value (potentially modified by application code) as well as original value which was read from the database. During UPDATE Pony checks that the value of column in the database is still the same. If the value is changed, Pony assumes that some concurrent transaction did it, and throw exception in order to avoid the "lost update" situation.
If you execute some raw SQL query, Pony does not know what exactly was modified in the database. So when Pony encounters that the counter value was changed, it mistakenly thinks that the value was changed by another transaction.
In order to avoid the problem you can mark order_id attribute as volatile. Then Pony will assume, that the value of attribute can change at any time (by trigger or raw SQL update), and will exclude that attribute from optimistic checks:
class Task(db.Entity):
text = Required(unicode)
heading = Required(int)
create_timestamp = Required(datetime)
done_timestamp = Optional(datetime)
order_id = Required(int, volatile=True)
Note that Pony will cache the value of volatile attribute and will not re-read the value from the database until the object was saved, so in some situation you can get obsolete value in Python.
Update:
Starting from release 0.7.4 you can also specify optimistic=False option to db_session to turn off optimistic checks for specific transaction that uses raw SQL queries:
with db_session(optimistic=False):
...
or
#db_session(optimistic=False)
def some_function():
...
Also it is possible now to specify optimistic=False option for attribute instead of specifying volatile=True. Then Pony will not make optimistic checks for that attribute, but will still consider treat it as non-volatile
I'm using Django 1.8, Mezzanine, Cartridge, and I use Postgresql as the database.
I've updated the num_in_stock directly from the database. The quantities are all correct in the database but not on my website. I know the solution is here, but I don't know what to do with that. I really need it spelled out for me.
How exactly would you use this in Cartridge to refresh the num_in_stock?
This should be all you need to do to update one object. Replace object_name with your object.
object_name.refresh_from_db()
I assume you're using an F expression.
According to the documentation an F expression:
...makes it possible to refer to model field values and perform
database operations using them without actually having to pull them
out of the database into Python memory.
You're working directly in the database. Python knows nothing about the values of the model fields. There's nothing on memory, everything is happening on the database.
The documentation's example:
from django.db.models import F
reporter = Reporters.objects.get(name='Tintin')
reporter.stories_filed = F('stories_filed') + 1
reporter.save()
Although reporter.stories_filed = F('stories_filed') + 1 looks like a
normal Python assignment of value to an instance attribute, in fact
it’s an SQL construct describing an operation on the database.
So, for Python to know about this value you need to reload the object.
To access the new value saved this way, the object must be reloaded:
reporter = Reporters.objects.get(pk=reporter.pk)
# Or, more succinctly:
reporter.refresh_from_db()
In your example:
object_name.refresh_from_db()
And one more thing...
F() assignments persist after Model.save()
F() objects assigned to
model fields persist after saving the model instance and will be
applied on each save().
reporter = Reporters.objects.get(name='Tintin')
reporter.stories_filed = F('stories_filed') + 1
reporter.save()
reporter.name = 'Tintin Jr.'
reporter.save()
stories_filed will be updated twice in this case. If it’s initially
1, the final value will be 3. This persistence can be avoided by
reloading the model object after saving it, for example, by using
refresh_from_db().
I assume the num_in_stock is an attribute of your model class. If true you should get an instance of the class (i.e object_name) then
object_name.refresh_from_db()
After which, you can access it like object_name.num_in_stock
Reading code modules of Tryton, I met a lot this method but I did not figure out what this is for.
What's the use of this function in Tryton?
#classmetod
def __register__(cls,module_name):
TableHandler = backend.get('TableHandler')
cursor = Transaction().cursor
table = TableHandler(cursor,cls,module_name)
super(Adress,cls).__register__(module_name)
table.not_null_action('sequence', action='remove')
The __register__ method is called every time the model is updated, and it's used to alter the database structure of the current module. Normally tryton, creates all the missing fields for you (this is done on ModelSQL class), but some actions are not possible to be detected automatically, so you must write a migration for it. This is done on the __register__ method of the model.
The code you copied ensures that the sequence field is nullable and if not, it alters the column from null to not null.
I have a problem about insert function. If i have a array of objects to insert[bad,good,good]. if the first object is bad, and object insert action will fail, then the rest of the objects will never hit the database even the object is good.
How can i deal with it ?
You can validate the model instances before saving to ensure they are valid eg:
valid_docs = [d for d in docs if d.validate()]
Or pass in the continue_on_error=True as a write_options eg:
Doc.objects.insert(docs, write_options={"continue_on_error": True})
Working a django project and trying to speed up the calls. I noticed that Django automatically does a second query to evaulate any foreign key relationships. For instance if my models look like:
Model Person:
name = model.CharField("blah")
Model Address:
person = model.ForeignKey(Person)
Then I make:
p1 = Person("Bob")
address1 = Address(p1)
print (p1.id) #let it be 1 cause it is the first entry
then when I call:
address1.objects.filter(person_id = "1")
I get:
Query #1: SELECT address.id, address.person_id FROM address
Query #2: SELECT person.id, person.name FROM person
I want to get rid of the 2nd call, query #2. I have tried using "defer" from django documentation, but that did not work (in fact it makes even more calls). "values" is a possibility but in actual practice, there are many more fields I want to pull. The only thing I want it to do is not evaluate the FOREIGN KEY. I would be happy to get the person_id back, or not. This drastically reduces the runtime especially when I do a command like: Address.objects.all(), because it Django evaluates every foreign key.
Having just seen your other question on the same issue, I'm going to guess that you have defined a __unicode__ method that references the ForeignKey field. If you query for some objects in the shell and output them, the __unicode__ method will be called, which requires a query to get the ForeignKey. The solution is to either rewrite that method so it doesn't need that reference, or - as I say in the other question - use select_related().
Next time, please provide full code, including some that actually demonstrates the problem you are having.