We Keep Coding

Unable to insert extracted objects in SQLAlchemy ORM

I'm trying to insert/update the list of objects extracted using SQLAlchemy ORM.
def truncate_mytable(self):
with self.session.begin():
current_records = self.session.query(MyTable).all()
self.session.query(MyTable).delete()
self.session.expunge_all()
return current_records
def compensate_truncate_mytable(self, objects):
with self.session.begin():
self.session.bulk_save_objects(objects)
But while the objects have been extracted correctly, they are not getting written to the DB.
Could it be because there are also some protected attributes inside the objects, such as <sqlalchemy.orm.state.InstanceState object at 0x11471bf70> and <ClassManager of <class 'lib.kaizen_models.models.MyTable'> at 1146673b0>? The objects' type in the list is <lib.kaizen_models.models.MyTable object at 0x11471bd00>.
(I'm writing compensation methods, following Saga pattern.)

The problem is that the objects are in a detached state, and this means bulk_save_objects will try to update rather than insert them*.
The state can be reset to transient by calling orm.make_transient on each object, after which they can be saved by bulk_save_objects.
def truncate_mytable(self):
with self.session.begin():
current_records = self.session.query(MyTable).all()
self.session.query(MyTable).delete()
self.session.expunge_all()
for record in current records:
orm.make_transient(record)
return current_records
Alternatively, you could merge the objects back into the session before calling bulk_save_objects, but this might reduce the performance benefits that you want to obtain from the bulk operation.
* By default bulk_create_objects' update_changed_only argument is True, and since there a no changes in the objects' attribute histories no updates are attempted. Setting it to False will emit UPDATE statements, but result in a StaleDataError because the UPDATE matches no rows in the empty table.

Django / Factoryboy - unable to delete instances in test

I am writings tests for a django application and I've run into a problem were deleted objects still exist in the test database after I've supposedly deleted them.
I'm using the following factory
class CMSPageFactory(factory.DjangoModelFactory):
class Meta:
model = CMSPage
title = factory.Faker('company')
tenant = factory.SubFactory(TenantFactory)
key = factory.Faker('slug')
protected = False
in_navigation = False
active = True
This is the test I am running
def test_example_for_so(self):
page = CMSPageFactory()
page.delete()
self.assertFalse(page)
And it raises the following error:
AssertionError: <CMSPage: Fletcher LLC> is not false
I must be missing something very obvious but for the life of me I cannot figure out what. Does anyone know what I am doing wrong?

Are you sure, page still exists in the db?
Calling delete() on a django model instance (which your factory supposedly creates) will delete the database row, but nor your local python representation:
https://docs.djangoproject.com/en/2.1/ref/models/instances/#django.db.models.Model.delete
Issues an SQL DELETE for the object. This only deletes the object in the database; the Python instance will still exist and will still have data in its fields.

The object was deleted from the database but still exists in memory. From the Model delete docs:
Issues an SQL DELETE for the object. This only deletes the object in
the database; the Python instance will still exist and will still have
data in its fields. This method returns the number of objects deleted
and a dictionary with the number of deletions per object type
What you can do in the test is get the id and then try to get the object from the database, or count the objects in the database and expect 0.

Django how to query objects one by one

I have to query an object from a model, this object is called "exercise" and has many fields(title, body, answers, etc.) I need to get "exercise objects" one by one with all their fields, do some rendering on them and give back a pdf of the exercise as result.
The problem is, if I do:
exercises = ExerciseModel.objects.all()
I get all of them. If I do:
some_exercises = ExerciseModel.objects.filter(something=something)
I get "some of them" depending on filter. And if i do:
exercise = ExerciseModel.objects.get()
I get
error get() returned more than one exercise
How can I get them one by one? I must be able to use them as input for another function.

If you need to perform the task on only 1 exercise, use get() instead. In such case, you need a criteria to make sure get() will return 1 and only 1 result.
Example:
ex = ExerciseModel.objects.get(pk=123)
From the doc:
Returns the object matching the given lookup parameters, which should be in the format described in Field lookups.
get() raises MultipleObjectsReturned if more than one object was found. The MultipleObjectsReturned exception is an attribute of the model class.
get() raises a DoesNotExist exception if an object wasn’t found for the given parameters. This exception is an attribute of the model class.
When you have a QuerySet with filters that ensure the underlying SQL request will return only 1 row, you can call get() without argument on the QuerySet. This return the row as model instance, instead of returning it as a list containing 1 elemnt.
Original answer:
Both filter() and all() methods return a QuerySet instance. You can iterate on it to perform a task for each "exercise" returned by your request
for exercise in ExerciseModel.objects.filter(something=something):
# ... do what you need on your exercise
From the documentation:
A QuerySet is iterable, and it executes its database query the first time you iterate over it. For example, this will print the headline of all entries in the database

What you need is .iterator() which will turn a queryset to an iterator:
exercises = ExerciseModel.objects.all().iterator()
then you get iterate over it in a for loop:
for exercise in exercises:
...
This will improve performance when you have large number of items. However, it has some downsides as well as stated in the docs
Hope it helps!

.get() must return a single object, not a QuerySet instance. If you want to get a single exercise then you must pass a parameter to the .get instance. For example, retrieving an object by name would require the following code:
exercise = ExerciseModel.objects.get(name="an_exercise")
If you want to iterate through all the objects without actually retrieving a QuerySet containing the objects, you could use the following code:
for i in range(Exercise.objects.all().count()):
exercise = Exercise.objects.get(pk=i)
... operate on object ...

What is the correct way to use refresh_from_db in Django?

I'm using Django 1.8, Mezzanine, Cartridge, and I use Postgresql as the database.
I've updated the num_in_stock directly from the database. The quantities are all correct in the database but not on my website. I know the solution is here, but I don't know what to do with that. I really need it spelled out for me.
How exactly would you use this in Cartridge to refresh the num_in_stock?

This should be all you need to do to update one object. Replace object_name with your object.
object_name.refresh_from_db()

I assume you're using an F expression.
According to the documentation an F expression:
...makes it possible to refer to model field values and perform
database operations using them without actually having to pull them
out of the database into Python memory.
You're working directly in the database. Python knows nothing about the values of the model fields. There's nothing on memory, everything is happening on the database.
The documentation's example:
from django.db.models import F
reporter = Reporters.objects.get(name='Tintin')
reporter.stories_filed = F('stories_filed') + 1
reporter.save()
Although reporter.stories_filed = F('stories_filed') + 1 looks like a
normal Python assignment of value to an instance attribute, in fact
it’s an SQL construct describing an operation on the database.
So, for Python to know about this value you need to reload the object.
To access the new value saved this way, the object must be reloaded:
reporter = Reporters.objects.get(pk=reporter.pk)
# Or, more succinctly:
reporter.refresh_from_db()
In your example:
object_name.refresh_from_db()
And one more thing...
F() assignments persist after Model.save()
F() objects assigned to
model fields persist after saving the model instance and will be
applied on each save().
reporter = Reporters.objects.get(name='Tintin')
reporter.stories_filed = F('stories_filed') + 1
reporter.save()
reporter.name = 'Tintin Jr.'
reporter.save()
stories_filed will be updated twice in this case. If it’s initially
1, the final value will be 3. This persistence can be avoided by
reloading the model object after saving it, for example, by using
refresh_from_db().

I assume the num_in_stock is an attribute of your model class. If true you should get an instance of the class (i.e object_name) then
object_name.refresh_from_db()
After which, you can access it like object_name.num_in_stock

Python peewee save() doesn't work as expected

I'm inserting/updating objects into a MySQL database using the peewee ORM for Python. I have a model like this:
class Person(Model):
person_id = CharField(primary_key=True)
name = CharField()
I create the objects/rows with a loop, and each time through the loop have a dictionary like:
pd = {"name":"Alice","person_id":"A123456"}
Then I try creating an object and saving it.
po = Person()
for key,value in pd.items():
setattr(po,key,value)
po.save()
This takes a while to execute, and runs without errors, but it doesn't save anything to the database -- no records are created.
This works:
Person.create(**pd)
But also throws an error (and terminates the script) when the primary key already exists. From reading the manual, I thought save() was the function I needed -- that peewee would perform the update or insert as required.
Not sure what I need to do here -- try getting each record first? Catch errors and try updating a record if it can't be created? I'm new to peewee, and would normally just write INSERT ... ON DUPLICATE KEY UPDATE or even REPLACE.

Person.save(force_insert=True)
It's documented: http://docs.peewee-orm.com/en/latest/peewee/models.html#non-integer-primary-keys-composite-keys-and-other-tricks

I've had a chance to re-test my answer, and I think it should be replaced. Here's the pattern I can now recommend; first, use get_or_create() on the model, which will create the database row if it doesn't exist. Then, if it is not created (object is retrieved from db instead), set all the attributes from the data dictionary and save the object.
po, created = Person.get_or_create(person_id=pd["person_id"],defaults=pd)
if created is False:
for key in pd:
setattr(fa,key,pd[key])
po.save()
As before, I should mention that these are two distinct transactions, so this should not be used with multi-user databases requiring a true upsert in one transaction.

I think you might try get_or_create()? http://peewee.readthedocs.org/en/latest/peewee/querying.html#get-or-create

You may do something like:
po = Person()
for key,value in pd.items():
setattr(po,key,value)
updated = po.save()
if not updated:
po.save(force_insert=True)