Duplicate entries in High Replication Datastore - python

We still have a rare case of duplicate entries when this POST method is called.
I had asked for advice previously on Stack overflow and was given a solution, that is utilising the parent/child methodology to retain strongly consistent queries.
I have migrated all data into that form and let it run for another 3 months.
However the problem was never solved.
The problem is right here with this conditional if recordsdb.count() == 1:
It should be true in order to update the entry, but instead HRD might not always find the latest entry and creates a new entry instead.
As you can see, we are writing/reading from the Record via Parent/Child methodology as recommended:
new_record = FeelTrackerRecord(parent=user.key,...)
And yet still upon retrieval, the HRD still doesn't always fetch the latest entry:
recordsdb = FeelTrackerRecord.query(ancestor = user.key).filter(FeelTrackerRecord.record_date == ... )
So we are quite stuck on this and don't know how to solve it.
#requires_auth
def post(self, ios_sync_timestamp):
user = User.query(User.email == request.authorization.username).fetch(1)[0]
if user:
json_records = request.json['records']
for json_record in json_records:
recordsdb = FeelTrackerRecord.query(ancestor = user.key).filter(FeelTrackerRecord.record_date == date_parser.parse(json_record['record_date']))
if recordsdb.count() == 1:
rec = recordsdb.fetch(1)[0]
if 'timestamp' in json_record:
if rec.timestamp < json_record['timestamp']:
rec.rating = json_record['rating']
rec.notes = json_record['notes']
rec.timestamp = json_record['timestamp']
rec.is_deleted = json_record['is_deleted']
rec.put()
elif recordsdb.count() == 0:
new_record = FeelTrackerRecord(parent=user.key,
user=user.key,
record_date = date_parser.parse(json_record['record_date']),
rating = json_record['rating'],
notes = json_record['notes'],
timestamp = json_record['timestamp'])
new_record.put()
else:
raise Exception('Got more than two records for the same record date - among REST post')
user.last_sync_timestamp = create_timestamp(datetime.datetime.today())
user.put()
return '', 201
else:
return '', 401
Possible Solution:
The very last idea I have to solve this would be, stepping away from Parent/Child strategy and using the user.key PLUS date-string as part of the key.
Saving:
new_record = FeelTrackerRecord(id=str(user.key) + json_record['record_date'], ...)
new_record.put()
Loading:
key = ndb.Key(FeelTrackerRecord, str(user.key) + json_record['record_date'])
record = key.get();
Now I could check if record is None, I shall create a new entry, otherwise I shall update it. And hopefully HRD has no reason not finding the record anymore.
What do you think, is this a guaranteed solution?

The Possible Solution appears to have the same problem as the original code. Imagine the race condition if two servers execute the same instructions practically simultaneously. With Google's overprovisioning, that is sure to happen once in a while.
A more robust solution should use Transactions and a rollback for when concurrency causes a consistency violation. The User entity should be the parent of its own Entity Group. Increment a records counter field in the User entity within a transaction. Create the new FeelTrackerRecord only if the Transaction completes successfully. Therefore the FeelTrackerRecord entities must have a User as parent.
Edit: In the case of your code the following lines would go before user = User.query(... :
Transaction txn = datastore.beginTransaction();
try {
and the following lines would go after user.put() :
txn.commit();
} finally {
if (txn.isActive()) {
txn.rollback();
}
}
That may overlook some flow control nesting detail, it is the concept that this answer is trying to describe.
With an active transaction, if multiple processes (for example on multiple servers executing the same POST concurrently because of overprovisioning) the first process will succeed with its put and commit, while the second process will throw the documented ConcurrentModificationException.
Edit 2: The transaction that increments the counter (and may throw an exception) must also create the new record. That way if the exception is thrown, the new record is not created.

Related

Am I correct in thinking I'll get read-after-write consistency with this approach?

I am writing a system to do lock-free reservation, using a Mongo database.
The system needs to ensure that for a given concert with N available tickets, no more than N can be reserved. I want to do this without locking.
The approach I'm taking is to create a reservation document, then read back and see if it was within the first N for the concert, and if not then delete it.
I'm not a Mongo expert, and I want to make sure that I've understood correctly that these read/write concerns will ensure that when I read back I will always have the reservation I just created and all reservations that have been created in other threads up to that point - this should be sufficient to ensure the system will work correctly.
def make_reservation(concert: Document, user: Document):
maximum_spaces = concert.maximum_spaces
# Will wait for acknowlegement before continuing
reservation_id = reservations.with_options(
write_concern=WriteConcern(fsync=True)
).insert_one({
"user": user.id,
"concert": concert.id,
"status": "UNRESERVED",
}).inserted_id
# Sorted list of ids ascending
all_reservations = sorted(reservations
.with_options(read_preference=ReadPreference.PRIMARY)
.find({"concert": concert.id}).only("id")
)
# If the id we just inserted is too far down the list we missed our spot
if all_reservations.index(reservation_id) >= maximum_spaces:
# Oh no, we missed out
reservations.delete_one({"id": reservation_id})
raise Exception("Sorry, too slow")
reservations.update_one({"id": reservation_id}, {"$set": {"status": "RESERVED"}})
return reservation_id
I've tried testing this a few ways and I can't coerce it to break, but I'm still not sure if it's theoretically watertight.

How to use self.env.commit() properly?

Could someone please explain to me how self.env.cr.commit() works, when to use it and some good practices?
From Odoo documentation seems the use of cr.commit is very dangerous. This is my first time using it and I am not sure how to use it properly for my use case.
Edit:
More information for my use case: I am creating shipments through shipping provider API. Let's say my API call is successful and I have created shipment but during handling of the response, I have to raise UserError for some reason and my changes are rollbacked. So now the state of the shipment is different in Odoo and on the shipping provider server which is unacceptable.
So if I am calling the method create_dhl_shipment() and the flag variable is True (an error occurred during last API call) then I would like to delete the original shipment and create a new one.
And my problem is: How do I make a change in the database and keep it from rollbacking.
During search on the internet, I came across cr.commit() but Odoo in documentation really discourages using it.
very simplified example:
Class StockPickingInherited(models.Model)
_inherit = 'stock.picking'
remnant_shipment = fields.Boolean("Possible remnant shipment")
packages = fields.One2many("stock.shipment.package", "picking_id")
def create_dhl_shipment(self):
response_from_shipping_provider = requests.get("API URL")
if response_from_shipping_provider != 200:
if not remnant_shipment:
raise UserError("Shipment creation failed")
else:
self.write({"packages" : [(5, 0, 0)]})
self.env.cr.commit() # write data into the db and keep the change from rollbacking due to raising UserError
raise UserError("Shipment creation failed")
Am I doing it right? Are there some potential dangers?
It is true that self.env.cr.commit() should be used sparingly. However, there are some legitimate use cases for it. For example:
#api.model
def some_cron_job(self):
for record in self.env[...].search(...):
record.do_some_process()
self.env.cr.commit()
The above is fine because you are doing some process on a batch of records and to avoid that the cron job does it over and over because of an error on one of the records, you can commit after each record has been processed. (PS: the above could be made safer with try...except... or marking a record as "failed" for example. This can be done in conjunction with commit())
In your use case you want to display a message to the user, but your problem is that once you raise the transaction will rollback, so to counter this you call a commit().
I have had similar situations and I sometimes find that using an #api.onchange works well for showing a message to the user.
#api.onchange('your_field')
def onchange_your_field(self):
if self.your_field > 100:
raise UserError("Thanks for entering the field")
However, I am no longer a fan of that approach since Odoo has gotten rid of most #api.onchange in favour of computed fields. (There are good reasons for the move, so try and avoid it too.)
Luckily there is a new way to do this, introduced in Odoo Version 13.0.
def some_method(self):
# Do something
self.write({"something" : False})
# Display message
return {
'type': 'ir.actions.client',
'tag': 'display_notification',
'params': {
'title': title,
'message': message,
'sticky': False,
}
}
The above is taken from here in the Odoo repo where a message is displayed to the user if the mail server credentials are correct.
The following rule is taken from the Odoo guidelines (never commit the transaction):
You should NEVER call cr.commit() yourself, UNLESS you have created your own database cursor explicitly! And the situations where you need to do that are exceptional!And by the way if you did create your own cursor, then you need to handle error cases and proper rollback, as well as properly close the cursor when you’re done with it.
In the following example, we create a new cursor to avoid rollback that could be caused by an upper method:
registry = odoo.registry(self.env.cr.dbname)
with registry.cursor() as cr:
env = api.Environment(cr, SUPERUSER_ID, {})
For more details check the well-documented Odoo guidelines
You can find an example in auth_ldap module

Inconsistent behavior of QSqlTableModel with OnRowSubmit

Premise: this question possibly refers to two distinct problems, but I believe they might be linked. If, after comments and further research we will find out that they are actually unrelated, I will open a separate question.
I'm experiencing some unexpected and odd behavior with some aspects of QSqlTableModel, and with subclassing in at least one case. I'm not an expert on Sql, but one of the problems doesn't seem what the expected behavior should be.
I can confirm this only for SQLite as I don't use other database systems.
I can also reproduce these problems with both [Py]Qt 5.15.2 and 6.2.2.
1. New row is "removed" after ignoring editor changes
With the default OnRowChange edit strategy, if a row is added, some data is inserted in a field, and editing of another field on the same row is cancelled using Esc, the whole row is then removed from the view.
The actual database, though, is still updated, and opening the program again shows the row that was previously "hidden", except for the field that has been cancelled.
from PyQt5 import QtWidgets, QtSql
class TestModel(QtSql.QSqlTableModel):
def __init__(self):
super().__init__()
QtSql.QSqlQuery().exec(
'CREATE TABLE IF NOT EXISTS test (name, value, data);')
self.setTable('test')
self.select()
app = QtWidgets.QApplication([])
db = QtSql.QSqlDatabase.addDatabase('QSQLITE')
db.setDatabaseName('test.db')
db.open()
win = QtWidgets.QWidget()
layout = QtWidgets.QVBoxLayout(win)
addButton = QtWidgets.QPushButton('Add row')
layout.addWidget(addButton)
table = QtWidgets.QTableView()
layout.addWidget(table)
model = TestModel()
table.setModel(model)
addButton.clicked.connect(lambda: model.insertRow(model.rowCount()))
app.aboutToQuit.connect(model.submitAll)
win.resize(640, 480)
win.show()
app.exec()
These are the steps to reproduce the problem:
add a row with the button;
edit at least one field, but not all fields;
start editing an empty field;
press Esc;
close and restart the program;
After step 4, you'll see that the added row is removed from the view, which is not completely unexpected: since the strategy is OnRowChange, cancelling reverts all cached changes (including insertRow()); I don't completely agree with the behavior (imagine filling dozens of fields and then hitting Esc by mistake), but that's not the point.
What's unexpected is that the model is actually updated with the new row and all fields that have been submitted before hitting Esc, and restarting the program will show that.
2. Implementing data() reverts to previous data for incomplete records
Editing an index that has empty (NULL) fields for its row brings different results whether data() has been implemented or not in the subclass, even if the override just calls the base implementation.
Add the following to the TestModel class above:
def data(self, index, role=QtCore.Qt.DisplayRole):
return super().data(index, role)
And a submit button before app.exec():
submitButton = QtWidgets.QPushButton('Submit')
layout.addWidget(submitButton)
submitButton.clicked.connect(model.submitAll)
To reproduce the problem follow these steps:
open a database with at least one row with an empty field at the bottom, similarly to what done above (note: with "empty field" I mean an item that has never been edited);
edit any field in that row and press Enter;
With the OnRowChange or OnFieldChange strategy, the result is that the whole row is made invalid: the vertical header shows "!" (a hint for an invalid record) and all fields are cleared, including those that have previous value from the database.
When the edit strategy is set to OnManualSubmit, calling submitAll() will revert to the original values of the database, just like as changes have been reverted.
The behavior is slightly different if the row with the empty field is not at the bottom; do the first two steps above, then:
press the submit button;
close and restart the program;
In this case, after step 3 the view seem to have accepted the changes, but restarting the program shows that no modification has been applied.
Depending on the edit strategy and the situation, the behavior changes. Usually, if a record with an empty field is followed by at least a record with all fields set, the view and model behave as expected when cancelling editing of that field.
In at least one case it was even impossible to edit an empty field at all (I've to admit, I did many random/speed tests and when I found out that I wasn't able to edit a field I couldn't remember the steps to reproduce it).
What's also strange is that both setData() and submitAll() return True, and there is no explicit lastError(). Despite of that, the shown (and stored) data reverts to the previous database content.
I believe that both issues are potentially caused by a common bug, but, before submitting something to the Qt bug report system I'd like to have some feedback, especially from people being more experienced in SQL and other db drivers, in order to provide a better report (and eventually know if those issues are in fact related or not).
Both issues are caused by bugs in Qt, but they aren't related.
Before explaining these issues, some clarification of the symbols used in the vertical header may be helpful, because they provide some important clues regarding the source of the problems. The symbols are documented thus:
If you insert rows programmatically using
QSqlTableModel::insertRows(), the new rows will be marked with an
asterisk (*) until they are submitted using submitAll() or
automatically when the user moves to another record (assuming the edit
strategy is QSqlTableModel::OnRowChange). Likewise, if you remove rows
using removeRows(), the rows will be marked with an exclamation mark
(!) until the change is submitted.
The first issue is caused by this sequence of events:
After pressing Esc whilst editing a new row (i.e. * is shown in the vertical header), the delegate will emit closeEditor with the RevertModelCache hint. This calls the closeEditor slot of the view, which in turn calls revert() on the table-model - and also, ultimately, the private revertCachedRow function. This function calls beginRemoveRows - but crucially before clearing the cache. Next, rowsAboutToBeRemoved is emitted, which removes the row from the view, causing currentRowChanged to be emitted, which in turn calls the submit() slot of the table-model. Oops! The still uncleared cache data is now inadvertently committed to the database, before endRemoveRows is called after the cache data is finally removed. So, in short, the bug here is that there is no guard to stop submit() being called during the execution of revert().
The second issue is much more subtle. The problem occurs because the SQL table is created without a primary key and the columns do not have an explicit type. This is all perfectly valid, but it exposes a critical bug in a small section of Qt code that builds SQL statements.
This happens in QSqlTableModel::selectRow, which needs to build a where-clause from the QSqlRecord returned by primaryValues. The sqlStatement function of the database driver is used for this, but that needs to know the exact type of the field values in order to quote them correctly. However, the table-model cache does not ensure that a sensible default type is used for columns without an explicit type. This means untyped values will pass through unquoted, allowing arbitrary SQL expressions to be evaluated whilst editing the table. Oops!
It's this that can sometimes make the bug hard to reproduce, because the exact behaviour depends on the precise values that are entered. A value like foo will cause an SQL error, because it's a valid column name that doesn't exist; yet a value like 6 won't raise an error, but will wrongly fail to return any rows, due to a type-mismatch (i.e. INT vs TEXT). If selectRow can't find the relevant row, it may call cache.refresh(), which will clear the values and mark the row for deletion (hence the ! shown in the vertical header). Note also that QSqlQuery is used to execute the problematic statement, so any errors will pass silently and won't be available via the database or driver.
I have provided a re-write below of the original example with some fixes that can be switched on via the command-line (1 to fix the first issue, 2 to fix the second, and 3 to fix both). These are mainly meant for debugging, but could also be adapted as work-arounds if required. The second fix is rather hackish (because primaryValues can't be reimplemented in PyQt) - but it's only needed if you don't have control over the database schema. If the table has a typed primary key and/or all the columns have an explicit type, the second issue won't occur at all. Hopefully the output from the script should make it clear what is going on.
PyQt5:
import sys
from PyQt5 import QtCore, QtWidgets, QtSql
BUGFIX = int(sys.argv[1]) if len(sys.argv) > 1 else 0
class TestModel(QtSql.QSqlTableModel):
def __init__(self):
super().__init__()
self._select_row = None
self._reverting = False
QtSql.QSqlQuery().exec(
'CREATE TABLE IF NOT EXISTS test (name, value, data);')
self.setTable('test')
self.select()
def selectRow(self, row):
if BUGFIX & 2:
self._select_row = row
result = super().selectRow(row)
print(f'selectRow: {result}')
return result
def select(self):
return super().select() if self._select_row is None else False
def selectStatement(self):
if self._select_row is not None:
record = self.primaryValues(self._select_row)
for index in range(record.count()):
field = record.field(index)
if (not field.isNull() and
field.type() == QtCore.QVariant.Invalid):
field.setType(QtCore.QVariant.String)
record.replace(index, field)
where = self.database().driver().sqlStatement(
QtSql.QSqlDriver.WhereStatement,
self.tableName(), record, False)
if where[:6].upper() == 'WHERE ':
where = where[6:]
self.setFilter(where)
self._select_row = None
statement = super().selectStatement()
print(f'selectStatement: {statement!r}')
query = self.database().exec(statement)
if query.lastError().isValid():
print(f' query-lastError: {query.lastError().text()!r}')
else:
print(f' query-next: {query.next()}')
return statement
def revert(self):
if BUGFIX & 1:
self._reverting = True
print('reverting ...')
super().revert()
self._reverting = False
print('reverted')
def submit(self):
print('submitting ...')
result = False if self._reverting else super().submit()
print(f'submitted: {result}')
return result
app = QtWidgets.QApplication(['Test'])
db = QtSql.QSqlDatabase.addDatabase('QSQLITE')
db.setDatabaseName('test.db')
db.open()
win = QtWidgets.QWidget()
layout = QtWidgets.QVBoxLayout(win)
addButton = QtWidgets.QPushButton('Add row')
layout.addWidget(addButton)
table = QtWidgets.QTableView()
layout.addWidget(table)
model = TestModel()
table.setModel(model)
submitButton = QtWidgets.QPushButton('Submit')
layout.addWidget(submitButton)
submitButton.clicked.connect(model.submitAll)
addButton.clicked.connect(lambda: model.insertRow(model.rowCount()))
app.aboutToQuit.connect(model.submitAll)
win.setGeometry(1000, 50, 640, 480)
win.show()
app.exec()
PyQt6:
import sys
from PyQt6 import QtCore, QtWidgets, QtSql
BUGFIX = int(sys.argv[1]) if len(sys.argv) > 1 else 0
class TestModel(QtSql.QSqlTableModel):
def __init__(self):
super().__init__()
self._select_row = None
self._reverting = False
QtSql.QSqlQuery().exec(
'CREATE TABLE IF NOT EXISTS test (name, value, data);')
self.setTable('test')
self.select()
def selectRow(self, row):
if BUGFIX & 2:
self._select_row = row
result = super().selectRow(row)
print(f'selectRow: {result}')
return result
def select(self):
return super().select() if self._select_row is None else False
def selectStatement(self):
if self._select_row is not None:
record = self.primaryValues(self._select_row)
MetaType = QtCore.QMetaType.Type
MetaString = QtCore.QMetaType(MetaType.QString.value)
for index in range(record.count()):
field = record.field(index)
if (not field.isNull() and
field.metaType().id() == MetaType.UnknownType.value):
field.setMetaType(MetaString)
record.replace(index, field)
where = self.database().driver().sqlStatement(
QtSql.QSqlDriver.StatementType.WhereStatement,
self.tableName(), record, False)
if where[:6].upper() == 'WHERE ':
where = where[6:]
self.setFilter(where)
self._select_row = None
statement = super().selectStatement()
print(f'selectStatement: {statement!r}')
query = self.database().exec(statement)
if query.lastError().isValid():
print(f' query-lastError: {query.lastError().text()!r}')
else:
print(f' query-next: {query.next()}')
return statement
def revert(self):
if BUGFIX & 1:
self._reverting = True
print('reverting ...')
super().revert()
self._reverting = False
print('reverted')
def submit(self):
print('submitting ...')
result = False if self._reverting else super().submit()
print(f'submitted: {result}')
return result
app = QtWidgets.QApplication(['Test'])
db = QtSql.QSqlDatabase.addDatabase('QSQLITE')
db.setDatabaseName('test.db')
db.open()
win = QtWidgets.QWidget()
layout = QtWidgets.QVBoxLayout(win)
addButton = QtWidgets.QPushButton('Add row')
layout.addWidget(addButton)
table = QtWidgets.QTableView()
layout.addWidget(table)
model = TestModel()
table.setModel(model)
submitButton = QtWidgets.QPushButton('Submit')
layout.addWidget(submitButton)
submitButton.clicked.connect(model.submitAll)
addButton.clicked.connect(lambda: model.insertRow(model.rowCount()))
app.aboutToQuit.connect(model.submitAll)
win.setGeometry(1000, 50, 640, 480)
win.show()
app.exec()

Flask SQLAlchemy Sessions commit() not working "sometimes"

I've been working in a web app for a while and this is the first time I realize this problem, I think it could be related with how SQLAlchemy sessions are handled, so some clarification in simple term would be helpful.
My configuration for work with flask sqlAlchemy is:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
My problem: db.session.commit() sometimes doesn't save changes. I wrote some flask endpoints which are reached via the front end requests in the user browser.
In this particular case, I'm editing a hotel "Booking" object altering the "Rooms" columns which is a Text field.
the function does the following:
1-Query the Booking object from the dates in the request
2- Edit the Rooms column of this Booking object
3- Commit the changes "db.session.commit()"
4- If a user has X functionality active, I make some checks calling a second function:
·4.1- This functions make some checks and query and edit another object in the database different from the "Booking" object I edited previously.
·4.2- At the end of this secondary function I call db.session.commit() "Note this changes always got saved correctly in the database"
·4.3- Return the results to the previous function
5- Return results to the front end ("just before this return, I print the Booking.Rooms to make sure it looks as it should, and it does... I even tried to make a second commit after the print but before the return... But after this, sometimes Booking.Rooms are updated as expected but some other times it doesn't... I noted if repeat the action many times it finally works, but given the intermediate function "described in point 4" saves all his changes correctly, this causes an inconsistency in the data and drives me mad because if I repeat the action and procedure in the function of point 4 worked correctly, I can't repeat the mod Rooms action...
So, I'm now really confused if this is something I don't understand from flask sessions, for what I understand, whenever I make a new request to flask, it's an isolated session, right?
I mean, if 2 concurrent users are storing some changes in the database, a db.session.commit() from one of the users won't commit the changes from the other one, right?
Same way, if I call db.session.commit() in one request, that changes are stored in the database, and if after that "in the same request", I keep modding things, it's like another session, right? And the committed changes are there already safely stored? And I can still use previous objects for further modifications
Anyway, all of this shouldn't be a problem because after the commit() I print the Booking.Rooms and looks as expected... And some times it works getting stored correctly and some times it doesn't...
Also note: When I return this result to the client, the client makes instantly a second request to the server to request updated Booking data, and then the data is returned without this expected changes committed... I suppose flask handled all the commit() before it gets the second request "other way it wouldn't have returned the result previously..."
Can this be a limitation of the flask development server which can't handle correctly many requests and that when deployed with gunicorn it doesn't happen?
Any hint or clarification about Sessions would be nice, because this is pretty strange behaviour, especially that sometimes works and others don't...
And as requested here is the code, I know is not possible to reproduce, have a lot of setup behind and would need a lot of data to works as intended under same circumstances as in my case, but this should provide an overview of how the functions looks like and where are the commits I mention above. Any ideas of where can be the problem is very helpful.
#Main function hit by the frontend
#users.route('/url_endpoint1', methods=['POST'], strict_slashes=False)
#login_required
def add_room_to_booking_Api():
try:
bookingData = request.get_json()
roomURL=bookingData["roomSafeURL"]
targetBooking = bookingData["bookingURL"]
startDate = bookingData["checkInDate"]
endDate = bookingData["checkOutDate"]
roomPrices=bookingData["roomPrices"]
booking = Bookings.query.filter_by(SafeURL=targetBooking).first()
alojamiento = Alojamientos.query.filter_by(id=reserva.CodigoAlojamiento).first() #owner of the booking
room=Rooms.query.filter_by(SafeURL=roomURL).first()
roomsInBooking=ast.literal_eval(reserva.Habitaciones) #I know, should be json.loads() and json.dumps() for better performance probably...
#if room is available for given days add it to the booking
if CheckIfRoomIsAvailableForBooking(alojamiento.id, room, startDate, endDate, booking) == "OK":
roomsInBooking.append({"id": room.id, "Prices": roomPrices, "Guests":[]}) #add the new room the Rooms column of the booking
booking.Habitaciones = str(roomsInBooking)#save the new rooms data
print(booking.Habitaciones) # check changes applied
room.ReservaAsociada = booking.id # associate booking and room
for ocupante in room.Ocupantes: #associate people in the room with the booking
ocupante.Reserva = reserva.id
#db.session.refresh(reserva) # test I made to check if something changes but didn't worked
if some_X_function() == True: #if user have some functionality enabled
#db.session.begin() #another test which didn't worked
RType = WuBook_Rooms.query.filter_by(ParentType=room.Tipo).first()
RType=[RType] #convert to list because I resuse the function in cases with multiple types
resultAdd = function4(RType, booking.Entrada.replace(hour=0, minute=0, second=0), booking.Salida.replace(hour=0, minute=0, second=0))
if resultAdd["resultado"] == True: # "resultado":error, "casos":casos
return (jsonify({"resultado": "Error", "mensaje": resultAdd["casos"]}))
print(booking.Habitaciones) #here I still get expected result
db.session.commit()
#I get this return of averything correct in my frontend but not really stored in the database
return jsonify({"resultado": "Ok", "mensaje": "Room " + str(room.Identificador) + " added to the booking"})
else:
return (jsonify({"resultado": "Error", "mensaje": "Room " + str(room.Identificador) + " not available to book in target dates"}))
except Exception as e:
#some error handling which is not getting hit now
db.session.rollback()
print(e, ": en linea", lineno())
excepcion = str((''.join(traceback.TracebackException.from_exception(e).format()).replace("\n","</br>"), "</br>Excepcion emitida ne la línea: ", lineno()))
sendExceptionEmail(excepcion, current_user)
return (jsonify({"resultado":"Error","mensaje":"Error"}))
#function from point 4
def function4(RType, startDate, endDate):
delta = endDate - startDate
print(startDate, endDate)
print(delta)
for ind_type in RType:
calendarUpdated=json.loads(ind_type.updated_availability_by_date)
calendarUpdatedBackup=calendarUpdated
casos={}
actualizar=False
error=False
for i in range(delta.days):
day = (startDate + timedelta(days=i))
print(day, i)
diaString=day.strftime("%d-%m-%Y")
if day>=datetime.now() or diaString==datetime.now().strftime("%d-%m-%Y"): #only care about present and future dates
disponibilidadLocal=calendarUpdated[diaString]["local"]
yaReservadas=calendarUpdated[diaString]["local_booked"]
disponiblesChannel=calendarUpdated[diaString]["avail"]
#adjust availability data
if somecondition==True:
actualizar=True
casos.update({diaString:"Happened X"})
else:
actualizar=False
casos.update({diaString:"Happened Y"})
error="Error"
if actualizar==True: #this part of the code is hit normally and changes stored correctly
ind_type.updated_availability_by_date=json.dumps(calendarUpdated)
wubookproperty=WuBook_Properties.query.filter_by(id=ind_type.PropertyCode).first()
wubookproperty.SyncPending=True
ind_type.AvailUpdatePending=True
elif actualizar==False: #some error occured, revert changes
ind_type.updated_availability_by_date = json.dumps(calendarUpdatedBackup)
db.session.commit()#this commit persists
return({"resultado":error, "casos":casos}) #return to main function with all this chnages stored
Realized nothing was wrong at session level, it was my fault in another function client side which sends a request to update same data which is just being updated but with the old data... so in fact, I was getting the data saved correctly in the database but overwrote few milliseconds later. It was just a return statement missing in a javascript file to avoid this outcome...

sqlalchemy query after flushed delete

Given this piece of code:
record = session.query(Foo).filter(Foo.id == 1).first()
session.delete(record)
session.flush()
has_record = session.query(Foo).filter(Foo.id == 1).first()
I think the 'has_record' should be None here, but it turns out to be the same row as record.
Did I miss something to get the assumed result. Or is there any way that can make the delete take effect without commit?
Mysql would behave in a different way under similar process.
start transaction;
select * from Foo where id = 1; # Hit one record
delete from Foo where id = 1; # Nothing goes to the disk
select * from Foo where id = 1; # Empty set
commit; # Everything geos to the disk
I made a stupid mistake here. The session I'm using is a routing session, which has a master/slave session behind it. The fact might be that the delete is flushed to master and the query still goes to slave, so of course I can query the record again.

Categories