Slow query msg using python motor client and mongodb - python

I have a MongoDB pod with millions of records in k8s and using AsyncIOMotorClient to iterate over the cursor with 1000 batch size. However, after a few iterations, the fetching got stuck and I can see the Slow query message from MongoDB logs.
The iteration is like below after getting collection:
cursor = collection.aggregate_raw_batches(pipeline=pipeline, batchSize=c['QUERY_BATCH_SIZE'])
async for batch in cursor:
...
Even I tried to paginate the fetched records by adding $skip $limit into pipeline, but the same result appears again.
I also tried to track resource usage such as CPU or memory in Kubernetes but everything looks fine except MongoDB itself.
Example log of MongoDB:
{"t":{"$date":"2022-03-24T13:02:41.042+00:00"},"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn1214224","msg":"Slow query","attr":{"type":"command","ns":"product.feed_1648113467","command":{"getMore":4946197083994343333,"collection":"feed_1648113467","batchSize":1000,"lsid":{"id":{"$uuid":"bec21ebb-f305-4438-8e1d-16badfc22bd3"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648126959,"i":1}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$db":"product"},"originatingCommand":{"aggregate":"feed_1648113467","pipeline":[{"$match":{"_disapproved":{"$ne":true},"_gpla_feed_name":"detske-zbozi/kocarky","_delivery.direct":true,"$or":[{"_restricted":{"$ne":true}},{"_restriction_exception_for_google":true}]}},{"$limit":1000000},{"$project":{"_id":0,"g:description":1,"g:id":1,"title":"$g:name","g:price":{"$concat":[{"$toString":"$g:price_rounded"}," CZK"]},"g:brand":"$g:producer_name","g:image_link":"$g:image_url","g:additional_image_link":1,"link":"$g:url","g:availability":"in stock","g:condition":"new","g:material":1,"g:color":1,"g:gender":1,"g:size":1,"g:product_type":1,"g:google_product_category":1,"g:energy_efficiency_class":1,"g:unit_pricing_measure":1,"g:unit_pricing_base_measure":1,"g:gtin":"$g:ean","g:adult":"$_restricted","g:product_detail":1}}],"cursor":{"batchSize":0},"lsid":{"id":{"$uuid":"bec21ebb-f305-4438-8e1d-16badfc22bd3"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648126959,"i":1}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$db":"product","$readPreference":{"mode":"primaryPreferred"}},"planSummary":"IXSCAN { _disapproved: 1, _gpla_feed_name: 1 }","cursorid":4946197083994343333,"keysExamined":861,"docsExamined":859,"cursorExhausted":true,"numYields":8,"nreturned":960,"reslen":642671,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":9}},"Global":{"acquireCount":{"r":9}},"Database":{"acquireCount":{"r":9}},"Collection":{"acquireCount":{"r":9}},"Mutex":{"acquireCount":{"r":1}}},"storage":{"data":{"bytesRead":21129204,"timeReadingMicros":112254}},"protocol":"op_msg","durationMillis":138}}
Thanks for any help

Related

Retrieve Status of Rows Processed for long query using PYODBC / SQLAlchemy in Python

I have several SQL queries that I run regularly but can take a while to complete. I've noticed that different applications which query SQL Server might show a status update (E.g. 1200 rows received).
I was wondering if within SQL Alchemy or PYODBC there was a way to retrieve this, perhaps treat the query as a threaded worker and poll it as it completes.
Thanks,

getting pymongo.errors.CursorNotFound: cursor id "..." not found

I am reading over 100 million records from mongodb and creating nodes and relationships in neo4j.
whenever I run this after executing certain records I am getting pymongo.errors.CursorNotFound: cursor id "..." not found
earlier when I was executing it without "no_cursor_timeout=True" in the mongodb query then at every 64179 records I was getting the same error but after looking for this on StackOverflow I had tried this adding no_cursor_timeout=True but now also at 2691734 value I am getting the same error. HOW CAN I GET RID OF THIS ERROR I had also tried by defining the batch size.

Per the ticket Belly Buster mentioned you should try:
manually specifying the session to use with all your operations, and
periodically pinging the server using that session id to keep it alive on the server

Fetching more then 1200 items is not possible in GAE

I'm trying to fetch more then 1200 records from a query in GAE but this fails, just keeps loading forever. I can see in the debug that it keeps doing GET requests to Google but I never see any results
This works perfectly:
for lcr in deactivation_list.fetch(1200,offset=0, batch_size=1000):
This keeps loading:
for lcr in deactivation_list.fetch(1201,offset=0, batch_size=1000):
Tried increasing the batchsize, didnt help. I'm using NDB models

The only solution I have found is to use cursors, as suggested in the comments earlier. The reason is that the Google Remote API has a 1mb limit, using cursors you can query it multiple times.
recordQuery = model.query()
record, cursor, more = recordQuery.fetch_page(1000)
while more:
record, cursor, more = recordQuery.fetch_page(1000, start_cursor=cursor)

Temporary SQlite malformed pragma / disk image

We use SQLite databases to store the results coming out of data analysis pipelines. The database files sit on our high performance scale-out filestore which is connected to the same switch as our cluster nodes to ensure a good connection.
However recently I've been having trouble querying the database via python. This is particularly the case when many jobs are trying to query the database at once. I get error messages such as
sqlite3.DatabaseError: malformed database schema (primary_digest_joint) - index primary_digest_joint already exists
or
sqlite3.DatabaseError: database disk image is malformed
Note that these jobs are only reading the database, not writing to it (nothing is writing to the database), which I thought should be fine with SQLite.
Generally if I stop the pipeline, I can access the database fine and it appears to be perfectly intact. If I restart the pipeline again a number of jobs will successfully complete before I get the error again.
Any idea why this is happening or what can be done to stop it? Is there any chance that the database is actually being damaged, even though I seems to be fine, and running a PRAGMA integrity_check doesn't suggest anything is wrong.

MySQL select query not working with limit, offset parameters

I am running MySQL 5.1 on my windows vista installation. The table in question uses MyISAM, has about 10 million rows. It is used to store text messages posted by users on a website.
I am trying to run the following query on it,
query = "select id, text from messages order by id limit %d offset %d" %(limit, offset)
where limit is set to a fixed value (in this case 20000) and offset is incremented in steps of 20000.
This query goes into an infinite loop when offset = 240000. This particular value and not any other value.
I isolated this query into a script and ran it, and got the same results. I then tried to run the last query (with offset = 240000) directly, and it worked !
I then tried executing the same queries directly in a mysql client to make sure that the error was not in the python DB accessor module. All the queries returned results, except the one with offset = 240000.
I then looked at the mysql server logs and saw the following.
[ERROR] C:\Program Files\MySQL\MySQL Server 5.1\bin\mysqld: Sort aborted
This probably means that when I stopped the python process (out of frustration), the mysqld process was 'sorting' something. When I looked at the my.ini file, I saw a lot of MAX_* options. I am currently experimenting with these, but just throwing it out there in the meanwhile.
Any help appreciated!

Have you checked the table with myisamchk?

Categories