I have a problem for several hours and didnt find an answer yet. My Problem is that i want to delete a key/value pair in subdictionary.
Structure is the following in MongoDB :
'_id' : objectId
'title: 'title'
'words': {
'word1' : [pos0,pos1,pos2,pos3],
'word2' : [pos0,pos1,pos2,pos3],
.
.
.
}
When I run this query :
client = pymongo.MongoClient(f"MONGODB_CONNECTION_LINK")
db = client.database
cursor = db['templates'] #OR cursor = db['templates'][index]
query = { 'words': { f'{word}' : f'{pos}' }}
x = cursor.delete_many(query)
and print out the cursor i get the place back where the deleteResult Object is located. Also when i delete them and print out the return, it says that something was deleted.. But when i go to my database its still there.
Index is the object where i want to delete from. Like Index[0] should be the first Object in the database. But when i try to delete with or without Index its the same result.
BTW also tried some other querys but probably not the right one.
Thanks for help guys
[
There were 2 Problems inside my Code:
I didnt access the entries properly according to the Post:
MongoDB - finding entries using a nested dictionary
I saved my Word Values with dots. Therefore i allowed mongodb to store with dots. In this Case its not possible to access the Entries correctly.
Now i just added an new Regex:
regex3 = re.compile(r'[,.]')
re.sub(regex3,'',words.decode_contents()))
I think the Cursor just points to the expected Location of the data. Even if the location exists or not. You dont get back an Error or something like this. Same with the DeleteResult.
Related
I'm a beginner in mongodb and pymongo and I'm working on a project where I have a students mongodb collection . What I want is to add a new field and specifically an adrress of a student to each element in my collection (the field is obviously added everywhere as null and will be filled by me later).
However when I try using this specific example to add a new field I get a the following syntax error:
client = MongoClient('mongodb://localhost:27017/') #connect to local mongodb
db = client['InfoSys'] #choose infosys database
students = db['Students']
students.update( { $set : {"address":1} } ) #set address field to every column (error happens here)
How can I fix this error?
You are using the update operation in wrong manner. Update operation is having the following syntax:
db.collection.update(
<query>,
<update>,
<options>
)
The main parameter <query> is not at all mentioned. It has to be at least empty like {}, In your case the following query will work:
db.students.update(
{}, // To update the all the documents.
{$set : {"address": 1}}, // Update the address field.
{multi: true} // To do multiple updates, otherwise Mongo will just update the first matching document.
)
So, in python, you can use update_many to achieve this. So, it will be like:
students.update_many(
{},
{"$set" : {"address": 1}}
)
You can read more about this operation here.
The previous answer here is spot on, but it looks like your question may relate more to PyMongo and how it manages updates to collections. https://pymongo.readthedocs.io/en/stable/api/pymongo/collection.html
According to the docs, it looks like you may want to use the 'update_many()' function. You will still need to make your query (all documents, in this case) as the first argument, and the second argument is the operation to perform on all records.
client = MongoClient('mongodb://localhost:27017/') #connect to local mongodb
db = client['InfoSys'] #choose infosys database
students = db['Students']
sudents.update_many({}, {$set : {"address":1}})
I solved my problem by iterating through every element in my collection and inserting the address field to each one.
cursor = students.find({})
for student in cursor :
students.update_one(student, {'$set': {'address': '1'}})
I currently have a dictionary with data being pulled from an API, where I have given each datapoint it's own variable (job_id, jobtitle, company etc.):
output = {
'ID': job_id,
'Title': jobtitle,
'Employer' : company,
'Employment type' : emptype,
'Fulltime' : tid,
'Deadline' : deadline,
'Link' : webpage
}
that I want to add to my database, easy enough:
db.jobs.insert_one(output)
but this is all in a for loop that will create 30-ish unique new documents, with names, titles, links and whatnot, this script will be run more than once, so what I would like for it to do is only insert the "output" as a document if it doesn't already exist in the database, all of these new documents do have their own unique ID's coming from the job_id variable am I able to check against that?
You need to try two things :
1) Doing .find() & if no document found for given job_id then writing to DB is a two way call - Instead you can have an unique-index on job_id field, that will throw an error if your operation tries to insert duplicate document (Having unique index is much more safer way to avoid duplicates, even helpful if your code logic fails).
2) If you've 30 dict's - You no need to iterate for 30 times & use insert_one to make 30 database calls, instead you can use insert_many which takes in an array of dict's & writes to database.
Note : By default all dict's are written in the order they're in the array, in case if a dict fails cause of duplicate error then insert_many fails at that point without inserting rest others, So to overcome this you need to pass an option
ordered=False that way all dictionaries will be inserted except duplicates.
EDIT:
replace
db.jobs.insert_one(output)
with
db.jobs.replace_one({'ID': job_id}, output, upsert=True)
ORIGINAL ANSWER with worked example:
Use replace_one() with upsert=True. You can run this multiple times and it will with insert if the ID isn't found or replace if it is found. It wasn't quite what you were asking as the data is always updated (so newer data will overwrite any existing data).
from pymongo import MongoClient
db = MongoClient()['mydatabase']
for i in range(30):
db.employer.replace_one({'ID': i},
{
'ID': i,
'Title': 'jobtitle',
'Employer' : 'company',
'Employment type' : 'emptype',
'Fulltime' : 'tid',
'Deadline' : 'deadline',
'Link' : 'webpage'
}, upsert=True)
# Should always print 30 regardless of number of times run.
print(db.employer.count_documents({}))
I'm trying to write a cloud function that returns users near a specific location. get_nearby() returns a list of tuples containing upper and lower bounds for a geohash query, and then this loop should query firebase for users within those geohashes.
user_ref = db.collection(u'users')
db_response = []
for query_range in get_nearby(lat, long, radius):
query = user_ref.where(u'geohash', u'>=', query_range[0]).where(u'geohash', u'<=', query_range[1]).get()
for el in query:
db_response.append(el.to_dict())
For some reason when I run this code, it returns only one document from my database, even though there are three other documents with the same geohash as that one. I know the documents are there, and they do get returned when I request the entire collection. What am I missing here?
edit:
The database currently has 4 records in it, 3 of which should be returned in this query:
{
{name: "Trevor", geohash: "dnvtz"}, #this is the one that gets returned
{name: "Test", geohash: "dnvtz"},
{name: "Test", geohash: "dnvtz"}
}
query_range is a tuple with two values. A lower and upper bound geohash. In this case, it's ("dnvt0", "dnvtz").
I decided to clear all documents from my database and then generate a new set of sample data to work with (everything there was only for testing anyway, nothing important). After pushing the new data to Firestore, everything is working. My only assumption is that even though the strings matched up, I'd used the wrong encoding on some of them.
Let's take this simple collection col with 2 documents:
{
"_id" : ObjectId("5ca4bf475e7a8e4881ef9dd2"),
"timestamp" : 1551736800,
"score" : 10
}
{
"_id" : ObjectId("5ca4bf475e7a8e4881ef9dd3"),
"timestamp" : 1551737400,
"score" : 12
}
To access the last timestamp (the one of the second document), I first did this request
a = db['col'].find({}).sort("_id", -1)
and then a[0]['timestamp']
But as there will be a lot of documents in this collection, i think that it would be more efficient to request only the last one with the limit function, like
a = db['col'].find({}).sort("_id", -1).limit(1)
and then
for doc in a:
lastTimestamp = doc['timestamp']
as there will be only one, i can declare the variable inside the loop.
So three questions :
Do i have to worry about memory / speed issues if i continue to use the first request and get the first element in the dic ?
Is there a smarter way to access the first element of the cursor instead of using a loop, when using the limit request ?
Is there another way to get that timestamp that i don't know ?
Thanks !
Python 3.6 / Pymongo 3.7
If you are using any field with an unique index in the selection criteria, you should use find_one method which will return the only document that matches your query.
That being said, the find method returns a Cursor object and does not load the data into memory.
You might get a better performance if you where using a filter option. Your query as it is now will do a collection scan.
if you are not using a filter, and want to retrieve the last document, then the clean way is with the Python built-in next function. You could also use the next method.
cur = db["col"].find().sort({"_id": -1}).limit(1):
with cur:
doc = next(cur, None) # None when we have empty collection.
find().sort() is so fast and don't worry about the speed and it's the best access the first element of the cursor.
I need tranfer data from a database in MySQL to MongoDB.
I have a Mysql's query with a few data:
SELECT data FROM table where data BETWEEN r1 AND r2
that i storaged in a list
so my problem is
when i try to find the data in mongodb (the data was in) i made this:
datamongo = collection.find({"data" : data[x]})
and the result is nothing, literaly.
I need to create a loop with the list range and search every data in the list in mongodb.
I tried with find() and find_one() but no one of them works.
BUT all works if i put a constant instead of a variable.
I hope anyone can help me
Here comes the regex part. You need to use that value in a way as explained below
collection.find({"data" : new RegExp(data[x], 'i')})
Or you can also use
collection.find( { 'data' : { '$regex' : data[x], '$options' : 'i' } } )
Note that i in RegExp, this is just for case insensitive comparison. Remove it if you want strict comparison.
Hope it helps.
Thanks