I'm attempting to create a web service using MongoDB and Flask (using the pymongo driver). A query to the database returns documents with the "_id" field included, of course. I don't want to send this to the client, so how do I remove it?
Here's a Flask route:
#app.route('/theobjects')
def index():
objects = db.collection.find()
return str(json.dumps({'results': list(objects)},
default = json_util.default,
indent = 4))
This returns:
{
"results": [
{
"whatever": {
"field1": "value",
"field2": "value",
},
"whatever2": {
"field3": "value"
},
...
"_id": {
"$oid": "..."
},
...
}
]}
I thought it was a dictionary and I could just delete the element before returning it:
del objects['_id']
But that returns a TypeError:
TypeError: 'Cursor' object does not support item deletion
So it isn't a dictionary, but something I have to iterate over with each result as a dictionary. So I try to do that with this code:
for object in objects:
del object['_id']
Each object dictionary looks the way I'd like it to now, but the objects cursor is empty. So I try to create a new dictionary and after deleting _id from each, add to a new dictionary that Flask will return:
new_object = {}
for object in objects:
for key, item in objects.items():
if key == '_id':
del object['_id']
new_object.update(object)
This just returns a dictionary with the first-level keys and nothing else.
So this is sort of a standard nested dictionaries problem, but I'm also shocked that MongoDB doesn't have a way to easily deal with this.
The MongoDB documentation explains that you can exclude _id with
{ _id : 0 }
But that does nothing with pymongo. The Pymongo documentation explains that you can list the fields you want returned, but "(“_id” will always be included)". Seriously? Is there no way around this? Is there something simple and stupid that I'm overlooking here?
To exclude the _id field in a find query in pymongo, you can use:
db.collection.find({}, {'_id': False})
The documentation is somewhat missleading on this as it says the _id field is always included. But you can exclude it like shown above.
Above answer fails if we want specific fields and still ignore _id. Use the following in such cases:
db.collection.find({'required_column_A':1,'required_col_B':1, '_id': False})
You are calling
del objects['_id']
on the cursor object!
The cursor object is obviously an iterable over the result set and not single
document that you can manipulate.
for obj in objects:
del obj['_id']
is likely what you want.
So your claim is completely wrong as the following code shows:
import pymongo
c = pymongo.Connection()
db = c['mydb']
db.foo.remove({})
db.foo.save({'foo' : 42})
for row in db.foo.find():
del row['_id']
print row
$ bin/python foo.py
> {u'foo': 42}
Related
I'm working on this REST application in python Flask and a driver called pymongo. But if someone knows mongodb well he/she maybe able to answer my question.
Suppose Im inserting a new document in a collection say students. I want to get the whole inserted document as soon as the document is saved in the collection. Here is what i've tried so far.
res = db.students.insert_one({
"name": args["name"],
"surname": args["surname"],
"student_number": args["student_number"],
"course": args["course"],
"mark": args["mark"]
})
If i call:
print(res.inserted_id) ## i get the id
How can i get something like:
{
"name": "student1",
"surname": "surname1",
"mark": 78,
"course": "ML",
"student_number": 2
}
from the res object. Because if i print res i am getting <pymongo.results.InsertOneResult object at 0x00000203F96DCA80>
Put the data to be inserted into a dictionary variable; on insert, the variable will have the _id added by pymongo.
from pymongo import MongoClient
db = MongoClient()['mydatabase']
doc = {
"name": "name"
}
db.students.insert_one(doc)
print(doc)
prints:
{'name': 'name', '_id': ObjectId('60ce419c205a661d9f80ba23')}
Unfortunately, the commenters are correct. The PyMongo pattern doesn't specifically allow for what you are asking. You are expected to just use the inserted_id from the result and if you needed to get the full object from the collection later do a regular query operation afterwards
I have a JSON text grabbed from an API of a website:
{"result":"true","product":{"made":{"Taiwan":"Taipei","HongKong":"KongStore","Area":"Asia"}}}
I want to capture "Taiwan" and "Taipei" but always fail.
Here is my code:
import json
weather = urllib2.urlopen('url')
wjson = weather.read()
wjdata = json.loads(wjson)
print wjdata['product']['made'][0]['Taiwan']
I always get the following error:
Keyword 0 error
Whats the correct way to parse that json?
You are indexing an array where there are none.
The JSON is the following:
{
"result":"true",
"product": {
"made": {
"Taiwan":"Taipei",
"HongKong":"KongStore",
"Area":"Asia"
}
}
}
And the above contains no arrays.
You are assuming the JSON structure to be something like this:
{
"result":"true",
"product": {
"made": [
{"Taiwan":"Taipei"},
{"HongKong":"KongStore"},
{"Area":"Asia"}
]
}
}
From a brief look at the doc pages for the json package, I found this conversion table: Conversion table using json.loads
It tells us that a JSON object translates to a dict. And a dict has a method called keys, which returns a list of the keys.
I suggest you try something like this:
#... omitted code
objectKeys = wjdata['product']['made'].keys()
# You should now have a list of the keys stored in objectKeys.
for key in objectKeys:
print key
if key == 'Taiwan':
print 'Eureka'
I haven't tested the above code, but I think you get the gist here :)
wjdata['product']['made']['Taiwan'] works
In my MongoDB, a bunch of these documents exist:
{ "_id" : ObjectId("5341eaae6e59875a9c80fa68"),
"parent" : {
"tokeep" : 0,
"toremove" : 0
}
}
I want to remove the parent.toremove attribute in every single one.
Using the MongoDB shell, I can accomplish this using:
db.collection.update({},{$unset: {'parent.toremove':1}},false,true)
But how do I do this within Python?
app = Flask(__name__)
mongo = PyMongo(app)
mongo.db.collection.update({},{$unset: {'parent.toremove':1}},false,true)
returns the following error:
File "myprogram.py", line 46
mongo.db.collection.update({},{$unset: {'parent.toremove':1}},false,true)
^
SyntaxError: invalid syntax
Put quotes around $unset, name the parameter you're including (multi) and use the correct syntax for true:
mongo.db.collection.update({}, {'$unset': {'parent.toremove':1}}, multi=True)
Just found weird to have to attach an arbitrary value for the field to remove, such as a small number (1), an empty string (''), etc, but it's really mentioned in MongoDB doc, with sample in JavaScript:
$unset
The $unset operator deletes a particular field. Consider the following syntax:
{ $unset: { field1: "", ... } }
The specified value in the $unset expression (i.e. "") does not impact
the operation.
For Python/PyMongo, I'd like to put a value None:
{'$unset': {'field1': None}}
So, for OP's question, it would be:
mongo.db.collection.update({}, {'$unset': {'parent.toremove': None}}, multi=True)
I'm trying to do something like this:
info = ([request.form['author'], request.form['title'], request.form['text']])
mongo.db.posts.insert(info, safe=True)
but I get a TypeError:'unicode' object does not support item assignment.
That's the last line in the stack trace:
if not "_id" in son:
son["_id"] = ObjectId()
return son
When using Pymongo your document must be a dict, not list.
Try:
info = {"author": request.form['author'], "title": request.form['title'], "text": request.form['text']}
or:
info = {"info": [request.form['author'], request.form['title'], request.form['text']]}
whatever is closer to your idea of model.
Actually this kind of error is documented for save() and update() methods of collection, but not for insert():
Raises TypeError if to_save is not an instance of dict.
pymongo collection docs
I am somewhat new to python and I am wondering what the best way is to generate json in a loop. I could just mash a bunch of strings together in the loop, but I'm sure there is a better way. Here's some more specifics. I am using app engine in python to create a service that returns json as a response.
So as an example, let's say someone requests a list of user records from the service. After the service queries for the records, it needs to return json for each record it found. Maybe something like this:
{records:
{record: { name:bob, email:blah#blah.com, age:25 } },
{record: { name:steve, email:blah#blahblah.com, age:30 } },
{record: { name:jimmy, email:blah#b.com, age:31 } },
}
Excuse my poorly formatted json. Thanks for your help.
Creating your own JSON is silly. Use json or simplejson for this instead.
>>> json.dumps(dict(foo=42))
'{"foo": 42}'
My question is how do I add to the
dictionary dynamically? So foreach
record in my list of records, add a
record to the dictionary.
You may be looking to create a list of dictionaries.
records = []
record1 = {"name":"Bob", "email":"bob#email.com"}
records.append(record1)
record2 = {"name":"Bob2", "email":"bob2#email.com"}
records.append(record2)
Then in app engine, use the code above to export records as json.
Few steps here.
First import simplejson
from django.utils import simplejson
Then create a function that will return json with the appropriate data header.
def write_json(self, data):
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(simplejson.dumps(data))
Then from within your post or get handler, create a python dictionary with the desired data and pass that into the function you created.
ret = {"records":{
"record": {"name": "bob", ...}
...
}
write_json(self, ret)