I'm attempting to retrieve a random result from a collection of JSON data using PyMongo. I'm using Flask and MongoDB. Here is how it is set up:
def getData():
dataCollection = db["data"]
for item in dataCollection.find({},{"Category":1,"Name":1,"Location":1,"_id":0}):
return (jsonify(item)
return (jsonify(item) returns 1 result and it is always the first one. How can I randomize this?
I tried importing the random module (import random) and switched the last line to random.choice(jsonify(item) but that results in an error.
Here is what the data looks like that was imported into MongoDB:
[
{
"Category":"Tennis",
"Name":"ABC Courts",
"Location":"123 Fake St"
},
{
"Category":"Soccer",
"Name":"XYZ Arena",
"Location":"319 Ace Blvd"
},
{
"Category":"Basketball",
"Name":"Dome Courts",
"Location":"8934 My Way"
},
]
You're always getting one result because return jsonify(item) ends the request. jsonify returns a response it does not only just turn result from Mongo into a json object. if you want to turn your Mongo result into a sequence use list then random.choice
item = random.choice(list(dataCollection.find({},{"Category":1,"Name":1,"Location":1,"_id":0}))
return jsonify(item)
Related
I have this example of an object in a movies collection:
{
"_id":{"$oid":"5f5101c31a05d8a343f944b1"},
"title":"Mother to Earth",
"year":2020,
"description":"A group of simps tries to find the source of an obscure meme game.",
"screenings":
[
{
"screeningID":{"$oid":"5f5101c31a05d8a343f944b0"},
"timedate":"2020-09-29, 18:00PM",
"tickets":46
}
]
}
And I want this to be the output of a find() function, with title as the query. However, when I include _id and screeningID, I get a TypeError: Object of type ObjectId is not JSON serializable error. I need the screeningID's value, in order to use it in a later part of my code, and preferably as a string. How do I do that?
EDIT: Here are the two lines of code in question:
result = movies.find_one({'title':data['title']})
result = {'title': result['title'],"year": result['year'],'description': result['description'],'screenings': [result['screenings']]}
I skipped the conditional checks I had in there, for simplicity's sake. As is, this produces the error I showed above. The only solution is to add {'_id':0, 'screenings.screeningID':0} in the projection of the first line, but this means losing the ObjectIds, and especially screeningID, which I need for later.
I ran your code and got no errors. If you run this do you get any errors?
from pymongo import MongoClient
from bson import ObjectId
data = {
"_id": ObjectId("5f5101c31a05d8a343f944b1"),
"title":"Mother to Earth",
"year":2020,
"description":"A group of simps tries to find the source of an obscure meme game.",
"screenings":
[
{
"screeningID": ObjectId("5f5101c31a05d8a343f944b0"),
"timedate":"2020-09-29, 18:00PM",
"tickets":46
}
]
}
db = MongoClient(port=27019)['testdatabase']
db.testcollection.delete_many({})
db.testcollection.insert_one(data)
result = db.testcollection.find_one({'title':data['title']})
result = {'title': result['title'],"year": result['year'],'description': result['description'],'screenings': [result['screenings']]}
I'am new to coding and try to build an HTTP-API with python. I have an Array of Objects in JSON format. I want to read a value of one of these objects.
In my python script I append a database table to an array of objects. I am looking for a solution to select a single value in one of these objects.
I have a function:
cur.execute(<SELECT STATEMENT>)
row_headers=[x[0] for x in cur.description]
response = cur.fetchall()
json_data=[]
for result in response:
json_data.append(dict(zip(row_headers,result)))
return jsonify(json_data)
the return looks like:
[
{
"ID": 123,
"CODE": 4117,
"STATUS": "off",
},
{
"ID": 345,
"CODE": 5776,
"STATUS": "on",
}
]
I am looking for a function(inputID):
where ID = inputID
set currentcode = <CODE>
set currentstatus = <STATUS>
<Do something with currentcode and currentstatus>
I believe you're looking for this:
def return_with_id(input_id):
for x in json_data:
if x['ID'] == input_id:
return x
This will loop over every index in json_data and checks to see if the ID of that index is equal to what you asked for, if it finds something that matches, it'll return that and the function completes.
If you want to do other things with this, you can do so by just editing the code the code before it returns.
I think I found the error. I do not exactly know what happend with:
return jsonify(json_data)
But it seems, that this makes my array not usable/iterable. I tried it with
return(json_data)
The output looks the same but know I can use
for x in json_data:
I tried to prevent pymongo.errors.InvalidOperation: No operations to execute after getting aggregate function's result. However, the way I used still shows pymongo.errors.InvalidOperation: No operations to execute for db.collection.insert(). I believe the result is not null at the beginning case.
Here is my code:
import sys
from pymongo import MongoClient
client = MongoClient()
db = client['movies']
for i in range(1,db.movies.count() + 1):
res = db.ratings.aggregate([
{"$match": {"movieID":str(i)}},
{"$group": {"_id": "$movieID", "avg": {"$avg":"$rating"}}}
])
if list(res) != None:
db.question1.insert(res)
So how to check the document is empty in MongoDB?
There are a couple things that may be wrong here: one is that a lot of pymongo's functions return iterators. For example:
query = db.customers.find({ 'name': 'John' })
# will print the number of Johns
print len(list(query))
# will always print 0, because the query results have already been
# consumed
print len(list(query))
Python's list function takes an iterator and creates a list with all the elements returned by the iterator, so list(query) will create a list with everything returned by the query. But, if we don't store the result of list(query) in a variable, they are lost forever and can't be accessed again ;-)
I'm not sure whether this is the case for aggregate or not. If it is, the moment you called list(res) you basically consumed all the results, so you can't access them a second time when you call insert(res) because they are already gone.
The other issue is that list(res) is never None. If there are no results, list(res) will be [] so you actually want to check simply if list(res) or if len(list(res)) > 0. Putting these two suggestions together, we get:
import sys
from pymongo import MongoClient
client = MongoClient()
db = client['movies']
for i in range(1,db.movies.count() + 1):
# Convert db.ratings.aggregate directly to a list
# Now `res` is a simple list, which can be accessed
# and traversed many times
res = list(db.ratings.aggregate([
{"$match": {"movieID":str(i)}},
{"$group": {"_id": "$movieID", "avg": {"$avg":"$rating"}}}
]))
if len(res) > 0:
db.question1.insert(res)
I spent several hours on this, tried everything I found online, pulled some of the hair left on my head...
I have this JSON sent to a Flask webservice I'm writing :
{'jsonArray': '[
{
"nom":"0012345679",
"Start":"2018-08-01",
"Finish":"2018-08-17",
"Statut":"Validee"
},
{
"nom":"0012345679",
"Start":"2018-09-01",
"Finish":"2018-09-10",
"Statut":"Demande envoyée au manager"
},
{
"nom":"0012345681",
"Start":"2018-04-01",
"Finish":"2018-04-08",
"Statut":"Validee"
},
{
"nom":"0012345681",
"Start":"2018-07-01",
"Finish":"2018-07-15",
"Statut":"Validee"
}
]'}
I want to simply loop through the records :
app = Flask(__name__)
#app.route('/graph', methods=['POST'])
def webhook():
if request.method == 'POST':
req_data = request.get_json()
print(req_data) #-> shows JSON that seems to be right
##print(type(req_data['jsonArray']))
#j1 = json.dumps(req_data['jsonArray'])
#j2 = json.loads(req_data['jsonArray'])
#data = json.loads(j1)
#for rec in data:
# print(rec) #-> This seems to consider rec as one of the characters of the whole JSON string, and prints every character one by one
#for key in data:
# value = data[key]
# print("The key and value are ({}) = ({})".format(key, value)) #-> TypeError: string indices must be integers
for record in req_data['jsonArray']:
for attribute, value in rec.items(): #-> Gives error 'str' object has no attribute 'items'
print(attribute, value)
I believe I am lost between JSON object, python dict object, strings, but I don't know what I am missing. I really tried to put the JSON received through json.dumps and json.loads methods, but still nothing. What am I missing ??
I simply want to loop through each record to create another python object that I will feed to a charting library like this :
df = [dict(Task="0012345678", Start='2017-01-01', Finish='2017-02-02', Statut='Complete'),
dict(Task="0012345678", Start='2017-02-15', Finish='2017-03-15', Statut='Incomplete'),
dict(Task="0012345679", Start='2017-01-17', Finish='2017-02-17', Statut='Not Started'),
dict(Task="0012345679", Start='2017-01-17', Finish='2017-02-17', Statut='Complete'),
dict(Task="0012345680", Start='2017-03-10', Finish='2017-03-20', Statut='Not Started'),
dict(Task="0012345680", Start='2017-04-01', Finish='2017-04-20', Statut='Not Started'),
dict(Task="0012345680", Start='2017-05-18', Finish='2017-06-18', Statut='Not Started'),
dict(Task="0012345681", Start='2017-01-14', Finish='2017-03-14', Statut='Complete')]
The whole thing is wrapped in single quotes, meaning it's a string and you need to parse it.
for record in json.loads(req_data['jsonArray']):
Looking at your commented code, you did this:
j1 = json.dumps(req_data['jsonArray'])
data = json.loads(j1)
Using json.dumps on a string is the wrong idea, and moreover json.loads(json.dumps(x)) is just the same as x, so that just got you back where you started, i.e. data was the same thing as req_data['jsonArray'] (a string).
This was the right idea:
j2 = json.loads(req_data['jsonArray'])
but you never used j2.
As you've seen, iterating over a string gives you each character of the string.
I am stuck on an issue where I am trying to parse for the id string in JSON that exists more than 1 time. I am using the requests library to pull json from an API. I am trying to retrieve all of the values of "id" but have only been able to successfully pull the one that I define. Example json:
{
"apps": [{
"id": "app1",
"id": "app2",
"id": "new-app"
}]
}
So what I have done so far is turn the json response into dictionary so that I am actually parse the first iteration of "id". I have tried to create for loops but have been getting KeyError when trying to find string id or TypeError: list indices must be integers or slices, not str. The only thing that I have been able to do successfully is define which id locations to output.
(data['apps'][N]['id']) -> where N = 0, 1 or 2
This would work if there was only going to be 1 string of id at a time but will always be multiple and the location will change from time to time.
So how do return the values of all strings for "id" from this single json output? Full code below:
import requests
url = "http://x.x.x.x:8080/v2/apps/"
response = requests.get(url)
#Error if not 200 and exit
ifresponse.status_code!=200:
print("Status:", response.status_code, "CheckURL.Exiting")
exit()
#Turn response into a dict and parse for ids
data = response.json()
for n in data:
print(data['apps'][0]['id'])
OUTPUT:
app1
UPDATE:
Was able to get resolution thanks to Robᵩ. Here is what I ended up using:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
print(value)
data = response.json(object_pairs_hook = list_hook)
Also The API that I posted as example is not a real API. It was just supposed to be a visual representation of what I was trying to achieve. I am actually using Mesosphere's Marathon API . Trying to build a python listener for port mapping containers.
Your best choice is to contact the author of the API and let him know that his data format is silly.
Your next-best choice is to modify the behavior of the the JSON parser by passing in a hook function. Something like this should work:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
else:
result[name] = value
return result
data = response.json(object_pairs_hook = list_hook)
for i in range(3):
print(i, data['apps'][0]['id'][i])