Convert stream of records from NEO4j get by NEO4J python driver into JSON. simply, I want to convert the results of cypher queries in neo4j into a JSON format.
Below is output got from NEO4J
<Record n=<Node id=3099 labels=frozenset({'BusinessData'}) properties={'Description': 'sample description.', 'x-NodeCreated': neo4j.time.Date(2023, 1, 11), 'name': 'Example', 'GUID': 'KL12822', 'Notes': 'Deployed', 'x-LastEdit': '16445676677MN'}>>
The result from a query looks something like above and When I perform a simple json.dumps() it return with TypeError: Object of type is not JSON serializable.
json_data = json.dumps(info,default = str)
What I am asking here is a way to get, What is the correct way to convert the query results to JSON!
One pattern that I quite like for this type of use case is to use Cypher's map projection feature to return a map/dictionary/object which can be more easily converted to JSON. Here's an example:
WITH point({latitude: $latitude, longitude:$longitude}) AS radiusCenter
MATCH (p:Point)-[:HAS_GEOMETRY]-(poi:PointOfInterest)-[:HAS_TAGS]->(t:Tags)
WHERE point.distance(p.location, radiusCenter) < $radius
RETURN p {
latitude: p.location.latitude,
longitude: p.location.longitude,
name: poi.name,
categories: labels(poi),
tags: t{.*}
} AS point
The above Cypher query will return a list of records, where each record is a map/dictionary/object:
{
latitude: 45.0,
longitude: -137.43,
name: "West Side Bakery",
categories: ['Bakery', 'PointOfInterest'],
tags: [
address: "1234 Main St",
commercial: "Bakery"
]
}
Then in your python code you can add each point record to a list and convert to JSON (another option would be to COLLECT in your Cypher statement and return a single array/list of objects):
point_list = []
with driver.session() as session:
results = session.execute_read(
lambda tx: tx.run(MY_CYPHER_QUERY).data()
)
for record in results:
point_list.append(record)
Related
I have the following data frame
lat long batchItems
0 64.427482 -161.457786 {"query": "?query=64.42748213233087,-161.45778...
1 31.271877 130.331371 {"query": "?query=31.271877,130.331371"}
which I need in the following format
{
"batchItems": [
{"query": "?query=64.42748213233087,-161.45778592219781"},
{"query": "?query=31.271877,130.331371"}
]
}
using the inbuilt to_json method in pandas gives me unwanted backslashes ("{") which get rejected by the API i'm trying to post this to. I don't need the lat and long fields.
Your batchItems is str type, so if you go to JSON, it's normal that quotes are escaped because they are not delimiting some string, they are inside the content itself.
You may change the type to dict using json.loads, then get the column as a list
import json
result = {"batchItems": df['batchItems'].apply(json.loads).tolist()}
print(result) # {'batchItems': [{'query': '?query=64,-161'}, {'query': '?query=31,130'}]}
df[['batchItems']].to_dict(orient='list') will return a dictionary in the required format
{
"batchItems": [
{"query": "?query=64.42748213233087,-161.45778592219781"},
{"query": "?query=31.271877,130.331371"}
]
}
You have to send this dictionary to the function that sends the data to the external API.
I have some JSON data like:
{
"status": "200",
"msg": "",
"data": {
"time": "1515580011",
"video_info": [
{
"announcement": "{\"announcement_id\":\"6\",\"name\":\"INS\\u8d26\\u53f7\",\"icon\":\"http:\\\/\\\/liveme.cms.ksmobile.net\\\/live\\\/announcement\\\/2017-08-18_19:44:54\\\/ins.png\",\"icon_new\":\"http:\\\/\\\/liveme.cms.ksmobile.net\\\/live\\\/announcement\\\/2017-10-20_22:24:38\\\/4.png\",\"videoid\":\"15154610218328614178\",\"content\":\"FOLLOW ME PLEASE\",\"x_coordinate\":\"0.22\",\"y_coordinate\":\"0.23\"}",
"announcement_shop": "",
etc.
How do I grab the content "FOLLOW ME PLEASE"? I tried using
replay_data = raw_replay_data['data']['video_info'][0]
announcement = replay_data['announcement']
But now announcement is a string representing more JSON data. I can't continue indexing announcement['content'] results in TypeError: string indices must be integers.
How can I get the desired string in the "right" way, i.e. respecting the actual structure of the data?
In a single line -
>>> json.loads(data['data']['video_info'][0]['announcement'])['content']
'FOLLOW ME PLEASE'
To help you understand how to access data (so you don't have to ask again), you'll need to stare at your data.
First, let's lay out your data nicely. You can either use json.dumps(data, indent=4), or you can use an online tool like JSONLint.com.
{
'data': {
'time': '1515580011',
'video_info': [{
'announcement': ( # ***
"""{
"announcement_id": "6",
"name": "INS\\u8d26\\u53f7",
"icon": "http:\\\\/\\\\/liveme.cms.ksmobile.net\\\\/live\\\\/announcement\\\\/2017-08-18_19:44:54\\\\/ins.png",
"icon_new": "http:\\\\/\\\\/liveme.cms.ksmobile.net\\\\/live\\\\/announcement\\\\/2017-10-20_22:24:38\\\\/4.png",
"videoid": "15154610218328614178",
"content": "FOLLOW ME PLEASE",
"x_coordinate": "0.22",
"y_coordinate": "0.23"
}"""),
'announcement_shop': ''
}]
},
'msg': '',
'status': '200'
}
*** Note that the data in the announcement key is actually more json data, which I've laid out on separate lines.
First, find out where your data resides. You're looking for the data in the content key, which is accessed by the announcement key, which is part of a dictionary inside a list of dicts, which can be accessed by the video_info key, which is in turn accessed by data.
So, in summary, "descend" the ladder that is "data" using the following "rungs" -
data, a dictionary
video_info, a list of dicts
announcement, a dict in the first dict of the list of dicts
content residing as part of json data.
First,
i = data['data']
Next,
j = i['video_info']
Next,
k = j[0] # since this is a list
If you only want the first element, this suffices. Otherwise, you'd need to iterate:
for k in j:
...
Next,
l = k['announcement']
Now, l is JSON data. Load it -
import json
m = json.loads(l)
Lastly,
content = m['content']
print(content)
'FOLLOW ME PLEASE'
This should hopefully serve as a guide should you have future queries of this nature.
You have nested JSON data; the string associated with the 'annoucement' key is itself another, separate, embedded JSON document.
You'll have to decode that string first:
import json
replay_data = raw_replay_data['data']['video_info'][0]
announcement = json.loads(replay_data['announcement'])
print(announcement['content'])
then handle the resulting dictionary from there.
The content of "announcement" is another JSON string. Decode it and then access its contents as you were doing with the outer objects.
I have developed a post request with requests module and I retrieve a JSON, this is code
headers_ = {'Authorization': acc_token}
params_ = (('start_time', '2016-08-01T00:00:00+00:00'),('limit', '2'))
r = requests.get('https://server.com/API/v1', headers=headers_, params=params_)
data_ = r.json()
json_result = (json.dumps(data_, indent=4, sort_keys=False))
and this is json result
{
"loc":[
{
"origin":{
"lat":39.72417,
"lng":-104.99984,
"eta_seconds":null,
"address":""
},
"destination":{
"lat":39.77446,
"lng":-104.9379,
"eta_seconds":null,
"address":null
}
},
{
"origin":{
"lat":39.77481,
"lng":-104.93618,
"eta_seconds":null,
"address":"10 Albion Street"
},
"destination":{
"lat":39.6984,
"lng":-104.9652,
"eta_seconds":null,
"address":null
}
}
]
}
now I'm trying to append into array the lat and lng value of each json items array and I have try with this code
for i in json_result:
print(i['loc']['origin']['lat'])
dict.append(i['loc']['origin']['lat'])
but I have this error
TypeError: string indices must be integers, not str
How can I retrieve the value lat and lng for each element of the array and append into dict?
Thanks
json_result is a string, because it is the result of json.dumps().
You don't need that at all. Remove the whole line, and operate directly on data_.
(As a style point, please stop adding those unnecessary trailing underscores.)
So the problem is this for i in json_result, which will return you each of the keys in your json. You can simply do:
for entry in json_result.get('loc'):
do stuff here....
Just a different solution , if you are still getting error then you can also try regex to fetch only lat , lng part :
import re
convert=str(json_result)
pattern=r"(l([ng]|[at]).'):\s[-]?\d{0,15}.\d{0,15}"
match=re.finditer(pattern,convert)
print([i.group() for i in match])
I am stuck on an issue where I am trying to parse for the id string in JSON that exists more than 1 time. I am using the requests library to pull json from an API. I am trying to retrieve all of the values of "id" but have only been able to successfully pull the one that I define. Example json:
{
"apps": [{
"id": "app1",
"id": "app2",
"id": "new-app"
}]
}
So what I have done so far is turn the json response into dictionary so that I am actually parse the first iteration of "id". I have tried to create for loops but have been getting KeyError when trying to find string id or TypeError: list indices must be integers or slices, not str. The only thing that I have been able to do successfully is define which id locations to output.
(data['apps'][N]['id']) -> where N = 0, 1 or 2
This would work if there was only going to be 1 string of id at a time but will always be multiple and the location will change from time to time.
So how do return the values of all strings for "id" from this single json output? Full code below:
import requests
url = "http://x.x.x.x:8080/v2/apps/"
response = requests.get(url)
#Error if not 200 and exit
ifresponse.status_code!=200:
print("Status:", response.status_code, "CheckURL.Exiting")
exit()
#Turn response into a dict and parse for ids
data = response.json()
for n in data:
print(data['apps'][0]['id'])
OUTPUT:
app1
UPDATE:
Was able to get resolution thanks to Robᵩ. Here is what I ended up using:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
print(value)
data = response.json(object_pairs_hook = list_hook)
Also The API that I posted as example is not a real API. It was just supposed to be a visual representation of what I was trying to achieve. I am actually using Mesosphere's Marathon API . Trying to build a python listener for port mapping containers.
Your best choice is to contact the author of the API and let him know that his data format is silly.
Your next-best choice is to modify the behavior of the the JSON parser by passing in a hook function. Something like this should work:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
else:
result[name] = value
return result
data = response.json(object_pairs_hook = list_hook)
for i in range(3):
print(i, data['apps'][0]['id'][i])
I'm attempting to retrieve a random result from a collection of JSON data using PyMongo. I'm using Flask and MongoDB. Here is how it is set up:
def getData():
dataCollection = db["data"]
for item in dataCollection.find({},{"Category":1,"Name":1,"Location":1,"_id":0}):
return (jsonify(item)
return (jsonify(item) returns 1 result and it is always the first one. How can I randomize this?
I tried importing the random module (import random) and switched the last line to random.choice(jsonify(item) but that results in an error.
Here is what the data looks like that was imported into MongoDB:
[
{
"Category":"Tennis",
"Name":"ABC Courts",
"Location":"123 Fake St"
},
{
"Category":"Soccer",
"Name":"XYZ Arena",
"Location":"319 Ace Blvd"
},
{
"Category":"Basketball",
"Name":"Dome Courts",
"Location":"8934 My Way"
},
]
You're always getting one result because return jsonify(item) ends the request. jsonify returns a response it does not only just turn result from Mongo into a json object. if you want to turn your Mongo result into a sequence use list then random.choice
item = random.choice(list(dataCollection.find({},{"Category":1,"Name":1,"Location":1,"_id":0}))
return jsonify(item)