I have a list that has a couple of dicts inside and they are all separated with a comma ",". What is happening is that when i create this list i am adding , inside a for loop, a comma after every dict to separate them but it is also adding a last comma after the last dictionary. Something like:
"guests": [{
"age": "18",
"birthDate": null,
"emailAddress": null,...
....
"name": {
"prefix": "Mr.",
"firstName": "James",
"middleName": "",
"lastName": "Jones",
"suffix": ""
}
},----------------------------->This comma
]
I think that last comma is creating some issues when trying to make a post request to the web service. So, How can i delete just that last comma inside the list?
Thanks
Edit
The creation of the list is happening inside a for loop. Something like:
participants_body = ''
for guest in guests_info:
post_body = '{"profile": {"name": {"title": "' + guest["title"] + '","firstName": "' \
+ guest["first_name"] + '","lastName": "' + guest["last_name"] \
+ '"},"age": 18},"preferences": {"avatarIdentifier": "15655408","favoriteCharacterIdentifier":' \
' "15655408"},"friendsAndFamily": {"groupClassification": {"name": "TRAVELLING_PARTY"},' \
'"accessClassification": {"name": "PLAN_VIEW_SHARED"}}}'
response = requests.post(url, data=post_body, headers=headers)
json_response = response.json()
participants_body = '{"age": "' + str(json_response["profile"]["age"]) + '","birthDate": null,"emailAddress": null,' \
'"phone": null,"primary": false,"swid": null,"guid": "' + guid + '","gender": null,"type": null,' \
'"participantId": "' + p_id + '","profileLink": "https://env5.nge.api.go.com' + profileLink + '", ' \
'"infantSittingWithAdult": false,"avatar": null,"itemsAssigned": ' \
'["' + item_id + '"],"address": null,"phoneNumber": null,"dataType": "basic",' \
'"isRoomAssigned": true,"isVacationOfferAssigned": true,"ageGroup": "","name": {' \
'"prefix": "' + json_response["profile"]["name"]["title"] + '","firstName": "' \
+ json_response["profile"]["name"]["firstName"] + '","middleName": "","lastName": "' \
+
json_response["profile"]["name"]["lastName"] + '","suffix": ""}},'------------> HERE IS THE COMA
post_body_participants += participants_body
So, that´s why i´m getting the coma. I just need to delete it after the for loop
EDIT
I´m creating a Post message and i´m getting this error:
{u'errors': [{u'message': u'org.codehaus.jackson.map.JsonMappingException: Can not instantiate value of type [simple type, class com.disney.wdpro.service.booking.webservice.resource.ParticipantWithAssignmentResourceCollection] from JSON String; no single-String constructor/factory method'}]}
I read a couple of SO questions and they mentioned that maybe this is happening because an error with the json format.
Also i can see how the body of the post is created in other messages in the logs and that last comma is not there, so maybe that´s what´s happening
I'm not sure why you're creating this as a string. You'll be happier to create the dicts as dicts. The code is much more readable, which will help you when you have to change it later. Besides that, it will eliminate little typo bugs like you're experiencing.
post_body = {
'profile': {
'name': {
'title': guest['title'],
'firstName': guest['first_name'],
'lastName': guest['last_name'] },
'age': 18 },
'preferences': {
'avatarIdentifier': 15655408,
'favoriteCharacterIdentifier': 15655408 },
'friendsAndFamily': {
'groupClassification': {
'name': 'TRAVELLING_PARTY' },
'accessClassification': {
'name': 'PLAN_VIEW_SHARED' }
}
}
It's easy to turn that dict into a JSON string:
import json
post_body = json.dumps(post_body)
You can do the same thing with creating a list from the participants_body response. Just create the one dict as above, and append it with post_body_participants.append(participants_body). Again, you can access that list in the form of a JSON string with json.dumps(post_body_participants).
You will save yourself a great deal of pain if you use the built in json encoders/decoders to build your json strings. Building them by hand is error prone. Why not stand on the shoulders of giants?
import requests
import json
participants =[]
for guest in guests_info:
#Build Python objects and not json strings
#Convert it all to json later
post_body = {
'profile': {
'name': {
'title': guest['title'],
'firstName': guest['first_name'],
'lastName': guest['last_name'] },
'age': 18 },
'preferences': {
'avatarIdentifier': 15655408,
'favoriteCharacterIdentifier': 15655408 },
'friendsAndFamily': {
'groupClassification': {
'name': 'TRAVELLING_PARTY' },
'accessClassification': {
'name': 'PLAN_VIEW_SHARED' }
}
}
#The requests module has json encoding/decoding built in
response = requests.post(url, json=post_body, headers=headers)
#Or you could use Python's built in json module
#response = requests.post(url, data=json.dumps(post_body), headers=headers)
json_response = response.json() #This decodes the json string in the response to a Python object
participant = {
"age": json_response["profile"]["age"],
"birthDate": None,
"emailAddress": None,
"phone": None,
"primary": False,
"swid": None,
"guid": guid,
"gender": None,
"type": None,
"participantId": p_id,
"profileLink": "https://env5.nge.api.go.com" + profileLink + ,
"infantSittingWithAdult": False,
"avatar": None,
"itemsAssigned": [item_id],
"address": None,
"phoneNumber": None,
"dataType": "basic",
"isRoomAssigned": True,
"isVacationOfferAssigned": True,
"ageGroup": "",
"name": {
"prefix": json_response["profile"]["name"]["title"],
"firstName": json_response["profile"]["name"]["firstName"],
"middleName": "",
"lastName": json_response["profile"]["name"]["lastName"],
"suffix": ""}
}
}
participants.append(participant)
Related
I am working on a new project in HubSpot that returns nested JSON like the sample below. I am trying to access the associated contacts id, but am struggling to reference it correctly (the id I am looking for is the value '201' in the example below). I've put together this script, but this script only returns the entire associations portion of the JSON and I only want the id. How do I reference the id correctly?
Here is the output from the script:
{'contacts': {'paging': None, 'results': [{'id': '201', 'type': 'ticket_to_contact'}]}}
And here is the script I put together:
import hubspot
from pprint import pprint
client = hubspot.Client.create(api_key="API_KEY")
try:
api_response = client.crm.tickets.basic_api.get_page(limit=2, associations=["contacts"], archived=False)
for x in range(2):
pprint(api_response.results[x].associations)
except ApiException as e:
print("Exception when calling basic_api->get_page: %s\n" % e)
Here is what the full JSON looks like ('contacts' property shortened for readability):
{
"results": [
{
"id": "34018123",
"properties": {
"content": "Hi xxxxx,\r\n\r\nCan you clarify on how the blocking of script happens? Is it because of any CSP (or) the script will decide run time for every URL’s getting triggered from browser?\r\n\r\nRegards,\r\nLogan",
"createdate": "2019-07-03T04:20:12.366Z",
"hs_lastmodifieddate": "2020-12-09T01:16:12.974Z",
"hs_object_id": "34018123",
"hs_pipeline": "0",
"hs_pipeline_stage": "4",
"hs_ticket_category": null,
"hs_ticket_priority": null,
"subject": "RE: call followup"
},
"createdAt": "2019-07-03T04:20:12.366Z",
"updatedAt": "2020-12-09T01:16:12.974Z",
"archived": false
},
{
"id": "34018892",
"properties": {
"content": "Hi Guys,\r\n\r\nI see that we were placed back on the staging and then removed again.",
"createdate": "2019-07-03T07:59:10.606Z",
"hs_lastmodifieddate": "2021-12-17T09:04:46.316Z",
"hs_object_id": "34018892",
"hs_pipeline": "0",
"hs_pipeline_stage": "3",
"hs_ticket_category": null,
"hs_ticket_priority": null,
"subject": "Re: Issue due to server"
},
"createdAt": "2019-07-03T07:59:10.606Z",
"updatedAt": "2021-12-17T09:04:46.316Z",
"archived": false,
"associations": {
"contacts": {
"results": [
{
"id": "201",
"type": "ticket_to_contact"
}
]
}
}
}
],
"paging": {
"next": {
"after": "35406270",
"link": "https://api.hubapi.com/crm/v3/objects/tickets?associations=contacts&archived=false&hs_static_app=developer-docs-ui&limit=2&after=35406270&hs_static_app_version=1.3488"
}
}
}
You can do api_response.results[x].associations["contacts"]["results"][0]["id"].
Sorted this out, posting in case anyone else is struggling with the response from the HubSpot v3 Api. The response schema for this call is:
Response schema type: Object
String results[].id
Object results[].properties
String results[].createdAt
String results[].updatedAt
Boolean results[].archived
String results[].archivedAt
Object results[].associations
Object paging
Object paging.next
String paging.next.after
String paging.next.linkResponse schema type: Object
String results[].id
Object results[].properties
String results[].createdAt
String results[].updatedAt
Boolean results[].archived
String results[].archivedAt
Object results[].associations
Object paging
Object paging.next
String paging.next.after
String paging.next.link
So to access the id of the contact associated with the ticket, you need to reference it using this notation:
api_response.results[1].associations["contacts"].results[0].id
notes:
results[x] - reference the result in the index
associations["contacts"] -
associations is a dictionary object, you can access the contacts item
by it's name
associations["contacts"].results is a list - reference
by the index []
id - is a string
In my case type was ModelProperty or CollectionResponseProperty couldn't reach dict anyhow.
For the record this got me to go through the results.
for result in list(api_response.results):
ID = result.id
I have a Python script that parses a JSON file like below:
[
{
"_index": "bulletins",
"_type": "bulletin",
"_id": "OPENWRT-SA-000001",
"_score": null,
"_source": {
"lastseen": "2016-09-26T15:45:23",
"references": [
"http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3193",
],
"affectedPackage": [
{
"OS": "OpenWrt",
"OSVersion": "15.05",
"packageVersion": "9.9.8-P3-1",
"packageFilename": "UNKNOWN",
"arch": "all",
"packageName": "bind",
"operator": "lt"
}
],
"edition": 1,
"description": "value in here,
"reporter": "OpenWrt Project",
"published": "2016-01-24T13:33:41",
"title": "bind: Security update (4 CVEs)",
"type": "openwrt",
"bulletinFamily": "unix",
"cvelist": [
"CVE-2015-8704",
],
"modified": "2016-01-24T13:33:41",
"id": "OPENWRT-SA-000001",
"href": "https://lists.openwrt.org/pipermail/openwrt-security-announce/2016-January/000001.html",
"cvss": {
"score": 7.1,
"vector": "AV:NETWORK/AC:MEDIUM/Au:NONE/C:NONE/I:NONE/A:COMPLETE/"
}
},
"sort": [
34872
]
},
I have removed some of the values to keep the post shorter but leaving some in to try to keep the structure.
I want to take all sub keys from the _source key and move them up to the same level as _source and then delete the _source key.
My code to parse the JSON is:
import json
import logging
import logging.handlers
import os
import pymongo
from pymongo import MongoClient
def import_json(mongo_server,mongo_port, vuln_folder):
try:
logging.info('Connecting to MongoDB')
client = MongoClient(mongo_server, mongo_port)
db = client['vuln_sets']
coll = db['vulnerabilities']
logging.info('Connected to MongoDB')
basepath = os.path.dirname(__file__)
filepath = os.path.abspath(os.path.join(basepath, ".."))
archive_filepath = filepath + vuln_folder
filedir = os.chdir(archive_filepath)
file_count = 0
for item in os.listdir(filedir):
if item.endswith('.json'):
file_name = os.path.abspath(item)
with open(item, 'r') as currentfile:
vuln_counter = 0
duplicate_count = 0
logging.info('Currently processing ' + item)
file_count +=1
json_data = currentfile.read()
vuln_content = json.loads(json_data)
for vuln in vuln_content:
try:
del vuln['_type']
coll.insert(vuln, continue_on_error=True)
vuln_counter +=1
except pymongo.errors.DuplicateKeyError:
duplicate_count +=1
logging.info('Added ' + str(vuln_counter) + ' vulnerabilities for ' + item)
logging.info('Found ' + str(duplicate_count) + ' duplicate records!')
os.remove(file_name)
logging.info('Processed ' + str(file_count) + ' files')
except Exception as e:
logging.exception(e)
Which you can see already deletes one key that is not needed but that key has no needed data where as I need the sub keys from _source. I am not sure on the best way to achieve this, whether it would be programmatically correct to just re-create the JSON file with the new info but I need to keep the order of the keys and structure apart from moving the sub keys up one level.
You can use the dictionary update() function to achieve what you're trying to do, but it's important to note that dictionaries don't have an "order of the keys" - see: Key Order in Python Dictionaries.
Here's an example of one way to do this, starting with a dictionary definition.
d = {
"_index": "bulletins",
"_type": "bulletin",
"_id": "OPENWRT-SA-000001",
"_score": None,
"_source": {
"lastseen": "2016-09-26T15:45:23",
"references": [
"http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3193",
],
"affectedPackage": [
{
"OS": "OpenWrt",
"OSVersion": "15.05",
"packageVersion": "9.9.8-P3-1",
"packageFilename": "UNKNOWN",
"arch": "all",
"packageName": "bind",
"operator": "lt"
}
],
"edition": 1,
"description": "value in here",
"reporter": "OpenWrt Project",
"published": "2016-01-24T13:33:41",
"title": "bind: Security update (4 CVEs)",
"type": "openwrt",
"bulletinFamily": "unix",
"cvelist": [
"CVE-2015-8704",
],
"modified": "2016-01-24T13:33:41",
"id": "OPENWRT-SA-000001",
"href": "https://lists.openwrt.org/pipermail/openwrt-security-announce/2016-January/000001.html",
"cvss": {
"score": 7.1,
"vector": "AV:NETWORK/AC:MEDIUM/Au:NONE/C:NONE/I:NONE/A:COMPLETE/"
}
}
}
# create a new dictionary with everything except the key "_source"
new_d = {key: d[key] for key in d if key != '_source'}
# add the keys/values from "_source" to new dictionary
new_d.update(d['_source']) # This will overwriting any existing keys
The output of new_d:
{'_id': 'OPENWRT-SA-000001',
'_index': 'bulletins',
'_score': None,
'_type': 'bulletin',
'affectedPackage': [{'OS': 'OpenWrt',
'OSVersion': '15.05',
'arch': 'all',
'operator': 'lt',
'packageFilename': 'UNKNOWN',
'packageName': 'bind',
'packageVersion': '9.9.8-P3-1'}],
'bulletinFamily': 'unix',
'cvelist': ['CVE-2015-8704'],
'cvss': {
'score': 7.1,
'vector': 'AV:NETWORK/AC:MEDIUM/Au:NONE/C:NONE/I:NONE/A:COMPLETE/'},
'description': 'value in here',
'edition': 1,
'href': 'https://lists.openwrt.org/pipermail/openwrt-security-announce/2016-January/000001.html',
'id': 'OPENWRT-SA-000001',
'lastseen': '2016-09-26T15:45:23',
'modified': '2016-01-24T13:33:41',
'published': '2016-01-24T13:33:41',
'references': ['http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-
3193'],
'reporter': 'OpenWrt Project',
'title': 'bind: Security update (4 CVEs)',
'type': 'openwrt'}
I managed to get it working by using the following code:
for vuln in vuln_content:
try:
del vuln['_type']
new_vuln = {key: vuln[key] for key in vuln if key != '_source'}
new_vuln.update(vuln['_source'])
coll.insert(new_vuln, continue_on_error=True)
vuln_counter +=1
except pymongo.errors.DuplicateKeyError:
duplicate_count +=1
i want to append below json with the data
meta = [{
"output_metadata": {
"api_URL": apiURL,
"query_execution_time": queryExecTime,
"api_execution_time": apiExecTime,
}
}]
jsondata = json.dumps([dict(ix) for ix in Data], default=str)
json data:
{"data": [{"id": "1234", "name": "jhon", "dept": "APA"}]}
meta.append(jsondata)
expected result:
{"output_metadata": {"api_url": "xxxxx", "query_execution_time":"xxxxx", "api_execution_time":"xxxxx"}},{"data": "[{"id": "1234", "name": "jhon", "dept": "APA"}]}
output:
{"output_metadata": {"api_url": "XXXXXX", "query_execution_time": "XXXXXX", "api_execution_time":"XXXXXX" }},{"data": "[{"\id": "1234\", "\name": "\jhon", "\dept": "\APA"}]}
How to remove \ from the final output?
If this thing you wrote above is python the meta variable you create is invalid because before every " you should use an escape character and every time you go in a new line. For example you should write:
meta = ["{\
\"output_metadata\": {\
\"api_URL\": apiURL,\
\"query_execution_time\": queryExecTime,\
\"api_execution_time\": apiExecTime, \
}\
}"]
data = ["{\"data\": {\"id\": \"1234\", \"name\": \"jhon\", \"dept\": \"APA\"}]}"]
meta.append(data)
Where you handle the json's as strings and then append them in one list. Is this what you want?
EDIT: if you run something like
data = [{"id": 1234, "name": "jhon", "dept": "APA" }]
jdata= json.dumps([dict(ix) for ix in data], default=str)
apiURL = 'url'
queryExecTime = 1
apiExecTime = 1
meta = [{ "output_metadata": { "api_url": apiURL,
"query_execution_time": queryExecTime,
"api_execution_time": apiExecTime, } }]
jdata = { "data": jdata }
meta.append(jdata)
res = json.dumps(meta)
print(res)
the result will be:
'[{"output_metadata": {"api_url": "url", "query_execution_time": 1, "api_execution_time": 1}}, {"data": "[{\\"id\\": 1234, \\"name\\": \\"jhon\\", \\"dept\\": \\"APA\\"}]"}]'
The \ are used as escape characters for the ". You see the result as a literal string.
I'm using the following python code to connect to a jsonrpc server and nick some song information. However, I can't work out how to get the current title in to a variable to print elsewhere. Here is the code:
TracksInfo = []
for song in playingSongs:
data = { "id":1,
"method":"slim.request",
"params":[ "",
["songinfo",0,100, "track_id:%s" % song, "tags:GPASIediqtymkovrfijnCYXRTIuwxN"]
]
}
params = json.dumps(data, sort_keys=True, indent=4)
conn.request("POST", "/jsonrpc.js", params)
httpResponse = conn.getresponse()
data = httpResponse.read()
responce = json.loads(data)
print json.dumps(responce, sort_keys=True, indent=4)
TrackInfo = responce['result']["songinfo_loop"][0]
TracksInfo.append(TrackInfo)
This brings me back the data in json format and the print json.dump brings back:
pi#raspberrypi ~/pithon $ sudo python tom3.py
{
"id": 1,
"method": "slim.request",
"params": [
"",
[
"songinfo",
"0",
100,
"track_id:-140501481178464",
"tags:GPASIediqtymkovrfijnCYXRTIuwxN"
]
],
"result": {
"songinfo_loop": [
{
"id": "-140501481178464"
},
{
"title": "Witchcraft"
},
{
"artist": "Pendulum"
},
{
"duration": "253"
},
{
"tracknum": "1"
},
{
"type": "Ogg Vorbis (Spotify)"
},
{
"bitrate": "320k VBR"
},
{
"coverart": "0"
},
{
"url": "spotify:track:2A7ZZ1tjaluKYMlT3ItSfN"
},
{
"remote": 1
}
]
}
}
What i'm trying to get is result.songinfoloop.title (but I tried that!)
The songinfo_loop structure is.. peculiar. It is a list of dictionaries each with just one key.
Loop through it until you have one with a title:
TrackInfo = next(d['title'] for d in responce['result']["songinfo_loop"] if 'title' in d)
TracksInfo.append(TrackInfo)
A better option would be to 'collapse' all those dictionaries into one:
songinfo = reduce(lambda d, p: d.update(p) or d,
responce['result']["songinfo_loop"], {})
TracksInfo.append(songinfo['title'])
songinfo_loop is a list not a dict. That means you need to call it by position, or loop through it and find the dict with a key value of "title"
positional:
responce["result"]["songinfo_loop"][1]["title"]
loop:
for info in responce["result"]["songinfo_loop"]:
if "title" in info.keys():
print info["title"]
break
else:
print "no song title found"
Really, it seems like you would want to have the songinfo_loop be a dict, not a list. But if you need to leave it as a list, this is how you would pull the title.
The result is really a standard python dict, so you can use
responce["result"]["songinfoloop"]["title"]
which should work
I'm reading in a JSON string which is littered with u'string' style strings. Example:
[
{
"!\/award\/award_honor\/honored_for": {
"award": {
"id": "\/en\/spiel_des_jahres"
},
"year": {
"value": "1996"
}
},
"guid": "#9202a8c04000641f80000000003a0ee6",
"type": "\/games\/game",
"id": "\/en\/el_grande",
"name": "El Grande"
},
{
"!\/award\/award_honor\/honored_for": {
"award": {
"id": "\/en\/spiel_des_jahres"
},
"year": {
"value": "1995"
}
},
"guid": "#9202a8c04000641f80000000000495ec",
"type": "\/games\/game",
"id": "\/en\/settlers_of_catan",
"name": "Settlers of Catan"
}
]
If I assign name = result.name. Then when I log of pass that value to a Django template, it displays as u'Dominion'
How do I format it to display as Dominion?
++ UPDATE ++
I think the problem has to do with printing values from a list or dictionary. For example:
result = freebase.mqlread(query)
games = {}
count = 0
r = result[0]
name = r.name
games["name"] = name,
self.response.out.write(games["name"])
self.response.out.write(name)
This displays as:
(u'Dominion',) // saved response to dictionary, and then printed
Dominion // when calling the value directly from the response
I need to iterate through an array of JSON items and the values are being shown with the unicode. Why?
The comma at the end of games["name"] = name, makes it a 1-tuple. Remove it.
>>> # example
>>> s = u"Jägermütze"
>>> s.encode("utf-8")
'J\xc3\xa4germ\xc3\xbctze'
>>> print s.encode("utf-8") # on a utf-8 terminal
Jägermütze
Don't know much about Django, but not accepting snicode strings seems unpythonic to me.
You can use str(your string) to do this.