I am running this line of code,
m =requests.post(api_url + 'accounts', json=order, auth=auth)
print m.json()
which produces the following output:
[{u'available': u'0.4', u'balance': u'0.5'}, {u'available': u'6.8', u'balance': u'9.0'}]
My goal is to be able to save the different "availables" and "balances" into FLOAT variables, so I can use them further in the code.
Currently, I am trying to do
output = float(m['available'])
print(output)
But this is not working ... and I am also unsure how to separate them both, given that there is a { curly bracket} between the two "availables" and "balances"
I think the json string you get looks like that
[
{
"available": "0.4",
"balance": "0.5"
},
{
"available": "6.8",
"balance": "9.0"
}
]
Which means m.json() will return a list with two dictionaries in it .
You should use float(json[0]['available']) to get the value .
Related
I have a file of hundreds of json schemas that I need to parse through and fix the schema format in a way that the "isRelatedto" property will have an object(dictionary) without the extra strings outside the object. Also, with the strings that startswith "pubmed", I want to change it to "pmid" and then insert it into the object inside of "isRelatedto" array(list). The problem I have is that some of the "isRelatedto" array has object and some of them do not as shown below. When I tried to use the code I wrote to make the change, the schema without an object does not change. I would really appreciate your help!
"isRelatedto": [
"pubmed:18984613",
"pubmed:25392406",
"pubmed:33147627"
]
"isRelatedto": [
"pubmed:20947564",
"pmcid:PMC3013774",
{
"#id": "https://doi.org/10.1093/nar/gkq901",
"#type": "sc:CreativeWork"
}
]
The expected result for the first one will be:
"isRelatedto": [
{
"pmid": "18984613",
"pmid": "25392406",
"pmid": "33147627"
}
]
and for the second one will be:
"isRelatedto": [
{
"#id": "https://doi.org/10.1093/nar/gkq901",
"#type": "sc:CreativeWork"
"pmid": "20947564"
}
]
Background
I have data stored in the following format
{
"player_id": "VU3R5HNTAGMK",
"markers": {
"BICF2P964092": "GC",
"BICF2G630653981": "CG",
"BICF2P483996": "CT",
"BICF2S23452916": "CG",
"chr26_19147949": "TC",
}
}
You can imagine i have data stored for multiple players and each has a unique player_id and they all have varying number of markers with different marker values.
In the above case a marker is BICF2P964092 and it's marker value is GC.
I am trying to query my mongo db in various ways. One obvious way is by using player_id. To do that I do the following col.find({"player_id": "VU3R5HNTAGMK"})
Another thing i want to do is maybe I just want to know value of a specific marker for a specific player. So for that I can do the following col.find({"player_id": "VU3R5HNTAGMK"}, {'markers.BICF2P964092'})
ISSUE
I also want to be able to get values for multiple markers for a specific player and i am not able to do so. I have tried the following with no luck.
col.find({"player_id": "VU3R5HNTAGMK"},{'markers': {'$in': ["BICF2P964092", "chr26_19147949"]}})
col.find({"player_id": "VU3R5HNTAGMK"}, {'markers.BICF2P964092'}, {'markers.chr26_19147949'})
I would really appreciate it if someone can help me write a query where i can get multiple marker values for specified marker and player_id
You can simply do the following
col.find({“player_id”: “VU3R5HNTAGMK”}, {“markers.” + m: 1 for m in [“ BICF2P964092", “BICF2G630653981”]})
As you've tagged this pymongo, you might be as best to process the marker values in python after the find; e.g.
docs = col.find({"player_id": "VU3R5HNTAGMK"})
for doc in docs:
for marker, value in doc.get('markers').items():
if marker in ["BICF2P964092", "chr26_19147949"]:
print(marker, value)
#Belly Buster solution is good if you want to handle this using python.
But, there is a way to completely handle this on the MongoDB side using Aggregation.
You can combine $objectToArray, $filter, and $arrayToObject operators in $project stage.
collection.aggregate([
{
"$match": {
"player_id": "VU3R5HNTAGMK" # <-- All your match conditons
}
},
{
"$project": {
"player_id": 1, # All the other keys which you want to project
"markers": {
"$arrayToObject": {
"$filter": {
"input": {
"$objectToArray": "$markers"
},
"as": "elem",
"cond": {
"$in": [
"$$elem.k",
[
# <-- List of key names you want to project
"BICF2G630653981",
"BICF2P483996"
]
]
},
},
},
},
}
},
])
Note: You have to use MongoDB version >= 3.4.4 for this aggregation query to work.
I am using elasticsearch on a large indexed database. One of the queries requires to find an integer value and a string such as:
s = Search(using=es, index="index1").extra(size=500) \
.query("match_phrase", name={"query": "john".casefold()})\
.query("match", age="46")
This will search for a data record that contains "John white" and "46". However, If the age is not correct, I would like to get a record that contains "John white" and age that is the closest to "46" (assuming I have those records, otherwise it will return nothing).
The above query however only returns records of age EXACTLY "46".
A similar question already exists on SO: how to find the nearest / closest number using Query DSL in elasticsearch
But I am not sure how to incorporate the JSON in my query since I am using specific python modules.
A case in point is the fact that I can use fuzziness on a string. But I think fuzziness on an integer is not possible in the same manner in elasticsearch.
I would recommend using script based sorting to accomplish this, as described here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting
Working under the assumption that you're only matching on the first name - if you want to match the name exactly, I'd recommend using a filter based match. I used three different 'users' in the index, defined as follows:
POST index1/_doc
{
"name": "John White",
"age": 46
}
POST index1/_doc
{
"name": "John White",
"age": 40
}
POST index1/_doc
{
"name": "John Black",
"age": 47
}
I find it easier to write something a little more complex like this using Kibana's Dev Tools for testing, and then convert it to the Python Elasticsearch DSL compatible format - so in Kibana, I ultimately came up with the following:
GET index1/_search
{
"query": {
"match_phrase": {
"name": {
"query": "john"
}
}
},
"sort": {
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": "Math.abs(doc['age'].value - params.target_age)",
"params": {
"target_age": 46
}
},
"order": "asc"
}
}
}
Note using the absolute value of the difference will give you the closest value in either direction (i.e. younger or older). Some tweaks might be necessary if your requirements are different. Simply adjust the parameter as your queries change to accommodate different target ages.
Once tested and validated, converting to Python Elasticsearch DSL is quite easy - you can use the 'Auto Indent' function to flatten the complexity of the sort and drop it right into your existing statement.
s = Search(using=es, index="index1").extra(size=500) \
.query("match_phrase", name={"query": "john".casefold()}) \
.sort({"_script":{"type":"number","script":{"lang":"painless","source": \
"Math.abs(doc['age'].value - params.target_age)", \
"params":{"target_age":46}},"order":"asc"}})
Executing this returns the expected response:
<Response: [<Hit(index1/_doc/VR3e7WkBsHIsqLp6vfx_): {'name': 'John White', 'age': 46}>, <Hit(index1/_doc/Vx3f7WkBsHIsqLp6DPxM): {'name': 'John Black', 'age': 47}>, <Hit(index1/_doc/Vh3e7WkBsHIsqLp6yfxd): {'name': 'John White', 'age': 40}>]>
However, as you indicated you want the closest value, I'd recommend changing the size parameter to 1.
I have JSON output as follows:
{
"service": [{
"name": ["Production"],
"id": ["256212"]
}, {
"name": ["Non-Production"],
"id": ["256213"]
}]
}
I wish to find all ID's where the pair contains "Non-Production" as a name.
I was thinking along the lines of running a loop to check, something like this:
data = json.load(urllib2.urlopen(URL))
for key, value in data.iteritems():
if "Non-Production" in key[value]: print key[value]
However, I can't seem to get the name and ID from the "service" tree, it returns:
if "Non-Production" in key[value]: print key[value]
TypeError: string indices must be integers
Assumptions:
The JSON is in a fixed format, this can't be changed
I do not have root access, and unable to install any additional packages
Essentially the goal is to obtain a list of ID's of non production "services" in the most optimal way.
Here you go:
data = {
"service": [
{"name": ["Production"],
"id": ["256212"]
},
{"name": ["Non-Production"],
"id": ["256213"]}
]
}
for item in data["service"]:
if "Non-Production" in item["name"]:
print(item["id"])
Whatever I see JSON I think about functionnal programming ! Anyone else ?!
I think it is a better idea if you use function like concat or flat, filter and reduce, etc.
Egg one liner:
[s.get('id', [0])[0] for s in filter(lambda srv : "Non-Production" not in srv.get('name', []), data.get('service', {}))]
EDIT:
I updated the code, even if data = {}, the result will be [] an empty id list.
I have a json response from an API in this way:-
{
"meta": {
"code": 200
},
"data": {
"username": "luxury_mpan",
"bio": "Recruitment Agents👑👑👑👑\nThe most powerful manufacturers,\nwe have the best quality.\n📱Wechat:13255996580💜💜\n📱Whatsapp：+8618820784535",
"website": "",
"profile_picture": "https://scontent.cdninstagram.com/t51.2885-19/10895140_395629273936966_528329141_a.jpg",
"full_name": "Mpan",
"counts": {
"media": 17774,
"followed_by": 7982,
"follows": 7264
},
"id": "1552277710"
}
}
I want to fetch the data in "media", "followed_by" and "follows" and store it in three different lists as shown in the below code:--
for r in range(1,5):
var=r,st.cell(row=r,column=3).value
xy=var[1]
ij=str(xy)
myopener=Myopener()
url=myopener.open('https://api.instagram.com/v1/users/'+ij+'/?access_token=641567093.1fb234f.a0ffbe574e844e1c818145097050cf33')
beta=json.load(url)
for item in beta['data']:
list1.append(item['media'])
list2.append(item['followed_by'])
list3.append(item['follows'])
When I run it, it shows the error TypeError: string indices must be integers
How would my loop change in order to fetch the above mentioned values?
Also, Asking out of curiosity:- Is there any way to fetch the Watzapp no from the "BIO" key in data dictionary?
I have referred questions similar to this and still did not get my answer. Please help!
beta['data'] is a dictionary object. When you iterate over it with for item in beta['data'], the values taken by item will be the keys of the dictionary: "username", "bio", etc.
So then when you ask for, e.g., item['media'] it's like asking for "username"['media'], which of course doesn't make any sense.
It isn't quite clear what it is that you want: is it just the stuff inside counts? If so, then instead of for item in beta['data']: you could just say item = beta['data']['counts'], and then item['media'] etc. will be the values you want.
As to your secondary question: I suggest looking into regular expressions.