We Keep Coding

Iterate / Loop thru a json file using python multiple times

Ive a json file,
{
"IGCSE":[
{
"rolename": "igcsesubject1",
"roleid": 764106550863462431
},
{
"rolename": "igcsesubject2",
"roleid": 764106550863462431
}
],
"AS":[
{
"rolename": "assubject1",
"roleid": 854789476987546
},
{
"rolename": "assubject2",
"roleid": 854789476987546
}
],
"A2":[
{
"rolename": "a2subject1",
"roleid": 854789476987856
},
{
"rolename": "a2subject2",
"roleid": 854789476987856
}
]
}
I want to fetch the keys [igcse, as, a2..] and then fetch the rolename and roleids under the specific keys. How do i do it?
Below is the python code for how i used to do it without the keys.
with open(fileloc) as f:
data = json.load(f)
for s in range(len(data)):
d1 = data[s]
rname = d1["rolename"]
rid = d1["roleid"]
any help would be appreciated :)

First you can have a list of keys, under which you will get them:
l = ['A1','A2']
Then iterate like this:
for x in data:
if x in l:
for y in range(len(data[x])):
print(j[x][y]['rolename'])
print(j[x][y]['roleid'])

hi you can use for and you will get the keys:
with open(fileloc) as f:
data = json.load(f)
for s in data:
d1 = data[s]
rname = d1["rolename"]
rid = d1["roleid"]

The following would work for what you need:
with open(file) as f:
json_dict = json.load(f)
for key in json_dict:
value_list = json_dict[key]
for item in value_list:
rname = item["rolename"]
rid = item["roleid"]
If you need to filter for specific keys in the JSON, you can have a list of keys you want to obtain and filter for those keys as you iterate through the keys (similar to Wasif Hasan's suggestion above).

How to get an element value in Python dictionary

I am trying to get the value of DomainName from the below dictionary.
print(domain_name)
# output
{
'DomainNames': [
{
'DomainName': 'some-value'
},
]
}
I have tried:
print(domain_name['DomainNames'][0]['DomainName'])
but it doesn't give that value. I even tried:
print(domain_name['DomainNames']['DomainName'])
Here is my code:
def add_es_tags():
for region in get_regions_depending_on_account():
pass
es_client = boto3.client('es', region_name="us-east-1")
response = es_client.list_domain_names()
get_es_domain_ARN("us-east-1", response)
def get_es_domain_ARN(region, domain_names):
es_client = boto3.client('es', region_name=region)
arns = []
print(len(domain_names))
for domain_name in domain_names:
# print(type(domain_name))
print(domain_name['DomainNames'][0]['DomainName'])

Like this:
domain_name = {
'DomainNames': [
{
'DomainName': 'some-value'
},
]
}
print(domain_name)
print(domain_name['DomainNames'][0]['DomainName'])
Yes, the answer is: it works exactly as you suggested!
Edit: Never mind, I'll update this when you've formulated a full question that actually matches what you're doing.

Python GraphQL query issue

I'm using Python to make requests to Pipefy GraphQL API.
I already read the documentation and make search in pipefy forum, but
I could not figure what is wrong with the query bellow:
pipeId = '171258'
query ="""
{
"query": "{allCards(pipeId: %s, first: 30, after: 'WyIxLjAiLCI1ODAuMCIsMzI0OTU0NF0'){pageInfo{endCursor hasNextPage}edges{node{id title}}}}"
}
"""%(pipeid)
The query worked pretty well until I added the after parameter.
I already tried variations like:
after: "WyIxLjAiLCI1ODAuMCIsMzI0OTU0NF0"
after: \"WyIxLjAiLCI1ODAuMCIsMzI0OTU0NF0\"
after: \n"WyIxLjAiLCI1ODAuMCIsMzI0OTU0NF0\n"
I know the issue is related with the escaping, because the API return messages like this:
'{"errors":[{"locations":[{"column":45,"line":1}],"message":"token recognition error at: \'\'\'"},{"locations":[{"column":77,"line":1}],"message":"token recognition error at: \'\'\'"}]}\n'
(this message is returned when the request is made with after: 'WyIxLjAiLCI1ODAuMCIsMzI0OTU0NF0')
Any help here would be immensely handful!
Thanks

I had the same problem as you today (and saw your post on Pipefy's Support page). I personally entered in contact with Pipefy's developers but they weren't helpful at all.
I solved it by escaping the query correctly.
Try like this:
query = '{"query": "{ allCards(pipeId: %s, first: 30, after: \\"WyIxLjAiLCI1ODAuMCIsMzI0OTU0NF0\\"){ pageInfo{endCursor hasNextPage } edges { node { id title } } } }"}'
Using single quotes to define the string and double-backslashes before the doublequotes included in the cursor.

With the code snippet below you are able to call the function get_card_list passing the authentication token (as String) and the pipe_id (as integer) and retrieve the whole card list of your pipe.
The get_card_list function will call the function request_card_list until the hasNextpage is set to False, updating the cursor in each call.
# Function responsible to get cards from a pipe using Pipefy's GraphQL API
def request_card_list(auth_token, pipe_id, hasNextPage=False, endCursor=""):
url = "https://api.pipefy.com/graphql"
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer %s' %auth_token
}
if not hasNextPage:
payload = '{"query": "{ allCards(pipeId: %i, first: 50) { edges { node { id title phases_history { phase { name } firstTimeIn lastTimeOut } } cursor } pageInfo { endCursor hasNextPage } } }"}' %pipe_id
else:
payload = '{"query": "{ allCards(pipeId: %i, first: 50, after: \\"%s\\") { edges { node { id title phases_history { phase { name } firstTimeIn lastTimeOut } } cursor } pageInfo { endCursor hasNextPage } } }"}' % (pipe_id, endCursor)
response = requests.request("POST", url, data=payload, headers=headers)
response_body = response.text
response_body_dict = json.loads(response_body)
response_dict_list = response_body_dict['data']['allCards']['edges']
card_list = []
for d in response_dict_list:
for h in d['node']['phases_history']:
h['firstTimeIn'] = datetime.strptime(h['firstTimeIn'], date_format)
if h['lastTimeOut']:
h['lastTimeOut'] = datetime.strptime(h['lastTimeOut'], date_format)
card_list.append(d['node'])
return_list = [card_list, response_body_dict['data']['allCards']['pageInfo']['hasNextPage'], response_body_dict['data']['allCards']['pageInfo']['endCursor']]
return return_list
# Function responsible to get all cards from a pipe using Pipefy's GraphQL API and pagination
def get_card_list(auth_token, pipe_id):
card_list = []
response = request_card_list(auth_token, pipe_id)
card_list = card_list + response[0]
while response[1]:
response = request_card_list(auth_token, pipe_id, response[1], response[2])
card_list = card_list + response[0]
return(card_list)

Thanks for Lodi answer, I was able to do the next step.
How to use a variable to pass the "after" parameter for the query
As it was quite difficult I decide to share it here for those facing the same challenge.
end_cursor = 'WyIxLjAiLCI2NTcuMCIsNDgwNDA2OV0'
end_cursor = "\\" + "\"" + end_cursor + "\\" + "\""
# desired output: end_cursor = '\"WyIxLjAiLCI2NTcuMCIsNDgwNDA2OV0\"'
query ="""
{
"query": "{allCards(pipeId: %s, first: 50, after: %s){pageInfo{endCursor hasNextPage}edges{node{id title}}}}"
}
"""%(pipeid, end_cursor)

How to search for specific value in Json array using Python

I'm using a website's API to get information. It outputs 100 values.
What is the best practice of searching for a specific value in this array?
The array looks something like this:
{
"success":true,
"data":
{
"array":
[
{
"id":"1","name":"Value1"
},
{
"id":"2","name":"Value2"
}
]
}
}
I try searching for the data using this snippet I found elsewhere, however it does not work:
for name in (r.json()['data']['array'][0]['name']):
if r.json()['data']['array'][0] == s_name:
ident = r.json()['data']['array'][0]['id']
name = r.json()['data']['array'][0]['name']
print (name + ' - ' + ident)
else:
print ('Nothing was found.')
So if I was trying to reference Value1 how would I do this using Python?
Please note: I'm a rookie developer, first time using Json, not very experienced with Python.
All help is greatly appreciated.

Here is a one-liner example for searching:
aaa = {
"success":True,
"data":
{
"array":
[
{
"id":"1","name":"Value1"
},
{
"id":"2","name":"Value2"
}
]
}
}
[a['name'] for a in aaa['data']['array'] if a['id']=='1']
This will return all the found cases, or an empty array if nothing is found

Not sure where that piece of code came from but it looks very wrong.
Just looking at the structure you can do something like:
for attrs in r.json()['data']['array']:
if attrs['name'] == s_name:
ident = attrs['id']
name = attrs['name']
print(name, '-', ident)
break
else:
print('Nothing found!')

Your code can be simplified a lot:
# no need to call `r.json` so many times, can simply save it to a variable
json_data = r.json()
for item in json_data["data"]["array"]:
if item["name"] == "Value1":
# do something...

Querying ElasticSearch with Python Requests not working fine

I'm trying to do full-text search on a mongodb db with the Elastic Search engine but I ran into a problem: no matters what search term I provide(or if I use query1 or query2), the engine always returns the same results. I think the problem is in the way I make the requests, but I don't know how to solve it.
Here is the code:
def search(search_term):
query1 = {
"fuzzy" : {
"art_text" : {
"value" : search_term,
"boost" : 1.0,
"min_similarity" : 0.5,
"prefix_length" : 0
}
},
"filter": {
"range" : {
"published": {
"from" : "20130409T000000",
"to": "20130410T235959"
}
}
}
}
query2 = {
"match_phrase": { "art_text": search_term }
}
es_query = json.dumps(query1)
uri = 'http://localhost:9200/newsidx/_search'
r = requests.get(uri, params=es_query)
results = json.loads( r.text )
data = [res['_source']['api_id'] for res in results['hits']['hits'] ]
print "results: %d" % len(data)
pprint(data)

The params parameter is not for data being sent. If you're trying to send data to the server you should specifically be using the data parameter. If you're trying to send query parameters, then you shouldn't be JSON-encoding them and just give it to params as a dict.
I suspect your first request should be the following:
r = requests.get(uri, data=es_query)
And before someone downvotes me, yes the HTTP/1.1 spec allows data to be sent with GET requests and yes requests does support it.

search = {'query': {'match': {'test_id':13} }, 'sort' {'date_utc':{'order':'desc'}} }
data = requests.get('http://localhost:9200/newsidx/test/_search?&pretty',params = search)
print data.json()
http://docs.python-requests.org/en/latest/user/quickstart/