We Keep Coding

Iterating over each json in the array

I have a query for SQL from which I need to prepare for each line json which I will use as a payload json for HTTP request. This json I need to rebuild a little bit, because for some keys I need to add another level of json. This is not a problem.
The problem is that I don't know how to iterate over each such object/row in the json. I need to do a for each and output as payload to separate HTTP requests. At this point I have:
result = []
for row in rows:
d = dict()
d['id'] = row[40]
d['email'] = row[41]
d['additional_level'] = dict()
d['additional_level']['key'] = row[42]
result.append(d)
payload = json.dumps(result, indent=3)
At this point, print(payload) looks like this:
[
{
"id": 01,
"email": "someemail01#gmail.com"
"additional_level": {
"key": 10
}
},
{
"id": 02,
"email": "someemail02#gmail.com"
"additional_level": {
"key": 10
}
}
]
Now I want to make a separate payload json from each id and use in request http. How can I refer to them and how to make for loop to separately refer to each "object" separately?

You can dump each object separately:
for row in rows:
d = dict()
d['id'] = row[40]
d['email'] = row[41]
d['additional_level'] = dict()
d['additional_level']['key'] = row[42]
result.append(json.dumpsd, indent = 3)
And then iterate over them and use them individually:
for payload in result:
# Use payload for a request

Convert json to formatted string

I want to convert this json :
{
"rate_limit_by":
[{ "type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
}
to this:
"{\"rate_limit_by\": [{\"type\": \"IP\", \"extract_from_header\": \"X-Forwarded-For\"}]}".
So that i can send it as part of payload in request in Python.
And i have tried multiple methods for the same. json.dumps doesnt work cause it doesnt escape characters in this case & .replace(""",r"\"") doesnt work cause it creates the string like this :
{\\"rate_limit_by\\": [{\\"type\\": \\"IP\\", \\"extract_from_header\\": \\"X-Forwarded-For\\"}]}
(Below is the example of curl but i want to send the data in specific format using python request.)
My upstream expects data in certain format, as of now am sending data to upstream as below:
curl -i --request POST --data "rule_name=only_ip" \
--data-binary "#data.txt" \
--url http://localhost:8001/plugin/rules
Where data.txt looks like this:
rule={
"rate_limit_by": [
{ "type":"IP", "extract_from_header": "X-Forwarded-For" }
]
}
Am trying to convert it to :
curl -i --request POST -H 'Content-Type: application/json' --data-binary #data.json http://localhost:8001/plugin/rules
Where data.json should like this
{
"rule_name" : "test_ip",
"rule":"{\"rate_limit_by\": [{\"type\": \"IP\", \"extract_from_header\": \"X-Forwarded-For\"}]}"
}
Now the value of "rule" is string with character escape.
This am trying to achieve & am doing post using python.
And below is the code for same:-
import requests
import json
import re
url = 'http://localhost:8001/plugin/rules'
rule = {
"rate_limit_by":
[{ "type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
}
rule = json.dumps(json.dumps(rule))
print(rule) #this output the data in correct format
obj = {
"rule_name" : "test_ip",
"rule": rule #but when used it here its get wrapped in two \\
}
headers = {'Content-Type': 'application/json', 'Accept': 'text/plain'}
print(obj)
r = requests.post(url, data=obj, headers=headers)
print(r.text)

desired is what you say you need in your something.json file. The following prints True. See https://repl.it/repls/DistantTeemingProtocol.
import json
desired = r'''{
"rule_name" : "test_ip",
"rule":"{\"rate_limit_by\": [{\"type\": \"IP\", \"extract_from_header\": \"X-Forwarded-For\"}]}"
}'''
d = {
"rate_limit_by": [{
"type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
}
s = json.dumps(d)
xxx = json.dumps({"rule_name": "test_ip", "rule": s}, indent=4)
o = json.loads(desired)
yyy = json.dumps(o, indent=4)
print(xxx == yyy)
If you're going to POST using requests, then you should not be posting the string but should instead be posting the dictionary.
I.e.,
r = requests.post(url, json={"rule_name": "test_ip", "rule": s})

Do you mean you want to access the items inside somehow?
You should drop the "[]" because that part doesn't really make sense.
import json
x = str({
"rate_limit_by":
[{ "type": "IP",
"extract_from_header": "X-Forwarded-For"
}]
})
x = x.replace("[","")
x = x.replace("]","")
x = eval(x)
d = json.dumps(x)
l = json.loads(d)
l['rate_limit_by']['type']
This outputs "IP". Now you have a dictionary of all you need called l.

Listing data inside a JSON object

I am trying to extract data from an API with python.
With this code, I am trying to print the content of dictionaries within a list.
response = requests.get(url, params=payload)
data = response.json()
log = json.dumps(data['result'], indent = 1)
print log
So far so good, it prints:
[
{
"line": "something#gmail.com:birthdaydate"
},
{
"line": "something#gmail.com:birthdaydate"
},
{
"line": "something#gmail.com:birthdaydate"
},
]
Is there a way for the output to look like below?
"something#gmail.com:birthdaydate"
"something#gmail.com:birthdaydate"
"something#gmail.com:birthdaydate"

Try this one:
response = requests.get(url, params=payload)
data = response.json()
log = data['result']
for l in log:
print(l["line"])

parsing and getting list from response of get request

I'm trying to parse a website with the requests module:
import requests
some_data = {'a':'',
'b':''}
with requests.Session() as s:
result = s.post('http://website.com',data=some_data)
print(result.text)
The page is responding as below:
{
"arrangetype":"U",
"list": [
{
"product_no":43,
"display_order":4,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
},
{
"product_no":44,
"display_order":6,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
}
],
"length":2
}
I found that instead of parsing full HTML, it would be better to deal with the response as all the data I want is in that response.
What I want to get is a list of the values of product_no, so the expected result is:
[43,44]
How do I do this?

Convert your JSON response to a dictionary with json.loads(), and collect your results in a list comprehension.
Demo:
from json import loads
data = """{
"arrangetype":"U",
"list": [
{
"product_no":43,
"display_order":4,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
},
{
"product_no":44,
"display_order":6,
"is_selling":"T",
"product_empty":"F",
"fix_position":null,
"is_auto_sort":false
}
],
"length":2
}"""
json_dict = loads(data)
print([x['product_no'] for x in json_dict['list']])
# [43, 44]
Full Code:
import requests
from json import loads
some_data = {'a':'',
'b':''}
with requests.Session() as s:
result = s.post('http://website.com',data=some_data)
json_dict = loads(result.text)
print([x["product_no"] for x in json_dict["list"]])

List indicies must be integers. not str JSON reponse

I am baffled and do not know how to solve this error. I'm trying to grab every name inside a JSON response list.
My code looks like this.
def extract_strucutres_ids(expected_structures):
response = requests.get(JIRA_REST + "/structures", verify=False)
response = response.json()
for structure in response['structures']:
print structure['name']
The Json reponse looks like this.
{
"structures": [{
"id": 165,
"name": "6.2 External Notifications Refactor",
"description": ""
}, {
"id": 364,
"name": "6.4 Day/Night Mode and Idle Scene Mode",
"description": "",
"readOnly": true
}, {
"id": 140,
"name": "ACC 5 Regression",
"description": ""
}
]
}
I keep getting List indicies must be integers, not str.
Python version 2.7.10

try this -
import json
def extract_strucutres_ids(expected_structures):
response = requests.get(JIRA_REST + "/structures", verify=False)
if response.status_code==200:
response_json = json.loads(response.text)
for structure in response_json['structures']:
print structure['name']
else:
print("Response is {}".format(response.status_code))
Let me know,if it worked!

Use json.loads()
response = requests.get(..)
response = json.loads(response.text) # response.text is a string
for structure in response['structures']:
# Do something