We Keep Coding

Python GraphQL API call composition

I've recently started learning how to use python and i'm having some trouble with a graphQL api call.
I'm trying to set up a loop to grab all the information using pagination, and my first request is working just fine.
values = """
{"query" : "{organizations(ids:) {pipes {id name phases {id name cards_count cards(first:30){pageInfo{endCursor hasNextPage} edges {node {id title current_phase{name} assignees {name} due_date createdAt finished_at fields{name value filled_at updated_at} } } } } }}}"}
"""
but the second call using the end cursor as a variable isn't working for me. I assume that it's because i'm not understanding how to properly escape the string of the variable. But for the life of me I'm unable to understand how it should be done.
Here's what I've got for it so far...
values = """
{"query" : "{phase(id: """ + phaseID+ """ ){id name cards_count cards(first:30, after:"""\" + pointer + "\"""){pageInfo{endCursor hasNextPage} edges {node {id title assignees {name} due_date createdAt finished_at fields{name value datetime_value updated_at phase_field { id label } } } } } } }"}
"""
the second one as it loops just returns a 400 bad request.
Any help would be greatly appreciated.

As a general rule you should avoid building up queries using string manipulation like this.
In the GraphQL query itself, GraphQL allows variables that can be placeholders in the query for values you will plug in later. You need to declare the variables at the top of the query, and then can reference them anywhere inside the query. The query itself, without the JSON wrapper, would look something like
query = """
query MoreCards($phase: ID!, $cursor: String) {
phase(id: $phase) {
id, name, cards_count
cards(first: 30, after: $cursor) {
... CardConnectionData
}
}
}
"""
To actually supply the variable values, they get passed as an ordinary dictionary
variables = {
"phase": phaseID,
"cursor": pointer
}
The actual request body is a straightforward JSON structure. You can construct this as a dictionary too:
body = {
"query": query,
"variables": variables
}
Now you can use the standard json module to format it to a string
print(json.dumps(body))
or pass it along to something like the requests package that can directly accept the object and encode it for you.

I had a similar situation where I had to aggregate data through paginating from a GraphQL endpoint. Trying the above solution didn't work for me that well.
to start my header config for graphql was like this:
headers = {
"Authorization":f"Bearer {token}",
"Content-Type":"application/graphql"
}
for my query string, I used the triple quote with a variable placeholder:
user_query =
"""
{
user(
limit:100,
page:$page,
sort:[{field:"email",order:"ASC"}]
){
list{
email,
count
}
}
"""
Basically, I had my loop here for the pages:
for page in range(1, 9):
formatted_query = user_query.replace("$page",f'{page}')
response = requests.post(api_url, data=formatted_query,
headers=headers)
status_code, json = response.status_code, response.json()

How to remove the first and last portion of a string in Python?

How can i cut from such a string (json) everything before and including the first [ and everything behind and including the last ] with Python?
{
"Customers": [
{
"cID": "w2-502952",
"soldToId": "34124"
},
...
...
],
"status": {
"success": true,
"message": "Customers: 560",
"ErrorCode": ""
}
}
I want to have at least only
{
"cID" : "w2-502952",
"soldToId" : "34124",
}
...
...

String manipulation is not the way to do this. You should parse your JSON into Python and extract the relevant data using normal data structure access.
obj = json.loads(data)
relevant_data = obj["Customers"]

Addition to #Daniel Rosman answer, if you want all the list from JSON.
result = []
obj = json.loads(data)
for value in obj.values():
if isinstance(value, list):
result.append(*value)

While I agree that Daniel's answer is the absolute best way to go, if you must use string splitting, you can try .find()
string = #however you are loading this json text into a string
start = string.find('[')
end = string.find(']')
customers = string[start:end]
print(customers)
output will be everything between the [ and ] braces.

If you really want to do this via string manipulation (which I don't recommend), you can do it this way:
start = s.find('[') + 1
finish = s.find(']')
inner = s[start : finish]

Python Proper JSON Format

I need to post data to a REST API. One field, incident_type, needs to be passed in the below JSON format ( must include brackets, can't be just curly brackets ):
"incident_type_ids": [{
"name": "Phishing - General"
}],
When I try to force this in my code, it doesn't come out quite right. There will usually be some extra quote-escapes ( ex. output: "incident_type_ids": "[\\"{ name : Phishing - General }\\"]")and I realized that was because I was double-encoding the JSON data in the incident type variable to forcibly add the brackets ( in line 6 which has since been commented out ):
#incident variables
name = 'Incident Name 2'
description = 'This is the description'
corpID = 'id'
incident_type = '{ name : Phishing - General }'
#incident_type = json.dumps([incident_type])
incident_owner = 'Security Operations Center'
payload = {
'name':name,
'discovered_date':'0',
'owner_id':incident_owner,
'description':description,
'exposure_individual_name':corpID,
'incident_type_ids':incident_type
}
body=json.dumps(payload)
create = s.post(url, data=body, headers=headers, verify=False)
However since I commented out the line, I can't get incident_type in the format I need ( with brackets ).
So, my question is: How can I get the incident_type variable in the proper format in the final payload?
Input I manually got to work using product's interactive REST API:
{
"name": "Incident Name 2",
"incident_type_ids": [{
"name": "Phishing - General"
}],
"description": "This is the description",
"discovered_date": "0",
"exposure_individual_name": "id",
"owner_id": "Security Operations Center"
}
I figure my approach is wrong and I'd appreciate any help. I'm new to Python so I'm expecting this is a beginner's mistake.
Thanks for your help.

JSON square brackets are for arrays, which correspond to Python lists. JSON curly braces are for objects, which correspond to Python dictionaries.
So you need to create a list containing a dictionary, then convert that to JSON.
incident_type = [{"name": "Phishing - General"}]
incident_owner = 'Security Operations Center'
payload = {
'name':name,
'discovered_date':'0',
'owner_id':incident_owner,
'description':description,
'exposure_individual_name':corpID,
'incident_type_ids':incident_type
}
body=json.dumps(payload)
It's only slightly coincidental that the Python syntax for this is similar to the JSON syntax.

How to get json data by name using python

I'm kinda new JSON and python and i wish to use the keys and values of JSON to compare it.
I'm getting the JSON from a webpage using requests lib.
Recently, I've done this:
import requests;
URL = 'https://.../update.php';
PARAMS = { 'platform':'0', 'authcode':'code', 'app':'60' };
request = requests.get( url=URL, params=PARAMS );
data = request.json( );
I used this loop to get the keys and values from that json:
for key, value in data.items( ):
print( key, value );
it return JSON part like this:
rescode 0
desc success
config {
"app":"14",
"os":"2",
"version":"6458",
"lang":"zh-CN",
"minimum":"5",
"versionName":"3.16.0-6458",
"md5":"",
"explain":"",
"DiffUpddate":[ ]
}
But in Firefox using pretty print i get different result look like this:
{
"rescode": 0,
"desc": "success",
"config": "{
\n\t\"app\":\"14\",
\n\t\"os\":\"2\",
\n\t\"version\":\"6458\",
\n\t\"lang\":\"zh-CN\",
\n\t\"minimum\":\"5\",
\n\t\"versionName\":\"3.16.0-6458\",
\n\t\"md5\":\"\",
\n\t\"explain\":\"\",
\n\t\"DiffUpddate\":[\n\t\t\n\t]\n
}"
}
What I'm planing to do is:
if data['config']['version'] == '6458':
print('TRUE!');
But everytime i get this error:
TypeError: string indices must be integers

You need to parse the config
json.loads(data['config'])['version']
Or edit the PHP to return an associative array rather than a string for the config object

String format a JSON string gives KeyError

Why does this code give a KeyError?
output_format = """
{
"File": "{filename}",
"Success": {success},
"ErrorMessage": "{error_msg}",
"LogIdentifier": "{log_identifier}"
}
"""
print output_format.format(filename='My_file_name',
success=True,
error_msg='',
log_identifier='123')
Error message:
KeyError: ' "File"'

You need to double the outer braces; otherwise Python thinks { "File".. is a reference too:
output_format = '{{ "File": "{filename}", "Success": {success}, "ErrorMessage": "{error_msg}", "LogIdentifier": "{log_identifier}" }}'
Result:
>>> print output_format.format(filename='My_file_name',
... success=True,
... error_msg='',
... log_identifier='123')
{ "File": "My_file_name", "Success": True, "ErrorMessage": "", "LogIdentifier": "123" }
If, indicentally, you are producing JSON output, you'd be better off using the json module:
>>> import json
>>> print json.dumps({'File': 'My_file_name',
... 'Success': True,
... 'ErrorMessage': '',
... 'LogIdentifier': '123'})
{"LogIdentifier": "123", "ErrorMessage": "", "Success": true, "File": "My_file_name"}
Note the lowercase true in the output, as required by the JSON standard.

As mentioned by Tudor in a comment to another answer, the Template class was the solution that worked best for me. I'm dealing with nested dictionaries or list of dictionaries and handling those were not as straightforward.
Using Template though the solution is quite simple.
I start with a dictionary that is converted into a string. I then replace all instances of { with ${ which is the Template identifier to substitute a placeholder.
The key point of getting this to work is using the Template method safe_substitute. It will replace all valid placeholders like ${user_id} but ignore any invalid ones that are part of the dictionary structure, like ${'name': 'John', ....
After the substitution is done I remove any leftovers $ and convert the string back to a dictionary.
In the code bellow, resolve_placeholders returns a dictionary where each key matches a placeholder in the payload string and the value is substituted by the Template class.
from string import Template
.
.
.
payload = json.dumps(payload)
payload = payload.replace('{', '${')
replace_values = self.resolve_placeholders(payload)
if replace_values:
string_template = Template(payload)
payload = string_template.safe_substitute(replace_values)
payload = payload.replace('${', '{')
payload = json.loads(payload)

To extend on Martijn Pieters answer and comment:
According to MArtijn' comment, escaping the {..} pairs that are not placeholders is they way to go with nested dictionaries. I haven't succeded in doing that, so I suggest the following method.
For nested dictionaries I tried doubling up on any { and } of the nested dictionaries.
a='{{"names":{{"a":"{name}"}}}}'
a.format(name=123) output:
output: '{"names":{"a":"123"}}'
But this makes using format to change values inside a json string, a over-complex method, so I use a twist on the format command.
I replace ${param_name} in a json string. For example:
My predefined JSON looks like this:
my_json_dict = {
'parameter': [
{
'name': 'product',
'value': '${product}'
},
{
'name': 'suites',
'value': '${suites}'
},
{
'name': 'markers',
'value': '${markers}'
}
]
}
I provide this dictionary as values to replace instead of the parameters
parameters = {
'product': 'spam',
'suites': 'ham',
'markers': 'eggs'
}
And use this code to do the replacment
json_str = json.dumps(my_json_dict)
for parameter_name, parameter_value in parameters.iteritems():
parameter_name = '${'+parameter_name+'}'
json_str = json_str.replace(parameter_name, parameter_value)
json_dict = json.loads(json_str)