I've recently started learning how to use python and i'm having some trouble with a graphQL api call.
I'm trying to set up a loop to grab all the information using pagination, and my first request is working just fine.
values = """
{"query" : "{organizations(ids:) {pipes {id name phases {id name cards_count cards(first:30){pageInfo{endCursor hasNextPage} edges {node {id title current_phase{name} assignees {name} due_date createdAt finished_at fields{name value filled_at updated_at} } } } } }}}"}
"""
but the second call using the end cursor as a variable isn't working for me. I assume that it's because i'm not understanding how to properly escape the string of the variable. But for the life of me I'm unable to understand how it should be done.
Here's what I've got for it so far...
values = """
{"query" : "{phase(id: """ + phaseID+ """ ){id name cards_count cards(first:30, after:"""\" + pointer + "\"""){pageInfo{endCursor hasNextPage} edges {node {id title assignees {name} due_date createdAt finished_at fields{name value datetime_value updated_at phase_field { id label } } } } } } }"}
"""
the second one as it loops just returns a 400 bad request.
Any help would be greatly appreciated.
As a general rule you should avoid building up queries using string manipulation like this.
In the GraphQL query itself, GraphQL allows variables that can be placeholders in the query for values you will plug in later. You need to declare the variables at the top of the query, and then can reference them anywhere inside the query. The query itself, without the JSON wrapper, would look something like
query = """
query MoreCards($phase: ID!, $cursor: String) {
phase(id: $phase) {
id, name, cards_count
cards(first: 30, after: $cursor) {
... CardConnectionData
}
}
}
"""
To actually supply the variable values, they get passed as an ordinary dictionary
variables = {
"phase": phaseID,
"cursor": pointer
}
The actual request body is a straightforward JSON structure. You can construct this as a dictionary too:
body = {
"query": query,
"variables": variables
}
Now you can use the standard json module to format it to a string
print(json.dumps(body))
or pass it along to something like the requests package that can directly accept the object and encode it for you.
I had a similar situation where I had to aggregate data through paginating from a GraphQL endpoint. Trying the above solution didn't work for me that well.
to start my header config for graphql was like this:
headers = {
"Authorization":f"Bearer {token}",
"Content-Type":"application/graphql"
}
for my query string, I used the triple quote with a variable placeholder:
user_query =
"""
{
user(
limit:100,
page:$page,
sort:[{field:"email",order:"ASC"}]
){
list{
email,
count
}
}
"""
Basically, I had my loop here for the pages:
for page in range(1, 9):
formatted_query = user_query.replace("$page",f'{page}')
response = requests.post(api_url, data=formatted_query,
headers=headers)
status_code, json = response.status_code, response.json()
Related
I have a mongo database including the following collection:
"
"_id": {
"$oid": "12345"
},
"id": "333555",
"token": [
{
"access_token": "ac_33bc",
"expires_in": 3737,
"token_type": "bearer",
"expires_at": {
"$date": "2021-07-02T13:37:28.123Z"
}
}
]
}
In the next python script I'm trying to return and print only the access_token but can't figure out how to do so. I've tried various methods which none of the worked.I've given the "id" as a parameter
def con_mongo():
try:
client = pymongo.MongoClient("mongodb:localhost")
#DB name
db = client["db1"]
#Collection
coll = db["coll1"]
#1st method
x = coll.find({"id":"333555"},{"token":"access_token"})
for data in x:
print(x)
#2nd method
x= coll.find({"id":"333555"})
tok=x.distinct("access_token")
#print(x[0])
for data in tok:
print(data)
except Exception:
logging.info(Exception)
It doesn't work this way, although if I replace (or remove) the "access_token" with simply "token" it works but I get back all the informations included in the field "token" where I only need the value of the "access_token".
Since access_token is an array element, you need to qualify it's name with the name of the array, to properly access its value.
Actually you can first extract the whole document and get the desired value through simple list and dict indexing.
So, assuming you are retrieving many documents with that same id:
x = [doc["token"][0]["access_token"] for doc in coll.find({"id":"333555"})]
The above, comprehensively creates a list with the access_tokens of all the documents matching the given id.
If you just need the first (and maybe only) occurrence of a document with that id, you can use find_one() instead:
x = coll.find_one({"id":"333555"})["token"][0]["access_token"]
# returns ac_33bc
token is a list so you have to reference the list element, e.g.
x = coll.find({"id":"333555"},{"token.access_token"})
for data in x:
print(data.get('token')[0].get('access_token'))
prints:
ac_33bc
I'm kinda new JSON and python and i wish to use the keys and values of JSON to compare it.
I'm getting the JSON from a webpage using requests lib.
Recently, I've done this:
import requests;
URL = 'https://.../update.php';
PARAMS = { 'platform':'0', 'authcode':'code', 'app':'60' };
request = requests.get( url=URL, params=PARAMS );
data = request.json( );
I used this loop to get the keys and values from that json:
for key, value in data.items( ):
print( key, value );
it return JSON part like this:
rescode 0
desc success
config {
"app":"14",
"os":"2",
"version":"6458",
"lang":"zh-CN",
"minimum":"5",
"versionName":"3.16.0-6458",
"md5":"",
"explain":"",
"DiffUpddate":[ ]
}
But in Firefox using pretty print i get different result look like this:
{
"rescode": 0,
"desc": "success",
"config": "{
\n\t\"app\":\"14\",
\n\t\"os\":\"2\",
\n\t\"version\":\"6458\",
\n\t\"lang\":\"zh-CN\",
\n\t\"minimum\":\"5\",
\n\t\"versionName\":\"3.16.0-6458\",
\n\t\"md5\":\"\",
\n\t\"explain\":\"\",
\n\t\"DiffUpddate\":[\n\t\t\n\t]\n
}"
}
What I'm planing to do is:
if data['config']['version'] == '6458':
print('TRUE!');
But everytime i get this error:
TypeError: string indices must be integers
You need to parse the config
json.loads(data['config'])['version']
Or edit the PHP to return an associative array rather than a string for the config object
I am trying to pass json field as input for my graphql mutation.
I have been trying and searching but just no luck. I can pass array fine with I know by defining graphene.List(graphene.String) would work for passing array of strings.
I figured there's a type named graphene.JSONstring() which I thought would work if I use it with graphene.List(graphene.JSONstring) but no luck, still getting errors saying type is not right.
I have something like this during the mutation
mutation {
create(data:{
field1: [
{
"first": "first",
"last": "last"
},
{
"first":"first1",
"last":"last1"
}
]
})
}
as for input class
class NameInput(graphene.InputObjectType):
# please ignore the same field names, just listing what I have tried
field1 = graphene.JSONString()
field1 = graphene.List(graphene.JSONString)
field1 = graphene.List(graphene.String)
Does anyone has an idea how this would work?
Thanks in advance
Seems like you are trying to have nested input objects. Unfortunately I have never used graphene but maybe I can answer in terms of the GraphQL specification and then make an educated guess about the graphene code:
type Mutation {
create(data: NameInput): Boolean # <- Please don't return just Boolean
}
input NameInput {
field1: FistLastInput[]
}
input FirstLastInput {
first: String!
last: String!
}
This means you will need two input objects to describe the structure of your input. Create a new class for you object that takes the fields first and last:
class FirstLastInput(graphene.InputObjectType):
first = graphene.NonNull(graphene.String)
last = graphene.NonNull(graphene.String)
Now we can use the input object in our initial query:
class NameInput(graphene.InputObjectType):
field1 = graphene.List(FirstLastInput)
You could try like this:
class NameInput(graphene.InputObjectType):
field1 = graphene.JSONString()
And then:
mutation {
create(data:{
field1: "[
{
\"first\": \"first\",
\"last\": \"last\"
},
{
\"first\":\"first1\",
\"last\":\"last1\"
}
]"
})
}
So basically send json as string.
Graphene provides a GenericScalar type. You can use it to input/output generic types like int, str, dict, list, etc.
from graphene import InputObjectType, Mutation
from graphene.types.generic import GenericScalar
class InputObject(InputObjectType):
field1 = GenericScalar()
class Create(Mutation):
class Arguments:
data = InputObject()
def mutate(root, info, data):
# do something with data.field1
Then your input would look like
mutation {
create (
data: {
field1: [
{
first: "first",
last: "last"
},
{
first: "first1",
last: "last1"
}
]
}
)
}
Note that field1 can accept any generic input, so make sure to validate it.
Also, when using a GenericScalar field for output (query), you won't be able to query its subfields. But you can write a resolver for that field to make sure only specific subfields are returned.
Here is the link to the corresponding GitHub issue.
I'm trying to access certain fields of information in JSON dict. My code is set up as the following:
Views.py
def viewIssues(request):
r = requests.get(bucket_url)
issue_payload = r.json()
issue = json.loads(str(issue_payload))
context = {
"issue_title": issue['issues']['title'],
"issue_content": issue['issues']['content'],
"title": "View Issues",
}
return render(request, "view_issues.html", context)
str(issue_payload) gives me this:
{
'search':None,
'count':1,
'filter':{
},
'issues':[
{
'priority':'major',
'comment_count':0,
'utc_created_on':'2016-11-12 01:48:16+00:00',
'utc_last_updated':'2016-11-12 01:48:16+00:00',
'status':'new',
'title':'example issue',
'reported_by':{
'is_staff':False,
'display_name':'display name',
'is_team':False,
'resource_uri':'/1.0/users/username',
'avatar':'https://bitbucket.org/account/username/avatar/32/?ts=1479493904',
'first_name':'firstname',
'username':'username',
'last_name':'lastname'
},
'is_spam':False,
'content':'blah blah',
'metadata':{
'milestone':None,
'component':None,
'version':None,
'kind':'bug'
},
'local_id':1,
'created_on':'2016-11-12T02:48:16.052',
'resource_uri':'/1.0/repositories/username/supportal2016test/issues/1',
'follower_count':1
}
]
}
However when I try to use the json.loads and indices ['issues']['title'] and ['issues']['title'] I get an error:
JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
I'm wondering if it's because the converted payload has quotations on each field (i.e. 'issues'). Any help would be much appreciated.
The .json() call already parses the JSON result and returns a Python structure in this case a dictionary. Then your call
issue = json.loads(str(issue_payload))
forces the dictionary into a string and tries to parse it again. But the dictionary string representation contains ' around strings and not " as required in JSON.
To cut the long story short: issue_payload is what you want already.
Can someone tell me how to write Python statements that will aggregate (sum and count) stuff about my documents?
SCRIPT
from datetime import datetime
from elasticsearch_dsl import DocType, String, Date, Integer
from elasticsearch_dsl.connections import connections
from elasticsearch import Elasticsearch
from elasticsearch_dsl import Search, Q
# Define a default Elasticsearch client
client = connections.create_connection(hosts=['http://blahblahblah:9200'])
s = Search(using=client, index="attendance")
s = s.execute()
for tag in s.aggregations.per_tag.buckets:
print (tag.key)
OUTPUT
File "/Library/Python/2.7/site-packages/elasticsearch_dsl/utils.py", line 106, in __getattr__
'%r object has no attribute %r' % (self.__class__.__name__, attr_name))
AttributeError: 'Response' object has no attribute 'aggregations'
What is causing this? Is the "aggregations" keyword wrong? Is there some other package I need to import? If a document in the "attendance" index has a field called emailAddress, how would I count which documents have a value for that field?
First of all. I notice now that what I wrote here, actually has no aggregations defined. The documentation on how to use this is not very readable for me. Using what I wrote above, I'll expand. I'm changing the index name to make for a nicer example.
from datetime import datetime
from elasticsearch_dsl import DocType, String, Date, Integer
from elasticsearch_dsl.connections import connections
from elasticsearch import Elasticsearch
from elasticsearch_dsl import Search, Q
# Define a default Elasticsearch client
client = connections.create_connection(hosts=['http://blahblahblah:9200'])
s = Search(using=client, index="airbnb", doc_type="sleep_overs")
s = s.execute()
# invalid! You haven't defined an aggregation.
#for tag in s.aggregations.per_tag.buckets:
# print (tag.key)
# Lets make an aggregation
# 'by_house' is a name you choose, 'terms' is a keyword for the type of aggregator
# 'field' is also a keyword, and 'house_number' is a field in our ES index
s.aggs.bucket('by_house', 'terms', field='house_number', size=0)
Above we're creating 1 bucket per house number. Therefore, the name of the bucket will be the house number. ElasticSearch (ES) will always give a document count of documents fitting into that bucket. Size=0 means to give use all results, since ES has a default setting to return 10 results only (or whatever your dev set it up to do).
# This runs the query.
s = s.execute()
# let's see what's in our results
print s.aggregations.by_house.doc_count
print s.hits.total
print s.aggregations.by_house.buckets
for item in s.aggregations.by_house.buckets:
print item.doc_count
My mistake before was thinking an Elastic Search query had aggregations by default. You sort of define them yourself, then execute them. Then your response can be split b the aggregators you mentioned.
The CURL for the above should look like:
NOTE: I use SENSE an ElasticSearch plugin/extension/add-on for Google Chrome. In SENSE you can use // to comment things out.
POST /airbnb/sleep_overs/_search
{
// the size 0 here actually means to not return any hits, just the aggregation part of the result
"size": 0,
"aggs": {
"by_house": {
"terms": {
// the size 0 here means to return all results, not just the the default 10 results
"field": "house_number",
"size": 0
}
}
}
}
Work-around. Someone on the GIT of DSL told me to forget translating, and just use this method. It's simpler, and you can just write the tough stuff in CURL. That's why I call it a work-around.
# Define a default Elasticsearch client
client = connections.create_connection(hosts=['http://blahblahblah:9200'])
s = Search(using=client, index="airbnb", doc_type="sleep_overs")
# how simple we just past CURL code here
body = {
"size": 0,
"aggs": {
"by_house": {
"terms": {
"field": "house_number",
"size": 0
}
}
}
}
s = Search.from_dict(body)
s = s.index("airbnb")
s = s.doc_type("sleepovers")
body = s.to_dict()
t = s.execute()
for item in t.aggregations.by_house.buckets:
# item.key will the house number
print item.key, item.doc_count
Hope this helps. I now design everything in CURL, then use Python statement to peel away at the results to get what I want. This helps for aggregations with multiple levels (sub-aggregations).
I do not have the rep to comment yet but wanted to make a small fix on Matthew's comment on VISQL's answer regarding from_dict. If you want to maintain the search properties, use update_from_dict rather the from_dict.
According to the Docs , from_dict creates a new search object but update_from_dict will modify in place, which is what you want if Search already has properties such as index, using, etc
So you would want to declare the query body before the search and then create the search like this:
query_body = {
"size": 0,
"aggs": {
"by_house": {
"terms": {
"field": "house_number",
"size": 0
}
}
}
}
s = Search(using=client, index="airbnb", doc_type="sleep_overs").update_from_dict(query_body)