The problem I'm having is with mixing serialized Django models using django.core.serializers with some other piece of data and then trying to serialize the entire thing using json.dumps.
Example code:
scores = []
for indicator in indicators:
score_data = {}
score_data["indicator"] = serializers.serialize("json", [indicator,])
score_data["score"] = evaluation.indicator_percent_score(indicator.id)
score_data["score_descriptor"] = \
serializers.serialize("json",
[form.getDescriptorByPercent(score_data["score"]),],
fields=("order", "value", "title"))
scores.append(score_data)
scores = json.dumps(scores)
return HttpResponse(scores)
Which returns a list of objects like this:
{
indicator: "{"pk": 345345, "model": "forms.indicator", "fields": {"order": 1, "description": "Blah description.", "elements": [10933, 4535], "title": "Blah Title"}}",
score: 37.5,
score_descriptor: "{"pk": 66666, "model": "forms.descriptor", "fields": {"order": 1, "value": "1.00", "title": "Unsatisfactory"}}"
}
The problem I'm having can be seen in the JSON with the serialized Django models being wrapped in multiple sets of quotations. This makes the it very hard to work with on the client side as when I try to do something like
indicator.fields.order
it evaluates to nothing because the browser thinks I'm dealing with a string of some sort.
Ideally I would like valid JSON without the conflicting quotations that make it unreadable. Something akin to a list of objects like this:
{
indicator: {
pk: 12931231,
fields: {
order: 1,
title: "Blah Title",
}
},
etc.
}
Should I be doing this in a different order, using a different data structure, different serializer?
My solution involved dropping the use of django.core.serializers and instead using django.forms.model_to_dict and django.core.serializers.json.DjangoJSONEncoder.
The resulting code looked like this:
for indicator in indicators:
score_data = {}
score_data["indicator"] = model_to_dict(indicator)
score_data["score"] = evaluation.indicator_percent_score(indicator.id)
score_data["score_descriptor"] = \
model_to_dict(form.getDescriptorByPercent(score_data["score"]),
fields=("order", "value", "title"))
scores.append(score_data)
scores = json.dumps(scores, cls=DjangoJSONEncoder)
The problem seemed to arise from the fact that I was essentially serializing the Django models twice. Once with the django.core.serializers function and once with the json.dumps function.
The solution solved this by converting the model to a dictionary first, throwing it in the dictionary with the other data and then serializing it once using the json.dumps using the DjangoJSONEncoder.
Hope this helps someone out because I couldn't find my specific issue but was able to piece it together using other answer to other stackoverflow posts.
Related
I'm building an API using Flask and marshmallow to format the query results but for some reason Filter.query.all() is returning empty, but when I replace that with Filter.query.first() it returns the first filter. I've also double checked my database to make sure multiple entries are there. Is there anything I'm doing wrong here?
from Model import db, Filter
class FilterSchema(ma.Schema):
id = fields.Integer()
filter_schema = FilterSchema()
### returns list of all filters in Filter db
def get(self):
filters = Filter.query.all()
filters = filter_schema.dump(filters).data
return {'status': 'success', 'data': filters}, 200
returns:
{
"status": "success",
"data": {}
}
while Filter.query.first() returns:
{
"status": "success",
"data": {
"id": 1
}
}
It turns out for a schema in marshmallow you need to specify whether the schema is dumping multiple entries or not. When there is more than one, add many=True to your FilterSchema():
ie: filter_schema = FilterSchema(many=True)
or even better, add a different variable called
filters_schema = FilterSchema(many=True)
and choose which one to use depending on the data you want returned.
I am trying to pass json field as input for my graphql mutation.
I have been trying and searching but just no luck. I can pass array fine with I know by defining graphene.List(graphene.String) would work for passing array of strings.
I figured there's a type named graphene.JSONstring() which I thought would work if I use it with graphene.List(graphene.JSONstring) but no luck, still getting errors saying type is not right.
I have something like this during the mutation
mutation {
create(data:{
field1: [
{
"first": "first",
"last": "last"
},
{
"first":"first1",
"last":"last1"
}
]
})
}
as for input class
class NameInput(graphene.InputObjectType):
# please ignore the same field names, just listing what I have tried
field1 = graphene.JSONString()
field1 = graphene.List(graphene.JSONString)
field1 = graphene.List(graphene.String)
Does anyone has an idea how this would work?
Thanks in advance
Seems like you are trying to have nested input objects. Unfortunately I have never used graphene but maybe I can answer in terms of the GraphQL specification and then make an educated guess about the graphene code:
type Mutation {
create(data: NameInput): Boolean # <- Please don't return just Boolean
}
input NameInput {
field1: FistLastInput[]
}
input FirstLastInput {
first: String!
last: String!
}
This means you will need two input objects to describe the structure of your input. Create a new class for you object that takes the fields first and last:
class FirstLastInput(graphene.InputObjectType):
first = graphene.NonNull(graphene.String)
last = graphene.NonNull(graphene.String)
Now we can use the input object in our initial query:
class NameInput(graphene.InputObjectType):
field1 = graphene.List(FirstLastInput)
You could try like this:
class NameInput(graphene.InputObjectType):
field1 = graphene.JSONString()
And then:
mutation {
create(data:{
field1: "[
{
\"first\": \"first\",
\"last\": \"last\"
},
{
\"first\":\"first1\",
\"last\":\"last1\"
}
]"
})
}
So basically send json as string.
Graphene provides a GenericScalar type. You can use it to input/output generic types like int, str, dict, list, etc.
from graphene import InputObjectType, Mutation
from graphene.types.generic import GenericScalar
class InputObject(InputObjectType):
field1 = GenericScalar()
class Create(Mutation):
class Arguments:
data = InputObject()
def mutate(root, info, data):
# do something with data.field1
Then your input would look like
mutation {
create (
data: {
field1: [
{
first: "first",
last: "last"
},
{
first: "first1",
last: "last1"
}
]
}
)
}
Note that field1 can accept any generic input, so make sure to validate it.
Also, when using a GenericScalar field for output (query), you won't be able to query its subfields. But you can write a resolver for that field to make sure only specific subfields are returned.
Here is the link to the corresponding GitHub issue.
I was wondering if it's possible to store an array in a Django model?
I'm asking this because I need to store an array of int (e.g [1,2,3]) in a field and then be able to search a specific array and get a match with it or by it's possible combinations.
I was thinking to store that arrays as strings in CharFields and then, when I need to search something, concatenate the values(obtained by filtering other model) with '[', ']' and ',' and then use a object filter with that generated string. The problem is that I will have to generate each possible combination and then filter them one by one until I get a match, and I believe that this might be inefficient.
So, I hope you can give me other ideas that I could try.
I'm not asking for code, necessarily, any ideas on how to achieve this will be good.
I'd have two advices for you:
1) Use ArrayField if you are using PostgreSQL as your database. You can read more about ArrayField here.
2) Encode your array as JSON and store it either as a plain string or using a JSONField as found here.
I'd personally prefer option number 1 since that is the cleaner and nicer way but depending on what you are actually using to store your data that might not be available to you.
Yes, you can use it like this:
from django.contrib.postgres.fields import ArrayField
class Board(models.Model):
pieces = ArrayField(ArrayField(models.IntegerField()))
However, it can only be available when using PostgreSQL for the database.
If you aren't using Postgres, I recommend Django's validate_comma_separated_integer_list validator.
https://docs.djangoproject.com/en/dev/ref/validators/#django.core.validators.validate_comma_separated_integer_list
You use is as a validator on a CharField().
I don't know why nobody has suggested it, but you can always pickle things and put the result into a binary field.
The advantages of this method are that it will work with just about any database, it's efficient, and it's applicable to more than just arrays. The downside is that you can't have the database run queries on the pickled data (not easily, anyway).
you can store a json and good to go with sub arrays of that JSON:
if (data != "attachmentto") {
formData.append(data, this.recipe[data])
console.log('rec data ', data)}
else if (data == "attachmentto") {
console.log('rec data434 ', this.recipe.attachmentto)
var myObj = { name: this.recipe.attachmentto.name, age: 31, city: "New York" };
let kokos = JSON.stringify(myObj);
// this.recipe.attachmentto.name = this.recipe.attachmentto.name
formData.append('attachmentto', kokos)
}
Django backend:
class Video(models.Model):
objects = models.Manager()
title = models.CharField(max_length=80)
description = models.TextField(max_length=300)
picture = JSONField(encoder=None)
price = models.IntegerField(default=0)
url = models.URLField()
category = models.CharField(max_length=50)
subcategory = models.TextField(max_length=50)
attachmentto = JSONField(encoder=None)
# attachmentto2 = models.JSONField()
user = models.ForeignKey(User, on_delete=models.CASCADE, default=1)
and result on backend API:
{
"id": 174,
"title": "ads",
"description": "ads",
"picture": {
"name": "https://v3.vuejs.org/logo.png"
},
"price": 0,
"user": 1,
"rating_average": 0,
"attachmentto": {
"age": 31,
"city": "New York",
"name": [
"https://restdj.herokuapp.com/media/uploads/ftf_3_cJ0V7TF.png",
"https://restdj.herokuapp.com/media/uploads/ftf_3.jpg"
]
}
},
I call it noicely(nicely). Notice that we send a JSON and we have a array in that JSON
Kokos is the full JSON disagned for djangoo:
var myObj = { name: this.recipe.attachmentto.name, age: 31, city: "New York" };
let kokos = JSON.stringify(myObj);
formData.append('attachmentto', kokos)
Above; name: this.recipe.attachmentto.name is an array
Here is the array:
"name": [
"https://restdj.herokuapp.com/media/uploads/ftf_3_cJ0V7TF.png",
"https://restdj.herokuapp.com/media/uploads/ftf_3.jpg"
]
I've got a json file that I've pulled from a web service and am trying to parse it. I see that this question has been asked a whole bunch, and I've read whatever I could find, but the json data in each example appears to be very simplistic in nature. Likewise, the json example data in the python docs is very simple and does not reflect what I'm trying to work with. Here is what the json looks like:
{"RecordResponse": {
"Id": blah
"Status": {
"state": "complete",
"datetime": "2016-01-01 01:00"
},
"Results": {
"resultNumber": "500",
"Summary": [
{
"Type": "blah",
"Size": "10000000000",
"OtherStuff": {
"valueOne": "first",
"valueTwo": "second"
},
"fieldIWant": "value i want is here"
The code block in question is:
jsonFile = r'C:\Temp\results.json'
with open(jsonFile, 'w') as dataFile:
json_obj = json.load(dataFile)
for i in json_obj["Summary"]:
print(i["fieldIWant"])
Not only am I not getting into the field I want, but I'm also getting a key error on trying to suss out "Summary".
I don't know how the indices work within the array; once I even get into the "Summary" field, do I have to issue an index manually to return the value from the field I need?
The example you posted is not valid JSON (no commas after object fields), so it's hard to dig in much. If it's straight from the web service, something's messed up. If you did fix it with proper commas, the "Summary" key is within the "Results" object, so you'd need to change your loop to
with open(jsonFile, 'w') as dataFile:
json_obj = json.load(dataFile)
for i in json_obj["Results"]["Summary"]:
print(i["fieldIWant"])
If you don't know the structure at all, you could look through the resulting object recursively:
def findfieldsiwant(obj, keyname="Summary", fieldname="fieldIWant"):
try:
for key,val in obj.items():
if key == keyname:
return [ d[fieldname] for d in val ]
else:
sub = findfieldsiwant(val)
if sub:
return sub
except AttributeError: #obj is not a dict
pass
#keyname not found
return None
I have a json object which I would like to filter for misspelled key name. So for the example below, I would like to have a json object without the misspelled test_name key. What is the easiest way to do this?
json_data = """{
"my_test": [{
"group_name": "group-A",
"results": [{
"test_name": "test1",
"time": "8.556",
"status": "pass"
}, {
"test_name": "test2",
"time": "45.909",
"status": "pass"
}, {
"test_nameZASSD": "test3",
"time": "9.383",
"status": "fail"
}]
}]
}"""
This is an online test, and looks like i'm not allowed to use jsonSchema.
So far my code looks like this:
if 'test_suites' in data:
for suites in data["test_suites"]:
if 'results' in suites and 'suite_name' in suites:
for result in suites["results"]:
if 'test_name' not in result or 'time' not in result or 'status' not in result:
result.clear()
else:
....
else:
print("Check 'suite_name' and/or 'results'")
else:
print("Check 'test_suites'")
It kind of works, but result.clear() leaves a empty {}, which get annoying later. What can I do here?
It looks like your data have a consistent schema. So I would try using json schema to solve your problem. With that you can set up a schema and only allow objects with certain key names.
If you just want to check if a certain key is in the dictionary and make sure that you only get the ones that are according to spec you could do something like this:
passed = []
for item in result:
if 'test_name' in item.keys():
passed.append(item)
But if you have a lot of different keys you need to check for it will become unwieldy. So for bigger projects I would say that json schema is the way to go.