Just wondering if there is a way to extend the FormatChecker that is passed to the jsonschema validator? I can't seem to find anything in the documentation.
Basically, I am trying to check if a string value is a valid timezone. I'm using pytz for the timezone side. But, I need to check the provided json string property is contained within that list.
The only other way I can think of is to extract the list as an enum field in the schema. But, it's a huge list and seems a pretty clunky way of doing it. Ideally, I'd like something like this:
from pytz import common_timezones
from jsonschema import validate, FormatChecker
timezone_checker = FormatChecker(formats=["timezone"])
timezone_checker.extend(check_timezone)
instance = { "timezone": "Australia/Sydney" }
schema = {
"properties": {
"timezone": {"type": "string", "format": "timezone"}
}
}
validate(instance=instance, schema=schema, format_checker=timezone_checker)
...
def check_timezone(p):
if not isinstance(p, str):
return False
return p in common_timezones
Thanks in advance.
The function to do so is called FormatChecker.checks.
Related
I'm testing an endpoint which naturally returns a JSON containing the datetime as a string.
I compare the response content in test as such:
assert serializer_instance.data == {
"created_at": str(model_instance.created_at),
"updated_at": str(model_instance.updated_at),
}
created_at and updated_at are surely DateTimeFields. However, in this case, test fails saying:
E Differing items:
E {'created_at': '2020-06-24T12:42:03.578207+03:00'} != {'created_at': '2020-06-24 09:42:03.578207+00:00'}
E {'updated_at': '2020-06-24T12:42:03.578231+03:00'} != {'updated_at': '2020-06-24 09:42:03.578231+00:00'}
So str uses a different formatting on datetimes. Sure, the test case can be passed successfully using strftime, but there should be an internal function that does it easily in either Django or Django Rest Framework and I'd like to learn it.
Thanks in advance.
Environment
Python 3.8.3
Django 2.2.12
Django Rest Framework 3.11.0
I am a bit late to the party, But, better late than never!
I am using this method to assert datetime response in DRF
from rest_framework.fields import DateTimeField
drf_str_datetime = DateTimeField().to_representation
assert serializer_instance.data == {
"created_at": drf_str_datetime(model_instance.created_at),
"updated_at": drf_str_datetime(model_instance.updated_at),
}
For DRF I normally use
obj.ts_updated.astimezone(timezone(settings.TIME_ZONE)).isoformat()
This matches the DRF format.
I've found a way. It uses parse_datetime method and, instead of converting DateTimeField fields on model instance with str, I thought it's better both stay as datetime.
from django.utils.dateparse import parse_datetime
data = serializer_instance.data
data["created_at"] = parse_datetime(data["created_at"])
# ... and the others ...
assert data == {
# ... and the others ...
"created_at": model_instance.created_at,
# ... and the others ...
}
While this is okay, we mutate serializer_instance.data like this. I don't think it is going to be a problem in tests though.
you can use:
myDate.strftime('%m/%d/%Y')
or
'{:%m/%d/%Y}'.format(myDate)
For Django Rest Framework 3.11.0 you can use the following helper function to convert a Python datetime object into a string representation used by DRF:
from pytz import timezone as pytz_timezone
def convert_datetime_to_drf_str(date_time: datetime) -> str:
return date_time.astimezone(pytz_timezone(settings.TIME_ZONE)).isoformat().replace("+00:00", "Z")
So, for your specific case it would be:
assert serializer_instance.data == {
"created_at": convert_datetime_to_drf_str(model_instance.created_at),
"updated_at": convert_datetime_to_drf_str(model_instance.updated_at),
}
or directly without the helper function:
from pytz import timezone as pytz_timezone
assert serializer_instance.data == {
"created_at": model_instance.created_at.astimezone(pytz_timezone(settings.TIME_ZONE)).isoformat().replace("+00:00", "Z"),
"updated_at": model_instance.updated_at.astimezone(pytz_timezone(settings.TIME_ZONE)).isoformat().replace("+00:00", "Z"),
}
I am trying to figure out on how to convert the following JSON object in to a dataframe
[
{
"id":"123",
"sources":[
{
"name":"ABC",
"first":"2020-02-26T03:19:23.247Z",
"last":"2020-02-26T03:19:23.247Z"
},
{
"name":"XYZ",
"first":"2020-02-26T03:19:23.247Z",
"last":"2020-02-26T03:19:23.247Z"
}
]
}
]
The dataframe should appear like this.
id ABC.first ABC.last XYZ.first XYZ.last
123 2020-02-26.. 2020-02-26.. 2020-02-26.. 2020-02-26T03:19:23.247Z
Thanks in advance for your help
Python includes the useful json module. You can find some documentation for it here.
To use it , you'll need to import it into your Python script using the import json command.
From there, you can use the json.loads() method and pass it a some JSON. The loads() method returns a Python dictionary which then lets you access the properties of your JSON object by name. Some example code might look like:
import json
# some JSON:
personJSON = '{"name":"John", "age":30, "city":"New York", "address": {"street": "Fake Street", "streetNumber": "123"}}'
# parse personJSON:
personDict = json.loads(personJSON)
# the result is a Python dictionary:
print(personDict["name"]) # Will print "John"
print(personDict["address"]["street"]) # Will print "Fake Street"
Good luck!
I've done some coding in RoR, and in Rails, when I return a JSON object via an API call, it returns as
{ "id" : "1", "name" : "Dan" }.
However in Python (with Flask and Flask-SQLAlchemy), when I return a JSON object via json.dumps or jsonpickle.encode it is returned as
"{ \"id\" : \"1\", \"name\": \"Dan\" }" which seems very unwieldily as it can't easily be parsed on the other end (by an iOS app in this case - Obj-C).
What am I missing here, and what should I do to return it as a JSON literal, rather than a JSON string?
This is what my code looks like:
people = models.UserRelationships.query.filter_by(user_id=user_id, active=ACTIVE_RECORD)
friends = people.filter_by(friends=YES)
json_object = jsonpickle.encode(friends.first().as_dict(), unpicklable=False, keys=True)
print(json_object) # this prints here, i.e. { "id" : "1", "name" : "Dan" }
return json_object # this returns "{ \"id\" : \"1\", \"name\": \"Dan\" }" to the browser
What is missing in your understanding here is that when you use the JSON modules in Python, you're not working with a JSON object. JSON is by definition just a string that matches a certain standard.
Lets say you have the string:
friends = '{"name": "Fred", "id": 1}'
If you want to work with this data in python, you will want to load it into a python object:
import json
friends_obj = json.loads(friends)
At this point friends_obj is a python dictionary.
If you want to convert it (or any other python dictionary or list) then this is where json.dumps comes in handy:
friends_str = json.dumps(friends_obj)
print friends_str
'{"name": "Fred", "id": 1}'
However if we attempt to "dump" the original friends string you'll see you get a different result:
dumped_str = json.dumps(friends)
print dumped_str
'"{\\"name\\": \\"Fred\\", \\"id\\": 1}"'
This is because you're basically attempting to encode an ordinary string as JSON and it is escaping the characters. I hope this helps make sense of things!
Cheers
Looks like you are using Django here, in which case do something like
from django.utils import simplejson as json
...
return HttpResponse(json.dumps(friends.first().as_dict()))
This is almost always a sign that you're double-encoding your data somewhere. For example:
>>> obj = { "id" : "1", "name" : "Dan" }
>>> j = json.dumps(obj)
>>> jj = json.dumps(j)
>>> print(obj)
{'id': '1', 'name': 'Dan'}
>>> print(j)
{"id": "1", "name": "Dan"}
>>> print(jj)
"{\"id\": \"1\", \"name\": \"Dan\"}"
Here, jj is a perfectly valid JSON string representation—but it's not a representation of obj, it's a representation of the string j, which is useless.
Normally you don't do this directly; instead, either you started with a JSON string rather than an object in the first place (e.g., you got it from a client request or from a text file), or you called some function in a library like requests or jsonpickle that implicitly calls json.dumps with an already-encoded string. But either way, it's the same problem, with the same solution: Just don't double-encode.
You should be using flask.jsonify, which will not only encode correctly, but also set the content-type headers accordingly.
people = models.UserRelationships.query.filter_by(user_id=user_id, active=ACTIVE_RECORD)
friends = people.filter_by(friends=YES)
return jsonify(friends.first().as_dict())
Is there a python library for converting a JSON schema to a python class definition, similar to jsonschema2pojo -- https://github.com/joelittlejohn/jsonschema2pojo -- for Java?
So far the closest thing I've been able to find is warlock, which advertises this workflow:
Build your schema
>>> schema = {
'name': 'Country',
'properties': {
'name': {'type': 'string'},
'abbreviation': {'type': 'string'},
},
'additionalProperties': False,
}
Create a model
>>> import warlock
>>> Country = warlock.model_factory(schema)
Create an object using your model
>>> sweden = Country(name='Sweden', abbreviation='SE')
However, it's not quite that easy. The objects that Warlock produces lack much in the way of introspectible goodies. And if it supports nested dicts at initialization, I was unable to figure out how to make them work.
To give a little background, the problem that I was working on was how to take Chrome's JSONSchema API and produce a tree of request generators and response handlers. Warlock doesn't seem too far off the mark, the only downside is that meta-classes in Python can't really be turned into 'code'.
Other useful modules to look for:
jsonschema - (which Warlock is built on top of)
valideer - similar to jsonschema but with a worse name.
bunch - An interesting structure builder thats half-way between a dotdict and construct
If you end up finding a good one-stop solution for this please follow up your question - I'd love to find one. I poured through github, pypi, googlecode, sourceforge, etc.. And just couldn't find anything really sexy.
For lack of any pre-made solutions, I'll probably cobble together something with Warlock myself. So if I beat you to it, I'll update my answer. :p
python-jsonschema-objects is an alternative to warlock, build on top of jsonschema
python-jsonschema-objects provides an automatic class-based binding to JSON schemas for use in python.
Usage:
Sample Json Schema
schema = '''{
"title": "Example Schema",
"type": "object",
"properties": {
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"age": {
"description": "Age in years",
"type": "integer",
"minimum": 0
},
"dogs": {
"type": "array",
"items": {"type": "string"},
"maxItems": 4
},
"gender": {
"type": "string",
"enum": ["male", "female"]
},
"deceased": {
"enum": ["yes", "no", 1, 0, "true", "false"]
}
},
"required": ["firstName", "lastName"]
} '''
Converting the schema object to class
import python_jsonschema_objects as pjs
import json
schema = json.loads(schema)
builder = pjs.ObjectBuilder(schema)
ns = builder.build_classes()
Person = ns.ExampleSchema
james = Person(firstName="James", lastName="Bond")
james.lastName
u'Bond' james
example_schema lastName=Bond age=None firstName=James
Validation :
james.age = -2
python_jsonschema_objects.validators.ValidationError: -2 was less
or equal to than 0
But problem is , it is still using draft4validation while jsonschema has moved over draft4validation , i filed an issue on the repo regarding this .
Unless you are using old version of jsonschema , the above package will work as shown.
I just created this small project to generate code classes from json schema, even if dealing with python I think can be useful when working in business projects:
pip install jsonschema2popo
running following command will generate a python module containing json-schema defined classes (it uses jinja2 templating)
jsonschema2popo -o /path/to/output_file.py /path/to/json_schema.json
more info at: https://github.com/frx08/jsonschema2popo
I am somewhat new to python and I am wondering what the best way is to generate json in a loop. I could just mash a bunch of strings together in the loop, but I'm sure there is a better way. Here's some more specifics. I am using app engine in python to create a service that returns json as a response.
So as an example, let's say someone requests a list of user records from the service. After the service queries for the records, it needs to return json for each record it found. Maybe something like this:
{records:
{record: { name:bob, email:blah#blah.com, age:25 } },
{record: { name:steve, email:blah#blahblah.com, age:30 } },
{record: { name:jimmy, email:blah#b.com, age:31 } },
}
Excuse my poorly formatted json. Thanks for your help.
Creating your own JSON is silly. Use json or simplejson for this instead.
>>> json.dumps(dict(foo=42))
'{"foo": 42}'
My question is how do I add to the
dictionary dynamically? So foreach
record in my list of records, add a
record to the dictionary.
You may be looking to create a list of dictionaries.
records = []
record1 = {"name":"Bob", "email":"bob#email.com"}
records.append(record1)
record2 = {"name":"Bob2", "email":"bob2#email.com"}
records.append(record2)
Then in app engine, use the code above to export records as json.
Few steps here.
First import simplejson
from django.utils import simplejson
Then create a function that will return json with the appropriate data header.
def write_json(self, data):
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(simplejson.dumps(data))
Then from within your post or get handler, create a python dictionary with the desired data and pass that into the function you created.
ret = {"records":{
"record": {"name": "bob", ...}
...
}
write_json(self, ret)