We Keep Coding

How to custom indent JSON?

I want to be able to indent a JSON file in a way where the first key is on the same line as the opening bracket, by deafult, a json.dump() function puts it in a new line.
For example, if the original file was:
[
{
"statusName": "CO_FILTER",
"statusBar": {}
I want it to start like this:
[
{ "statusName": "CO_FILTER",
"statusBar":{}

I believe if you wanted to custom format it, you will most likely have to make/add onto the JSON library, just like the Google Json formatter and many others. But for the most part, its not like a value you can change on demand, it will either be included in another Json library apart from the default library from python. Maybe search for different python Json libraries that dumps Json the way you like.

Send metadata in mqtt message

I am using mqtt for the first time to transfer some binary files, so far I have no issues transferring it using a code like bellow
import paho.mqtt.client as paho
f=open("./file_name.csv.gz","rb")
filename= f.read()
f.close()
byteArray = bytearray(filename)
mqttc = paho.Client()
mqttc.will_set("/event/dropped", "Sorry, I seem to have died.")
mqttc.connect(*connection definition here*)
mqttc.publish("hello/world", byteArray )
However together with the file itself there is some extra info I want to send (the original file name, creation date,etc...), I can't find any proper way to transfer it using mqtt, is there any way to do that or do I need to add that info to the message byteArray itself? How would I do that?

You need to build your own data structor to hold the file and it's meta data.
How you build that structure is up to you. A couple of options would be:
base64/uuencode encode the file and add it as a field in a JSON object and save the meta data as other fields then publish the JSON object.
Build a Python map with the file as a field and other meta data as other fields. Then use pickle to serialise the map.

loading Behave test results in JSON file in environment.py's after_all() throws error

I'm trying to send my Behave test results to an API Endpoint. I set the output file to be a new JSON file, run my test, and then in the Behave after_all() send the JSON result via the requests package.
I'm running my Behave test like so:
args = ['--outfile=/home/user/nathan/results/behave4.json',
'--for mat=json.pretty']
from behave.__main__ import main as behave_main
behave_main(args)
In my environment.py's after_all(), I have:
def after_all(context):
data = json.load(open('/home/user/myself/results/behave.json', 'r')) # This line causes the error
sendoff = {}
sendoff['results'] = data
r = requests.post(MyAPIEndpoint, json=sendoff)
I'm getting the following error when running my Behave test:
HOOK-ERROR in after_all: ValueError: Expecting object: line 124 column 1
(char 3796)
ABORTED: By user.
The reported error is here in my JSON file:
[
{
...
} <-- line 124, column 1
]
However, behave.json is outputted after the run and according to JSONLint it is valid JSON. I don't know the exact details of after_all(), but I think the issue is that the JSON file isn't done writing by the time I try to open it in after_all(). If I try json.load() a second time on the behave.json file after the file is written, it runs without error and I am able to view my JSON file at the endpoint.
Any better explanation as to why this is happening? Any solution or change in logic to get past this?

Yes, it seems as though the file is still in the process of being written when I try to access it in after_all(). I put in a small delay before I open the file in my code, then I manually viewed the behave.json file and saw that there was no closing ] after the last }.
That explains that. I will create a new question to find out how to get by this, or if a change in a logic is required.

Retrieving a JSON valid string from a python requests.post()

I am extremely confused after trying a few possible solutions and getting various errors that just lead me in circles. I have a function that will grab a tweet, put it in a dictionary, then write that dictionary to a file using dumps like so:
jsonFile = {}
jsonFile["tweet"] = tweet
jsonFile["language"] = language
with open('jsonOutputfile.txt', 'w') as f:
json.dump(jsonFile, f)
I then have another python file that has a function that will return the value of this jsonOutputfile.txt if I want to use it elsewhere. I do that like so:
with open('jsonOutputfile.txt') as f:
jsonObject = json.load(f)
return jsonObject
This function sits on my localhost. The above two functions that have to do with saving and retrieving the JSON file are separate from the rest of my functions below, as I want them to be.
I have another function that will retrieve the values of the returned status using python requests, like so:
def grab_tweet():
return requests.post("http://gateway:8080/function/twittersend")
and then after grabbing the tweet I want to manipulate it, and I want to do so using the JSON that I should have received from this request.
r = grab_tweet()
data = json.dumps(r.text)
return data.get('tweet')
I want this function above to return just the value that is associated with the tweet key in the JSON that I received from when I saved and loaded it. However, I keep on getting the following error: AttributeError: 'str' object has no attribute 'get' which I am confused about because from my understanding using json.dumps() should create a JSON valid string that I can call get on. Is there an encoding error when I am transferring this to and from a file, or maybe when I am receiving my request?
Any help is appreciated.
EDIT:
Here is a sample of a response from my requests.post when I use r.text, it also looks like there is some Unicode in the response so I put an example at the end of the tweet section. (This also doesn't look like a JSON which is what my question is centered around. There should at least be double quotes and no U's right?):
{u'tweet': u'RT THIS IS THE TWEET BLAH BLAH\u2026', u'language': u'en'}

Use .json() in requests module to get response as JSON
Ex:
data = r.json()
return data.get('tweet')
Note: json.dumps convert your response to a string object
Edit as per comment - Try using the ast module.
Ex:
import ast
data = ast.literal_eval(r.text)

You will need to use the .json() method. See requests' documentation: JSON Response Content
Also, for future reference, rather than do
f.write(json.dumps(jsonFile))
You could simply use:
json.dump(jsonFile, f)
Same with using load instead of loads:
jsonObject = json.load(f)

Trouble retrieving data from kivy's jsonstore

I'm having issues retrieving data from a '.json' file if the key contains non-ascii characters.
To explain better i want to illustrate this issue with an example.
Say if i want to save data into a json file as follows
store = JsonStore('example.json')
store.put('André Rose', type = 'sparkling wine', comment = 'favourite')
Then I want to retrieve it as follows
store.get('André Rose')
this returns an error that says:
KeyError: 'Andr\xc3\xa9'
I believe the problem is the non-ascii character " é ".
so my question is how can I save stuffs like this into a json file, and retrieve without getting this key error?

"There is a bug in kivy 1.8.0 under Python 3. When you are using Kivy 1.8.0 and Python 3, URlRequest fails to convert the incoming data to JSON. If you are using this combination you'll need to add:" (Philips, Creating Apps in Kivy)
import json
data = json.loads(data.decode())
I'm not sure if this will help your particular problem, but I thought I might throw it out there.