We Keep Coding

pyArango bulkImport_json is complaining about improper indicies

I'm testing the ability to store PyTest results, generated by the json plugin for that test harness, into ArangoDB. I am attempting to import as follows
import pyArango.connection as adbConn
dbConn = adbConn.Connection(...)
db = dbConn['mydb']
collection = db.collections['PyTestResults']
collection.bulkImport_json('/path/to/results.json')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/site-packages/pyArango/collection.py", line 777, in bulkImport_json
errorMessage = "At least: %d errors. The first one is: '%s'\n\n more in <this_exception>.data" %
(len(data), data[0]["errorMessage"])
TypeError: string indices must be integers
What isn't making sense is that the JSON file is properly formed. In fact, using the regular Python JSON module, it works just fine:
import json
with open('/path/to/results.json') as fd:
data = json.load(fd)
print(data)
This works. The beginning of the file is
{"report":
{"environment":
{
"Python": "3.6.9", "Platform": "Linux-4.4.0-17763-Microsoft-x86_64-with-Ubuntu-18.04-bionic"
},
It seems that the library, pyArango, is wanting the keys to be integers. I tried this, that is I tried changing "report" to 0. However, this resulted in invalidating the JSON structure.
How is one to use the pyArango library to import JSON? The overall structure of this JSON file doesn't look much different than any of the examples in this page. Any pointers are greatly appreciated.

TypeError when trying to get data from JSON

I would like to print specific data in a JSON but I get the following error:
Traceback (most recent call last):
File "script.py", line 47, in <module>
print(link['data.file.url.short'])
TypeError: 'int' object has no attribute '__getitem__'
Here is the JSON:
{
"status":true,
"data":{
"file":{
"url":{
"full":"https://anonfile.com/y000H35fn3/yuh_txt",
"short":"https://anonfile.com/y000H35fn3"
},
"metadata":{
"id":"y000H35fn3",
"name":"yuh.txt",
"size":{
"bytes":0,
"readable":"0 Bytes"
}
}
}
}
}
I'm trying to get data.file.url.short which is the short value of the url
Here is the script in question:
post = os.system('curl -F "file=#' + save_file + '" https://anonfile.com/api/upload')
link = json.loads(str(post))
print(link['data.file.url.short'])
Thanks

Other than os.system() return value mentioned by #John Gordon I think correct syntax to access data.file.url.short is link['data']['file']['url']['short'], since json.loads returns dict.

os.system() does not return the output of the command; it returns the exit status of the command, which is an integer.
If you want to capture the command's output, see this question.

You are capturing the return code of the process created by os.system which is an integer.
Why dont you use the request class in the urllib module to perform that action within python?
import urllib.request
import json
urllib.request.urlretrieve('https://anonfile.com/api/upload', save_file)
json_dict = json.load(save_file)
print(json_dict['data']['file']['url']['short']) # https://anonfile.com/y000H35fn3
Or if you don't need to save the file you can use the requests library:
import requests
json_dict = requests.get('https://anonfile.com/api/upload').json()
print(json_dict['data']['file']['url']['short']) # https://anonfile.com/y000H35fn3

Python parsing Json - "X is None" doesn't catch empty objects

I am new to Python and I am trying to parse a Json file using Python.
The Json file is a nested file. While I am trying to exact the "conversation_id" item, the list that contains this item, and the list above sometimes can be empty. I am hoping to replace empty list as string "N/A", otherwise grab the item. Code I am using is as following:
for log in data['logs']:
print("Processing log "+log['log_id'])
logcolumns=[]
if log['request'] is None:
logcolumns.append("N/A")
elif log['request']['context'] is None:
logcolumns.append("N/A")
else:
logcolumns.append(log['request']['context']['conversation_id'])
try:
print("\t".join(logcolumns),file = conv_tsv)
except KeyError:pass
del logcolumns
Traceback error I got is
Processing log cafa1077-f479-4c55-ac34-3bc3ebbb41fc
Traceback (most recent call last):
File "conversation_log_2.py", line 43, in <module>
logcolumns.append(log['request']['context']['conversation_id'])
KeyError: 'conversation_id'
The "request" list that is associated with this log id is shown as below in the json file:
{"request": {"input": {}, "context": {}},
A full request list would be like this:
{"request": {"input": {"text": "haha"}, "context": {"conversation_id": "328d2320-f488-4f46-b71f-6cdfb1b79106", "system": {"dialog_stack": [{"dialog_node_s": "root"}], "dialog_turn_counter": 1, "dialog_request_counter": 1, "_node_output_map_s": "{\"Welcome\":[0,1,0]}", "branch_exited_s": "true", "branch_exited_reason_s": "completed"}}},
When I went to the output file, which is conv.tsv, there is N/A in the output.

You seem to have the syntax quite moodled up. Is the try/except supposed to be wrapping the if/elif? Do you actually want if/elifs?
Note that log['request'] is None does not test that the key's value is an empty dict.
You can use the .get method that returns a default when the key is not found:
logcolumns.append(log.get('request', {}).get('context', {}).get('conversation', 'N/A'))
Or better still, use a try/except to append the default value if any of the keys in missing:
try:
logcolumns.append(log['request']['context']['conversation_id'])
except KeyError:
logcolumns.append('N/A')

Json 'list' object has no attribute 'get'

I have a json list like this:
{
"usuarios": [
{
"id": 1,
"nome": "vitor",
"email": "vitor#"
}
]
}
And I try to list this items using Python like this:
#!/usr/bin/python
import requests
import json
import sys
def list_users():
response = json.loads(requests.get("http://127.0.0.1:3000/usuarios/")._content)
for r in response.get("usuarios"):
print r["id"],r["nome"],r['email']
if __name__ == '__main__':
list_users()
And when I run, appear this error:
python rest_cli.py
Traceback (most recent call last):
File "rest_cli.py", line 62, in <module>
listar_usuarios()
File "rest_cli.py", line 10, in listar_usuarios
for r in response.get("usuarios"):
AttributeError: 'list' object has no attribute 'get'
How can I fix it?

URL in the following code is already giving you an array as response.
response = json.loads(requests.get("http://127.0.0.1:3000/usuarios/")._content)
Don't do a get on the response. Just use a simple for loop on it like :
for r in response:
print r["id"],r["nome"],r['email']

That is a namespace issue as you are importing everything from requests I believe. Remove the .get and just use requests.
requests("http://127.0.0.1:3000/usuarios/")._content

Unable to read MongoDB JSON file from Python

I have a mongo collection called test.json,I am unable to read the test.json from python.If i run the below code its giving the error
ValueError: No JSON object could be decoded
from bson import ObjectId
import json
from pprint import pprint
with open('E:/Work/Paths/Production/test.json') as data_file:
data = json.load(data_file)
pprint(data)
test.json
{
"_id" : ObjectId("582c2011fe5dc80c8f2f8077"),
"menuNumber" : NumberInt(14603),
"imageurl" : "menu/test.png",
"imageurl_thumb" : "master/14603_thumb.png"
}
{
"_id" : ObjectId("582c2018fe5dc80c8f2f8078"),
"menuNumber" : NumberInt(14614),
"imageurl" : "menu/test1.png",
"imageurl_thumb" : "master/14614_thumb.png"
}

Actually the test.json file that you have put up is not a valid json it can be treated something like each line is a json object starting with '{' and ending with '}' but not the entire file as a whole. You should read it as a normal file and then apply some techniques to load it as json.