I know there is no order in a Python's dictionary, but I have a JSON file which contains dictionaries, and I want to insert an element in every dictionary In a way that when I will open the JSON I will see the element added in the top.
Exemple: File JSON contains:
{"b":2, "c":3}
Should be like this after adding:
{"a":1, "b":2, "c":3}
You can use a little trick with OrderedDict to do this.
Step 1: Have your old data stored somewhere.
import json
s = {"b":2, "c":3}
json.dump(s, open('file.json', 'w'))
The data looks like this: '{"b": 2, "c": 3}'
Step 2: When adding new data, use an OrderedDict to load your existing data.
from collections import OrderedDict
new_data = OrderedDict({'a' : 1})
new_data.update(json.loads(open('file.json'), object_pairs_hook=OrderedDict))
json.dump(new_data, open('file.json', 'w'))
And now, the data looks like this: '{"a": 1, "b": 2, "c": 3}'
Related
I am using csv module to convert json to csv and store it in a file or print it to stdout.
def write_csv(data:list, header:list, path:str=None):
# data is json format data as list
output_file = open(path, 'w') if path else sys.stdout
out = csv.writer(output_file)
out.writerow(header)
for row in data:
out.writerow([row[attr] for attr in header])
if path: output_file.close()
I want to store the converted csv to a variable instead of sending it to a file or stdout.
say I want to create a function like this:
def json_to_csv(data:list, header:list):
# convert json data into csv string
return string_csv
NOTE: format of data is simple
data is list of dictionaries of string to string maping
[
{
"username":"srbcheema",
"name":"Sarbjit Singh"
},
{
"username":"testing",
"name":"Test, user"
}
]
I want csv output to look like:
username,name
srbcheema,Sarbjit Singh
testing,"Test, user"
Converting JSON to CSV is not a trivial operation. There is also no standardized way to translate between them...
For example
my_json = {
"one": 1,
"two": 2,
"three": {
"nested": "structure"
}
}
Could be represented in a number of ways...
These are all (to my knowledge) valid CSVs that contain all the information from the JSON structure.
data
'{"one": 1, "two": 2, "three": {"nested": "structure"}}'
one,two,three
1,2,'{"nested": "structure"}'
one,two,three__nested
1,2,structure
In essence, you will have to figure out the best translation between the two based on your knowledge of the data. There is no right answer on how to go about this.
I'm relatively knew to Python so there's probably a better way, but this works:
def get_safe_string(string):
return '"'+string+'"' if "," in string else string
def json_to_csv(data):
csv_keys = data[0].keys()
header = ",".join(csv_keys)
res = list(",".join(get_safe_string(row.get(k)) for k in csv_keys) for row in data)
res.insert(0,header)
return "\n".join(r for r in res)
This question already has answers here:
Convert a String representation of a Dictionary to a dictionary
(11 answers)
Closed 8 months ago.
I have a bytes type object like this:
b"{'one': 1, 'two': 2}"
I need to get the dictionary from that using python code. I am converting it into string and then converting into dictionary as follows.
string = dictn.decode("utf-8")
print(type(string))
>> <class 'str'>
d = dict(toks.split(":") for toks in string.split(",") if toks)
But I am getting the below error:
------> d = dict(toks.split(":") for toks in string.split(",") if toks)
TypeError: 'bytes' object is not callable
I think a decode is also required to get a proper dict.
a= b"{'one': 1, 'two': 2}"
ast.literal_eval(a.decode('utf-8'))
**Output:** {'one': 1, 'two': 2}
The accepted answer yields
a= b"{'one': 1, 'two': 2}"
ast.literal_eval(repr(a))
**output:** b"{'one': 1, 'two': 2}"
The literal_eval hasn't done that properly with many of my code so I personally prefer to use json module for this
import json
a= b"{'one': 1, 'two': 2}"
json.loads(a.decode('utf-8'))
**Output:** {'one': 1, 'two': 2}
All you need is ast.literal_eval. Nothing fancier than that. No reason to mess with JSON unless you are specifically using non-Python dict syntax in your string.
# python3
import ast
byte_str = b"{'one': 1, 'two': 2}"
dict_str = byte_str.decode("UTF-8")
mydata = ast.literal_eval(dict_str)
print(repr(mydata))
See answer here. It also details how ast.literal_eval is safer than eval.
You could try like this:
import json
import ast
a= b"{'one': 1, 'two': 2}"
print(json.loads(a.decode("utf-8").replace("'",'"')))
print(ast.literal_eval(a.decode("utf-8")))
There are the doc of module:
1.ast doc
2.json doc
You can use Base64 library to convert string dictionary to bytes, and although you can convert bytes result to a dictionary using json library. Try this below sample code.
import base64
import json
input_dict = {'var1' : 0, 'var2' : 'some string', 'var1' : ['listitem1','listitem2',5]}
message = str(input_dict)
ascii_message = message.encode('ascii')
output_byte = base64.b64encode(ascii_message)
msg_bytes = base64.b64decode(output_byte)
ascii_msg = msg_bytes.decode('ascii')
# Json library convert stirng dictionary to real dictionary type.
# Double quotes is standard format for json
ascii_msg = ascii_msg.replace("'", "\"")
output_dict = json.loads(ascii_msg) # convert string dictionary to dict format
# Show the input and output
print("input_dict:", input_dict, type(input_dict))
print()
print("base64:", output_byte, type(output_byte))
print()
print("output_dict:", output_dict, type(output_dict))
Simple 😁
data = eval(b"{'one': 1, 'two': 2}")
I have a JSON File which looks like this:
{"one":"Some data", "two":"Some data",...}
and so on...
I want to split all the ID's into separate files according to the name of the ID, For example:
one.json
{"one":"Some data"}
two.json
{"two":"Some data"}
and so on.
I got a reference from this. But my problem is slightly different. What can I modify to achieve the separate text files?
I won't teach you how to do file I/O and assume you can do that yourself.
Once you have loaded the original file as a dict with the json module, do
>>> org = {"one":"Some data", "two":"Some data"}
>>> dicts = [{k:v} for k,v in org.items()]
>>> dicts
[{'two': 'Some data'}, {'one': 'Some data'}]
which will give you a list of dictionaries that you can dump to a file (or separate files named after the keys), if you wish.
After loading the JSON file you can treat it as a dictionary in python and then save the contents in file by looping through as you would in normal python dictionary.
Here is an example related to what you want to achieve
Data = {"one":"Some data", "two":"Some data"}
for item in Data:
name = item + '.json'
file = open(name, 'w')
file.write('{"%s":"%s"}' % (item, Data[item]))
file.close()
after getting the json data into a variable,do
a = {"one":"Some data", "two":"Some data"}
for k,v in a.items():
with open(k+".json","w") as f:
f.write('{"%s" : "%s"}' %(k,v))
and output is :
one.json => {"one":"Some data"}
and
two.json => {"two":"Some data"}
If I have dictionary like:
{
"cats": {
"sphinx": 3,
"british": 2
},
"dogs": {}
}
And try to save it to a text file, I get something like this:
{"cats": {"sphinx": 3}, {"british": 2}, "dogs": {}}
How can I save a dictionary in pretty format, so it will be easy to read by human eye?
You can import json and specify an indent level:
import json
d = {
"cats": {
"sphinx": 3,
"british": 2
},
"dogs": {}
}
j = json.dumps(d, indent=4)
print(j)
{
"cats": {
"sphinx": 3,
"british": 2
},
"dogs": {}
}
Note that this is a string, however:
>>> j
'{\n "cats": {\n "sphinx": 3, \n "british": 2\n }, \n "dogs": {}\n}'
You can use pprint for that:
import pprint
pprint.pformat(thedict)
If you want to save it in a more standard format, you can also use, for example, a yaml file (and the related python package http://pyyaml.org/wiki/PyYAMLDocumentation), and the code would look like:
import yaml
dictionary = {"cats": {"sphinx": 3}, {"british": 2}, "dogs": {}}
with open('dictionary_file.yml', 'w') as yaml_file:
yaml.dump(dictionary, stream=yaml_file, default_flow_style=False)
dump creates a string in the yaml format to be written to the file. Note that it is possible to specify the stream and write the content immediately to the file. If it is necessary to get the string for some reason before writing to the file, just don't specify it and write it after using write function for the file.
Note also that the parameter default_flow_style allows to have a nicer format; in the example the file looks:
cats:
british: 2
sphinx: 3
dogs: {}
To load again the yaml file in a dictionary:
import yaml
with open('dictionary_file.yml', 'r') as yaml_file:
dictionary = yaml.load(yaml_file)
You can dump it by using the Python Object Notation module (pon: disclaimer I am the author of that module)
from pon import PON, loads
data = {
"cats": {
"sphinx": 3,
"british": 2
},
"dogs": {}
}
pon = PON(obj=data)
pon.dump()
which gives:
dict(
cats=dict(
sphinx=3,
british=2,
),
dogs=dict( ),
)
which again is correct Python, but trading the quoted strings needed for keys by using dict .
You can load this again with:
read_back = loads(open('file_name.pon').read())
print(read_back)
giving:
{'cats': {'sphinx': 3, 'british': 2}, 'dogs': {}}
Please note that loads() does not evaluate the string, it actually parses it safely using python's built-in parser.
PON also allows you to load python dictionaries from files, that have commented entries, and dump them while preserving the comments. This is where it's real usefulness comes into action.
Alternatively, if you would like something, arbitrarily more readable like the YAML format, you can use ruamel.yaml and do:
import ruamel.yaml
ruamel.yaml.round_trip_dump(data, stream=open('file_name.yaml', 'wb'), indent=4)
which gives you a file file_name.yaml with contents:
cats:
sphinx: 3
british: 2
dogs: {}
which uses the indent you seem to prefer (and is more efficient than #alberto's version)
I have a txt file that contains a dictionary in Python and I have opened it in the following manner:
with open('file') as f:
test = list(f)
The result when I look at test is a list of one element. This first element is a string of the dictionary (which also contains other dictionaries), so it looks like:
["ID": 1, date: "2016-01-01", "A": {name: "Steve", id: "534", players:{last: "Smith", first: "Joe", job: "IT"}}
Is there any way to store this as the dictionary without having to find a way to determine the indices of the characters where the different keys and corresponding values begin/end? Or is it possible to read in the file in a way that recognizes the data as a dictionary?
If you are reading a json file then you can use the json module.
import json
with open('data.json') as f:
data = json.load(f)
If you are sure that the file you are reading contains python dictionaries, then you can use the built-in ast.literal_eval to convert those strings to a Python dictionary:
>>> import ast
>>> a = ast.literal_eval("{'a' : '1', 'b' : '2'}")
>>> a
{'a': '1', 'b': '2'}
>>> type(a)
<type 'dict'>
There is an alternative method, eval. But using ast.literal_eval would be better. This answer will explain why.
You can use json module
for Writing
import json
data = ["ID": 1, date: "2016-01-01", "A": {name: "Steve", id: "534", players:{last: "Smith", first: "Joe", job: "IT"}}
with open("out.json", "w") as f:
json.dump(data)
for Reading
import json
with open("out.json", "w") as f:
data = json.load(f)
print data
Just use eval() when you read it from your file.
for example:
>>> f = open('file.txt', 'r').read()
>>> mydict = eval(f)
>>> type(f)
<class 'str'>
>>> type(mydict)
<class 'dict'>
The Python interpreter thinks you are just trying to read an external file as text. It does not know that your file contains formatted content. One way to import easily as a dictionary be to write a second python file that contains the dictionary:
# mydict.py
myImportedDict = {
"ID": 1,
"date": "2016-01-01",
"A": {
"name": "Steve",
"id": "534",
"players": {
"last": "Smith",
"first": "Joe",
"job": "IT"
}
}
}
Then, you can import the dictionary and use it in another file:
#import_test.py
from mydict import myImportedDict
print(type(myImportedDict))
print(myImportedDict)
Python also requires that folders containing imported files also contain a file called
__init__.py
which can be blank. So, create a blank file with that name in addition to the two files above.
If your source file is meant to be in JSON format, you can use the json library instead, which comes packaged with Python: https://docs.python.org/2/library/json.html