I require 2 things to be done.
First, take the request object and save the object attribute values
to a file as values of some known keys. This file needs to be editable
after saving, ie, a user can modify the values of the keys(So I used
json format). This is handled in function
save_auth_params_to_file().
Second, get the file contents in a such a format that I can retrieve
the values using the keys. This is handled in function
get_auth_params_from_file.
import json
import os
SUCCESS_AUTH_PARAM_FILE = '/auth/success_auth_params.json'
def save_auth_params_to_file(request):
auth_params = {}
if request is not None:
auth_params['token'] = request.token
auth_params['auth_url'] = request.auth_url
auth_params['server_cert'] = request.server_cert
auth_params['local_key'] = request.local_key
auth_params['local_cert'] = request.local_cert
auth_params['timeout'] = request.timeout_secs
with open(SUCCESS_AUTH_PARAM_FILE, 'w') as fout:
json.dump(auth_params, fout, indent=4)
def get_auth_params_from_file():
auth_params = {}
if os.path.exists(SUCCESS_AUTH_PARAM_FILE):
with open(SUCCESS_AUTH_PARAM_FILE, "r") as fin:
auth_params = json.load(fin)
return auth_params
Question:
Is there a more pythonic way to achieve the 2 things ?
Any potential issues in the code which I have overlooked?
Any error conditions I have to take care ?
There are some things to be noted, yes:
i) When your request is None for some reason, you are saving an empty JSON object to your file. Maybe you'll want to write to your file only if request is not None?
auth_params = {}
if request is not None:
auth_params['token'] = request.token
auth_params['auth_url'] = request.auth_url
auth_params['server_cert'] = request.server_cert
auth_params['local_key'] = request.local_key
auth_params['local_cert'] = request.local_cert
auth_params['timeout'] = request.timeout_secs
with open(SUCCESS_AUTH_PARAM_FILE, 'w') as fout:
json.dump(auth_params, fout, indent=4)
ii) Why not create the dict all at once?
auth_params = {
'token': request.token,
'auth_url': request.auth_url,
'server_cert': request.server_cert,
'local_key': request.local_key,
'local_cert': request.local_cert,
'timeout': request.timeout,
}
iii) Make sure this file is in a SAFE location with SAFE permissions. This is sensitive data, like anything related to authentication.
iv) You are overwriting your file everytime save_auth_params_to_file is called. Maybe you mean to append your JSON to the file instead of overwriting? If that's the case:
with open(SUCCESS_AUTH_PARAM_FILE, 'a') as fout:
Related
I'm trying to make a feature with a Discord bot where it counts how many commands have been run and it stores it on a JSON file named data.json. Here is the code:
import json
# Read 'data.json'
with open("data.json", "r") as d:
datar = json.load(d)
com = datar.get("commands-run")
# Functions
def command_increase():
commands_run = com + 1
dataw = open("data.json", "w")
dataw["commands-run"] = commands_run
And here is the JSON file:
{
"commands-run": 0
}
And this is the error I get when I run a command and it tries to increase the value in the JSON file:
TypeError: '_io.TextIOWrapper' object does not support item assignment
On top of that, it also completely wipes the JSON file. By that, I mean that it just clears everything. Even the brackets.
When you do a json.load, it loads your json data into a dict.
You can increase your command counter in this dict and re-write the dict back into your json file at the end of it.
import json
# Read 'data.json'
with open("data.json", "r") as d:
datar = json.load(d)
com = datar.get("commands-run")
# Functions
def command_increase():
commands_run = com + 1
datar["commands-run"] = commands_run
with open("data.json", "w") as dataw:
dataw.write(json.dumps(datar, indent=4))
command_increase()
I have a file, memory.txt, and I want to store an instance of the class Weapon() in a dictionary, on the second line.
with open(memorypath(), "r") as f:
lines = f.readlines()
inv = inventory()
if "MAINWEAPON" not in inv or inv["MAINWEAPON"] == "":
inv["MAINWEAPON"] = f"""Weapon(sw, 0, Ability(0, "0"), ["{name}'s first weapon."], dmg=30, cc=20, str=15)"""
lines[1] = str(inv) + "\n"
with open(memorypath(), "w") as f:
f.writelines(lines)
(inventory and memorypath are from another file I have for utility functions)
Though, with what I have, if I get inv["MAINWEAPON"] I'll just get the string, not the class. And I have to store it like a string, or else I'll be getting something like <__main\__.Weapon object at (hexadecimal path thing)>.
How do I get the class itself upon getting inv["MAINWEAPON"]?
Another thing, too, I feel like I'm making such confusion with newlines, because file memory.txt has 6 lines but gets shortened to 5, please tell me if I'm doing anything wrong.
If you have a class then you can represent it as a dict and save it as json format.
class Cat:
name: str
def __init__(self, name: str):
self.name = name
def dict(self):
return {'name': self.name}
#classmethod
def from_dict(cls, d):
return cls(name = d['name'])
Now you can save the class as a json to a file like this:
import json
cat = Cat('simon')
with open('cat.json', 'w') as f:
json.dump(cat.dict(), f)
And you can load the json again like this:
with open('cat.json', 'r') as f:
d = json.load(f)
cat = Cat.from_dict(d)
Update
Since python 3.7 the possilility to make dataclasses has been made, and I am here giving an example of how you can use that to save the classes into a json format.
If you want to use the json file as a database and be able to append new entities to it then you will have to load the file into memory and append the new data and finally override the old json file, the code below will do exactly that.
from dataclasses import dataclass, asdict
import json
#dataclass
class Cat:
name: str
def load_cats() -> list[Cat]:
try:
with open('cats.json', 'r') as fd:
return [Cat(**x) for x in json.load(fd)]
except FileNotFoundError:
return []
def save_cat(c):
data = [asdict(x) for x in load_cats() + [c]]
with open('cats.json', 'w') as fd:
json.dump(data, fd)
c = Cat(name='simon')
save_cat(c)
cats = load_cats()
print(cats)
A simplest approach I can suggest would be dataclasses.asdict as mentioned; or else, using a serialization library that supports dataclasses. There are a lot of good ones out there, but for this purpose I might suggest dataclass-wizard. Further, if you want to transform an arbitrary JSON object to dataclass structure, you can use the included CLI tool. When serializing, it will autoamtically apply a key transform (snake_case to camelCase) but this is easily customizable as well.
Disclaimer: I am the creator (and maintener) of this library.
I have a JSON file and i need to read it into dictionary or list without using and library.This is my file content.
{
"101":"Break and Enter Commercial",
"102":"Break and Enter Residential/Other",
"103":"Vehicle Collision or Pedestrian Struck (with Fatality)",
"104":"Vehicle Collision or Pedestrian Struck (with Injury)"
}
This is what i try
def read_codes(filename):
jsonData = {}
# empty list to append to it later
file = open(filename, "r")
for key in file:
print(key)
return jsonData
print(read_codes('codes.json'))
What about such way:
with open(file) as f:
your_dict = eval(f.read().replace('\n', ''))
You could open it as a text file. It would return you list, then filter the list as you require.
with open('file.json', 'r') as jsonFile:
json_obj = jsonFile.readlines()
json_obj = [(obj.rstrip()).lstrip()[:-1] for obj in json_obj[1:-1]]
print(json_obj)
The task:
Your program should read from the file, storing the names and
corresponding email addresses in a dictionary as key-value pairs.
Then, the program should display a menu that lets the user enter the
numbers 1 through 5, each corresponding to a different menu item:
When the user enters 5, the program should write the names and email
addresses in alphabetical order by first name to the file
phonebook.out You can use the sorted() function which accepts a
dictionary argument to sort a dictionary based on Key
This is my code:
def write_to_file(contact):
file = open("phonebook.out", "w")
contactsort = dict(sorted(contact.items()))
phonebook.write(contact)
phonebook.close
However, this code isn't working. I'm not sure why, so any help is appreciated. thank you.
Have you tried json file?
Like this:
import json
filename = "phonebook.json"
def write_to_file(contact):
with open(filename, 'w') as f_obj:
contactsort = dict(sorted(contact.items()))
json.dump(contact, f_obj)
This is your code:
def write_to_file(contact):
file = open("phonebook.out", "w")
contactsort = dict(sorted(contact.items()))
phonebook.write(contact)
phonebook.close
As #Cheche mentioned, you are declaring the output as file but using it as phonebook. Simply replace file = open("phonebook.out", "w") with phonebook = open("phonebook.out", "w"). Also, you are storing the sorted names to contactsort but writing contact to the file. As a side note, phonebook.close needs to be be phonebook.close() with the parentheses to call the function.
The way you sort the dict is incorrect. Try:
contactsort = {key: contact[key] for key in sorted(contact.iterkeys())}
Also, you should try to use with when possible. with takes care of closing the file for you. Final code:
def write_to_file(contact):
with open("phonebook.out", "w") as phonebook:
contactsort = {key: contact[key] for key in sorted(contact.iterkeys())}
phonebook.write(str(contactsort))
def write_to_file(contact):
phonebook = open("phonebook.out", "w")
contactsort = dict(sorted(contact.items()))
phonebook.write(str(contactsort))
phonebook.close()
write_to_file({"name":9090909090, "name_a":8080808080})
here You go
Hi I am trying to take the data from a json file and insert and id then perform POST REST.
my file data.json has:
{
'name':'myname'
}
and I would like to add an id so that the json data looks like:
{
'id': 134,
'name': 'myname'
}
So I tried:
import json
f = open("data.json","r")
data = f.read()
jsonObj = json.loads(data)
I can't get to load the json format file.
What should I do so that I can convert the json file into json object and add another id value.
Set item using data['id'] = ....
import json
with open('data.json', 'r+') as f:
data = json.load(f)
data['id'] = 134 # <--- add `id` value.
f.seek(0) # <--- should reset file position to the beginning.
json.dump(data, f, indent=4)
f.truncate() # remove remaining part
falsetru's solution is nice, but has a little bug:
Suppose original 'id' length was larger than 5 characters. When we then dump with the new 'id' (134 with only 3 characters) the length of the string being written from position 0 in file is shorter than the original length. Extra chars (such as '}') left in file from the original content.
I solved that by replacing the original file.
import json
import os
filename = 'data.json'
with open(filename, 'r') as f:
data = json.load(f)
data['id'] = 134 # <--- add `id` value.
os.remove(filename)
with open(filename, 'w') as f:
json.dump(data, f, indent=4)
I would like to present a modified version of Vadim's solution. It helps to deal with asynchronous requests to write/modify json file. I know it wasn't a part of the original question but might be helpful for others.
In case of asynchronous file modification os.remove(filename) will raise FileNotFoundError if requests emerge frequently. To overcome this problem you can create temporary file with modified content and then rename it simultaneously replacing old version. This solution works fine both for synchronous and asynchronous cases.
import os, json, uuid
filename = 'data.json'
with open(filename, 'r') as f:
data = json.load(f)
data['id'] = 134 # <--- add `id` value.
# add, remove, modify content
# create randomly named temporary file to avoid
# interference with other thread/asynchronous request
tempfile = os.path.join(os.path.dirname(filename), str(uuid.uuid4()))
with open(tempfile, 'w') as f:
json.dump(data, f, indent=4)
# rename temporary file replacing old file
os.rename(tempfile, filename)
There is really quite a number of ways to do this and all of the above are in one way or another valid approaches... Let me add a straightforward proposition. So assuming your current existing json file looks is this....
{
"name":"myname"
}
And you want to bring in this new json content (adding key "id")
{
"id": "134",
"name": "myname"
}
My approach has always been to keep the code extremely readable with easily traceable logic. So first, we read the entire existing json file into memory, assuming you are very well aware of your json's existing key(s).
import json
# first, get the absolute path to json file
PATH_TO_JSON = 'data.json' # assuming same directory (but you can work your magic here with os.)
# read existing json to memory. you do this to preserve whatever existing data.
with open(PATH_TO_JSON,'r') as jsonfile:
json_content = json.load(jsonfile) # this is now in memory! you can use it outside 'open'
Next, we use the 'with open()' syntax again, with the 'w' option. 'w' is a write mode which lets us edit and write new information to the file. Here s the catch that works for us ::: any existing json with the same target write name will be erased automatically.
So what we can do now, is simply write to the same filename with the new data
# add the id key-value pair (rmbr that it already has the "name" key value)
json_content["id"] = "134"
with open(PATH_TO_JSON,'w') as jsonfile:
json.dump(json_content, jsonfile, indent=4) # you decide the indentation level
And there you go!
data.json should be good to go for an good old POST request
try this script:
with open("data.json") as f:
data = json.load(f)
data["id"] = 134
json.dump(data, open("data.json", "w"), indent = 4)
the result is:
{
"name":"mynamme",
"id":134
}
Just the arrangement is different, You can solve the problem by converting the "data" type to a list, then arranging it as you wish, then returning it and saving the file, like that:
index_add = 0
with open("data.json") as f:
data = json.load(f)
data_li = [[k, v] for k, v in data.items()]
data_li.insert(index_add, ["id", 134])
data = {data_li[i][0]:data_li[i][1] for i in range(0, len(data_li))}
json.dump(data, open("data.json", "w"), indent = 4)
the result is:
{
"id":134,
"name":"myname"
}
you can add if condition in order not to repeat the key, just change it, like that:
index_add = 0
n_k = "id"
n_v = 134
with open("data.json") as f:
data = json.load(f)
if n_k in data:
data[n_k] = n_v
else:
data_li = [[k, v] for k, v in data.items()]
data_li.insert(index_add, [n_k, n_v])
data = {data_li[i][0]:data_li[i][1] for i in range(0, len(data_li))}
json.dump(data, open("data.json", "w"), indent = 4)
This implementation should suffice:
with open(jsonfile, 'r') as file:
data = json.load(file)
data[id] = value
with open(jsonfile, 'w') as file:
json.dump(data, file)
using context manager for the opening of the jsonfile.
data holds the updated object and dumped into the overwritten jsonfile in 'w' mode.
Not exactly your solution but might help some people solving this issue with keys.
I have list of files in folder, and i need to make Jason out of it with keys.
After many hours of trying the solution is simple.
Solution:
async def return_file_names():
dir_list = os.listdir("./tmp/")
json_dict = {"responseObj":[{"Key": dir_list.index(value),"Value": value} for value in dir_list]}
print(json_dict)
return(json_dict)
Response look like this:
{
"responseObj": [
{
"Key": 0,
"Value": "bottom_mask.GBS"
},
{
"Key": 1,
"Value": "bottom_copper.GBL"
},
{
"Key": 2,
"Value": "copper.GTL"
},
{
"Key": 3,
"Value": "soldermask.GTS"
},
{
"Key": 4,
"Value": "ncdrill.DRD"
},
{
"Key": 5,
"Value": "silkscreen.GTO"
}
]
}