We Keep Coding

implement a simple NoSQL database interface based on a text file

In this test, you will have to implement a simple NoSQL database interface based on a text file. The get_() function takes the file path and key (any string) as input and returns the value for this key. If the value is missing, None is returned. The set_() function also accepts the path to the file, the key and the value (any string), and writes the value to the database using the key.
The sizes of the key and values are set by the constants KEY_LEN and VALUE_LEN.
What i should make:
db = open('./nosql.db', 'w')
path = db.name # ./nosql.db
set_(path, 'key1', 'value1')
get_(path, 'key1') # 'value1'
set_(path, 'key1', 'value2')
get_(path, 'key1') # 'value2'
My code (yes, I haven't done much, but I think the idea is clear):
KEY_LEN = 10
VALUE_LEN = 20
# BEGIN (write your solution here)
def set_(path, key, value):
fr = open(path)
fw = open(path, 'w')
for l in fr:
ls = l.split(' ')
if ls[0] == key:
ls[1] = value
fr.writelines(' '.join(ls))
# END

Python shelve not saving nested keys

I am using the shelve module to save some Python objects (strings in the example).
When I am trying to save an object as a nested key, then it is not being saved.
class Car:
def __init__(self, ID):
self.id = ID
def save_to_shelf(self):
with shelve.open('shelf') as shelf:
if not shelf.get('users'):
shelf['users'] = {}
print(shelf['users'])
try:
shelf['users'][self.id] = "hello"
except Exception as e:
print(e)
print('saved')
print(shelf['users'])
car = Car(123)
car.save_to_shelf()
The code should print:
{}
saved
{123: hello}
But instead it prints:
{}
saved
{}
Meaning that it is not getting saved
If I print the value of the key, it gives KeyError
shelf['users'][self.id] = "hello"
print(shelf['users'][self.id])
Error
Traceback (most recent call last):
File "P:/Python practice/Shelll/main.py", line 3, in <module>
car.save_to_shelf()
File "P:/Python practice/Shelll\db.py", line 20, in save_to_shelf
print(shelf['users'][self.id])
KeyError: '123'
I am able to save it if I do the following while saving
Instead of
with shelve.open('shelf') as shelf:
shelf['users'][self.id] = "hello"
This works
with shelve.open('shelf') as shelf:
USERS = shelf['users']
USERS[self.id] = self.id
shelf['users'] = USERS
# or this works
# with shelve.open('shelf') as shelf:
# shelf['users'] = {self.id:"hello"}
I want to understand the reason behind this. As far as I know, shelve objects work as a dictionary. So the way I was saving earlier should work but does not.

Python script ran OK for weeks using cron job and it's now giving a KeyError

Here's the code I'm trying to run. Works when executed from PyCharm. I set up a cronjob and it worked wonders for weeks. It's now giving a KeyError out of the bloom. Can't find what's wrong since I haven't touched anything in the cronjob.
import csv
import json
import os
import random
import time
from urllib.parse import urlencode
import requests
API_URL = "https://community.koodomobile.com/widget/pointsLeaderboard?"
LEADERBOARD_FILE = "leaderboard_data.json"
def get_leaderboard(period: str = "allTime", max_results: int = 20) -> list:
payload = {"period": period, "maxResults": max_results}
return requests.get(f"{API_URL}{urlencode(payload)}").json()
def dump_leaderboard_data(leaderboard_data: dict) -> None:
with open("leaderboard_data.json", "w") as jf:
json.dump(leaderboard_data, jf, indent=4, sort_keys=True)
def read_leaderboard_data(data_file: str) -> dict:
with open(data_file) as f:
return json.load(f)
def parse_leaderboard(leaderboard: list) -> dict:
return {
item["name"]: {
"id": item["id"],
"score_data": {
time.strftime("%Y-%m-%d %H:%M"): item["points"],
},
"rank": item["leaderboardPosition"],
} for item in leaderboard
}
def update_leaderboard_data(target: dict, new_data: dict) -> dict:
for player, stats in new_data.items():
target[player]["rank"] = stats["rank"]
target[player]["score_data"].update(stats["score_data"])
return target
def leaderboard_to_csv(leaderboard: dict) -> None:
data_rows = [
[player] + list(stats["score_data"].values())
for player, stats in leaderboard.items()
]
random_player = random.choice(list(leaderboard.keys()))
dates = list(leaderboard[random_player]["score_data"])
with open("the_data.csv", "w") as output:
w = csv.writer(output)
w.writerow([""] + dates)
w.writerows(data_rows)
def script_runner():
if os.path.isfile(LEADERBOARD_FILE):
fresh_data = update_leaderboard_data(
target=read_leaderboard_data(LEADERBOARD_FILE),
new_data=parse_leaderboard(get_leaderboard()),
)
leaderboard_to_csv(fresh_data)
dump_leaderboard_data(fresh_data)
else:
dump_leaderboard_data(parse_leaderboard(get_leaderboard()))
if __name__ == "__main__":
script_runner()
Here's the error taht it gives me. Seems like there's a problem with dictionary hence the KeyError.
File "/Users/Rob/PycharmProjects/Koodo/TEST.Json.py", line 75, in <module>
script_runner()
File "/Users/Rob/PycharmProjects/Koodo/TEST.Json.py", line 64, in script_runner
fresh_data = update_leaderboard_data(
File "/Users/Rob/PycharmProjects/Koodo/TEST.Json.py", line 44, in update_leaderboard_data
target[player]["rank"] = stats["rank"]
KeyError: 'triggered123'
Here's the data in JSON file : https://pastebin.com/HQyL4Kyx

It looks to me like the first time your code is run, it caches the leaderboard.
On subsequent runs, it will update the existing leaderboard with the new values, by looping over each of the new entries, finding their username, and looking up that key in the older dictionary. However, if there's a new user, that key will not exist in the old dictionary.
You're getting the error because the player triggered123 is a new user to the leaderboard, and was listed after you first ran the script.
You need to update update_leaderboard_data to handle this case (by checking if the key exists in the dictionary before attempting to access it).

That's because the key named triggered123 is NOT present in the dictionary target.
Source: https://wiki.python.org/moin/KeyError
The problem is that in update_leaderboard_data you iterate over the new data and use them to lookup in the old data. If a new key does not already exists in the old data, you'll get the KeyError.
Try to print the dict target to see if the key triggered123 is in it.
Or use the get method of the dict with a default value: so no error wil be raised, and you'll be able to search this default value in your output.

Is there a way to write a correct-form json info to a .json file and read from it locally?

I am planning to use JSON file as a simple database, i am trying to append to it new entries and try to take my entries later.
This is the code i have:
import json
import time
try:
with open('json_database.json', 'r') as json_database:
profiles = json.load(json_database)
except FileNotFoundError:
profiles = []
while True:
answer = input('list info (l), write info (w), new info (a)').lower()
if answer == 'w':
break
elif answer == 'l':
print(profiles)
else:
username = input('username: ')
email = input('Email: ')
rating = input('Rating: ')
lichess_profiles.append({
'profile':{
'username': lichess_username,
'email': email,
'rating': rating
}
})
with open('json_database.json', 'w') as json_database:
json.dump(profiles, json_database)
Now i want to call the info from the JSON info ! thats what i added :
with open('json_database.json') as json_1:
result = json.load(json_1)
print(result['profile']['email'])
what is the reason of that ? what shall i add ?
i tried that code but it raise this error :
TypeError: list indices must be integers or slices, not str

The base item you are writing to the json file is a list, but you're treating it like a dictionary. It contains dictionaries, but you have to access it like a list:
print(result[0]['profile']['email'])
print(result[1]['profile']['email'])
# etc.

dictionary reading from module

I have following module root_file.py. This file contains number of blocks like.
Name = {
'1':'a'
'2':'b'
'3':'c'
}
In other file I am using
f1= __import__('root_file')
Now the requirement is that I have to read values a,b,c at runtime using variables like
for reading a
id=1
app=Name
print f1[app][id]
but getting error that
TypeError: unsubscriptable object

How about
import root_file as f1
id = 1
app = 'Name'
print getattr(f1, app)[id] # or f1.Name[id]

Uh, well, if I understand what you are trying to do:
In root_file.py
Name = {
'1':'a', #note the commas here!
'2':'b', #and here
'3':'c', #the last one is optional
}
Then, in the other file:
import root_file as mymodule
mydict = getattr(mymodule, "Name")
# "Name" could be very well be stored in a variable
# now mydict eqauls Name from root_file
# and you can access its properties, e.g.
mydict['2'] == 'b' # is a True statement