I am using tcdb to hold a large key-value store. The keys are strings representing user IDs, the values are dicts of the form
{'coord':0,'node':0,'way':0,'relation':0}
The store is filled iterating over a data file that has coord, node, way and relation objects, each linked to a specific user. Here's my code for incrementing the fields:
def increment(self,uid,typ):
uid = str(uid)
type = str(typ)
try:
self.cache[uid][typ] += 1
except KeyError:
try:
self.cache[uid][typ] = 1
except KeyError:
try:
print 'creating record for %s' % uid
self.cache[uid] = {'coord':0,'node':0,'way':0,'relation':0}
except KeyError:
print 'something\'s messed up'
This does not work. I end up with a table that has all zero values:
def result(self):
print 'cache is now %i records' % len(self.cache)
for key in self.cache:
print key + ': ' + str(self.cache[key])
yields:
...
4951: {'node': 0, 'coord': 0, 'relation': 0, 'way': 0}
409553: {'node': 0, 'coord': 0, 'relation': 0, 'way': 0}
92274: {'node': 0, 'coord': 0, 'relation': 0, 'way': 0}
259040: {'node': 0, 'coord': 0, 'relation': 0, 'way': 0}
...
Why?
The last exception is never called.
EDIT This code in the first try block:
tempdict = self.cache[uid]
tempdict[typ] = tempdict.get(typ,0) + 1
self.cache[uid] = tempdict
instead of the original
self.cache[uid][typ] += 1
works, but looks really ugly to me.
After this line:
self.cache[uid] = {'coord':0,'node':0,'way':0,'relation':0}
Add this:
self.cache[uid][type] = 1
Also, please don't use type as a variable name as it hides the built-in of the same name.
Related
I tried to update main dictionary with every new one in the loop, but every time it overwrite values inside the nested dictionary.
I mean I want smth like this:
{'Barbour': {1900: 73041, 1910: 895427, 1920: 1531624, 1930: 1617086, 1940: 1420561, 1950: 1853223,
1960: 3092728, 1970: 3505193, 1980: 3659797, 1990: 2575561, 2000: 743757, 2010: 1730711},
'Berkeley': {1900: 0, 1910: 0, 1920: 0, 1930: 0, 1940: 0, 1950: 0, 1960: 0, 1970: 0, 1980: 0, 1990:
0, 2000: 0, 2010: 0}}
(for all of the cities)
def read_file_contents_coal():
return ['Barbour,73041,895427,1531624,1617086,1420561,1853223,3092728,3505193,3659797,2575561,743757,1730711\n',
'Berkeley,0,0,0,0,0,0,0,0,0,0,0,0\n',
'Boone,0,50566,1477560,3045056,3804527,5851267,6278609,11607216,13842525,27618152,32446186,23277998\n',
'Braxton,0,114422,286955,123991,13751,38414,218087,0,459517,3256906,1196489,439662\n',
]
def process_file_contents():
lst = read_file_contents_coal()
dic = {}
coal_dic = {}
yearcoaldic = {}
ycdic = {}
for stringdata in lst:
city_data = stringdata.strip().split(',')
year = 1900
for j in range(1, 13):
ycdic = {year: int(city_data[j])}
year += 10
yearcoaldic.update(ycdic)
dic = {city_data[0]: yearcoaldic}
coal_dic.update(dic)
print(coal_dic)
return coal_dic
[EDIT]: the issue is that you have to move yearcoaldic to the first loop and always set it to en empty dictionary otherwise you will always overwrite your values as you have experienced.
def process_file_contents():
lst = read_file_contents_coal()
dic = {}
coal_dic = {}
ycdic = {}
for stringdata in lst:
yearcoaldic = {}
city_data = stringdata.strip().split(',')
year = 1900
for j in range(1, 13):
ycdic = {year: int(city_data[j])}
year += 10
yearcoaldic.update(ycdic)
# dic = {city_data[0]: yearcoaldic}
dic[city_data[0]] = yearcoaldic
# coal_dic.update(dic)
# print(coal_dic)
return dic
Sample json is below. I want to save id which is completed (False and True) into seperated dictionaryies
todos = [{'userId': 1, 'id': 1, 'title': 'delectus aut autem', 'completed': False},
{'userId': 1, 'id': 2, 'title': 'quis ut nam facil ', 'completed': False},
{'userId': 1, 'id': 1, 'title': 'fugiat veniam minus', 'completed': False},
{'userId': 1, 'id': 2, 'title': 'et porro tempora', 'completed': True},
{'userId': 1, 'id': 1,'title': 'laprovident illum', 'completed': False}]
Expected out is below
todos_by_user_true = {1:0,2:1}
todos_by_user_false = {1:3,2:1}
code is below? Why my code not working. I am getting blank dictionary
todos_by_user_true = {}
todos_by_user_false = {}
# Increment complete TODOs count for each user.
for todo in todos:
if todo["completed"]==True:
try:
# Increment the existing user's count.
todos_by_user_true[todo["id"]] += 1
except KeyError:
# This user has not been seen. Set their count to 1.
todos_by_user_true[todo["id"]] = 0
elif todo["completed"]==False:
try:
# Increment the existing user's count.
todos_by_user_false[todo["id"]] += 1
except KeyError:
# This user has not been seen. Set their count to 1.
todos_by_user_false[todo["id"]] = 0
I am getting not proper dictionary
My output is below
todos_by_user_false {1: 2, 2: 0}
todos_by_user_true {2: 0}
Disclaimer: I need to take care of exception also
Looking at your input data, it is so that:
userId 1, id 1 has 0 true, and 3 false
userId 1, id 2 has 1 true, and 1 false
Given the required output, it looks like you really want to use id rather than userId in your lookups. Besides that, there's an issue with accounting the first time you insert the id in the resulting dictionary. I would fix it like this:
todos_by_user_true = {}
todos_by_user_false = {}
# Increment complete TODOs count for each user.
for todo in todos:
if todo["completed"]==True:
try:
# Increment the existing user's count.
todos_by_user_true[todo["id"]] += 1
except KeyError:
# This user has not been seen. Set their count to 1.
todos_by_user_true[todo["id"]] = 1
elif todo["completed"]==False:
try:
# Increment the existing user's count.
todos_by_user_false[todo["id"]] += 1
except KeyError:
# This user has not been seen. Set their count to 1.
todos_by_user_false[todo["id"]] = 1
which (btw) is already what's in your comments.
Personally, I would check the dictionary for the key before insertion, instead of using try..except, like this:
todos_by_user_true = {}
todos_by_user_false = {}
# Increment complete TODOs count for each user.
for todo in todos:
key = todo["id"]
if todo["completed"]: # true case
# If `id` not there yet, insert it to 0
if key not in todos_by_user_true:
todos_by_user_true[key] = 0
# increment
todos_by_user_true[key] += 1
else: # false case
# If `id` not there yet, insert it to 0
if key not in todos_by_user_false:
todos_by_user_false[key] = 0
# increment
todos_by_user_false[key] += 1
This gives out:
todos_by_user_true = {2:1}
todos_by_user_false = {1:3,2:1}
The logic being this, you cannot have:
todos_by_user_true = {1:0}
You account for the value when you find it; rather than iterating for id from a separate list.
Here is an example dataset
Firstly, I try to create a dict from values in rows:
import csv
who = set()
figure = set()
date = set()
action = []
activity = {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0}
with open(r'ShtrudelT.csv',
mode = 'r') as csv_file:
lines = csv_file.readlines()
for row in lines:
data = row.split(',')
who.add(data[1])
figure.add(data[2])
date.add(data[3][:7])
action.append(data[4].strip())
xdict = dict.fromkeys(who,
dict.fromkeys(figure,
dict.fromkeys(date, activity)))
The result is:
{'Googenhaim': {'Circle': {'2020-04': {'play': 0,'throw': 0, 'pin': 0, 'tap': 0},
'2020-06': {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0},
'2020-05': {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0}},
'Rectangle': {'2020-04': {'play': 0, 'throw': 0, 'pin': 0, 'tap': 0}...}
Secondly, I need to count actions divided by key to analyze data. For example, how many times Googenhaim use Circle by any type of action in every month.
Is there a solution without using Pandas?
import csv
count_dict = {}
with open(r'ShtrudelT.csv',
mode = 'r') as csv_file:
lines = csv_file.readlines()
for row in lines:
data = row.split(',')
key = data[1] + "_" + data[2] + "_" + data[3][:7] + "_" + data[4].strip()
if key in count_dict:
count_dict[key] += 1
else:
count_dict[key] = 1
print("\t".join(["Name", "Shape", "Month", "Action", "Count"]))
for element, count in count_dict.items():
items = element.split("_")
print("\t".join(items) + "\t" + str(count))
We use a dictionary where every key is the combination that we want to count. This combination is formed from name of the user, shape, month and the action. While processing every line, we form the key and store it in the dictionary. If it is encountered for the first time then we insert it or else we update the count.
After all the lines are processed, we can do any kind of post processing we want to do.
Hope that solves it.
Hello I'm VERY new in python. I just have to do 1 thing with it.
When i print my string names, this is what comes up:
{'id': 1, 'xd_id': 2, 'name': 'nameea', 'description': 'somethingveryweird', 'again_id': 6, 'some_id': None, 'everything': False, 'is_ready': False, 'test_on': None, 'something': None, 'something': [], 'count_count': 28, 'other_count': 0, 'again_count': 0, 'new_count': 0, 'why_count': 0, 'custom_count': 0, 'custom2_count': 0, 'custom3_count': 0, 'custom4_count': 0, 'custom5_count': 0, 'custom_status6_count': 0, 'custom7_count': 0, 'lol_id': 7, 'wtf_id': None, 'numbers_on': 643346, 'something_by': 99, 'site': 'google.com'}
I would to get this info to excel with the left row being the "id": and the right being the 1. And all the info like this. for example. "site" on the left and "google.com" on the right. my current code adds all this info to the first row on the excel and i can't seem to find any tutorial for this. Thanks for all answers. My current code:
f = open('test.csv', 'w')
s = str(names)
f.write(s)
f.close()
if python is not going to be your key skill and only this task needs to be done, then here is the answer.
f = open('test.csv', 'w')
csvwt = csv.writer(f)
for x in names.items():
csvwt.writerow(x)
f.close()
if you want to write to an excel, then you have to do this,
workbook = xlsxwriter.Workbook('test.xlsx')
worksheet = workbook.add_worksheet()
row = 0
col = 0
for x in names.items():
worksheet.write(row, col, str(x[0]))
worksheet.write(row, col + 1, str(x[1]))
row += 1
workbook.close()
The function's return not meeting expectations.
My env py2.7.11 centos6.5.
I hope the function return the return like this
{'www.baidu.com': {'3xx': 0, 'response_time': 0.126, '5xx': 0, '4xx': 1},
'www.google.com': {'3xx': 0, 'response_time': 0, '5xx': 0, '4xx': 0}}
But in fact it is return like that. The send_dict['www.google.com']['response_time'] and send_dict['www.google.com']['response_code'] shouldn't be assigned. But why?
{'www.baidu.com': {'3xx': 0, 'response_time': 0.126, '5xx': 0, '4xx': 1},
'www.google.com': {'3xx': 0, 'response_time': 0.126, '5xx': 0, '4xx': 1}}
The Python code:
#!/usr/bin/python
# -*- coding: utf-8 -*-
sink_dict = {'sink_zabbix_monitor_keys': '3xx,4xx,5xx,response_time',
'sink_zabbix_domain_keys': 'www.baidu.com,www.google.com'}
sub_send_dict = dict.fromkeys(sink_dict['sink_zabbix_monitor_keys'].split(','), 0)
send_dict = dict.fromkeys(sink_dict['sink_zabbix_domain_keys'].split(','), sub_send_dict)
def calculate_item(item):
response_code_dict = dict.fromkeys(sink_dict['sink_zabbix_domain_keys'].split(','), 0)
response_time_dict = dict.fromkeys(sink_dict['sink_zabbix_domain_keys'].split(','), 0)
domain = item['domain']
response_code_dict[domain] = int(item['response_code'])
response_time_dict[domain] = float(item['response_time'])
if domain in send_dict:
print domain
if response_time_dict[domain] > float(send_dict[domain]['response_time']):
send_dict[domain]['response_time'] = response_time_dict[domain]
send_dict[domain][str(response_code_dict[domain])[0] + "xx"] += 1
return send_dict
tmp_item = {'domain': 'www.baidu.com', 'response_time': '0.126', 'response_code': '401'}
tmp_item1 = {'domain': 'www.google.com', 'response_time': '0.126', 'response_code': '401'}
tmp_item2 = {'domain': 'www.baidu.com', 'response_time': '0.166', 'response_code': '401'}
print calculate_item(tmp_item)
Here you go:
send_dict = {k: sub_send_dict.copy() for k in sink_dict['sink_zabbix_domain_keys'].split(',')}
This is called dict comprehension. This works because I'm assigning a shallow copy of the dictionary to the key. dict.fromkeys() will assign the value to every key, all keys will share the same sub_send_dict reference.
From the (linked) python docs:
Assignment statements in Python do not copy objects, they create bindings between a target and an object. For collections that are mutable or contain mutable items, a copy is sometimes needed so one can change one copy without changing the other.