This is my file: test.txt
Amy|Female|Desc1|12
John|Male|Desc2|10
Mike|Male|Desc3|18
I tried to create nested dictionary and it's not sucessful.
This is the output:
{'Amy': '12', 'John': '10', 'Mike': '18'}
This is my code:
import csv
with open('test.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
d = {}
for row in tsvfile:
d[row[0]] = row[0] #this should be name
d[row[0]] = row[1] #this should be gender
d[row[0]] = row[3] #this should be desc
d[row[0]] = row[3] #this should be age
print(d)
My desired output as below but was not successful.
d={1{'Name':'Amy', 'Gender':'Female', 'Desc': 'Desc1', 'Age': '12'}
2{'Name':'John', 'Gender':'Male', 'Desc': 'Desc2', 'Age': '10'}
3{'Name':'Mike', 'Gender':'Male', 'Desc': 'Desc3', 'Age': '18'}}
and below (with name and age only
d1={1{'Name':'Amy','Age': '12'}
2{'Name':'John', 'Age': '10'}
3{'Name':'Mike', 'Age': '18'}}
Here's how to do it without csv import, given the data format is constant:
fixed = {}
i = 1
with open("test.txt", 'r') as f:
for line in f:
listDetails = line.strip().split('|')
fixed[i] = {"Name": listDetails[0]}
fixed[i].update({"Sex": listDetails[1]})
fixed[i].update({"Description": listDetails[2]})
fixed[i].update({"Age": listDetails[3]})
i+=1
print(fixed)
This should turn
Amy|Female|Desc1|12
John|Male|Desc2|10
Mike|Male|Desc3|18
To
{1: {'Name': 'Amy', 'Sex': 'Female', 'Description': 'Desc1', 'Age': '12'}, 2: {'Name': 'John', 'Sex': 'Male', 'Description': 'Desc2', 'Age': '10'}, 3: {'Name': 'Mike', 'Sex': 'Male', 'Description': 'Desc3', 'Age': '18'}}
Edit: Just as Nakor said though, it doesn't really make sense to make a dict of dicts here, just posted this if you really need to make it a dict.
1) Nested Dictionary, I have made some changes in the same code, it may help you.
import csv
with open('hello.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
final_dict = {}
counter = 1
for row in tsvfile:
d = {}
d['Name'] = row[0] #this should be name
d['Gender'] = row[1] #this should be gender
d['Desc'] = row[2] #this should be desc
d['Age'] = row[3] #this should be age
final_dict[counter] = d
counter+=1
print(final_dict)
Your desired output looks more like a list of dictionaries.
In this case, I would just modify your code like this:
import csv
with open('test.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
d = []
for row in tsvfile:
entry = {
'Name': row[0],
'Gender': row[1],
'Desc': row[2],
}
d.append(entry)
print(d)
Output:
[{'Name': 'Amy', 'Gender': 'Female', 'Desc': 'Desc1'},
{'Name': 'John', 'Gender': 'Male', 'Desc': 'Desc2'},
{'Name': 'Mike', 'Gender': 'Male', 'Desc': 'Desc3'}]
You can even write the loop in a more compact way like this:
keys = ["Name","Gender","Desc"]
for row in tsvfile:
entry = { key: value for (key,value) in zip(keys,row) }
d.append(entry)
EDIT: If you want a dictionary with the line number as keys, you can do:
import csv
with open('test.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
d = {}
keys = ["Name","Gender","Desc"]
for i,row in enumerate(tsvfile):
entry = {
'Name': row[0],
'Gender': row[1],
'Desc': row[2],
}
d[i+1] = entry
print(d)
Related
I'm trying to convert an array with a dictionary to a flattened dictionary and export it to a JSON file. I have an initial tab-delimited file, and have tried multiple ways but not coming to the final result. If there is more than one row present then save these as arrays in the dictionary
Name file code file_location
TESTLIB1 443 123 location1
TESTLIB2 444 124 location2
Current Output:
{'library': 'TESTLIB2', 'file': '444', 'code': '124', 'file_location': 'location2'}
Desired Output if num_lines > 1:
{'library': ['TEST1', 'TEST2'], 'file': ['443', '444'], 'code': ['123', 123], 'file_location': ['location1', 'location2]}
Code Snippet
data_dict = {}
with open('file.tmp') as input:
reader = csv.DictReader(input, delimiter='\t')
num_lines = sum(1 for line in open('write_object.tmp'))
for row in reader:
data_dict.update(row)
if num_lines > 1:
data_dict.update(row)
with open('output.json', 'w') as output:
output.write(json.dumps(data_dict))
print(data_dict)
create list for each column and iterate to append row by row
import csv
import json
# read file
d = {}
with open('write_object.tmp') as f:
reader = csv.reader(f, delimiter='\t')
headers = next(reader)
for head in headers:
d[head] = []
for row in reader:
for i, head in enumerate(headers):
d[head].append(row[i])
# save as json file
with open('output.json', 'w') as f:
json.dump(d, f)
output:
{'Name': ['TESTLIB1', 'TESTLIB2'],
'file': ['443', '444'],
'code': ['123', '124'],
'file_location': ['location1', 'location2']}
from collections import defaultdict
data_dict = defaultdict(list)
with open('input-file') as inp:
for row in csv.DictReader(inp, delimiter='\t'):
for key, val in row.items():
data_dict[key].append(val)
print(data_dict)
# output
{'Name': ['TESTLIB1', 'TESTLIB2'],
'file': ['443', '444'],
'code': ['123', '124'],
'file_location': ['location1', 'location2']}
there are list :
data = ['man', 'man1', 'man2']
key = ['name', 'id', 'sal']
man_res = ['Alexandra', 'RST01', '$34,000']
man1_res = ['Santio', 'RST009', '$45,000']
man2_res = ['Rumbalski', 'RST50', '$78,000']
the expected output will be nested output:
Expected o/p:- {'man':{'name':'Alexandra', 'id':'RST01', 'sal':$34,000},
'man1':{'name':'Santio', 'id':'RST009', 'sal':$45,000},
'man2':{'name':'Rumbalski', 'id':'RST50', 'sal':$78,000}}
Easy way would be using pandas dataframe
import pandas as pd
df = pd.DataFrame([man_res, man1_res, man2_res], index=data, columns=key)
print(df)
df.to_dict(orient='index')
name id sal
man Alexandra RST01 $34,000
man1 Santio RST009 $45,000
man2 Rumbalski RST50 $78,000
{'man': {'name': 'Alexandra', 'id': 'RST01', 'sal': '$34,000'},
'man1': {'name': 'Santio', 'id': 'RST009', 'sal': '$45,000'},
'man2': {'name': 'Rumbalski', 'id': 'RST50', 'sal': '$78,000'}}
Or you could manually merge them using dict + zip
d = dict(zip(
data,
(dict(zip(key, res)) for res in (man_res, man1_res, man2_res))
))
d
{'man': {'name': 'Alexandra', 'id': 'RST01', 'sal': '$34,000'},
'man1': {'name': 'Santio', 'id': 'RST009', 'sal': '$45,000'},
'man2': {'name': 'Rumbalski', 'id': 'RST50', 'sal': '$78,000'}}
#save it in 2D array
all_man_res = []
all_man_res.append(man_res)
all_man_res.append(man1_res)
all_man_res.append(man2_res)
print(all_man_res)
#Add it into a dict output
output = {}
for i in range(len(l)):
person = l[i]
details = {}
for j in range(len(key)):
value = key[j]
details[value] = all_man_res[i][j]
output[person] = details
output
The pandas dataframe answer provided by NoThInG makes the most intuitive sense. If you are looking to use only the built in python tools, you can do
info_list = [dict(zip(key,man) for man in (man_res, man1_res, man2_res)]
output = dict(zip(data,info_list))
import csv
keys = ["id", "name", "age", "height", "weight"]
with open('temp.csv', 'w') as temp_file:
dict_writer_obj = csv.DictWriter(temp_file, fieldnames = keys)
with open('dictReader.csv','r') as file:
dict_reader_obj = csv.DictReader(file)
dict_writer_obj.writeheader()
dict_writer_obj.writerows(file)
I want to convert a csv file called dictReader.csv file into dictionary based file:
However I am getting the following error. Any ideas?
AttributeError: 'str' object has no attribute 'keys'
My dictReader.csv file content:
id,name,age,height,weight
1,Alice,20,62,120.6
2,Freddie,21,74,190.6
3,Bob,17,68,120.0
Desired output file called temp.csv with this format
{'id': '1', 'name': 'Alice', 'age': '20', 'height': '62', 'weight': '120.6'}
{'id': '2', 'name': 'Freddie', 'age': '21', 'height': '74', 'weight': '190.6'}
{'id': '3', 'name': 'Bob', 'age': '17', 'height': '68', 'weight': '120.0'}
To improve on the other user's answer a bit, you can still use writerows like this.
import csv
keys = ["id", "name", "age", "height", "weight"]
with open('temp.csv', 'w') as temp_file:
dict_writer_obj = csv.DictWriter(temp_file, fieldnames = keys)
with open('dictReader.csv','r') as file:
dict_reader_obj = csv.DictReader(file)
dict_writer_obj.writeheader()
# Here:
dict_writer_obj.writerows(row for row in dict_reader_obj)
Just change:
dict_writer_obj.writerows(file)
to:
dict_writer_obj.writerows(row for row in dict_reader_obj)
Or row by row using .writerow():
for row in dict_reader_obj:
dict_writer_obj.writerow(row)
I am missing something small here and could use a pointer. I am trying to generate data to save time for my work with CRUD work in a database via pymonogo and other pythonic database libraries. Below is the code that I am having trouble with. I would like to create a function which creates a dictionary of length n but I cannot figure out how to append the dictionary appropriately. As you can see, it only enters in the last item of the list generated. Any input would be great!
import names
import random
import numpy as np
age_choices = np.arange(18, 90)
gender_choices = ['male', 'female']
salary_choices = np.arange(10000, 200000)
def create_data(n=20):
age_choices = np.arange(18, 90)
gender_choices = ['male', 'female']
salary_choices = np.arange(10000, 200000)
person_values = []
data_dict = {}
unique_id = 0
while unique_id < n:
age = random.choice(age_choices)
gender = random.choice(gender_choices)
salary = random.choice(salary_choices)
person_keys = ['id', 'name', 'gender', 'age', 'salary']
person_values = [unique_id, names.get_full_name(gender), gender, age, salary]
for k, v in zip(person_keys, person_values):
data_dict[k] = v
unique_id += 1
return person_values, data_dict
data_list, data_dict = create_data(5)
print(data_list)
print()
print(data_dict)
current outputs:
[4, 'Anthony Shultz', 'male', 29, 188503] # This is the last item of the list generated in the while loop
{'id': 4, 'name': 'Anthony Shultz', 'gender': 'male', 'age': 29, 'salary': 188503} # This is the "whole" dictionary generated but should have length 5 since n=5
The desired out put should be a dictionary of length n not just one.
You should introduce another variable in your function which would be a list or tuple and append each data_dict to it, every time you create one. You should also create a unique data_dict in your while loop, on every iteration. For example (check the lines with comments):
import names
import random
import numpy as np
age_choices = np.arange(18, 90)
gender_choices = ['male', 'female']
salary_choices = np.arange(10000, 200000)
def create_data(n=20):
age_choices = np.arange(18, 90)
gender_choices = ['male', 'female']
salary_choices = np.arange(10000, 200000)
person_values = []
all_data = [] # Make a list which will store all our dictionaries
unique_id = 0
while unique_id < n:
data_dict = {} # Create a dictionary with current values
age = random.choice(age_choices)
gender = random.choice(gender_choices)
salary = random.choice(salary_choices)
person_keys = ['id', 'name', 'gender', 'age', 'salary']
person_values = [unique_id, names.get_full_name(gender), gender, age,
salary]
for k, v in zip(person_keys, person_values):
data_dict[k] = v
all_data.append(data_dict) # Add newly created `data_dict` dictionary to our list
unique_id += 1
return person_values, data_dict, all_data # Return as desired
data_list, data_dict, all_data = create_data(5) # Just as an example
print(data_list)
print()
print(data_dict)
print()
print(all_data) # Print the output
This will result in list of dictionaries, which I assume you want as an output, e.g.:
[{'id': 0, 'name': 'David Medina', 'gender': 'male', 'age': 87, 'salary': 67957}, {'id': 1, 'name': 'Valentina Reese', 'gender': 'female', 'age': 68, 'salary': 132938}, {'id': 2, 'name': 'Laura Franklin', 'gender': 'female', 'age': 84, 'salary': 93839}, {'id': 3, 'name': 'Melita Pierce', 'gender': 'female', 'age': 21, 'salary': 141055}, {'id': 4, 'name': 'Brenda Clay', 'gender': 'female', 'age': 36, 'salary': 94385}]
my csv file is below
Uid,locate,category,act
Anna,NY,house,dance
Anna,LA,house,sing
Anna,CH,house,ride
John,NY,house,dance
John,LA,home,sing
John,CH,home,ride
and i want to create dictionary just like
{'Uid': 'Anna', 'infos':[{'locate': 'NY', 'category': 'house', 'act': 'dance'},
{'locate': 'LA', 'category': 'house', 'act': 'sing'},
{'locate': 'CH', 'category': 'house', 'act': 'ride'}]
},
{'Uid': 'John', 'infos':[{'locate': 'NY', 'category': 'house', 'act': 'dance'},
{'locate': 'LA', 'category': 'home', 'act': 'sing'},
{'locate': 'CH', 'category': 'home', 'act': 'ride'}]
},
my code is below:
result = {}
with open('test.csv') as fp:
reader = csv.DictReader(fp)
for row in reader:
result['test_uid'] = row['Uid']
result["test_locate"] = row['locate']
result["test_category"] = row['category']
result["test_act"] = row['act']
print(result)
how to append the infos datas to the same person?
how to fix my code that can print the result I want??
Need someone help please.
Please try the following:
payload = {}
# first let create a dict with uid as a key and a list of infos as a value.
with open('test.csv') as fp:
reader = csv.DictReader(fp)
for row in reader:
data = {"locate": row['locate'], 'category': row['category'],
'act': row['act']}
if row['Uid'] in payload.keys():
payload[row['Uid']].append(data)
else:
payload[row['Uid']] = [data]
# reconstruct the payload to be list of dicts in the structure you want
result = list(map(lambda uid, infos: {'Uid':uid, 'infos':infos}, payload.items()))
I would change the resulting datastructure a bit so it becomes easier to handle:
result = {}
with open('test.csv') as fp:
reader = csv.DictReader(fp)
for row in reader:
if row['Uid'] not in result:
result[row['Uid']] = [{
'test_locate': row['locate'],
'test_category': row['category'],
'test_act': row['act']}]
else:
result[row['Uid']].append({
'test_locate': row['locate'],
'test_category': row['category'],
'test_act': row['act']})
print(result)
Your output looks like a list, but you need a dictionary. Try this:
result = {}
with open('test.csv') as fp:
reader = csv.DictReader(fp)
for row in reader:
uid = row['Uid']
del row['Uid']
if uid in result:
result[uid]['infos'].append(row)
else:
result[uid] = {'infos': [row]}
print(result)
The result you are showing is actually a list of dictionaries. If that is what you want, then:
result = []
with open('test.csv') as fp:
reader = csv.DictReader(fp)
infos = []
last_uid = None
for row in reader:
uid = row['Uid']
if uid != last_uid:
if last_uid is not None:
result.append({'Uid': last_uid, 'infos': infos})
last_uid = uid
infos = []
last_uid = uid
infos.append({'locate': row['locate'], 'category': row['category'], 'act': row['act']})
if last_uid is not None:
result.append({'Uid': last_uid, 'infos': infos})