We Keep Coding

Convert tab delimited data into dictionary

I'm trying to convert an array with a dictionary to a flattened dictionary and export it to a JSON file. I have an initial tab-delimited file, and have tried multiple ways but not coming to the final result. If there is more than one row present then save these as arrays in the dictionary
Name file code file_location
TESTLIB1 443 123 location1
TESTLIB2 444 124 location2
Current Output:
{'library': 'TESTLIB2', 'file': '444', 'code': '124', 'file_location': 'location2'}
Desired Output if num_lines > 1:
{'library': ['TEST1', 'TEST2'], 'file': ['443', '444'], 'code': ['123', 123], 'file_location': ['location1', 'location2]}
Code Snippet
data_dict = {}
with open('file.tmp') as input:
reader = csv.DictReader(input, delimiter='\t')
num_lines = sum(1 for line in open('write_object.tmp'))
for row in reader:
data_dict.update(row)
if num_lines > 1:
data_dict.update(row)
with open('output.json', 'w') as output:
output.write(json.dumps(data_dict))
print(data_dict)

create list for each column and iterate to append row by row
import csv
import json
# read file
d = {}
with open('write_object.tmp') as f:
reader = csv.reader(f, delimiter='\t')
headers = next(reader)
for head in headers:
d[head] = []
for row in reader:
for i, head in enumerate(headers):
d[head].append(row[i])
# save as json file
with open('output.json', 'w') as f:
json.dump(d, f)
output:
{'Name': ['TESTLIB1', 'TESTLIB2'],
'file': ['443', '444'],
'code': ['123', '124'],
'file_location': ['location1', 'location2']}

from collections import defaultdict
data_dict = defaultdict(list)
with open('input-file') as inp:
for row in csv.DictReader(inp, delimiter='\t'):
for key, val in row.items():
data_dict[key].append(val)
print(data_dict)
# output
{'Name': ['TESTLIB1', 'TESTLIB2'],
'file': ['443', '444'],
'code': ['123', '124'],
'file_location': ['location1', 'location2']}

Dictionary to CSV using Python with headers style

For this same Stack Overflow question How do I output a list of dictionaries to an Excel sheet?.
For xlsx code by jossef-harush we can use this:
import xlsxwriter
# ...
def create_xlsx_file(file_path: str, headers: dict, items: list):
with xlsxwriter.Workbook(file_path) as workbook:
worksheet = workbook.add_worksheet()
worksheet.write_row(row=0, col=0, data=headers.values())
header_keys = list(headers.keys())
for index, item in enumerate(items):
row = map(lambda field_id: item.get(field_id, ''), header_keys)
worksheet.write_row(row=index + 1, col=0, data=row)
headers = {
'bank': 'Money in Bank',
'dailyWinners': 'Daily Winners',
'dailyFree': 'Daily Free',
'user': 'User',
}
players = [
{'dailyWinners': 3, 'dailyFreePlayed': 2, 'user': 'Player1', 'bank': 0.06},
{'dailyWinners': 3, 'dailyFreePlayed': 2, 'user': 'Player2', 'bank': 4.0},
{'dailyWinners': 1, 'dailyFree': 2, 'user': 'Player3', 'bank': 3.1},
{'dailyWinners': 3, 'dailyFree': 2, 'user': 'Player4', 'bank': 0.32}
]
create_xlsx_file("my xslx file.xlsx", headers, players)
For creating a CSV file, what are the changes I need to do in the above code?
Like create_csv_file("my csv file.csv", headers, players).

import csv
def create_csv_file(file_path, headers, items):
with open(file_path, "wt") as f:
dw = csv.DictWriter(f, headers.values(), extrasaction='ignore')
dw.writeheader()
for row in items:
dw.writerow({headers.get(k): v for (k, v) in row.items()})
writes
Money in Bank,Daily Winners,Daily Free,User
0.06,3,,Player1
4.0,3,,Player2
3.1,1,2,Player3
0.32,3,2,Player4
Note that this will ignore any dict keys not in headers; in your case, the dailyFreePlayeds.

Create a new list of dictionaries within a new dictionary in Python

I have a list of dictionaries and need to create a new one that contains new keys and also keys&values from my original dictionary. One of the keys will have to contain a list of dictionaries (those would be the values from original dictionary)
My data looks like the following:
data = [{'CloseDate': '2020-05-01',
'OpportunityID': '1',
'CustomerID': '10'},
{'CloseDate': '2020-07-31',
'OpportunityID': '2',
'CustomerID': '11'}]
I want my new list of dicts look like this:
new_data = [{'id': '39',
'Query': [{'records': '40', 'Order Name': '1', 'CustomerID': '10'}]},
{'id': '39',
'Query': [{'records': '40', 'Order Name': '2', 'CustomerID': '11'}]}]
I have tried the following:
new_data = []
for item in data:
params_dict = {}
params_dict["id"] = "39"
params_dict["Query"] = []
# push new_dicts in params_dict
new_dict = {}
new_dict["records"] = "40"
new_dict["Order Name"] = data["OpportunityID"]
params_dict.append(new_dict)
new_data.append(params_dict)
Error: TypeError: list indices must be integers or slices, not str

datas_list=[]
for get_dict in data:
new_dict={}
new_dict["id"] = 39
new_dict['Query']=[]
other_dictionary={}
other_dictionary['records']=40
for values in get_dict:
if values == "OpportunityID":
other_dictionary['Order Name'] = get_dict[values]
if values == "CustomerID" :
other_dictionary[values] = get_dict[values]
new_dict["Query"].append(other_dictionary)
datas_list.append(new_dict)

You were trying to iterate through item and not data inside the loop.
Also you need to append to Query.
Try:
new_data = []
for item in data:
params_dict = {}
params_dict["id"] = "39"
params_dict["Query"] = []
new_dict = {} # defined new_dict
new_dict["records"] = "40"
new_dict["Order Name"] = item["OpportunityID"] # here it should be item
params_dict["Query"].append(new_dict)
new_data.append(params_dict)
Also:
new_data = []
for item in data:
params_dict = {}
params_dict["id"] = "39"
params_dict["Query"] = [{"records" : "40","Order Name" :item["OpportunityID"] }]
new_data.append(params_dict)

Create nested dictionary from text file using Python 3

This is my file: test.txt
Amy|Female|Desc1|12
John|Male|Desc2|10
Mike|Male|Desc3|18
I tried to create nested dictionary and it's not sucessful.
This is the output:
{'Amy': '12', 'John': '10', 'Mike': '18'}
This is my code:
import csv
with open('test.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
d = {}
for row in tsvfile:
d[row[0]] = row[0] #this should be name
d[row[0]] = row[1] #this should be gender
d[row[0]] = row[3] #this should be desc
d[row[0]] = row[3] #this should be age
print(d)
My desired output as below but was not successful.
d={1{'Name':'Amy', 'Gender':'Female', 'Desc': 'Desc1', 'Age': '12'}
2{'Name':'John', 'Gender':'Male', 'Desc': 'Desc2', 'Age': '10'}
3{'Name':'Mike', 'Gender':'Male', 'Desc': 'Desc3', 'Age': '18'}}
and below (with name and age only
d1={1{'Name':'Amy','Age': '12'}
2{'Name':'John', 'Age': '10'}
3{'Name':'Mike', 'Age': '18'}}

Here's how to do it without csv import, given the data format is constant:
fixed = {}
i = 1
with open("test.txt", 'r') as f:
for line in f:
listDetails = line.strip().split('|')
fixed[i] = {"Name": listDetails[0]}
fixed[i].update({"Sex": listDetails[1]})
fixed[i].update({"Description": listDetails[2]})
fixed[i].update({"Age": listDetails[3]})
i+=1
print(fixed)
This should turn
Amy|Female|Desc1|12
John|Male|Desc2|10
Mike|Male|Desc3|18
To
{1: {'Name': 'Amy', 'Sex': 'Female', 'Description': 'Desc1', 'Age': '12'}, 2: {'Name': 'John', 'Sex': 'Male', 'Description': 'Desc2', 'Age': '10'}, 3: {'Name': 'Mike', 'Sex': 'Male', 'Description': 'Desc3', 'Age': '18'}}
Edit: Just as Nakor said though, it doesn't really make sense to make a dict of dicts here, just posted this if you really need to make it a dict.

1) Nested Dictionary, I have made some changes in the same code, it may help you.
import csv
with open('hello.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
final_dict = {}
counter = 1
for row in tsvfile:
d = {}
d['Name'] = row[0] #this should be name
d['Gender'] = row[1] #this should be gender
d['Desc'] = row[2] #this should be desc
d['Age'] = row[3] #this should be age
final_dict[counter] = d
counter+=1
print(final_dict)

Your desired output looks more like a list of dictionaries.
In this case, I would just modify your code like this:
import csv
with open('test.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
d = []
for row in tsvfile:
entry = {
'Name': row[0],
'Gender': row[1],
'Desc': row[2],
}
d.append(entry)
print(d)
Output:
[{'Name': 'Amy', 'Gender': 'Female', 'Desc': 'Desc1'},
{'Name': 'John', 'Gender': 'Male', 'Desc': 'Desc2'},
{'Name': 'Mike', 'Gender': 'Male', 'Desc': 'Desc3'}]
You can even write the loop in a more compact way like this:
keys = ["Name","Gender","Desc"]
for row in tsvfile:
entry = { key: value for (key,value) in zip(keys,row) }
d.append(entry)
EDIT: If you want a dictionary with the line number as keys, you can do:
import csv
with open('test.txt') as file:
tsvfile = csv.reader(file, delimiter='|')
d = {}
keys = ["Name","Gender","Desc"]
for i,row in enumerate(tsvfile):
entry = {
'Name': row[0],
'Gender': row[1],
'Desc': row[2],
}
d[i+1] = entry
print(d)

Python CSV append new row

So here is my issue, I have created a Dictionary for my lists and am trying to add the data I find and append it to each row but instead it is just appending to the same column with the same data type.
How could I get it so each new append add to new row.
data_dict = {'contact name': [], 'name': [], 'telephone': [], 'email': [],
'mobile': [], 'feedback average': []}
try:
data_dict['telephone'].append(soup.find('span',itemprop='telephone').text)
except AttributeError:
data_dict['telephone'].append('No telephone')
print data_dict
field_names = fn = data_dict.keys()
with open('./file.csv','w') as csvfile:
f = csv.DictWriter(csvfile, fieldnames=fn)
f.writeheader()
f.writerow(data_dict)

Try something like this:
data_dict = {'contact name': [], 'name': [], 'telephone': [], 'email': [],
'mobile': [], 'feedback average': []}
try:
data_dict['telephone'].append(soup.find('span',itemprop='telephone').text)
except AttributeError:
data_dict['telephone'].append('No telephone')
print data_dict
fn = data_dict.keys()
with open('./file.csv','w') as csvfile:
f = csv.reader(csvfile)
for row in f:
for i in len(fn):
data_dict[fn[i]].append(row[i])
This should work for you, if I got you right.
But care, this requires that one row in the csv contains exactly the elements of your dictionary, in the correct order.
If this isn't the case, you will need to find out which value is written in which column, and then add the value of this column to the list in your dictionary.
So you would need to replace
for i in len(fn):
data_dict[fn[i]].append(row[i])
by
for k in fn:
data_dict[k].append(row[columns[k]])
where columns is a dictionary that contains the same keys as data_dict, and as the values the columns in which the data of the specific key is stored in the csv-file. For an example, columns could look like this:
columns = {'contact name': 1, 'name': 3, 'telephone' : 6, 'email': 7, 'mobile':8, 'feedback average': 2}