Finding a specific record in a csv file - python

I'm making a project for school where users are quizzed on subjects and their results are saved into a report (CSV file called 'reportForFergusTwo') for the 'customer' (Fergus). The report shows the average score a certain quiz, the high score, and the username of the person who achieved it. And then, I'm meant to display the user's details but the user's details are saved in a different CSV file ('details').
So far I've made a function that finds the highest score and username of the person who achieved it in the CSV 'reportForFergusTwo'. I now need to work on a function that searches through another CSV file ('details').
def findHighScoreUser():
print(user)
with open ('details.csv', 'r') as stalking:
stalkingReader=csv.reader(stalking)
valid4=False
for row in stalkingReader:
if user in row:
valid4=True
print("Here are the details for user {}... ".format(user))
splitter=row.split(',')
name=splitter[0]
age=splitter[1]
year=splitter[2]
print("Name: {}".format(name))
print("Age: {}".format(age))
print("Year Group: {}".format(year))
postReport()
if valid4==False:
print("Sorry Fergus, this user doesn't seem to be in our records.")
'user' is a variable from my previous function, where I found the username of the person with the highest score. It holds the username. I made it global and printed it at the start of this function, and that's working, making me think this is an issue I've made in reading the CSV.
Here is the details CSV:
Name,Age,Year Group,Username,Password
Erin Jones,15,11,Eri15,Password
Lucas Jones,16,11,Luc16,Password2
Michael Jordan,11,7,Mic11,GetSomeHelp
La Roux,14,9,La 14,Bulletproof
And here is the reportForFergusTwo CSV:
Subject,Difficulty,Score,Username
language,easy,10,Luc16
chemistry,easy,10,Luc16
maths,easy,9,Luc16
chemistry,easy,5,Eri15
chemistry,easy,6,Mic11
chemistry,easy,0,Eri15
I'm still fairly new to code, so please excuse me if the answer is obvious.

If your user is no longer in scope you can provide it to your function like this:
def functionThatFindsHighestScoreUser():
# do stuff
return maxScoreUser
def findHighScoreUser(user):
print(user)
# the rest of your existing code
# get the user
maxScoreUser = functionThatFindsHighestScoreUser()
# find it and print details
findHighScoreUser(maxScoreUser)
There is almost always a way around global variables by using parameters in the function - don't use them, provide each function exactly with what it needs.
As for the csv-issues: csv module does the splitting for you as Tomalak mentioned. You can consume the reader line wise and get a list of all fields:
import csv
def checkUsers(name):
with open("users.csv",encoding="utf8",newline='\n') as u:
reader = csv.DictReader(u, delimiter=',', quotechar='|' )
found = False
for row in reader:
if row["Name"] == name:
print("Here are the details for user {}... ".format(row["Name"]))
print("Name: {}".format( row["Name"]))
print("Age: {}".format( row["Age"]))
print("Year Group: {}".format( row["Year Group"]))
found = True
break # leave for loop
if not found:
print("Not in users")
checkUsers("Michael Jordan")
checkUsers("Chris Cringle")
Output:
OrderedDict([('Name', 'Michael Jordan'), ('Age', '11'),
('Year Group', '7'), ('Username', 'Mic11'), ('Password', 'GetSomeHelp')])
Not in users

Try this?:
import pandas
#read csv data
stalking = pandas.read_csv('details.csv')
#Select all records from the stalking csv where Username equals the user you are looking for
user_details = stalking[stalking['Username']==user]
#Print values for each column from subset of data which only contains records where username == user
print('Name: {}'.format(user_details['Name']))
print('Age: {}'.format(user_details['Age']))
print('Year Group: {}'.format(user_details['Year Group']))
edit: FYI if there are multiple rows with the same username this will print all the names, ages and year groups associated with that username.
pandas makes accessing and working with csvs really easily. Once you've done var = pandas.read_csv('somefile.csv') you can access all the values in a given column using var['col1']. Subsetting can be done by doing var[var['col1'] == 'something'], which reads as 'Select records from dataset var where the values of col1 are 'something'.
The pandas docs can be found here: http://pandas.pydata.org/pandas-docs/stable/
Dictreader might also be a good way to go if pandas seems a bit too complex for what you're tyring to do.

Related

Error is being shown when I run the code and I am unable to figure out the second question

import pickle
med = {}
medfile = open("Medicines.dat","wb")
while True:
name = input("Enter the name: ")
company = input("Enter the company: ")
chemical = input("Enter the chemical: ")
price = input("Enter the price: ")
med['name'] = name
med['company'] = company
med['chemical'] = chemical
med['price'] = price
pickle.dump(med,medfile)
ans = input("Wouldyou like to add more(y/n) ")
if ans == "y":
continue
elif ans == "n":
break
medfile = open("Medicines.dat","r+")
print(pickle.load(medfile))
medfile.close()
The question is as follows:
A binary file "Medicines.dat has structure [Name, Company, Chemical, Price] a) Write a user defined function add_data() to input the data for a record and store in the file b) Write a function search() which accepts a company name and displays the details of all the Medicines by that company
There are a few problems here:
1st Opening the file correctly
medfile = open("Medicines.dat","r+")
You mean rb. The difference is explained here, but pickle parsing requires the file to be in "binary" mode, hence the "b".
2nd Closing the file correctly
You should close the file before re-opening it for writing, as a matter of best practce. (medfile.close()). Even better, python will take care of when the file gets closed if you use the "with" syntax to create a context
3rd Having the right values
While the code should now run, I doubt it will do what you want. Your query asks "Wouldyou [sic] like to add more(y/n)", but it does not look to me like it is adding more values, since you use the same "med" dictionary over and over. Consider how the "new" fields would ever be distinguishable from the "old" ones, based on their key

Python Dictionaries stuck

hope you are having a nice day, so I need to write a code that takes an input, which is going to be a key in an already existing dictionary, and prints the value as a str as described in the pic. I am stuck rn, I am a beginner and tried everything I see on the web related, appriciate any help, thank you.
[and these are the codes I tried][1]
def read_dataset(filename):
lines = open(filename).readlines()
lines_splitted = [line.strip().split(',') for line in lines]
return {lst[0]: lst[1:] for lst in lines_splitted}
movie_dict = read_dataset('dataset.txt')
# DO_NOT_EDIT_ANYTHING_ABOVE_THIS_LINE
hey = input('Enter a movie name: ')
actors = movie_dict[hey]
# DO_NOT_EDIT_ANYTHING_BELOW_THIS_LINE
print(','.join(sorted(actors)))
So this is my assignment
If you begin, try using pandas to make a dataframe of your base. It will be easier to use, and you should reuse it in a lot of cases.
Here an example of what you could do. If you don't understand a point, do not hesitate to ask me:
# import the package
import pandas as pd
# build the dataframe for the example
film = pd.DataFrame({'film':['Goldorak','Mickey Mouse'],'actors':[['Actarus','Alkor'],['Mickey, Minnie, Pluto, Goofy']]})
# the input
query = input('Enter the film')
# the 'search' engine
if len(film[film.film==query].actors)>0:
print(film[film.film==query].actors)
else:
print('not found in base')
EDIT with Dictionary :
FILM = {'Forest Gump':['Tom Hanks', 'Gary Sinise']}
query = None
query = input('Enter the film')
try:
print(FILM.get(query))
except:
pass

Which file mode to choose and creating lists in a loop

back again with another problem. I am trying to make a "table maker" - for now just creating files with dictionaries and checking keys.
I have two problems: firstly i don't know if there is a file format for writing in binary but not overwriting a previous file if it already exists? I tried the (try, except) method but the excepted FileExistsError does never come up :/ (in the createTable function)
Secondly I have a problem with creating lists. I made a loop which asks for entries and values for them to be stored in separate lists. Those lists will later be zipped into a dictionary and pickled into a file. (in the createTable function)
Of course if there are other mistakes I'd love them pointed out :)
import pickle
def checkTable(nameOfTable) :
try :
#seeing what is it they are looking for
prompt = input("What do you want to check?\n")
with open("%s.pkl" % nameOfTable, "rb") as f:
data = pickle.load(f)
#getting what they want from the data from table
whatTheyWant = data.get(prompt, "There is nothing like that in the table.\n")
print(whatTheyWant)
#if table doesn't exist
except IOError as e:
print("Sorry such a directory doesn't exist.\n")
def createTable(nameOfYourTable) :
try :
#opens a new file with the table
with open("%s.pkl" %nameOfYourTable, "wb+") as f :
decision = "yes"
if decision == "yes" :
#asking for entries and keys to put into the table
#creates lists with entries and values to be zipped together
entry.append = input("What is the entry?\n")
value.append = input("What is the value of the entry?\n")
decision = input("Do you want to go on? (yes/no)\n")
i += 1
else :
#getting it all into a dictionary and putting it into a file
table={dict(zip(entry, value))}
pickle.dump(table, f)
#if a file with the name already exists
except FileExistsError as e :
print("Sorry, a file with this name already exists.")
#what the person wants to do
answer = input("Hello. Do you want to create a table or check an existing one?\n")
#asking for the name of the new table
if answer == "create" :
nameOfYourTable = input("What do you want the table to be called?\n")
createTable(nameOfYourTable)
#asking what table to look in
elif answer == "check" :
nameOfTable = input("What is the name of the table?\n")
checkTable(nameOfTable)
else :
print("\nThat's not a valid option.\n")
print("Thank you for using me. It was very nice.")
There is a file mode just for what you want, open(file, "x"), add b or t as per your need.
In x mode a file is created only when it does not already exist, raises exception otherwise. The createTable function does not really make sense to me. decision = "yes" then if decision == "yes":? decision is supposed to be a global? It's very fuzzy.
You need to pickle read the list from the file first, add to the list, and then write a new pickle file at the end. Btw, you can check if a file exists with os.path.exists. Appending items to a list is done with list_name.append(new_item). You need to have initialized the list first with list_name = [].

How to unpickle objects from .pkl file using python

I created a student class.I stored every student details in a file.I used serialization concept to store the details.The problem is while unpickling the data it gives first object only.Can anyone say how to retrieve all objects which are in .pkl file
and my code is
this is my code
'''
Created on Apr 20, 2015
#author: murali
'''
import pickle
from pip._vendor.distlib.compat import raw_input
class Student:
def __init__(self, name, roll, sex, subject, total):
"""Return a new Car object."""
self.name = name
self.roll = roll
self.sex = sex
self.subject = subject
self.total = total
print ("*************MENU*******************************")
print ("\t 1.Add New Student")
print ("\t 2.Display all Students")
print ("\t 3.Delete Student")
print ("\t 4.Display all Boys")
print ("\t 5.Display all Girls")
print ("\t 6.Display ascending order of marks")
print ("\t 7.Display alphabetical order of names ")
print ("\t 0.exit")
print ("************************************************")
print ("\n Enter your choice")
ch=int(raw_input())
if ch==1:
print ("Enter Student Details")
print ("Enter Student Name")
n=raw_input()
print ("Enter Student Roll number")
r=raw_input()
print ("Male or Female")
s=raw_input()
print ("Enter Subject ")
su=raw_input()
print ("Enter Student marks ")
t=raw_input()
s=Student(n,r,s,su,t);
with open("save2.pkl",'ab') as outfile:
pickle.dump(s,outfile)
print ("Data saved Successfully")
elif ch==2:
st=[];
f=open("save2.pkl",'rb')
storedlist=pickle.load(f)
print (storedlist)
#st.append(storedlist)
print ("Students are-")
print (storedlist)
#for i in storedlist:
print(storedlist.name)
print(storedlist.roll)
print(storedlist.sex)
print(storedlist.subject)
print(storedlist.total)
f.close();
#s1=None
#with open("save2.pkl",'rb') as infile:
#s1=pickle.load(infile)
#print (s1.name)
#print (s1.roll)
#print (s1.sex)
#print (s1.subject)
The issue you're encountering is that pickle.load only reads one item from your pkl file. That's just how its designed. If you have more than one item in the same file, you'll need to repeatedly load them until you get an EOFError when there is nothing more to read. Something like:
students = []
try:
with open("save2.pkl",'rb') as f:
while True: # loop indefinitely
students.append(pickle.load(f)) # add each item from the file to a list
except EOFError: # the exception is used to break the loop
pass # we don't need to do anything special
Alternatively, you could maintain in memory a complete list of the items your file is intended to contain (e.g. all the Student instances). When the list is modified, you'd pickle the whole list and write it all out together in one call to pickle.dump, overwriting the previous contents of the your data file. Rather than reading the data when the user requested it, you'd read it at the start of the program, and already have it in memory if the user wants it later.
This is a bit harder to show, as I don't want to rewrite large chunks of your code. I think these snippets are all you need:
Just below definition of the Student class:
try:
with open("save2.pkl",'rb') as f:
students = pickle.load(f) # just one call, loads the whole list
except FileNotFoundError: # if the file doesn't exist, make an empty list
students = []
Where you are currently appending to the file:
s=Student(n,r,s,su,t)
students.append(s) # update the in-memory list of students
with open("save2.pkl",'wb') as outfile: # use mode "w" to overwrite existing file
pickle.dump(students, outfile) # dump the whole list, not only s
print ("Data saved Successfully")
In the later code you wouldn't need to re-read the file. It can just use the students list you already have in memory.
This second approach is not terribly useful given your current menu design which can only be passed through a single time. But if you later put the menu in a loop (as the 0 option suggests you eventually will want to do), you might appreciate not needing to read the data file over and over.
This approach is also much better than the other version if you ever extend your Student type to have references to other Student instances (perhaps you're recording lab partners, or teams for some group project). If you pickle such interconnected objects separately, the links between them will not be retained when you load them again. The loaded versions will still have references, but the references will be to copies of the other Student instances, not the same ones you'll get when the other Students are loaded themselves. If you dump and load a whole list of Students at once, however, their interconnections will be maintained. (This is fairly deep stuff, don't worry too much if you don't understand what I'm talking about.)

change/writing single value to csv file python

There is an error in my code :
_csv.Error: sequence expected
which i believe is because i am trying to write only one value not a list etc.
exRtFile = open ('exchangeRate.csv')
exchReader = csv.reader(exRtFile)
exchWriter = csv.writer(exRtFile)
loop2=0
while loop2==0:
selected=int(input("Please select an option: "))
if selected == 1:
change = input("What rate would you like to change: ")
changeRt = float(input("What would you like to change the rate to: "))
for row in exchReader:
currency = row[0]
if currency == change:
crntRt = row[1]
crntRt = changeRt
exchWriter.writerow(crntRt)
exRtFile.close()
what would be the best way to fix this, or is there a better wayy to change a value in an CSV file?
csv file:
Pound Sterling,1
Euro,1.22
US Dollar,1.67
Japanese Yen,169.948
Here is some code, not tested, that will do what you want. The idea is to read the text into memory, apply the updates, then write out the results over the original file.
You can further enhance this ask the user if they want to save their changes, and to add new currencies instead of just telling the user they're not known.
In the real world, I would break this code into three separate functions (or even classes), one for reading, one for writing, and one for editing the list.
import csv
rates = {}
# read file into dictionary
with open('csv_file.csv', 'r') as in_file:
rdr = csv.reader(in_file)
for item in reader:
rates[row[0]] = row[1]
# ask user for updates and apply to dictionary
while true:
cmd = raw_input('Enter exchange rate to adjust, or blank to exit')
if cmd is None or cmd.strip() == '':
break
if rates.has_key(cmd):
new_rate = float(raw_input('Enter new exchange rate:'))
rates[cmd] = new_rate
else:
print 'Currency {} is not known.'.format(cmd)
# Write the updated dictionary back over the same file.
with open('csv_file.csv', 'w') as out_file:
wrtr = csv_writer(out_file)
wrtr.writerows(rates)
Answering your question: Yes, the problem is that you were trying to write only a value, while writerow expects a list.
That said... Would you consider changing a bit the way your code works?
Here's what I've done (I've tested it now, so I know it works):
First, ask the user for all the changes to make and keep them in a dict where keys are the currency names (Euro, for instance) and the value is the new currency value (5.0, for instance) The user can get out of the loop pressing 0
Second, open and read your exchangeRate.csv file row by row. If the row[0] (name of the currency) is among the values to change, then change it in that row.
No matter what happens (regardless of whether the row needed to be changed or not) write that row in a new temporary file exchangeRate.csv.tmp
When all the rows in the original file are read, you'll have exchangeRate.csv.tmp with some rows unchanged and some rows changed. Swap (move) the .tmp file to exchangeRate.csv
Dunno... might be too much change maybe? Here it is, anyway:
import csv
import shutil
change_rates = {}
selected = 1
while selected:
selected=int(raw_input("Please select an option: (1 to change, 0 to exit)"))
if selected == 1:
change = raw_input("What rate would you like to change?: ")
changeRt = float(raw_input("What would you like to change the rate to: "))
change_rates[change] = changeRt
if len(change_rates) > 0:
with open('exchangeRate.csv', 'r') as f_in,\
open('exchangeRate.csv.tmp', 'w') as f_out:
exchReader = csv.reader(f_in)
exchWriter = csv.writer(f_out)
for row in exchReader:
if row[0] in change_rates:
row[1] = change_rates[row[0]]
exchWriter.writerow(row)
shutil.move('exchangeRate.csv.tmp', 'exchangeRate.csv')
And a sample execution below:
Please select an option: (1 to change, 0 to exit)1
What rate would you like to change?: Euro
What would you like to change the rate to: 5
Please select an option: (1 to change, 0 to exit)0
borrajax#borrajax:~/Documents/Tests$ cat ./exchangeRate.csv
Pound Sterling,1
Euro,5.0
US Dollar,1.67
Japanese Yen,169.948
You can always make more optimizations, such as... allow case insensitive searches, or check that the currency has actually been changed (like even if the user says he wants to change the currency Euro to 5.0, if that was the Euro's exchange rate then don't do anything)... Things like that.
EDIT 1:
I've just seen Larry Lustig's answer and I agree that for small files as it seems to be your case (files that you can fully load in memory) the continuous reading and writing from disk I posted is not optimal. His idea of keeping everything in memory and then do a bulk write to the same exchangeRate.csv file probably is a better fit for your needs.
EDIT 2:
To answer your questions in a comment to this answer:
what does .tmp do at the end of: exchangeRate.csv.tmp:
It's just a new name. I add the suffix .tmp to avoid a naming conflict with your original file (exchangeRate.csv). You could name it whatever you want, though (even foobar.baz)
What is the purpose of 'change' in the variable: change_rates[change] = changeRt:
change is a variable that contains the name of the currency to change (in the usage example I posted, change contains the string "Euro", because that's what the user (erm... me) typed on the console. Is just a way of accessing a dict.
What is the prupose of '[row[0]]' in: row1=change_rates[row[0]].
We agreed that when reading the file, row[0] (just like that, not [row[0]]) contains the name of the currency in the file (Euro, Pound Sterling... etcetera) right? So at a certain point of the execution row[0] will contain the string "Euro", which (in my test example) is the currency the user wanted to change. That string ("Euro") is also a key in the change_rates dictionary (because the user said he wanted to change it) so you are querying the value for the item with key "Euro" in the change_rates dictionary (which will give you 5.0). Is pretty much doing change_rates["Euro"] To see it a bit more clearer add the line print "Currencies to change: %s" % change_rates on the line right above if len(change_rates) > 0: (that'll show you how the dictionary looks like)
what does shutil.move('exchangeRate.csv.tmp', 'exchangeRate.csv') do?
It copies the file with the new currencies to exchangeRate.csv (see the shutil documentation)

Categories