I am currently doing a project for school that involves making a graphing editor. I am at a part where I have to be able to save and reopen the file. I can open the file but I have to iterate through it and regraph everything I saved. I am unsure however to actually iterate through the file because when print the file that I opened, i get a huge list that has all of my lists within it like this:
["['Rectangle', 5.168961201501877, 8.210262828535669, 7.6720901126408005, 6.795994993742178, 'red']['Line', 5.782227784730914, 5.269086357947434, 8.69837296620776, 4.993742177722153, 'red']['Circle', 2.6491232154288933, -0.8552572601656006, 6.687547623119292, 3.1831671475247982, 'red']"]
I am new at using this website so please bear with me.
def open_file(self,cmd):
filename=input("What is the name of the file? ")
File= open(filename,'r')
file= File.readlines()
print(file)
I had previously saved the file by using:
file.write(str(l)) where l is the name of a list of values I made
I have tried using split()
I tried using a for loop to save the data within the string into a list
and I have searched the web for hours to find some sort of explanation but I couldn't find any.
What you've provided is actually a list with one item consisting of a long string. Can you provide the code you're using to generate this?
If it actually is a list within a list, you can use a for loop inside another for loop to access each item in each list.
let's say your list is object l.
l[0] = ['Rectangle', 5.168961201501877, 8.210262828535669, 7.6720901126408005, 6.795994993742178, 'red']
and l[0][0] = 'Rectangle'
for i in l:
for x in i:
Would allow you to loop through all of them.
For the info you've provided, readlines() won't necessarily work, as there's nothing to delineate a new line in the text. Instead of saving the list as a converted string, you could use a for loop to save each item in the list as a line
for lne in l:
f.write(lne)
Which would write each item in the list on a new line in the file (depending on your python version, you might have to use f.write(lne+'\n') to add a new line). Then when you open the file and use readlines(), it will append each line as an item in a list.
You are apparently having problem with reading data you have created before.
Your task seem to require
1) creating some geometry in an editor
2) serialize all the geometry to a file
and later on (after the program is restarted and all old memory content is gone:
3) load geometries from the file
4) recreated the content (geometries) in your program
In step 2 you did something and you seem to be surprised by that. My proposal would be to use some other serialization option. Python offers many of them, e.g.
pickle - quick and easy, but is not interoperable with other than Python programs
JSON - easy, but might require some coding for serialization and loading your custom objects
Sample solution using JSON serialization could go like this:
import json
class Geometry():
def __init__(self, geotype="Geometry", color="blank", numbers=[]):
self.geotype = geotype
self.color = color
self.numbers = numbers
def to_list(self):
return [self.geotype, self.color, self.numbers]
def from_list(self, lst):
print "lst", lst
self.geotype, self.color, self.numbers = lst
return self
def __repr__(self):
return "<{self.geotype}: {self.color}, {self.numbers}>".format(self=self)
def test_create_save_load_recreate():
geoms = []
rect = Geometry("Rectange", "red", [12.34, 45])
geoms.append(rect)
line = Geometry("Line", "blue", [12.33, 11.33, 55.22, 22,41])
geoms.append(line)
# now serialize
fname = "geom.data"
with open(fname, "w") as f:
geoms_lst = [geo.to_list() for geo in geoms]
json.dump(geoms_lst, f)
# "geom.data are closed noe
del f
del geoms
del rect
del line
# after a while
with open(fname, "r") as f:
data = json.load(f)
geoms = [Geometry().from_list(itm) for itm in data]
print geoms
Related
I have a python script that goes out and pulls a huge chunk of JSON data and then iterates it to build 2 lists
# Get all price data
response = c.get_price_history_every_minute(symbol)
# Build prices list
prices = list()
for i in range (len(response.json()["candles"])):
prices.append (response.json()["candles"][i]["prices"])
# Build times list
times = list()
for i in range (len(response.json()["candles"])):
times.append (response.json()["candles"][i]["datetime"])
This works fine, but it takes a LONG time to pull in all of the data and build the lists. I am doing some testing trying to build out a complex script, and would like to save these two lists to two files, and then import the data from those files and recreate the lists when I run subsequent tests to skip generating, iterating and parsing the JSON.
I have been trying the following:
# Write Price to a File
a_file = open("prices7.txt", "w")
content = str(prices)
a_file.write(content)
a_file.close()
And then in future scripts:
# Load Prices from File
prices_test = array('d')
a_file = open("prices7.txt", "r")
prices_test = a_file.read()
The outputs from my json lists and the data loaded into the list created from the file output look identical, but when I try to do anything with the data loaded from a file it is garbage...
print (prices)
{The output looks like this} [69.73, 69.72, 69.64, ... 69.85, 69.82, etc]
print (prices_test)
The output looks identical
If I run a simple query like:
print (prices[1], prices[2])
I get the expected output {69.73, 69.72]
If I do the same on the list created from the file:
print (prices_test[1], prices_test[2])
I get the output ( [,6 )
It is pulling every character in the string individually instead of using the comma separated values as I would have expected...
I've googled every combination of search terms I could think of so any help would be GREATLY appreciated!!
I had to do something like this before. I used pickle to do it.
import pickle
def pickle_the_data(pickle_name, list_to_pickle):
"""This function pickles a given list.
Args:
pickle_name (str): name of the resulting pickle.
list_to_pickle (list): list that you need to pickle
"""
with open(pickle_name +'.pickle', 'wb') as pikd:
pickle.dump(list_to_pickle, pikd)
file_name = pickle_name + '.pickle'
print(f'{file_name}: Created.')
def unpickle_the_data(pickle_file_name):
"""This will unpickle a pickled file
Args:
pickle_file_name (str): file name of the pickle
Returns:
list: when we pass a pickled list, it will return an
unpickled list.
"""
with open(pickle_file_name, 'rb') as pk_file:
unpickleddata = pickle.load(pk_file)
return unpickleddata
so first pickle your list pickle_the_data(name_for_pickle, your_list)
then when you need to load the list unpickle_the_data(name_of_your_pickle_file)
This is what I'm trying to explain into the comments section. Note I replaced response.json() to jsonData, successfully taking it out of each for-loop, and reduced both loops into a single one for more efficiency. Now the code should run faster.
import json
def saveData(filename, data):
# Convert Data to a JSON String
data = json.dumps(data)
# Open the file, then save it
try:
file = open(filename, "wt")
except:
print("Failed to save the file.")
return False
else:
file.write(data)
file.close()
return True
def loadData(filename):
# Open the file, then load its contents
try:
file = open(filename, "rt")
except:
print("Failed to load the file.")
return None
else:
data = file.read()
file.close()
# Data is a JSON string, so now we convert it back
# to a Python Structure:
data = json.loads(data)
return data
# Get all price data
response = c.get_price_history_every_minute(symbol)
jsonData = response.json()
# Build prices and times list:
#
# As you're iterating over the same "candles" index on both loops
# when building those two lists, just reduce it to a single loop
prices = list()
times = list()
for i in range(len(jsonData["candles"])):
prices.append(jsonData["candles"][i]["prices"])
times.append(jsonData["candles"][i]["datetime"])
# Now, when you need, just save each list like this:
saveData("prices_list.json", prices)
saveData("times_list.json", times)
# And retrieve them back when you need it later:
prices = loadData("prices_list.json")
times = loadData("times_list.json")
Btw, pickle does the same thing, but it uses Binary Data instead of json, which is probably faster for save / load data. I don't know, didn't tested it.
In json, you have the advantage of readability, as you can open each file and read it directly, if you can understand JSON syntax.
First, sorry if the title is not clear. I (noob) am baffled by this...
Here's my code:
import csv
from random import random
from collections import Counter
def rn(dic, p):
for ptry in parties:
if p < float(dic[ptry]):
return ptry
else:
p -= float(dic[ptry])
def scotland(r):
r['SNP'] = 48
r['Con'] += 5
r['Lab'] += 1
r['LibDem'] += 5
def n_ireland(r):
r['DUP'] = 9
r['Alliance'] = 1
# SF = 7
def election():
results = Counter([rn(row, random()) for row in data])
scotland(results)
n_ireland(results)
return results
parties = ['Con', 'Lab', 'LibDem', 'Green', 'BXP', 'Plaid', 'Other']
with open('/Users/andrew/Downloads/msp.csv', newline='') as f:
data = csv.DictReader(f)
for i in range(1000):
print(election())
What happens is that in every iteration after the first one, the variable data seems to have vanished: the function election() creates a Counter object from a list obtained by processing data, but on every pass after the first one, this object is empty, so the function just returns the hard coded data from scotland() and n_ireland(). (msp.csv is a csv file containing detailed polling data). I'm sure I'm doing something stupid but would welcome anyone gently pointing out where...
I’m going to place a bet on your definition of newline. Are you sure you don’t want newline = “\n” ? Otherwise it will interpret the entire file as a single line, which explains what you’re seeing.
EDIT
I now see another issue. The file object in python acts as a generator for each line. The problem is once the generator is finished (you hit the end of the file), you have no more data generated. To solve this: reset your file pointer to the beginning of the file like so:
with open('/Users/andrew/Downloads/msp.csv') as f:
data = csv.DictReader(f)
for i in range(1000):
print(election())
f.seek(0)
Here the call to f.seek(0) will reset the file pointer to the beginning of your file. You are correct that data is a global object given the way you've defined it at the module level, there's no need to pass it as a parameter.
I agree with #smassey, you might need to change the code to
with open('/Users/andrew/Downloads/msp.csv', newline='\n') as f:
or simply try not use that argument
with open('/Users/andrew/Downloads/msp.csv') as f:
So, I am trying to make a combined list inside of Python for matching data of about 25,000 lines.
The first list data came from file mac.uid and looks like this
Mac|ID
The second list data came serial.uid and looks like this:
Serial|Mac
Mac from list 1 must equal the Mac from list 2 before it's joined.
This is what I am currently doing, I believe there is too much repetition going on.
combined = [];
def combineData():
lines = open('mac.uid', 'r+')
for line in lines:
with open('serial.uid', 'r+') as serial:
for each in serial:
a, b = line.strip().split('|')
a = a.lower()
x, y = each.strip().split('|')
y = y.lower()
if a == y:
combined.append(a+""+b+""+x)
The final list is supposed to look like this:
Mac(List1), ID(List1), Serial(List2)
So that I can import it into an excel sheet.
Thanks for any and all help!
Instead of your nested loops (which cause quadratic complexity) you should use dictionaries which will give you roughly O(n log(n)) complexity. To do so, first read serial.uid once and store the mapping of MAC addresses to serial numbers in a dict.
serial = dict()
with open('serial.uid') as istr:
for line in istr:
(ser, mac) = split_fields(line)
serial[mac] = ser
Then you can close the file again and process mac.uid looking up the serial number for each MAC address in the dictionary you've just created.
combined = list()
with open('mac.uid') as istr:
for line in istr:
(mac, uid) = split_fields(line)
combined.append((mac, uid, serial[mac]))
Note that I've changed combined from a list of strings to a list of tuples. I've also factored the splitting logic out into a separate function. (You'll have to put its definition before its use, of course.)
def split_fields(line):
return line.strip().lower().split('|')
Finally, I recommend that you start using more descriptive names for your variables.
For files of 25k lines, you should have no issues storing everything in memory. If your data sets become too large for that, you might want to start looking into using a database. Note that the Python standard library ships with an SQLite module.
Super new at Python let alone coding. I am stuck on this assignment dealing with PPMImage and creating a readfile function.
Our professor gave us these hints for the function:
"Create a PPMImage using the constructor and these three integers
above as arguments (store in a variable called image_variable). Loop
until end of file Read red, green and blue components of pixel,
Append these to the list self.__pixels via method addpixel(), Close
filename Return the image_variable to caller"
What I have so far is this
def readfile(filename):
fv = open(filename, "r") ### Open filename for reading.
line = fv.readline() ### Read line containing "P3"
line = fv.readline().rstrip().split() ###Read line containing integers width and height (hint: use rstrip() and split())
line = fv.readline()
close = fv.close()
return image_variable
Any tips on how to get the rest of the code that is needed?
Thank you!
We had our customer details spread over 4 legacy systems and have subsequently migrated all the data into 1 new system.
Our customers previously had different account numbers in each system and I need to check which account number has been used in the new system, which supports API calls.
I have a text file containing all the possible account numbers, structured like this:
30000001, 30000002, 30000003, 30000004
30010000, 30110000, 30120000, 30130000
34000000, 33000000, 32000000, 31000000
Where each row represents all the old account numbers for each customer.
I'm not sure if this is the best approach but I want to open the text file and create a nested list:
[['30000001', '30000002', '30000003', '30000004'], ['30010000', '30110000', '30120000', '30130000'], ['34000000', '33000000', '32000000', '31000000']]
Then I want to iterate over each list but to save on API calls, as soon as I have verified the new account number in a particular list, I want to break out and move onto the next list.
import json
from urllib2 import urlopen
def read_file():
lines = [line.rstrip('\n') for line in open('customers.txt', 'r')]
return lines
def verify_account(*customers):
verified_accounts = []
for accounts in customers:
for account in accounts:
url = api_url + account
response = json.load(urlopen(url))
if response['account_status'] == 1:
verified_accounts.append(account)
break
return verified_accounts
The main issue is when I read from the file it returns the data like below so I can't iterate over the individual accounts.
['30000001, 30000002, 30000003, 30000004', '30010000, 30110000, 30120000, 30130000', '34000000, 33000000, 32000000, 31000000']
Also, is there a more Pythonic way of using list comprehensions or similar to iterate and check the account numbers. There seems to be too much nesting being used for Python?
The final item to mention is that there are over 255 customers to check, well there is almost 1000. Will I be able to pass more than 255 parameters into a function?
What about this? Just use str.split():
l = []
with open('customers.txt', 'r') as f:
for i in f:
l.append([s.strip() for s in i.split(',')])
Output:
[['30000001', '30000002', '30000003', '30000004'],
['30010000', '30110000', '30120000', '30130000'],
['34000000', '33000000', '32000000', '31000000']]
How about this?
with open('customers.txt','r') as f:
final_list=[i.split(",") for i in f.read().replace(" ","").splitlines()]
print final_list
Output:
[['30000001', '30000002', '30000003', '30000004'],
['30010000', '30110000', '30120000', '30130000'],
['34000000', '33000000', '32000000', '31000000']]