We Keep Coding

Grouping data by an certain rank in Python

I have a file that looks like this. The numbers in the segment are x and y coordinates.The text file
I want to only read the records with rank=1 and store the coordinates in a x and y list. So I need to read and save the rank and the number of points. If the program knows the number of points it knows how much coordinates it have to read and store.
I have already the following code but I am stuck at the point that I don't know how I tell the program that it needs to read the number of points until the new segment.
file = "/Users/yuval/Desktop/test1.txt"
x = []
y = []
with open(file, "r") as f:
for lines in f:
line = lines.split()
if(line[0] == "segment"):
rank = int(line[3])
points = int(line[5])

After your first if block, you can use an additional if block to append to your lists when rank==1.
x = []
y = []
rank = None
with open(file, "r") as f:
for lines in f:
line = lines.strip().split()
if(line[0] == "segment"):
rank = int(line[3])
points = int(line[5])
continue
if rank==1:
x.append(float(line[0]))
y.append(float(line[1]))

I want to extract data from a txt file with python

I want to extract the initial residual values from the file attached:
http://www.filedropper.com/fixvel
But the initial residual must be referred to the "DICPCG" line and not to others (like "smoothsolver"). Then I want to store those values into a matrix that contains the values of inital residual at the same time step (on the same row) for every time-step.
Really thanks in advance

Please try this code:
import re
import sys
file = open("fixVel.txt")
textfile = file.readlines()
count = -1
matrix = []
for row in textfile:
if row.find("Time") == 0:
count = count + 1
matrix.append([])
if row.find("DICPCG") == 0:
index = row.find("Initial residual")
index1 = row[index:].find(",")
matrix[count].append(row[index+19:index+index1])
#print matrix
for i in matrix:
for j in i:
sys.stdout.write(j + " ")
print

Finding Maximum Value in CSV File

Have an assignment of finding average and maximum rainfall in file "BoulderWeatherData.csv". Have found the average using this code:
rain = open("BoulderWeatherData.csv", "r")
data = rain.readline()
print(rain)
data = rain.readlines()
total = 0
linecounter = 0
for rain in data:
linecounter = linecounter + 1
print("The number of lines is", linecounter)
for line in data:
r = line.split(",")
total = total + float(r[4])
print(total)
average = float(total / linecounter)
print("The average rainfall is ", "%.2f" % average)
However, can't seem to find maximum using this same process. Attempted using max, function but the answer that must be obtained is float number, which can not be iterated through max.
Any help would be appreciated.

This is my prefered way of handling this.
#!/usr/bin/env python3
rain = open("BoulderWeatherData.csv","r")
average = 0.0
total = 0
maxt = 0.0
for line in rain:
try:
p = float(line.split(",")[4])
average += p
total += 1
maxt = max(maxt,p)
except:
pass
average = average / float(total)
print("Average:",average)
print("Maximum:",maxt)
This will output:
Average: 0.05465272591486193
Maximum: 1.98

import csv
INPUT = "BoulderWeatherData.csv"
PRECIP = 4 # 5th column
with open(INPUT, "rU") as inf:
incsv = csv.reader(inf)
header = next(incsv, None) # skip header row
precip = [float(row[PRECIP]) for row in incsv]
avg_precip = sum(precip, 0.) / (1 and len(precip)) # prevent div-by-0
max_precip = max(precip)
print(
"Avg precip: {:0.3f} in/day, max precip: {:0.3f} in/day"
.format(avg_precip, max_precip)
)
returns
Avg precip: 0.055 in/day, max precip: 1.980 in/day

max=0
for line in data:
r = line.split(",")
if float(r[4]) > max:
max=float(r[4])
print(max)
something like that

You're already accumulating total across loop iterations.
To keep track of a maxvalue, it's basically the same thing, except instead of adding you're maxing:
total = 0
maxvalue = 0
for line in data:
r = line.split(",")
value = float(r[4])
total = total + value
maxvalue = max(maxvalue, value)
print(total)
print(maxvalue)
Or, if you don't want to use the max function:
for line in data:
r = line.split(",")
value = float(r[4])
total = total + value
if value > maxvalue:
maxvalue = value

This code will attempt to find the maximum value, and the average value, of floats stored in the 5th position in a .csv.
rainval = []
Initializes the empty array where we will store values.
with open ("BoulderWeatherData.csv", "r") as rain:
Opens the .csv file and names it "rain".
for lines in rain:
This reads every line in rain until the end of the file.
rainval += [float(lines.strip().split(",")[4])]
We append the float value found in the fifth position (fourth index) of the line.
We repeat the above for every line located in the .csv file.
print (sorted(rainval)[len(rainval)])
This sorts the values in the rainval array and then takes the last (greatest) value, and prints it. This is the maximum value and is better than max because it can handle floats and not just ints.
print (sum(rainval)/len(rainval))
This prints the average rainfall.
Alternatively, if we don't want to use arrays:
maxrain = -float("inf")
total, count = 0, 0
with open ("test.txt", "r") as rain:
for lines in rain:
temp = float(lines.strip().split(",")[4])
if maxrain < temp:
maxrain = temp
total += temp
count += 1
print (maxrain)
print (total/count)

Python, if statement to change a number

I am learning Python, and I made a script which searches for several lines that contain the "keyword" and then write/print in a new file the (previously) selected list-line (I used a second argument to select the line from the list).
Everything went well until I tried to add a statement in case my selected list-line is > than the actual len(list) then the selected list_line = len(list); for whatever reason, it does not work.
Can anyone please point out to why it is not working, this my script. Thanks a million for the help. (Here is a link with an example of the files that I am using as an input)
import sys
import re
filename = sys.argv[1]
line_select = int(sys.argv[2])
newfile = str(filename) + ".3d"
openold = open(filename,"r")
opennew = open(newfile,"w")
rline = openold.readlines()
energies = []
line_number = 0
for line in rline:
line_number += 1
if re.search( r"SCF Done", line ):
words = line.split()
energy = float( words[4] )
energies.append(str(line_number) + " : " + "The energy of the molecule is %f kcal mol-1" % energy)
len_list = len(energies)
if line_select > len_list:
line_select = len_list
print >>opennew, energies[line_select]
openold.close()
opennew.close()

The last element of the energies list is actually energies[len_list-1], since Python indexes start from 0.
So if you want to print "the last element of energies", you need to initialize line_select to one less than the list length:
if line_select >= len_list:
line_select = len_list-1

looping until the number of cells changed is neglible

This is probably a simple question, but it's driving me crazy! I have a python code that performs cellular automata on a land use grid. I've made a dictionary of cell id: land use code imported from a text file. I've also import of the adjacent neighbors of each cell from a text file. For each cell in the nested loop, I pick out the highest value, count the highest value of the neighboring cells. If this value is greater than the processing cell and occurred more than 4 times, then I update the dictionary for that cell id. The land use codes are ranked in priority. You will see < 6 in the code below...6 is water and wetlands which I do not want to be changed. The first time I run the code, 7509 cells changed land use based on adjacent neighbors land uses. I can comment out the reading the dictionary text file and run it again, then around 5,000 cells changed. Run it again, then even less and so on. What I would like to do is run this in a loop until only 0.0001 of the total cells change, after that break the loop.
I've tried several times using iterators like "for r in range(999)---something big; If End_Sim > count: break". But it breaks after the first one, because the count goes back to zero. I've tried putting the count = 0 inside the loop and it adds up...I want it to start back over every time so the number of cells gets less and less. I'm stump...hopefully this is trivial to somebody!
Here's my code (it's a clean slate...I've deleted my failed attempts to create the number of simulations loop):
import sys, string, csv
#Creating a dictionary of FID: LU_Codes from external txt file
text_file = open("H:\SWAT\NC\FID_Whole_Copy.txt", "rb")
#Lines = text_file.readlines()
FID_GC_dict = dict()
reader = csv.reader(text_file, delimiter='\t')
for line in reader:
FID_GC_dict[line[0]] = int(line[1])
text_file.close()
#Importing neighbor list file for each FID value
Neighbors_file = open("H:\SWAT\NC\Pro_NL_Copy.txt","rb")
Entries = Neighbors_file.readlines()
Neighbors_file.close()
Neighbors_List = map(string.split, Entries)
#print Neighbors_List
#creates a list of the current FID
FID = [x[0] for x in Neighbors_List]
#Calculate when to end of one sweep
Tot_Cells = len(FID)
End_Sim = int(0.0001*Tot_Cells)
gridList = []
for nlist in Neighbors_List:
row = []
for item in nlist:
row.append(FID_GC_dict[item])
gridList.append(row)
#print gridList
#Performs cellular automata rules on land use grid codes
i = iter(FID)
count = 0
for glist in gridList:
Cur_FID = i.next()
Cur_GC = glist[0]
glist.sort()
lr_Value = glist[-1]
if lr_Value < 6:
tie_LR = glist.count(lr_Value)
if tie_LR >= 4 and lr_Value > Cur_GC:
FID_GC_dict[Cur_FID] = lr_Value
#print "The updated gridcode for FID ", Cur_FID, "is ", FID_GC_dict[Cur_FID]
count += 1
print count
Thanks for any help!

use a while loop:
cnt_total = 1234 # init appropriately
cnt_changed = cnt_total
p = 0.001
while (cnt_changed > cnt_total*p):
# your code here
# remember to update the cnt_changed variable

Try with the while break statements
initialization stuff
while(1):
...
if x < 0.0001:
break
...
http://docs.python.org/tutorial/controlflow.html#break-and-continue-statements-and-else-clauses-on-loops

I fixed the code so the simulations stop once the number of cells change is less than 0.0001 of the total cells. I had the while loop in the wrong place. Here's the code if anyone is interested in cellular automata.
import sys, string, csv
#Creating a dictionary of FID: LU_Codes from external txt file
text_file = open("H:\SWAT\NC\FID_Whole_Copy.txt", "rb")
#Lines = text_file.readlines()
FID_GC_dict = dict()
reader = csv.reader(text_file, delimiter='\t')
for line in reader:
FID_GC_dict[line[0]] = int(line[1])
text_file.close()
#Importing neighbor list file for each FID value
Neighbors_file = open("H:\SWAT\NC\Pro_NL_Copy.txt","rb")
Entries = Neighbors_file.readlines()
Neighbors_file.close()
Neighbors_List = map(string.split, Entries)
#print Neighbors_List
#creates a list of the current FID
FID = [x[0] for x in Neighbors_List]
#print FID
#Calculate when to end the simulations (neglible change in land use)
tot_cells = len(FID)
end_sim = tot_cells
p = 0.0001
#Performs cellular automata rules on land use grid codes
while (end_sim > tot_cells*p):
gridList = []
for nlist in Neighbors_List:
row = []
for item in nlist:
row.append(FID_GC_dict[item])
gridList.append(row)
#print gridList
i = iter(FID)
count = 0
for glist in gridList:
Cur_FID = i.next()
Cur_GC = glist[0]
glist.sort()
lr_Value = glist[-1]
if lr_Value < 6:
tie_LR = glist.count(lr_Value)
if tie_LR >= 4 and lr_Value > Cur_GC:
FID_GC_dict[Cur_FID] = lr_Value
print "The updated gridcode for FID ", Cur_FID, "is ", FID_GC_dict[Cur_FID]
count += 1
end_sim = count
print count