We Keep Coding

Creating multiple plots in Python for loop

I want to create a plot of (T/Tmax vs R/R0) for different values of Pa in a single plot like below. I have written this code that appends values to a list but all values of (T/Tmax vs R/R0) are appended in single list which does not give a good plot. What can I do to have such a plot? Also how can I make an excel sheet from the data from the loop where column 1 is T/Tmax list and column 2,3,4...are corresponding R/R0 values for different pa?
KLMDAT1 = []
KLMDAT2 = []
for j in range(z):
pa[j] = 120000-10000*j
i = 0
R = R0
q = 0
T = 0
while (T<Tmax):
k1 = KLM_RKM(i*dT,R,q,pa[j])
k2 = KLM_RKM((i+0.5)*dT,R,q+0.5*dT*k1,pa[j])
k3 = KLM_RKM((i+0.5)*dT,R,q+0.5*dT*k2,pa[j])
k4 = KLM_RKM((i+1)*dT,R,q+dT*k3,pa[j])
q = q +1/6.0*dT*(k1+2*k2+2*k3+k4)
R = R+dT*q
if(R>0):
KLMDAT1.append(T / Tmax)
KLMDAT2.append(R / R0)
if(R>Rmax):
Rmax = R
if (abs(q)>c or R < 0):
break
T=T+dT
i = i+1
wb.save('KLM.xlsx')
np.savetxt('KLM.csv',[KLMDAT1, KLMDAT2])
plt.plot(KLMDAT1, KLMDAT2)
plt.show()

You are plotting it wrong. Your first variable needs to be T/Tmax. So initialize an empty T list, append T values to it, divide it by Tmax, and then plot twice: first KLMDAT1 and then KLMDAT2. Following pseudocode explains it
KLMDAT1 = []
KLMDAT2 = []
T_list = [] # <--- Initialize T list here
for j in range(z):
...
while (T<Tmax):
...
T=T+dT
T_list.append(T) # <--- Append T here
i = i+1
# ... rest of the code
plt.plot(np.array(T_list)/Tmax, KLMDAT1) # <--- Changed here
plt.plot(np.array(T_list)/Tmax, KLMDAT2) # <--- Changed here
plt.show()

Python3 pysal error message for large dataset

I am using a package called pysal to run the following Theil Decomposition to find within and between outputs.
When I create a small dataframe below the package works.
See code below:
import pysal
path="/Users/username/Desktop/file1.csv"
df_table=pd.read_table(path, sep=",")
df2=pd.DataFrame(df_table)
df= df2.sort_values(['exposure'], ascending=True)
rr = np.array(df['exposure'])
drop = pysal.inequality.theil.Theil(rr)
print ('drop.T', drop.T) # this is total theil
dropp = pysal.inequality.theil.TheilD(rr, df['race'] )
print ('WG', dropp.wg) #within group
print ("BG", dropp.bg) #between group
When I try to run the same code on a much larger file I get the following ERROR MESSAGE:
How do i fix error message??
TypeError: unorderable types: float() < str()
Below is the sourced code from the pysal package
The data types appear to be the same for both file types.
Im uncertain why it is working work a small file but not a large file.
def __init__(self, y, partition):
groups = np.unique(partition)
T = Theil(y).T
ytot = y.sum(axis=0)
#group totals
gtot = np.array([y[partition == gid].sum(axis=0) for gid in groups])
mm = np.dot
if ytot.size == 1: # y is 1-d
sg = gtot / (ytot * 1.)
sg.shape = (sg.size, 1)
else:
sg = mm(gtot, np.diag(1. / ytot))
ng = np.array([sum(partition == gid) for gid in groups])
ng.shape = (ng.size,) # ensure ng is 1-d
n = y.shape[0]
# between group inequality
sg = sg + (sg==0) # handle case when a partition has 0 for sum
bg = np.multiply(sg, np.log(mm(np.diag(n * 1. / ng), sg))).sum(axis=0)
self.T = T
self.bg = bg
self.wg = T - bg

Why is my interpolation not working properly in my function?

I have a fairly long code that processes spectra, and along the way I need an interpolation of some points. I used to have all this code written line-by-line without any functions, and it all worked properly, but now I'm converting it to two large functions so that I can call it on other models more easily in the future. Below is my code (I have more code after the last line here that plots some things, but that's not relevant to my issue, since I've tested this with a bunch of print lines and learned that my issue arises when I call the interpolation function inside my process function.
import re
import numpy as np
import scipy.interpolate
# Required files and lists
filename = 'bpass_spectra.txt' # number of columns = 4
extinctionfile = 'ExtinctionLawPoints.txt' # R_V = 4.0
datalist = []
if filename == 'bpass_spectra.txt':
filetype = 4
else:
filetype = 1
if extinctionfile == 'ExtinctionLawPoints.txt':
R_V = 4.0
else:
R_V = 1.0 #to be determined
# Constants
h = 4.1357e-15 # Planck's constant [eV s]
c = float(3e8) # speed of light [m/s]
# Inputs
beta = 2.0 # power used in extinction law
R = 1.0 # star formation rate [Msun/yr]
z = 1.0 # redshift
M_gas = 1.0 # mass of gas
M_halo = 2e41 # mass of dark matter halo
# Read spectra file
f = open(filename, 'r')
rawlines = f.readlines()
met = re.findall('Z\s=\s(\d*\.\d+)', rawlines[0])
del rawlines[0]
for i in range(len(rawlines)):
newlist = rawlines[i].split(' ')
datalist.append(newlist)
# Read extinction curve data file
rawpoints = open(extinctionfile, 'r').readlines()
def interpolate(R_V, rawpoints, Elist, i):
pointslist = []
if R_V == 4.0:
for i in range(len(rawpoints)):
newlst = re.split('(?!\S)\s(?=\S)|(?!\S)\s+(?=\S)', rawpoints[i])
pointslist.append(newlst)
pointslist = pointslist[3:]
lambdalist = [float(item[0]) for item in pointslist]
k_abslist = [float(item[4]) for item in pointslist]
xvallist = [(c*h)/(lamb*1e-6) for lamb in lambdalist]
k_interp = scipy.interpolate.interp1d(xvallist, k_abslist)
return k_interp(Elist[i])
# Processing function
def process(interpolate, filetype, datalist, beta, R, z, M_gas, M_halo, met):
speclist = []
if filetype == 4:
metallicity = float(met[0])
Elist = [float(item[0]) for item in datalist]
speclambdalist = [h*c*1e9/E for E in Elist]
met1list = [float(item[1]) for item in datalist]
speclist.extend(met1list)
klist, Tlist = [None]*len(speclist), [None]*len(speclist)
if metallicity > 0.0052:
DGRlist = [50.0*np.exp(-2.21)*metallicity]*len(speclist) # dust to gas ratio
elif metallicity <= 0.0052:
DGRlist = [((50.0*metallicity)**3.15)*np.exp(-0.96)]*len(speclist)
for i in range(len(speclist)):
if Elist[i] <= 4.1357e-3: # frequencies <= 10^12 Hz
klist[i] = 0.1*(float(Elist[i])/(1000.0*h))**beta # extinction law [cm^2/g]
elif Elist[i] > 4.1357e-3: # frequencies > 10^12 Hz
klist[i] = interpolate(R_V, rawpoints, Elist, i) # interpolated function's value at Elist[i]
print "KLIST (INTERPOLATION) ELEMENTS 0 AND 1000:", klist[0], klist[1000]
return
The output from the print line is KLIST (INTERPOLATION) ELEMENTS 0 AND 1000: 52167.31734159269 52167.31734159269.
When I run my old code without functions, I print klist[0] and klist[1000] like I do here and get different values for each. In this new code, I get back two values that are the same from this line. This shouldn't be the case, so it must not be interpolating correctly inside my function (maybe it's not performing it on each point correctly in the loop?). Does anyone have any insight? It would be unreasonable to post my entire code with all the used text files here (they're very large), so I'm not expecting anyone to run it, but rather examine how I use and call my functions.
Edit: Below is the original version of my code up to the interpolation point without the functions (which works).
import re
import numpy as np
import scipy.interpolate
filename = 'bpass_spectra.txt'
extinctionfile = 'ExtinctionLawPoints.txt' # from R_V = 4.0
pointslist = []
datalist = []
speclist = []
# Constants
h = 4.1357e-15 # Planck's constant [eV s]
c = float(3e8) # speed of light [m/s]
# Read spectra file
f = open(filename, 'r')
rawspectra = f.readlines()
met = re.findall('Z\s=\s(\d*\.\d+)', rawspectra[0])
del rawspectra[0]
for i in range(len(rawspectra)):
newlist = rawspectra[i].split(' ')
datalist.append(newlist)
# Read extinction curve data file
rawpoints = open(extinctionfile, 'r').readlines()
for i in range(len(rawpoints)):
newlst = re.split('(?!\S)\s(?=\S)|(?!\S)\s+(?=\S)', rawpoints[i])
pointslist.append(newlst)
pointslist = pointslist[3:]
lambdalist = [float(item[0]) for item in pointslist]
k_abslist = [float(item[4]) for item in pointslist]
xvallist = [(c*h)/(lamb*1e-6) for lamb in lambdalist]
k_interp = scipy.interpolate.interp1d(xvallist, k_abslist)
# Create new lists
Elist = [float(item[0]) for item in datalist]
speclambdalist = [h*c*1e9/E for E in Elist]
z1list = [float(item[1]) for item in datalist]
speclist.extend(z1list)
met = met[0]
klist = [None]*len(speclist)
Loutlist = [None]*len(speclist)
Tlist = [None]*len(speclist)
# Define parameters
b = 2.0 # power used in extinction law (beta)
R = 1.0 # star formation ratw [Msun/yr]
z = 1.0 # redshift
Mgas = 1.0 # mass of gas
Mhalo = 2e41 # mass of dark matter halo
if float(met) > 0.0052:
DGRlist = [50.0*np.exp(-2.21)*float(met)]*len(speclist)
elif float(met) <= 0.0052:
DGRlist = [((50.0*float(met))**3.15)*np.exp(-0.96)]*len(speclist)
for i in range(len(speclist)):
if float(Elist[i]) <= 4.1357e-3: # frequencies <= 10^12 Hz
klist[i] = 0.1*(float(Elist[i])/(1000.0*h))**b # extinction law [cm^2/g]
elif float(Elist[i]) > 4.1357e-3: # frequencies > 10^12 Hz
klist[i] = k_interp(Elist[i]) # interpolated function's value at Elist[i]
print "KLIST (INTERPOLATION) ELEMENTS 0 AND 1000:", klist[0], klist[1000]
The output from this print line is KLIST (INTERPOLATION) ELEMENTS 0 AND 1000 7779.275435560996 58253.589270674354.

You are passing i as an argument to interpolate, and then also using i in a loop within interpolate. Once i is used within the for i in range(len(rawpoints)) loop in interpolate, it will be set to some value: len(rawpoints)-1. The interpolate function will then always return the same value k_interp(Elist[i]), which is equivalent to k_interp(Elist[len(rawpoints)-1]). You will need to either define a new variable within your loop (e.g. for not_i in range(len(rawpoints))), or use a different variable for the Elist argument. Consider the following change to interpolate:
def interpolate(R_V, rawpoints, Elist, j):
pointslist = []
if R_V == 4.0:
for i in range(len(rawpoints)):
newlst = re.split('(?!\S)\s(?=\S)|(?!\S)\s+(?=\S)', rawpoints[i])
pointslist.append(newlst)
pointslist = pointslist[3:]
lambdalist = [float(item[0]) for item in pointslist]
k_abslist = [float(item[4]) for item in pointslist]
xvallist = [(c*h)/(lamb*1e-6) for lamb in lambdalist]
k_interp = scipy.interpolate.interp1d(xvallist, k_abslist)
return k_interp(Elist[j])

Matlab to Python conversion - Can't assign to function call

I have recently been trying to convert a piece of Matlab code into Python code.
I have made most of the changes that I need to however, the issue I am having is the line where it says:
y(index(m)) = 1-x(index(m));
I get the error:
"Can't assign to function call"
However I am not sure how to restructure it in order to remove this error.
I have had a look around and people mention "get item" and "set item" however I have tried to use them, but I can't get them to work (probably because I can't figure out the structure)
Here is the full code:
import numpy
N = 100;
B = N+1;
M = 5e4;
burnin = M;
Niter = 20;
p = ones(B,Niter+1)/B;
hit = zeros(B,1);
for j in range(1,Niter):
x = double(rand(1,N)>0.5);
bin_x = 1+sum(x);
index = ceil(N*rand(1,M+burnin));
acceptval = rand(1,M+burnin);
for m in range(1,M+burnin):
y = x;
y(index(m)) = 1-x(index(m));
bin_y = 1+sum(y);
alpha = min(1, p(bin_x,j)/p(bin_y,j) );
if acceptval(m)<alpha:
x = y; bin_x = bin_y;
end
if m > burnin: hit(bin_x) = hit(bin_x)+1; end
end
pnew = p[:,j];
for b in range(1,B-1):
if (hit(b+1)*hit(b) == 0):
pnew(b+1) = pnew(b)*(p(b+1,j)/p(b,j));
else:
g(b,j) = hit(b+1)*hit(b) / (hit(b+1)+hit(b));
g_hat(b) = g(b,j)/sum(g(b,arange(1,j)));
pnew(b+1) = pnew(b)*(p(b+1,j)/p(b,j))+((hit(b+1)/hit(b))^g_hat(b));
end
end
p[:,j+1] = pnew/sum(pnew);
hit[:] = 0;
end
Thanks in advance

The round brackets () indicate a function. For indexing you need [] square brackets - but that is only the first of many, many errors... I am currently going through line by line, but it's taking a while.
This code at least runs... you need to figure out whether the indexing is doing what you are expecting since Python arrays are indexed from zero, and Matlab arrays start at 1. I tried to fix that in a couple of places but didn't go through line by line - that's debugging.
Some key learnings:
There is no end statement... just stop indenting
When you import a library, you need to reference it (numpy.zeros, not zeros)
Lists are indexed from zero, not one
Indexing is done with [], not ()
Creating an array of random numbers is done with [random.random() for r in xrange(N)], not random(N).
... and many other things you will find as you look through the code below.
Good luck!
import numpy
import random
N = int(100);
B = N+1;
M = 5e4;
burnin = M;
Niter = 20;
p = numpy.ones([B,Niter+1])/B;
hit = numpy.zeros([B,1]);
g = numpy.zeros([B, Niter]);
b_hat = numpy.zeros(B);
for j in range(1,Niter):
x = [float(random.randint(0,1)>0.5) for r in xrange(N)];
bin_x = 1+sum(x);
index = [random.randint(0,N-1) for r in xrange(int(M+burnin))];
#acceptval = rand(1,M+burnin);
acceptval = [random.random() for r in xrange(int(M+burnin))];
for m in range(1,int(M+burnin)):
y = x;
y[index[m]] = 1-x[index[m]];
bin_y = 1+sum(y);
alpha = min(1, p[bin_x,j]/p[bin_y,j] );
if acceptval[m]<alpha:
x = y; bin_x = bin_y;
if m > burnin:
hit[bin_x] = hit[bin_x]+1;
pnew = p[:,j];
for b in range(1,B-1):
if (hit[b+1]*hit[b] == 0):
pnew[b+1] = pnew[b]*(p[b+1,j]/p[b,j]);
else:
g[b,j] = hit[b+1]*hit[b] / [hit[b+1]+hit[b]];
g_hat[b] = g[b,j]/sum(g[b,numpy.arange(1,j)]);
pnew[b+1] = pnew[b]*(p[b+1,j]/p[b,j])+((hit[b+1]/hit[b])^g_hat[b]);
p[:,j+1] = pnew/sum(pnew);
hit[:] = 0;

How to pull specific information from an output in Python

So I have a code that gives an output, and what I need to do is pull the information out in between the commas, assign them to a variable that changes dynamically when called... here is my code:
import re
data_directory = 'Z:/Blender_Roto/'
data_file = 'diving_board.shape4ae'
fullpath = data_directory + data_file
print("====init=====")
file = open(fullpath)
for line in file:
current_line = line
# massive room for optimized code here.
# this assumes the last element of the line containing the words
# "Units Per Second" is the number we are looking for.
# this is a non float number, generally.
if current_line.find("Units Per Second") != -1:
fps = line_split = float(current_line.split()[-1])
print("Frames Per Second:", fps)
# source dimensions
if current_line.find("Source Width") != -1:
source_width = line_split = int(current_line.split()[-1])
print("Source Width:", source_width)
if current_line.find("Source Height") != -1:
source_height = line_split = int(current_line.split()[-1])
print("Source Height:", source_height)
# aspect ratios
if current_line.find("Source Pixel Aspect Ratio") != -1:
source_px_aspect = line_split = int(current_line.split()[-1])
print("Source Pixel Aspect Ratio:", source_px_aspect)
if current_line.find("Comp Pixel Aspect Ratio") != -1:
comp_aspect = line_split = int(current_line.split()[-1])
print("Comp Pixel Aspect Ratio:", comp_aspect)
# assumption, ae file can contain multiple mocha shapes.
# without knowing the exact format i will limit the script
# to deal with one mocha shape being animated N frames.
# this gathers the shape details, and frame number but does not
# include error checking yet.
if current_line.find("XSpline") != -1:
# record the frame number.
frame = re.search("\s*(\d*)\s*XSpline", current_line)
if frame.group(1) != None:
frame = frame.group(1)
print("frame:", frame)
# pick part the part of the line that deals with geometry
match = re.search("XSpline\((.+)\)\n", current_line)
line_to_strip = match.group(1)
points = re.findall('(\(.*?\))', line_to_strip)
print(len(points))
for point in points:
print(point)
print("="*40)
file.close()
This gives me the output:
====init=====
Frames Per Second: 24.0
Source Width: 2048
Source Height: 778
Source Pixel Aspect Ratio: 1
Comp Pixel Aspect Ratio: 1
frame: 20
5
(0.793803,0.136326,0,0.5,0)
(0.772345,0.642332,0,0.5,0)
(0.6436,0.597615,0,0.5,0)
(0.70082,0.143387,0,0.5,0.25)
(0.70082,0.112791,0,0.5,0)
========================================
So what I need for example is to be able to assign (0.793803, 0.136326, 0, 0.5, 0) to (1x,1y,1z,1w,1s), (0.772345,0.642332,0,0.5,0) to (2x, 2y, 2z, 2w, 2s) etc so that no matter what numbers are filling those positions they will take on that value.
here is the code I need to put those numbers into:
#-------------------------------------------------------------------------------
# Name: Mocha Rotoscoping Via Blender
# Purpose: Make rotoscoping more efficient
#
# Author: Jeff Owens
#
# Created: 11/07/2011
# Copyright: (c) jeff.owens 2011
# Licence: Grasshorse
#-------------------------------------------------------------------------------
#!/usr/bin/env python
import sys
import os
import parser
sys.path.append('Z:\_protomotion\Prog\HelperScripts')
import GetDir
sys.path.append('Z:\_tutorials\01\tut01_001\prod\Blender_Test')
filename = 'diving_board.shape4ae'
infile = 'Z:\_tutorials\01\tut01_001\prod\Blender_Test'
import bpy
from mathutils import Vector
#below are taken from mocha export
x_width =2048
y_height = 778
z_depth = 0
frame = 20
def readText():
text_file = open('diving_board.shape4ae', 'r')
lines = text_file.readlines()
print (lines)
print (len.lines)
for line in lines:
print (line)
##sets points final x,y,z value taken from mocha export for blender interface
point1x = (0.642706 * x_width)
point1y = (0.597615 * y_height)
point1z = (0 * z_depth)
point2x = (0.770557 * x_width)
point2y = (0.647039 * y_height)
point2z = (0 * z_depth)
point3x = (0.794697 * x_width)
point3y = (0.0869024 * y_height)
point3z = (0 * z_depth)
point4x = (0.707973* x_width)
point4y = (0.0751348 * y_height)
point4z = (0 * z_depth)
w = 1 # weight
listOfVectors = [Vector((point1x,point1y,point1z)),Vector((point2x,point2y,point2z)),Vector((point3x,point3 y,point3z)),Vector((point4x,point4y,point4z)), Vector((point1x,point1y,point1z))]
def MakePolyLine(objname, curvename, cList):
curvedata = bpy.data.curves.new(name=curvename, type='CURVE')
curvedata.dimensions = '3D'
objectdata = bpy.data.objects.new(objname, curvedata)
objectdata.location = (0,0,0) #object origin
bpy.context.scene.objects.link(objectdata)
polyline = curvedata.splines.new('POLY')
polyline.points.add(len(cList)-1)
for num in range(len(cList)):
x, y, z = cList[num]
polyline.points[num].co = (x, y, z, w)
MakePolyLine("NameOfMyCurveObject", "NameOfMyCurve", listOfVectors)
So where I have my vector I would like to be able to place (p.x, p.y,0.z,p.w,p.s) then (p2.x,p2.y,p2.zp2.wp2.s) etc so that it can change per the number given
Any help will be great.. thank you in advance!
-jeff

Instead of printing each output, you can create point objects and index them by name. For example:
>>> class Point:
... def __init__(self, t):
... (self.x,self.y,self.z,self.w,self.s) = t
...
>>> p = Point( (3,4,5,3,1) )
>>> p.w
3
You can place these point objects in an array, then access components by
myPoints[3].x
ADDENDUM
If it is important to you not to pull the points from an array, but rather use actual variable names, you can do the following, where points is your array of tuples:
(p0x,p0y,p0z,p0w,p0s) = points[0]
(p1x,p1y,p1z,p1w,p1s) = points[1]
(p2x,p2y,p2z,p2w,p2s) = points[2]
...
and so on.
Do consider whether this is an appropriate approach though. Having a point class allows you to have any number of points. With defined variable names, creating an unbounded number of these things on the fly is possible but almost always a bad idea. Here is a caveat about doing so: http://mail.python.org/pipermail/tutor/2005-January/035232.html.
When you have an array of point objects you do what you want much better! For example you can do the following:
myPoints[i].y = 12
thereby changing the y-coordinate of the ith point. This is next to impossible when you have fixed the variable names. Hope that helps! (And hope I understand your clarification! Let me know if not....)

If I'm reading your code right, the relevant portion is the loop at the end that produces your tuples.
data = []
for point in points:
data.append(point)
print(point)
That will create a new list and add each tuple to the list. So, data[0] holds (0.793803,0.136326,0,0.5,0) and data[0][0] holds 0.793803.