We Keep Coding

How to create a numpy array filled with average value of a vector

I am not sure how to phrase my question in any way better. Basically, I have three lists of the same length x, y and z and I want to fill a 2D numpy array in the z/y plane with the average of the associated z values.
Here is how I can achieve what I wan to do:
import numpy as np
import matplotlib.pyplot as plt
x = [37.59390426045407, 38.00530354847739, 38.28412244348653, 38.74871247986305, 38.73175910429809, 38.869008864244016, 39.188234404976555, 39.92835838352555, 40.881394113153334, 41.686136269465884]
y = [0.1305391767832006, 0.13764519613447768, 0.14573326951792354, 0.15090729309032114, 0.16355823707239897, 0.17327106424274763, 0.17749746339532224, 0.17310384614773594, 0.16545780437882962, 0.1604752704890856]
z = [0.05738534353865021, 0.012572155256903583, -0.021709582561809437, -0.11191337750722108, -0.07931921785775153, -0.06241610118871843, 0.014216349927058225, 0.042002641153291886, -0.029354425271534645, 0.061894011359833856]
n = 5
image = np.zeros(shape=(n,n))
# Fill the 2D array
x0 = min(x)
y0 = min(y)
dx = (max(x) - min(x))/n
dy = (max(y) - min(y))/n
# Loop over each 2D cell
for index_x in range(n):
for index_y in range(n):
# find the limits of the cell
x1 = x0 + index_x * dx
x2 = x0 + (index_x+1) * dx
y1 = y0 + index_y * dy
y2 = y0 + (index_y+1) * dy
# find the points of z that lie within the range of the cell
vec_z = [z[idx] for idx in range(len(z)) if x[idx]>=x1 and x[idx]<x2 and y[idx]>=y1 and y[idx]<y2]
if vec_z:
image[index_x, index_y] = np.mean(vec_z)
# In the end, used to create a surface plot
fig, ax = plt.subplots()
ax.imshow(image, cmap=plt.cm.gray, interpolation='nearest')
plt.show()
Is there a more easy way to achieve this? I can imagine there is a numpy method for that.

If I understand correctly what you want to do, maybe a 2D interpolation from scipy.interpolate.interp2d is what you are looking for.
You define the interpolation function of your points:
f = interp2d(x = x, y = y, z = z)
Then you define the X and Y meshgrid:
N = 50
x_axis = np.linspace(np.min(x), np.max(x), N)
y_axis = np.linspace(np.min(y), np.max(y), N)
X, Y = np.meshgrid(x_axis, y_axis)
Finally you can compute Z interpolated values on the meshgrid:
Z = np.zeros((N, N))
for i in range(N):
for j in range(N):
Z[i, j] = f(X[i, j], Y[i, j])
If you plot in 3D the interpolated surface, you get:
fig = plt.figure()
ax = fig.add_subplot(projection = '3d')
ax.plot_surface(X, Y, Z, cmap = 'jet', shade = False)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
plt.show()
Interpolated surface compared to interpolation data points:
ax.scatter(x, y, z, color = 'black', s = 100, alpha = 1)

How to fix "not enough values to unpack" when trying to plot 3D data as colormesh?

With matplotlib I am trying to plot 3D data as a 2D colormap. Each point has a x and a y coordinate, and a 'height' z. This height should determine the color a certain x/y region is colored in.
Here is the code I have been trying:
import random
import numpy as np
import matplotlib.pyplot as plt
x = []
y = []
z = []
for index in range(100):
a = random.random()
b = random.random()
c = np.exp(-a*a - b*b)
x.append(a)
y.append(b)
z.append(c)
cmap = plt.get_cmap('PiYG')
fig, ax = plt.subplots()
ax.pcolormesh(x, y, z, cmap=cmap)
But it gives an error
ValueError: not enough values to unpack (expected 2, got 1)
Maybe I am trying the wrong thing?
Remark: The three lists x,y,z and calculated for the example above, but in reality I have just three lists with "random" numbers in it I want to vizualize. I cannot calculate z given x and y.
I could also use imshow to create the plot I want, but I have to convert my original data into a matrix first. Maybe there is a function I can use?

pcolormesh might not be the choice for this kind of problem. pcolormesh expects ordered cell edges as data rather than random data points. You could do this if you know your grid before hand e.g.
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 1, 51)
# meshgrid makes a 2D grid of points
xx, yy = np.meshgrid(x, x)
z = np.exp(-xx**2 - yy*2)
fig, ax = plt.subplots()
ax.pcolormesh(xx, yy, z, cmap="PiYG")
which will give you
Alternatively, you could use one of the tri functions such as tripcolor with your existing setup
import random
import numpy as np
import matplotlib.pyplot as plt
x = []
y = []
z = []
for index in range(100):
a = random.random()
b = random.random()
c = np.exp(-a*a - b*b)
x.append(a)
y.append(b)
z.append(c)
fig, ax = plt.subplots()
ax.tripcolor(x, y, z, cmap="PiYG")
which will give
Note it would be simpler to use np.random to generate your data
x, y = np.random.random(size=(2, 100))
z = np.exp(-x**2 - y**2)
fig, ax = plt.subplots()
ax.tripcolor(x, y, z, cmap="PiYG")

There is an issue with x, y and z shapes: they have to be 2D arrays (matrices) but they are 1-dimensional.
In order to generate x and y axis, you could use:
x = []
y = []
for index in range(100):
x.append(random.random())
y.append(random.random())
Then you have to create a meshgrid:
X, Y = np.meshgrid(x, y)
Finally you can compute Z over the meshgrid:
Z = np.exp(-X**2 - Y**2)
In this way, your code:
cmap = plt.get_cmap('PiYG')
fig, ax = plt.subplots()
ax.pcolormesh(X, Y, Z, cmap=cmap)
gives:
If you you cannot compute Z on the meshgrid, then you should not use pcolormesh.
Some alternative could be:
3D scatterplot:
import random
import numpy as np
import matplotlib.pyplot as plt
x = []
y = []
z = []
for index in range(100):
a = random.random()
b = random.random()
c = np.exp(-a*a - b*b)
x.append(a)
y.append(b)
z.append(c)
cmap = plt.get_cmap('PiYG')
fig = plt.figure()
ax = fig.add_subplot(projection = '3d')
ax.scatter(x, y, z, cmap=cmap)
plt.show()
2D colored scatterplot:
import random
import numpy as np
import matplotlib.pyplot as plt
x = []
y = []
z = []
for index in range(100):
a = random.random()
b = random.random()
c = np.exp(-a*a - b*b)
x.append(a)
y.append(b)
z.append(c)
cmap = plt.get_cmap('PiYG')
plt.style.use('seaborn-darkgrid')
fig, ax = plt.subplots()
ax.scatter(x, y, c = z, cmap=cmap)
plt.show()

Way to contour outer edge of selected grid region in Python

I have the following code:
import numpy as np
import matplotlib.pyplot as plt
x = np.linspace(-np.pi/2, np.pi/2, 30)
y = np.linspace(-np.pi/2, np.pi/2, 30)
x,y = np.meshgrid(x,y)
z = np.sin(x**2+y**2)[:-1,:-1]
fig,ax = plt.subplots()
ax.pcolormesh(x,y,z)
Which gives this image:
Now lets say I want to highlight the edge certain grid boxes:
highlight = (z > 0.9)
I could use the contour function, but this would result in a "smoothed" contour. I just want to highlight the edge of a region, following the edge of the grid boxes.
The closest I've come is adding something like this:
highlight = np.ma.masked_less(highlight, 1)
ax.pcolormesh(x, y, highlight, facecolor = 'None', edgecolors = 'w')
Which gives this plot:
Which is close, but what I really want is for only the outer and inner edges of that "donut" to be highlighted.
So essentially I am looking for some hybrid of the contour and pcolormesh functions - something that follows the contour of some value, but follows grid bins in "steps" rather than connecting point-to-point. Does that make sense?
Side note: In the pcolormesh arguments, I have edgecolors = 'w', but the edges still come out to be blue. Whats going on there?
EDIT:
JohanC's initial answer using add_iso_line() works for the question as posed. However, the actual data I'm using is a very irregular x,y grid, which cannot be converted to 1D (as is required for add_iso_line().
I am using data which has been converted from polar coordinates (rho, phi) to cartesian (x,y). The 2D solution posed by JohanC does not appear to work for the following case:
import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage
def pol2cart(rho, phi):
x = rho * np.cos(phi)
y = rho * np.sin(phi)
return(x, y)
phi = np.linspace(0,2*np.pi,30)
rho = np.linspace(0,2,30)
pp, rr = np.meshgrid(phi,rho)
xx,yy = pol2cart(rr, pp)
z = np.sin(xx**2 + yy**2)
scale = 5
zz = ndimage.zoom(z, scale, order=0)
fig,ax = plt.subplots()
ax.pcolormesh(xx,yy,z[:-1, :-1])
xlim = ax.get_xlim()
ylim = ax.get_ylim()
xmin, xmax = xx.min(), xx.max()
ymin, ymax = yy.min(), yy.max()
ax.contour(np.linspace(xmin,xmax, zz.shape[1]) + (xmax-xmin)/z.shape[1]/2,
np.linspace(ymin,ymax, zz.shape[0]) + (ymax-ymin)/z.shape[0]/2,
np.where(zz < 0.9, 0, 1), levels=[0.5], colors='red')
ax.set_xlim(*xlim)
ax.set_ylim(*ylim)

This post shows a way to draw such lines. As it is not straightforward to adapt to the current pcolormesh, the following code demonstrates a possible adaption.
Note that the 2d versions of x and y have been renamed, as the 1d versions are needed for the line segments.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
x = np.linspace(-np.pi / 2, np.pi / 2, 30)
y = np.linspace(-np.pi / 2, np.pi / 2, 30)
xx, yy = np.meshgrid(x, y)
z = np.sin(xx ** 2 + yy ** 2)[:-1, :-1]
fig, ax = plt.subplots()
ax.pcolormesh(x, y, z)
def add_iso_line(ax, value, color):
v = np.diff(z > value, axis=1)
h = np.diff(z > value, axis=0)
l = np.argwhere(v.T)
vlines = np.array(list(zip(np.stack((x[l[:, 0] + 1], y[l[:, 1]])).T,
np.stack((x[l[:, 0] + 1], y[l[:, 1] + 1])).T)))
l = np.argwhere(h.T)
hlines = np.array(list(zip(np.stack((x[l[:, 0]], y[l[:, 1] + 1])).T,
np.stack((x[l[:, 0] + 1], y[l[:, 1] + 1])).T)))
lines = np.vstack((vlines, hlines))
ax.add_collection(LineCollection(lines, lw=1, colors=color))
add_iso_line(ax, 0.9, 'r')
plt.show()
Here is an adaption of the second answer, which can work with only 2d arrays:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from scipy import ndimage
x = np.linspace(-np.pi / 2, np.pi / 2, 30)
y = np.linspace(-np.pi / 2, np.pi / 2, 30)
x, y = np.meshgrid(x, y)
z = np.sin(x ** 2 + y ** 2)
scale = 5
zz = ndimage.zoom(z, scale, order=0)
fig, ax = plt.subplots()
ax.pcolormesh(x, y, z[:-1, :-1] )
xlim = ax.get_xlim()
ylim = ax.get_ylim()
xmin, xmax = x.min(), x.max()
ymin, ymax = y.min(), y.max()
ax.contour(np.linspace(xmin,xmax, zz.shape[1]) + (xmax-xmin)/z.shape[1]/2,
np.linspace(ymin,ymax, zz.shape[0]) + (ymax-ymin)/z.shape[0]/2,
np.where(zz < 0.9, 0, 1), levels=[0.5], colors='red')
ax.set_xlim(*xlim)
ax.set_ylim(*ylim)
plt.show()

I'll try to refactor add_iso_line method in order to make it more clear an open for optimisations. So, at first, there comes a must-do part:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
x = np.linspace(-np.pi/2, np.pi/2, 30)
y = np.linspace(-np.pi/2, np.pi/2, 30)
x, y = np.meshgrid(x,y)
z = np.sin(x**2+y**2)[:-1,:-1]
fig, ax = plt.subplots()
ax.pcolormesh(x,y,z)
xlim, ylim = ax.get_xlim(), ax.get_ylim()
highlight = (z > 0.9)
Now highlight is a binary array that looks like this:
After that we can extract indexes of True cells, look for False neighbourhoods and identify positions of 'red' lines. I'm not comfortable enough with doing it in a vectorised manner (like here in add_iso_line method) so just using simple loop:
lines = []
cells = zip(*np.where(highlight))
for x, y in cells:
if x == 0 or highlight[x - 1, y] == 0: lines.append(([x, y], [x, y + 1]))
if x == highlight.shape[0] or highlight[x + 1, y] == 0: lines.append(([x + 1, y], [x + 1, y + 1]))
if y == 0 or highlight[x, y - 1] == 0: lines.append(([x, y], [x + 1, y]))
if y == highlight.shape[1] or highlight[x, y + 1] == 0: lines.append(([x, y + 1], [x + 1, y + 1]))
And, finally, I resize and center coordinates of lines in order to fit with pcolormesh:
lines = (np.array(lines) / highlight.shape - [0.5, 0.5]) * [xlim[1] - xlim[0], ylim[1] - ylim[0]]
ax.add_collection(LineCollection(lines, colors='r'))
plt.show()
In conclusion, this is very similar to JohanC solution and, in general, slower. Fortunately, we can reduce amount of cells significantly, extracting contours only using python-opencv package:
import cv2
highlight = highlight.astype(np.uint8)
contours, hierarchy = cv2.findContours(highlight, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
cells = np.vstack(contours).squeeze()
This is an illustration of cells being checked:

Matplotlib contour hatching not working if only two levels was used

I am trying to plot hatches over contours lines that
statisfy certian criteria folliwng the example found here. Yet, I got regular contours (the yellow lines) instead of the hatches. Any ideas how to resolve that. Thanks
import matplotlib.pyplot as plt
import numpy as np
# invent some numbers, turning the x and y arrays into simple
# 2d arrays, which make combining them together easier.
x = np.linspace(-3, 5, 150).reshape(1, -1)
y = np.linspace(-3, 5, 120).reshape(-1, 1)
z = np.cos(x) + np.sin(y)
# we no longer need x and y to be 2 dimensional, so flatten them.
x, y = x.flatten(), y.flatten()
fig2, ax2 = plt.subplots()
n_levels = 6
a=ax2.contourf(x, y, z, n_levels)
fig2.colorbar(a)
[m,n]=np.where(z > 0.5)
z1=np.zeros(z.shape)
z1[m,n]=99
cs = ax2.contour(x, y, z1,2,hatches=['','.'])
plt.show()enter code here

Use contourf() with proper parameters to get useful plot with hatching. See important comment within the working code below:
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(-3, 5, 150).reshape(1, -1)
y = np.linspace(-3, 5, 120).reshape(-1, 1)
z = np.cos(x) + np.sin(y)
x, y = x.flatten(), y.flatten()
fig2, ax2 = plt.subplots()
n_levels = 6
a = ax2.contourf(x, y, z, n_levels)
fig2.colorbar(a)
[m,n] = np.where(z > 0.5)
z1=np.zeros(z.shape)
z1[m, n] = 99
# use contourf() with proper hatch pattern and alpha value
cs = ax2.contourf(x, y, z1 ,3 , hatches=['', '..'], alpha=0.25)
plt.show()
The output plot:

Contour plot in Python importing txt table file

I am trying to make a contour plot like:
Using a table of data like 3 columns in a txt file, with a long number of lines.
Using this code:
import numpy as np
import matplotlib.pyplot as plt
import scipy.interpolate
data = np.loadtxt(r'dataa.txt')
a = [data[:,0]]
b = [data[:,1]]
n = [data[:,2]]
x = np.asarray(a)
y = np.asarray(b)
z = np.asarray(n)
print "x = ", x
print "y = ", y
print "z = ", z
fig=plt.figure()
CF = contour(x,y,z,colors = 'k')
plt.xlabel("X")
plt.ylabel("Y")
plt.colorbar()
plt.show()
I don't know why, it is not working. Python gives me the right axes for the values that I am expecting to see, but in the graph is just a blank and I know that it is importing the data in right way because it shows me my values before the plot.
Example of table: (the diference is because my table has 90000 lines)
Using this code:
import numpy as np
import matplotlib.pyplot as plt
import scipy.interpolate
N = 1000 #number of points for plotting/interpolation
x, y, z = np.genfromtxt(r'dataa.txt', unpack=True)
xi = np.linspace(x.min(), x.max(), N)
yi = np.linspace(y.min(), y.max(), N)
zi = scipy.interpolate.griddata((x, y), z, (xi[None,:], yi[:,None]), method='cubic')
fig = plt.figure()
plt.contour(xi, yi, zi)
plt.xlabel("X")
plt.ylabel("Y")
plt.show()
Ive got this result:
I think I've got the advices wrongly.

Followup from my comment... first, I would replace all these lines:
data = np.loadtxt(r'dataa.txt')
a = [data[:,0]]
b = [data[:,1]]
n = [data[:,2]]
x = np.asarray(a)
y = np.asarray(b)
z = np.asarray(n)
With:
x, y, z = np.genfromtxt(r'dataa.txt', unpack=True)
Your original code is adding an extra axis at the front, since [data[:,0]] is a list of arrays with one element. The result is that x.shape will be (1, N) instead if (N,). All of this can be done automatically using the last line above, or you could just use the same data loading and say:
x = data[:,0]
y = data[:,1]
z = data[:,2]
since those slices will give you an array back.
However, you're not quite done, because plt.contour expects you to give it a 2d array for z, not a 1d array of values. Right now, you seem to have z values at given x, y points, but contour expects you to give it a 2d array, like an image.
Before I can answer that, I need to know how x and y are spaced. If regularly, you can just populate an array pretty easily. If not regularly, you basically have to interpolate before you can make a contour plot.
To do the interpolation, use
import numpy as np
import matplotlib.pyplot as plt
import scipy.interpolate
N = 1000 #number of points for plotting/interpolation
x, y, z = np.genfromtxt(r'dataa.txt', unpack=True)
xi = np.linspace(x.min(), x.max(), N)
yi = np.linspace(y.min(), y.max(), N)
zi = scipy.interpolate.griddata((x, y), z, (xi[None,:], yi[:,None]), method='cubic')
fig = plt.figure()
plt.contour(xi, yi, zi)
plt.xlabel("X")
plt.ylabel("Y")
plt.show()

The code below worked for me:
import scipy.interpolate
import numpy as np
N = 500 #number of points for plotting/interpolation
x, y, z = np.genfromtxt(r'data.dat', unpack=True)
xll = x.min(); xul = x.max(); yll = y.min(); yul = y.max()
xi = np.linspace(xll, xul, N)
yi = np.linspace(yll, yul, N)
zi = scipy.interpolate.griddata((x, y), z, (xi[None,:], yi[:,None]), method='cubic')
contours = plt.contour(xi, yi, zi, 6, colors='black')
plt.clabel(contours, inline=True, fontsize=7)
plt.imshow(zi, extent=[xll, xul, yll, yul], origin='lower', cmap=plt.cm.jet, alpha=0.9)
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.clim(0, 1)
plt.colorbar()
plt.show()