We Keep Coding

Struggles with matplotlib geoprojections

This is my first question here so I'd appreciate if you go easy on me.
I'm a total newbie in python and in programming in general to be honest. My main passion is astrophotography and my working horse is a very old program "Iris" written by Cristian Buil.
Recently I took spherical panorama and I wanted to represent it in Hammer-Aitoff projection. Iris is able to do that but it struggles with very large images. Therefore I thought that I could try to use a Python for this task.
During my research of this question I found that Matplotlib library seems to be designed for this. In particular Basemap tool directly offers the features necessary such as projection selection, image warping e.t.c.
I've tried to cope with the manuals but the longer I've played with the examples the more my frustration grew. It seems that only half of the examples work for me! Although I've replicated al the steps and installed the libraries necessary.
For example if I try to replicate these two examples from here, the first one returns an empty circle and the second is working properly:
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
from matplotlib import image
import matplotlib.image as mpimg
import numpy as np
#empty circle
plt.figure(figsize=(8, 8))
m = Basemap(projection='ortho', resolution=None, lat_0=50, lon_0=-100)
m.bluemarble(scale=0.5);
#proper map
fig = plt.figure(figsize=(8, 8))
m = Basemap(projection='lcc', resolution=None,
width=8E6, height=8E6,
lat_0=45, lon_0=-100,)
m.etopo(scale=0.5, alpha=0.5)
#it works even if I change "m.etopo(scale=0.5, alpha=0.5)" by "m.bluemarble(scale=0.5);"
#so it seems that the problem in the projection and not something else
# Map (long, lat) to (x, y) for plotting
x, y = m(-122.3, 47.6)
plt.plot(x, y, 'ok', markersize=5)
plt.text(x, y, ' Seattle', fontsize=12);
with the image warping I also have the same inconsistent behavior. Lambert projection works fine but the hammer or mollweide projections return empty ellipses.
#works just fine
m = Basemap(width=20000000,height=10000000,projection='lcc', resolution=None,lat_1=-55.,lat_2=-55,lat_0=-0,lon_0=30.)
m.warpimage(image="DJI_0114.png")
plt.show()
#empty shells in the output
m = Basemap(projection='hammer', resolution=None, lat_0=0,lon_0=0.)
m = Basemap(projection='mall', resolution=None, lat_0=0,lon_0=0.)
#There are no width and height fields here since python shows this message:
#"warning: width and height keywords ignored for Hammer projection"
m.warpimage(image="DJI_0114.png")
Manuals say that the image has to cover the whole sky/ground i.e. be in equirectangular projection with the aspect ratio of 2:1. So it is.
Is it something wrong with me/my_code or is there some mistake in the library?.. I bet that I'm doing something wrong but I can't see what. So any help would be very welcomed!

twinx non-linear mapping between shared y axes on a plot in matplotlib

I have some 3d data that I am plotting with pcolormesh.
On the x-axis is time, on the y-axis is height.
The height has a potential (E) associated with it, but the mapping from height (y) to potential (E) is non-linear.
I want to add an axis on the right hand side of my figure showing the potential that is correct based on the values on the left hand side. I do not particularly care about the left and right ticks lining up (as is the case in this solution). If anything 'nice number' ticks on the right axis would be useful.
I have tried setting the ylim of the top and bottom points as per the celsius-farenheit example in the matplotlib docs, but this assumes a linear scale between the start and end point which is not the case here.
Finally I tried using a funcformatter, but the scaling for the potential requires two external constants to be given, and I can't find a way that constants can be passed to a funcformatter.
So far my code looks like:
import numpy as np
import matplotlib.pyplot as plt
time = np.arange(0.0, 11.0, 1.0)
y = np.arange(0.0, 11.0, 1.0)
data = np.random.randint(1,100, size=(11,11))
fig,ax=plt.subplots(1,1)
im=ax.pcolormesh(time,y,data,shading='nearest')
ax.set_xlabel('time')
ax.set_ylabel('height')
ax.set_ylim(y.min(),y.max())
ax_E = ax.twinx()
ax_E.set_ylabel('Potential E')
plt.savefig('test.png')
Currently the right hand y axis has a linear scale from 0 to 1.0.
I would like this replacing with a scale showing the potential correct according to the values of y on the left hand y-axis.
The function I want to use for the potential is something like:
def get_E(mu, ymax, y):
p2 = 2.0*mu/ymax**3
Jmin = 2.0*np.sqrt(p2)*ymax
pmin = Jmin/(2.0*y)
E = np.sqrt(mu**2*pmin**2 + mu**2) - mu
return E
i.e. highly nonlinear, with 2 constants (mu and ymax) passed to it.
Any help you can give would be greatly appreciated.
I have done my best to search for a solution to this specific problem already, but my apologies if I have missed anything.
Please do ask any questions to clarify.

Matplotlib plot_surface transparency artefact

I'm trying to plot a surface in 3D from a set of data which specifies the z-values. I get some weird transparency artefact though, where I can see through the surface, even though I set alpha=1.0.
The artefact is present both when plotting and when saved to file (both as png and pdf):
I have tried changing the line width, and changing the number of strides from 1 to 10 (in the latter case, the surface is not visible though due to too rough resolution).
Q: How can I get rid of this transparency?
Here is my code:
import sys
import numpy as np
import numpy.ma as ma
import matplotlib as mpl
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
y_label = r'x'
x_label = r'y'
z_label = r'z'
x_scale = 2.0*np.pi
y_scale = 2.0*np.pi
y_numPoints = 250
x_numPoints = 250
def quasiCrystal(x, y):
z = 0
for i in range(0,5):
z += np.sin(x * np.cos(float(i)*np.pi/5.0) +
y * np.sin(float(i)*np.pi/5.0))
return z
x = np.linspace(-x_scale, x_scale, x_numPoints)
y = np.linspace(-y_scale, y_scale, y_numPoints)
X,Y = np.meshgrid(x,y)
Z = quasiCrystal(X, Y)
f = plt.figure()
ax = f.gca(projection='3d')
surf = ax.plot_surface( X, Y, Z,
rstride=5, cstride=5,
cmap='seismic',
alpha=1,
linewidth=0,
antialiased=True,
vmin=np.min(Z),
vmax=np.max(Z)
)
ax.set_zlim3d(np.min(Z), np.max(Z))
f.colorbar(surf, label=z_label)
ax.set_xlabel(x_label)
ax.set_ylabel(y_label)
ax.set_zlabel(z_label)
plt.show()
Here is another picture of my actual data where it is easier to see the artefact:

Matplotlib is not a "real" 3D engine. This is a very well known problem and once in a while a similar question to yours appears appears (see this and this). The problem is that the same artefact can originate problems that seem to be different. I believe such is the case for you.
Before going on with my recommendations let me just quote this information from the maplotlib website:
My 3D plot doesn’t look right at certain viewing angles
This is probably the most commonly reported issue with mplot3d. The problem is
that – from some viewing angles – a 3D object would appear in front of
another object, even though it is physically behind it. This can
result in plots that do not look “physically correct.”
Unfortunately, while some work is being done to reduce the occurance
of this artifact, it is currently an intractable problem, and can not
be fully solved until matplotlib supports 3D graphics rendering at its
core.
The problem occurs due to the reduction of 3D data down to 2D +
z-order scalar. A single value represents the 3rd dimension for all
parts of 3D objects in a collection. Therefore, when the bounding
boxes of two collections intersect, it becomes possible for this
artifact to occur. Furthermore, the intersection of two 3D objects
(such as polygons or patches) can not be rendered properly in
matplotlib’s 2D rendering engine.
This problem will likely not be solved until OpenGL support is added
to all of the backends (patches are greatly welcomed). Until then, if
you need complex 3D scenes, we recommend using MayaVi.
It seems that Mayavi has finally moved on to Python 3, so its certainly a possibility. If you want to stick with matplotlib for this kind of plot my advice is that you work with rstride and cstride values to see which ones produce a plot satisfactory to you.
surf = ax.plot_surface( X, Y, Z,
rstride=5, cstride=5,
cmap='jet',
alpha=1,
linewidth=0,
antialiased=True,
vmin=0,
rstride=10,
cstride=10,
vmax=z_scale
)
Other possibility is to try to see if other kinds of 3D plots do better. Check plot_trisurf, contour or contourf. I know its not ideal but in the past I also managed to circumvent other type of artefacts using 3D polygons.
Sorry for not having a more satisfactory answer. Perhaps other SO users have better solutions for this. Best of luck.

I ran into some similar issues and found that they were antialiasing artifacts and could be fixed by setting antialiased=False in plot_surface.

Matplotlib can't render multiple contour plots on Django

Whenever (at least) 2 people try to generate a contour plot in my application, at least one of them will receive a random error depending on how far the first person managed to draw.. ("unknown element o", "ContourSet must be in current Axes" are just two of the possibilities)
The following is a cut down test that can produce the error, if you try to load this page in 2 or more tabs at once, the first will render correctly whilst the second will produce an error. (Easiest way I found to do this was to click the refresh page button in chrome with the middle mouse button a couple times)
views.py
def home(request):
return render(request, 'home.html', {'chart': _test_chart()})
def _test_chart():
import base64
import cStringIO
import matplotlib
matplotlib.use('agg')
from matplotlib.mlab import bivariate_normal
import matplotlib.pyplot as plt
import numpy as np
from numpy.core.multiarray import arange
delta = 0.5
x = arange(-3.0, 4.001, delta)
y = arange(-4.0, 3.001, delta)
X, Y = np.meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = (Z1 - Z2) * 10
fig = plt.figure(figsize=(10, 5))
plt.contour(X, Y, Z, 10, colors='k')
jpg_image_buffer = cStringIO.StringIO()
fig.savefig(jpg_image_buffer)
array = base64.b64encode(jpg_image_buffer.getvalue())
jpg_image_buffer.close()
return array
home.html (just this one line is enough)
<img src="data:image/png;base64,{{ chart }}" />
I've tried using mpld3 instead to handle the generation of the image and this still produces different errors so I know its definitely not the saving of the figure but more its generation. I've also tried using a ThreadPool and Threading to no avail, from what I can tell it seems like creating a contour plot in matplotlib cannot support multiple instances which will never work for a website...
My only clear solution I can think of right now is to replace matplotlib with something else which I really don't want to do.
Is there a way to generate contour plots with matplotlib that will work for me?

First, let me start by saying that this is much more easy to reproduce by calling _test_chart in a couple threads
from threading import Thread
for i in xrange(2):
Thread(target=_test_chart).start()
Doing the above, one will work as desired whilst the second one will crash.
The simple reason for this is that the pyplot module is not designed for multithreading and therefore the two charts are getting their data mixed up as they attempt to draw.
This can be better explained by mdboom
...pyplot is used for convenient plotting at the commandline and keeps around global state. For example, when you say plt.figure() it adds the figure to a global list and then sets the "current figure" pointer to the most recently created figure. Then subsequent plotting commands automatically write to that figure. Obviously, that's not threadsafe...
There are two ways to fix this issue,
Force these charts to be drawn in different processes.
for i in xrange(2):
pool = Pool(processes=1)
pool.apply(_test_chart)
Whilst this will work you will find that there is a noticable drop in performance since it will often take just as long to create the process as it will to generate the chart (which I didn't think was acceptable!)
The real solution is to use the OO interface modules of Matplotlib which will then allow you to work with the correct objects - essentially this works down to working with subplots rather than plots. For the given example in the question, this would look like the following
def _test_chart2():
delta = 0.5
x = arange(-3.0, 4.001, delta)
y = arange(-4.0, 3.001, delta)
X, Y = np.meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = (Z1 - Z2) * 10
fig = figure(figsize=(10, 5))
ax1 = fig.add_subplot(111)
extents = [x.min(), x.max(), y.min(), y.max()]
im = ax1.imshow(Z,
interpolation='spline36',
extent=extents,
origin='lower',
aspect='auto',
cmap=cm.jet)
ax1.contour(X, Y, Z, 10, colors='k')
jpg_image_buffer = cStringIO.StringIO()
fig.savefig(jpg_image_buffer)
array = base64.b64encode(jpg_image_buffer.getvalue())
jpg_image_buffer.close()
return array

set matplotlib 3d plot aspect ratio

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
Setting the aspect ratio works for 2d plots:
ax = plt.axes()
ax.plot([0,1],[0,10])
ax.set_aspect('equal','box')
But does not for 3d:
ax = plt.axes(projection='3d')
ax.plot([0,1],[0,1],[0,10])
ax.set_aspect('equal','box')
Is there a different syntax for the 3d case, or it's not implemented?

As of matplotlib 3.3.0, Axes3D.set_box_aspect seems to be the recommended approach.
import numpy as np
import matplotlib.pyplot as plt
xs, ys, zs = ...
ax = plt.axes(projection='3d')
ax.set_box_aspect((np.ptp(xs), np.ptp(ys), np.ptp(zs))) # aspect ratio is 1:1:1 in data space
ax.plot(xs, ys, zs)

I didn't try all of these answers, but this kludge did it for me:
def axisEqual3D(ax):
extents = np.array([getattr(ax, 'get_{}lim'.format(dim))() for dim in 'xyz'])
sz = extents[:,1] - extents[:,0]
centers = np.mean(extents, axis=1)
maxsize = max(abs(sz))
r = maxsize/2
for ctr, dim in zip(centers, 'xyz'):
getattr(ax, 'set_{}lim'.format(dim))(ctr - r, ctr + r)

Looks like this feature has since been added so thought I'd add an answer for people who come by this thread in the future like I did:
fig = plt.figure(figsize=plt.figaspect(0.5)*1.5) #Adjusts the aspect ratio and enlarges the figure (text does not enlarge)
ax = fig.add_subplot(projection='3d')
figaspect(0.5) makes the figure twice as wide as it is tall. Then the *1.5 increases the size of the figure. The labels etc won't increase so this is a way to make the graph look less cluttered by the labels.

I think setting the correct "box aspect" is a good solution:
ax.set_box_aspect(aspect = (1,1,1))
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(projection='3d')
ax.set_box_aspect(aspect = (1,1,1))
ax.plot(dataX,dataY,dataZ)
https://matplotlib.org/stable/api/_as_gen/mpl_toolkits.mplot3d.axes3d.Axes3D.html?highlight=3d%20set_box_aspect#mpl_toolkits.mplot3d.axes3d.Axes3D.set_box_aspect

If you know the bounds, eg. +-3 centered around (0,0,0), you can add invisible points like this:
import numpy as np
import pylab as pl
from mpl_toolkits.mplot3d import Axes3D
fig = pl.figure()
ax = fig.add_subplot(projection='3d')
ax.set_aspect('equal')
MAX = 3
for direction in (-1, 1):
for point in np.diag(direction * MAX * np.array([1,1,1])):
ax.plot([point[0]], [point[1]], [point[2]], 'w')

If you know the bounds you can also set the aspect ratio this way:
ax.auto_scale_xyz([minbound, maxbound], [minbound, maxbound], [minbound, maxbound])

Another helpful (hopefully) solution when, for example, it is necessary to update an already existing figure:
world_limits = ax.get_w_lims()
ax.set_box_aspect((world_limits[1]-world_limits[0],world_limits[3]-world_limits[2],world_limits[5]-world_limits[4]))
get_w_lims()
set_box_aspect()

My understanding is basically that this isn't implemented yet (see this bug in GitHub). I'm also hoping that it is implemented soon. See This link for a possible solution (I haven't tested it myself).

A follow-up to Matt Panzer's answer. (This was originally a comment on said answer.)
limits = np.array([getattr(ax, f'get_{axis}lim')() for axis in 'xyz'])
ax.set_box_aspect(np.ptp(limits, axis=1))
Now that this pull request has been merged, when the next release of Matplotlib drops, you should be able to just use ax.set_aspect('equal'). I will try to remember and update this answer when that happens.
Update: Matplotlib 3.6 has been released; ax.set_aspect('equal') will now work as expected.

As of matplotlib 3.6.0, this feature has been added with the shortcut
ax.set_aspect('equal'). Other options are 'equalxy', 'equalxz', and 'equalyz', to set only two directions to equal aspect ratios. This changes the data limits, example below.
In the upcoming 3.7.0, you will be able to change the plot box aspect ratios rather than the data limits via the command ax.set_aspect('equal', adjustable='box'). (Thanks to #tfpf on another answer here for implementing that!) To get the original behavior, use adjustable='datalim'.

Matt Panzer's answer worked for me, but it took me a while to figure out an issue I had.
If you're plotting multiple datasets into the same graph, you have to calculate the peak-to-peak values for the entire range of datapoints.
I used the following code to solve it for my case:
x1, y1, z1 = ..., ..., ...
x2, y2, z2 = ..., ..., ...
ax.set_box_aspect((
max(np.ptp(x1), np.ptp(x2)),
max(np.ptp(y1), np.ptp(y2)),
max(np.ptp(z1), np.ptp(y2))
))
ax.plot(x1, y1, z1)
ax.scatter(x2, y2, z2)
Note that this solution is not perfect. It will not work if x1 contains the most negative number and x2 contains the most positive one. Only if either x1 or x2 contains the greatest peak-to-peak range.
If you know numpy better than I do, feel free to edit this answer so it works in a more general case.

I tried several methods, such as ax.set_box_aspect(aspect = (1,1,1)) and it does not work. I want a sphere to show up as a sphere -- not ellipsoid. I wrote this function and tried it on a variety of data. It is a hack and it is not perfect, but pretty close.
def set_aspect_equal(ax):
"""
Fix the 3D graph to have similar scale on all the axes.
Call this after you do all the plot3D, but before show
"""
X = ax.get_xlim3d()
Y = ax.get_ylim3d()
Z = ax.get_zlim3d()
a = [X[1]-X[0],Y[1]-Y[0],Z[1]-Z[0]]
b = np.amax(a)
ax.set_xlim3d(X[0]-(b-a[0])/2,X[1]+(b-a[0])/2)
ax.set_ylim3d(Y[0]-(b-a[1])/2,Y[1]+(b-a[1])/2)
ax.set_zlim3d(Z[0]-(b-a[2])/2,Z[1]+(b-a[2])/2)
ax.set_box_aspect(aspect = (1,1,1))