matplotlib pcolormesh behaviour with large arrays - python

I am trying to plot an array, but run into a problem when the array gets too large. However, it seems to depend on whether or not the data is monotonic. In the example below, I increase the size of the array that I am plotting. The expected behaviour is the same image in the left and right hand sides, but this is not so. Furthermore, the size of the array I am able to plot correctly seems to depend on the size of the axis itself. When the figures are exported, there is no problem - just the display on the screen, which looks like the screenshot below:
len_xs = [4850,4860,4870] # sizes to try and plot
for j in np.arange(3):
x = np.arange(len_xs[j]); # dummy x variable
y = 2**np.linspace(-4.18676100,1.39659793,538) # log2 spaced y variable
z = np.random.random((len(y),len(x))) # random variable to plot
plt.subplot(3,2,j*2+1);
if j==0: plt.title('Right way up')
plt.pcolormesh(x,y,z,shading='Nearest');
plt.gca().set_yscale('log',base=2)```
plt.subplot(3,2,j*2+2);
if j==0: plt.title('Upside down')
plt.pcolormesh(x,y[::-1],z[::-1],shading='Nearest'); # flip y and z
plt.gca().set_yscale('log',base=2);
plt.text(1000,1,'nx = %g'%len_xs[j],bbox=dict(facecolor='white', alpha=0.8)) # annotate size of x
plt.savefig('demo.png',dpi=600)

The cause is unknown, but since the DPI of the saved image is high, it was displayed with the same content as the saved file, which matched the DPI of the display accordingly.
plt.rcParams['figure.dpi'] = 600

Related

Python - plt.savefig() resulting in blank image without plt.show()

I work on a crowd simulation, and I tried to get a simple representation at a given time like this : new to the site so here is the link. I work with Spyder and the code works wonderfully when I display the image in ipython with plt.show(), but when i try to save the images with plt.savefig() (I removed plt.show() prior to that, not the issue) i ended up with blank images. Here is the code :
p,v,t = resolve() #p[c][i] is the position vector of individual i at time t[c]
N = len(t) # number of instant
n = len(m) # number of individual
murs_x = [w[0] for w in W] # wall points x coordinates
murs_y = [w[1] for w in W] # wall points y coordinates
conv = 39.3701 #inch/m
L = longueur*conv/50 # width of figure
H = (largeur + decalage)*conv/50 # height of figure
for c in range(N):
fig1 = plt.figure(num="aff",figsize = (L,H), dpi = 200) # arbitrary num, allow to recreate the figure
ax = fig1.add_axes([1,1,1,1])
ax.set_axis_off() # visual purpose
ax.set_frame_on(False) # visual purpose
ax.axis([0,longueur,0,largeur+decalage])
ax.scatter(murs_x,murs_y,s=0.01,marker='.')
for i in range(n):
if p[c][i][1] <= (largeur + r[i]): # presence condition for individual i
ax.add_artist(plt.Circle((p[c][i][0], p[c][i][1]), r[i], alpha=1))
# drawing of the circle representing individual i
# here is the plt.show(), unused
fig1.savefig(str(c)+".png") # trying to save instant c visual represention
fig1.clf()
Moreover, without the 2 lines for visual purposes, the images are not totally blank but rather like this : another link.
I first attempted to use matplotlib.animation to create a video, however i had the same issue of a blank video with 2 cropped zeros in the upper right corner. I suppose that the issue is linked to the artist class (I had better results using scattered points instead of circles to represent each individual) but I am a beginner and do not know how to handle it precisely. At least the size of the image is the one expected one.
Thanks for reading this.

Plotting per-point alpha values in 3D scatterplot throws ValueError

I have data in form of a 3D array, with "intensities" at every point. Depending on the intensity, I want to plot the point with a higher alpha. There are a lot of low-value outliers, so color coding (with scalar floats) won't work since they eclipse the real data.
What I have tried:
#this generates a 3D array with higher values around the center
a = np.array([0,1,2,3,4,5,4,3,2,1])
aa = np.outer(a,a)
aaa = np.einsum("ij,jk,jl",aa,aa,aa)
x_,y_,z_,v_ = [],[],[],[]
from matplotlib.colors import to_rgb,to_rgba
for x in range(aaa.shape[0]):
for y in range(aaa.shape[1]):
for z in range(aaa.shape[2]):
x_.append(x)
y_.append(y)
z_.append(z)
v_.append(aaa[x,y,z])
r,g,b = to_rgb("blue")
color = np.array([[r,g,b,a] for a in v_])
fig = plt.figure()
ax = fig.add_subplot(projection = '3d')
ax.scatter(x_,y_,z_,c =color)
plt.show()
the scatterplot documentation says that color can be a 2D array of RGBA, which I do pass. Hoever when I try to run the code, I get the following error:
ValueError: 'c' argument has 4000 elements, which is inconsistent with 'x' and 'y' with size 1000.
I just found my own answer.
The "A 2D array in which the rows are RGB or RGBA." statement in the documentation was a bit confusing - one needs to convert the RGBA rows to RGBA objects first, so that list comprehension should read:
color = np.array([to_rgba([r,g,b,a]) for a in v_])

Changing the lognorm() base from base10 to base2 and saving the image

I have a script that plots out a heat map using matplotlib. Range of X-axis value = (-180 to +180) and Y-axis value =(0 to 180). The 2D heatmap colours areas in Rainbow according to the number of points occurring in a specified area in the x-y graph (defined by the 'bins' - see code below). In this case, x = values_Rot and y = values_Tilt (see below for code).
As of now, this script colours the 2D-heatmap on a log_base10 scale. However, the range of my data is small and I am thinking of changing the base from 10 to base 2.
Is there a way to colour the heatmap by changing the base from 10 to 2 in this code?
Also, I find that the image which has popped up after running this script clearly shows the heatmap. However, when I see the PNG image that has been saved, I see it is a blank image (pure white image) with nothing on it. Is there a way to save the figure - am I missing something while importing?
import matplotlib.pyplot as plt
import matplotlib as mpl
import numpy as np
values_Rot = []
values_Tilt = []
values_Psi = []
for line in data:
try:
values_Rot.append(float(line.split()[rot_number]))
values_Tilt.append(float(line.split()[tilt_number]))
values_Psi.append(float(line.split()[psi_number]))
except:
print ('This line didnt work, it may just be a blank space. The line is:' + line)
# Change the values here if you want to plot something else, such as psi.
# You can also change how the data is binned here.
plt.hist2d(values_Rot, values_Tilt, norm=mpl.colors.LogNorm(), bins=100,)
plt.set_cmap('jet')
plt.colorbar()
plt.show()
plt.savefig('name_of_output.png')

How to hack this Bokeh HexTile plot to fix the coords, label placement and axes?

Below is Bokeh 1.4.0 code that tries to draw a HexTile map of the input dataframe, with axes, and tries to place labels on each hex.
I've been stuck on this for two days solid, reading bokeh doc, examples and github known issues, SO, Bokeh Discourse and Red Blob Games's superb tutorial on Hexagonal Grids, and trying code. (I'm less interested in raising Bokeh issues for the future, and far more interested in pragmatic workarounds to known limitations to just get my map code working today.) Plot is below, and code at bottom.
Here are the issues, in rough decreasing order of importance (it's impossible to separate the root-cause and tell which causes which, due to the way Bokeh handles glyphs. If I apply one scale factor or coord transform it fixes one set of issues, but breaks another, 'whack-a-mole' effect):
The label placement is obviously wrong, but I can't seem to hack up any variant of either (x,y) coords or (q,r) coords to work. (I tried combinations of figure(..., match_aspect=True)), I tried 1/sqrt(2) scaling the (x,y)-coords, I tried Hextile(... size, scale) params as per redblobgames, e.g. size = 1/sqrt(3) ~ 0.57735).
Bokeh forces the origin to be top left, and y-coords to increase as you go down, however the default axis labels show y or r as being negative. I found I still had to use p.text(q, -r, .... I suppose I have to manually patch the auto-supplied yaxis labels or TickFormatter to be positive.
I use np.mgrid to generate the coord grid, but I still seem to have to assign q-coords right-to-left: np.mgrid[0:8, (4+1):0:-1]. Still no matter what I do, the hexes are flipped L-to-R
(Note: empty '' counties are placeholders to get the desired shape, hence the boolean mask [counties!=''] on grid coords. This works fine and I want to leave it as-is)
The source (q,r) coords for the hexes are integers, and I use 'odd-r' offset coords (not axial or hexagonal coords). No matter what HexTile(..., size, scale) args I use, one or both dimensions in the plot is wrong or squashed. Or whether I include the 1/sqrt(2) factor in coord transform.
My +q-axis is east and my +r-axis should be 120° SSE
Ideally I'd like to have my origin at bottom-left (math plot style, not computer graphics). But Bokeh apparently doesn't support that, I can live without that. However defaulting the y-axis labels to negative, while requiring a mix of positive and negative coords, is confusing. Anyway, how to hack an automatic fix to that with minimum grief? (manual p.yrange = Range1d(?, ?)?)
Bokeh's approach to attaching (hex) glyphs to plots is a hard idiom to use. Ideally I simply want to reference (q,r)-coords everywhere for hexes, labels, axes. I never want to see (x,y)-coords appearing on axes, label coords, tick-marks, etc. but seems Bokeh won't allow you. I guess you have to manually hack the axes and ticks later. Also, the plot<->glyph interface doesn't allow you to expose a (q,r) <-> (x,y) coord transform function, certainly not a bidirectional one.
The default axes don't seem to have any accessors to automatically find their current extent/limits; p.yaxis.start/end are empty unless you specified them. The result from p.yaxis.major_tick_in,p.yaxis.major_tick_out is also wrong, for this plot it gives (2,6) for both x and y, seems to be clipping those to the interior multiples of 2(?). How to automatically get the axes' extent?
My current plot:
My code:
import pandas as pd
import numpy as np
from math import sqrt
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource
from bokeh.models.glyphs import HexTile
from bokeh.io import show
# Data source is a list of county abbreviations, in (q,r) coords...
counties = np.array([
['TE','DY','AM','DN', ''],
['DL','FM','MN','AH', ''],
['SO','LM','CN','LH', ''],
['MO','RN','LD','WH','MH'],
['GA','OY','KE','D', ''],
['', 'CE','LS','WW', ''],
['LC','TA','KK','CW', ''],
['KY','CR','WF','WX', ''],
])
#counties = counties[::-1] # UNUSED: flip so origin is at bottom-left
# (q,r) Coordinate system is “odd/even-r” horizontal Offset coords
r, q = np.mgrid[0:8, (4+1):0:-1]
q = q[counties!='']
r = r[counties!='']
sqrt3 = sqrt(3)
# Try to transform odd-r (q,r) offset coords -> (x,y). Per Red Blob Games' tutorial.
x = q - (r//2) # this may be slightly dubious
y = r
counties_df = pd.DataFrame({'q': q, 'r': r, 'abbrev': counties[counties!=''], 'x': x, 'y': y })
counties_ds = ColumnDataSource(ColumnDataSource.from_df(counties_df)) # ({'q': q, 'r': r, 'abbrev': counties[counties != '']})
p = figure(tools='save,crosshair') # match_aspect=True?
glyph = HexTile(orientation='pointytop', q='x', r='y', size=0.76, fill_color='#f6f699', line_color='black') # q,r,size,scale=??!?!!? size=0.76 is an empirical hack.
p.add_glyph(counties_ds, glyph)
p.xaxis.minor_tick_line_color = None
p.yaxis.minor_tick_line_color = None
print(f'Axes: x={p.xaxis.major_tick_in}:{p.xaxis.major_tick_out} y={p.yaxis.major_tick_in}:{p.yaxis.major_tick_out}')
# Now can't manage to get the right coords for text labels
p.text(q, -r, text=["(%d, %d)" % (q,r) for (q, r) in zip(q, r)], text_baseline="middle", text_align="center")
# Ideally I ultimately want to fix this and plot `abbrev` column as the text label
show(p)
There is an axial_to_cartesian function that will just compute the hex centers for you. You can then attach the labels in a variety of orientations and anchoring from these.
Bokeh does not force the origin to be anywhere. There is one axial to cartesian mapping Bokeh uses, exactly what is given by axial_to_cartesian. The position of the Hex tiles (and hence the cartesian coordinates that the axes display) follows from this. If you want different ticks, Bokeh affords lots of control points over both tick location and tick labelling.
There is more than one convention for Axial coords. Bokeh picked the one that has the r-axis tile "up an to the left", i.e. the one explicitly shown here:
https://docs.bokeh.org/en/latest/docs/user_guide/plotting.html#hex-tiles
Bokeh expects up-and-to-the-left axial coords. You will need to convert whatever coordinate system you have to that. For "squishing" you will need to set match_aspect=True to ensure the "data space" aspect ratio matches the "pixel space" aspect ratio 1-1.
Alternatively, if you don't or can't use auto-ranging you will need to set the plot size carefully and also control the border sizes with min_border_left etc to make sure the borders are always big enough to accommodate any tick labels you have (so that the inner region will not be resized)
I don't really understand this question, but you have absolute control over what ticks visually appear, regardless of the underlying tick data. Besides the built-in formatters, there is FuncTickFormatter that lets you format ticks any way you want with a snippet of JS code. [1] (And you also have control of where ticks are located, if you want that.)
[1] Please note the CoffeeScript and from_py_func options are both deprecated and being removed in then next 2.0 release.
Again, you'll want to use axial_to_cartesian to position anything other then Hex tiles. No other glyphs in Bokeh understand axial coordinates (which is why we provide the conversion function).
You misunderstood what major_tick_in and major_tick_out are for. They are literally how far the ticks visually extend inside and outside the plot frame, in pixels.
Auto-ranging (with DataRange1d) is only computed in the browser, in JavaScript, which is why the start/end are not available on the "Python" side. If you need to know the start/end, you will need to explicitly set the start/end, yourself. Note, however that match_aspect=True only function with DataRange1d. If you explicitly set start/end manually, Bokeh will assume you know what you want, and will honor what you ask for, regardless of what it does to aspect.
Below are my solution and plot. Mainly per #bigreddot's advice, but there's still some coordinate hacking needed:
Expecting users to pass input coords as axial instead of offset coords is a major limitation. I work around this. There's no point in creating a offset_to_cartesian() because we need to negate r in two out of three places:
My input is even-r offset coords. I still need to manually apply the offset: q = q + (r+1)//2
I need to manually negate r in both the axial_to_cartesian() call and the datasource creation for the glyph. (But not in the text() call).
The call needs to be: axial_to_cartesian(q, -r, size=2/3, orientation='pointytop')
Need p = figure(match_aspect=True ...) to prevent squishing
I need to manually create my x,y axes to get the range right
Solution:
import pandas as pd
import numpy as np
from math import sqrt
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource, Range1d
from bokeh.models.glyphs import HexTile
from bokeh.io import curdoc, show
from bokeh.util.hex import cartesian_to_axial, axial_to_cartesian
counties = np.array([
['DL','DY','AM','', ''],
['FM','TE','AH','DN', ''],
['SO','LM','CN','MN', ''],
['MO','RN','LD','MH','LH'],
['GA','OY','WH','D' ,'' ],
['' ,'CE','LS','KE','WW'],
['LC','TA','KK','CW','' ],
['KY','CR','WF','WX','' ]
])
counties = np.flip(counties, (0)) # Flip UD for bokeh
# (q,r) Coordinate system is “odd/even-r” horizontal Offset coords
r, q = np.mgrid[0:8, 0:(4+1)]
q = q[counties!='']
r = r[counties!='']
# Transform for odd-r offset coords; +r-axis goes up
q = q + (r+1)//2
#r = -r # cannot globally negate 'r', see comments
# Transform odd-r offset coords (q,r) -> (x,y)
x, y = axial_to_cartesian(q, -r, size=2/3, orientation='pointytop')
counties_df = pd.DataFrame({'q': q, 'r': -r, 'abbrev': counties[counties!=''], 'x': x, 'y': y })
counties_ds = ColumnDataSource(ColumnDataSource.from_df(counties_df)) # ({'q': q, 'r': r, 'abbrev': counties[counties != '']})
p = figure(match_aspect=True, tools='save,crosshair')
glyph = HexTile(orientation='pointytop', q='q', r='r', size=2/3, fill_color='#f6f699', line_color='black') # q,r,size,scale=??!?!!?
p.add_glyph(counties_ds, glyph)
p.x_range = Range1d(-2,6)
p.y_range = Range1d(-1,8)
p.xaxis.minor_tick_line_color = None
p.yaxis.minor_tick_line_color = None
p.text(x, y, text=["(%d, %d)" % (q,r) for (q, r) in zip(q, r)],
text_baseline="middle", text_align="center")
show(p)

Index error due to the for-loop in python

As I am new to python programming. I have a problem in the for loop with index error. I have gone through the suggestions that you have given me. My problem is that in the for loop...
I didn't get any error with this code below...
for i in range(0,1):
But I have obtained an error if the limit exceeds for example (0,3)
for i in range(0,3):
The error is
IndexError: index 1 is out of bounds for axis 0 with size 1
I have tried to clear out this error and I am not sure that why this error occurs in the for loop if the limits exceed 1.
This is my code:
m=['paketone4000.dump.xlsx','paketone8000.dump.xlsx','paketone12000.dump.xlsx']
fig_name=['j4000','e8000','e12000']
fig=plt.figure(figsize=(6,6)) ##to obtain figure and dimensions of graph
for i in range(0,3):
#ax=fig.add_subplot(111,projection='3d') ## to have a broad view of figure
ax = fig.add_axes([0,0,1,1], projection='3d')
#plot planes
p = Rectangle((0,-0.7), 4.5,1.4, color="lightgrey", alpha=0.2) #plots the background frame
ax.add_patch(p)
art3d.pathpatch_2d_to_3d(p, z=0, zdir="z")
j=pd.read_excel(m[i]) ##to read the excel file format
X=j['x'] ## to import the variable on to axes from data set
Y=j['y']
Z=j['z']
#ax.scatter(X,Y,Z,c='g', marker='o') ## to specify the color and shape of point(marker) of the frame
a=j['x']##import centre of mass from excel file format
b=j['y']
c=j['z']
q1=j['q1'], ##attaining quaternons from excel file format. (comma(,) transformed series to tuple)
q2=j['q2'],
q3=j['q3'],
q4=j['q4'],
m,n,o,p=np.array([q1,q2,q3,q4]) ## assigning quaternions to variables had converted tuple to float
Rot_Mat=QtoR(m,n,o,p)
#cuboid initialising parameters
center = [a[0], b[0], c[0]] ##centre of the body
length = 0.3 ##defining length, breadth, height
width = 0.4
height = 0.1
side = np.zeros((8,3)) ###This numpy vector will be used to store the position of the sides
#rotate the axes and update
for angle in range(0, 360):
ax.view_init(90, angle)
cuboid(center, (length, width, height)) #to execute the defined cuboid
plt.savefig(fig_name[i])
plt.clf()
print("\nq1=",m,"q2=",n,"q3=",o,"q4=",p)
print('\nRotation Matrix=',Rot_Mat)
print ("\nCenter = \n",center)
My expected result is that I want to remove the error that was obtained and I am interested in to know why that error occurred when end limit is greater than one.
You're using the name m for two different variables in your code. At the top of the file you use it to create a list of filenames, which you read in the loop. But later in the loop, you reassign it with this line:
m,n,o,p=np.array([q1,q2,q3,q4])
That causes the error when you try to read later files, as the new m value doesn't contain what the code expects (and may not be the expected size).
You should use two different variable names. This kind of issue suggest that it might be a good idea to use longer, more descriptive variable name, as you are a lot less likely to have this kind of random namespace collision with names like filenames and first_quaternion (or whatever).
I'd also suggest using range(len(m)) so that if you change the size of the list at some future time, you won't need to remember to also change the hard-coded range size.
An image for this code execution. How about you try replacing
for i in range(0, 5):
with
for i in range(len(m)):
EDIT:
Does this work?
m=['paketone4000.dump.xlsx','paketone8000.dump.xlsx','paketone12000.dump.xlsx']
fig_name=['j4000','e8000','e12000']
fig=plt.figure(figsize=(6,6)) ##to obtain figure and dimensions of graph
for index, i in enumerate(m):
#ax=fig.add_subplot(111,projection='3d') ## to have a broad view of figure
ax = fig.add_axes([0,0,1,1], projection='3d')
#plot planes
p = Rectangle((0,-0.7), 4.5,1.4, color="lightgrey", alpha=0.2) #plots the background frame
ax.add_patch(p)
art3d.pathpatch_2d_to_3d(p, z=0, zdir="z")
j=pd.read_excel(i) ##to read the excel file format
X=j['x'] ## to import the variable on to axes from data set
Y=j['y']
Z=j['z']
#ax.scatter(X,Y,Z,c='g', marker='o') ## to specify the color and shape of point(marker) of the frame
a=j['x']##import centre of mass from excel file format
b=j['y']
c=j['z']
q1=j['q1'], ##attaining quaternons from excel file format. (comma(,) transformed series to tuple)
q2=j['q2'],
q3=j['q3'],
q4=j['q4'],
m2,n,o,p=np.array([q1,q2,q3,q4]) ## assigning quaternions to variables had converted tuple to float
Rot_Mat=QtoR(m2,n,o,p)
#cuboid initialising parameters
center = [a[0], b[0], c[0]] ##centre of the body
length = 0.3 ##defining length, breadth, height
width = 0.4
height = 0.1
side = np.zeros((8,3)) ###This numpy vector will be used to store the position of the sides
#rotate the axes and update
for angle in range(0, 360):
ax.view_init(90, angle)
cuboid(center, (length, width, height)) #to execute the defined cuboid
amount_of_files_to_rename=index
new_names = [i*1000 for i in range(4*amount_of_files_to_rename)[::4]]
for i in new_names:
plt.savefig('packetone {}.jpg'.format(i))
#plt.savefig(fig_name[b])
#plt.clf()
print("\nq1=",m2,"q2=",n,"q3=",o,"q4=",p)
print('\nRotation Matrix=',Rot_Mat)
print ("\nCenter = \n",center)

Categories