I have found some unintuitive behavior in the interaction between the point property of mark_line and the appearance of the color legend for Altair/Vega-Lite. I ran into this when attempting to create a line with very large and mostly-transparent points in order to increase the area that would trigger the line's tooltip, but was unable to preserve a visible type=gradient legend.
The following code is an MRE for this problem, showing 6 cases: the use of [False, True, and a custom OverlayMarkDef] for the point property and the use of plain and customized color encoding.
import pandas as pd
import altair as alt
# create data
df = pd.DataFrame()
df['x_data'] = [0, 1, 2] * 3
df['y2'] = [0] * 3 + [1] * 3 + [2] * 3
# initialize
base = alt.Chart(df)
markdef = alt.OverlayMarkDef(size=1000, opacity=.001)
color_encode = alt.Color(shorthand='y2', legend=alt.Legend(title='custom legend', type='gradient'))
marks = [False, True, markdef]
encodes = ['y2', color_encode]
plots = []
for i, m in enumerate(marks):
for j, c in enumerate(encodes):
plot = base.mark_line(point=m).\
encode(x='x_data', y='y2', color=c, tooltip=['x_data','y2']).\
properties(title=', '.join([['False', 'True', 'markdef'][i], ['plain encoding', 'custom encoding'][j]]))
plots.append(plot)
combined = alt.vconcat(
alt.hconcat(*plots[:2]).resolve_scale(color='independent'),
alt.hconcat(*plots[2:4]).resolve_scale(color='independent'),
alt.hconcat(*plots[4:]).resolve_scale(color='independent')
).resolve_scale(color='independent')
The resulting plot (the interactive tooltips work as expected):
The color data is the same for each of these plots, and yet the color legend is all over the place. In my real case, the gradient is preferred (the data is quantitative and continuous).
With no point on the mark_line, the legend is correct.
Adding point=True converts the legend to a symbol type - I'm not sure why this is the case since the default legend type is gradient for quantitative data (as seen in the first row) and this is the same data - but can be forced back to gradient by the custom encoding.
Attempting to make a custom point via OverlayMarkDef however renders the forced gradient colorbar invisible - matching the opacity of the OverlayMarkDef. But it is not simply a matter of the legend always inheriting the properties of the point, because the symbol legend does not attempt to reflect the opacity.
I would like to have the normal gradient colorbar available for the custom OverlayMarkDef, but I would also love to build up some intuition for what is going on here.
The transparency issue with the bottom right plot has been fixed since Altair 4.2.0, so now all occasions that include a point on the line changes the legend to 'Ordinal' instead of 'Quantitative'.
I believe the reason the legend is converted to a symbol instead of a gradient, is that your are adding filled points and the fill channel is not set to a quantitative field so it defaults to either ordinal or nominal with a sort:
plot = base.mark_line().encode(
x='x_data',
y='y2',
color='y2',
)
plot + plot.mark_circle(opacity=1)
mark_point gives a gradient legend since it has not fill, and if we set the fill for mark_circle explicitly we also get a gradient legend (one for fill and one for color.
plot = base.mark_line().encode(
x='x_data',
y='y2',
color='y2',
fill='y2'
)
plot + plot.mark_circle(opacity=1)
I agree with you that this is a bit unexpected and it would be more convenient if the encoding type of point=True was set to the same as that used for the lines. You might suggest this as an enhancement in VegaLite together with reporting the apparent bug that you can't override the legend type via type='gradient'.
I am trying to create a 2D histogram with square bins (instead of hex elements) and I'm using quad to do so.
First, I create a 2D histogram using numpy (not shown for brevity).
Then I plot it using the quad glyph:
def plotHist2D(self,name):
"""
Creates a 2D histogram figure in Bokeh.
Parameters
----------
name : string
The name of one of the column data sources in self.files.
Returns
-------
A bokeh plot object.
"""
cds = ColumnDataSource(self.makeHist2D(name))
plot_cmap = linear_cmap('tops','Turbo256',0,1,)
bar_cmap = LinearColorMapper(palette='Turbo256',low=0,high=1)
p = figure(plot_height=350,plot_width=350,
title='kW/mm^2', x_axis_label='x (m)',
y_axis_label='y (m)',
x_range=(-0.05,0.05),
y_range=(-0.05,0.05),
output_backend="webgl",
toolbar_location="above")
p.quad(top='yrights',bottom='ylefts',left='xlefts',right='xrights',
fill_color=plot_cmap,line_color=plot_cmap,
line_width=0.1,line_alpha=1.0,
alpha=1.0,source=cds)
color_bar = ColorBar(color_mapper=bar_cmap,width=8,
border_line_color=None,location=(0,0))
p.add_layout(color_bar,'right')
return p
This functions, in that it creates a figure, but the figure has unpleasant lines at the border between the quad elements.
I cannot find any combination of settings for the parameters line_width, line_color, and line_alpha that will get rid of these lines. Is there any combination of settings that will eliminate those lines entirely?
If not, how should I be creating this 2D histogram with square bins?
The trick to this is to use line_color with the same color mapper and not set line_width too small.
The following call to quad works.
p.quad(top='yrights',bottom='ylefts',left='xlefts',right='xrights',
fill_color=plot_cmap,
line_color=plot_cmap,
line_width=1.0,line_alpha=1.0,
alpha=1.0,source=cds)
The desired plot:
I thought I had tried every permutation before asking this question. I'm not sure why a smaller line_width makes the lines more visible. As #EugenePakhomov suggests you can also just leave out the line_width entirely. Presumably this is because the default setting is large enough not to cause the "meshing."
Below is Bokeh 1.4.0 code that tries to draw a HexTile map of the input dataframe, with axes, and tries to place labels on each hex.
I've been stuck on this for two days solid, reading bokeh doc, examples and github known issues, SO, Bokeh Discourse and Red Blob Games's superb tutorial on Hexagonal Grids, and trying code. (I'm less interested in raising Bokeh issues for the future, and far more interested in pragmatic workarounds to known limitations to just get my map code working today.) Plot is below, and code at bottom.
Here are the issues, in rough decreasing order of importance (it's impossible to separate the root-cause and tell which causes which, due to the way Bokeh handles glyphs. If I apply one scale factor or coord transform it fixes one set of issues, but breaks another, 'whack-a-mole' effect):
The label placement is obviously wrong, but I can't seem to hack up any variant of either (x,y) coords or (q,r) coords to work. (I tried combinations of figure(..., match_aspect=True)), I tried 1/sqrt(2) scaling the (x,y)-coords, I tried Hextile(... size, scale) params as per redblobgames, e.g. size = 1/sqrt(3) ~ 0.57735).
Bokeh forces the origin to be top left, and y-coords to increase as you go down, however the default axis labels show y or r as being negative. I found I still had to use p.text(q, -r, .... I suppose I have to manually patch the auto-supplied yaxis labels or TickFormatter to be positive.
I use np.mgrid to generate the coord grid, but I still seem to have to assign q-coords right-to-left: np.mgrid[0:8, (4+1):0:-1]. Still no matter what I do, the hexes are flipped L-to-R
(Note: empty '' counties are placeholders to get the desired shape, hence the boolean mask [counties!=''] on grid coords. This works fine and I want to leave it as-is)
The source (q,r) coords for the hexes are integers, and I use 'odd-r' offset coords (not axial or hexagonal coords). No matter what HexTile(..., size, scale) args I use, one or both dimensions in the plot is wrong or squashed. Or whether I include the 1/sqrt(2) factor in coord transform.
My +q-axis is east and my +r-axis should be 120° SSE
Ideally I'd like to have my origin at bottom-left (math plot style, not computer graphics). But Bokeh apparently doesn't support that, I can live without that. However defaulting the y-axis labels to negative, while requiring a mix of positive and negative coords, is confusing. Anyway, how to hack an automatic fix to that with minimum grief? (manual p.yrange = Range1d(?, ?)?)
Bokeh's approach to attaching (hex) glyphs to plots is a hard idiom to use. Ideally I simply want to reference (q,r)-coords everywhere for hexes, labels, axes. I never want to see (x,y)-coords appearing on axes, label coords, tick-marks, etc. but seems Bokeh won't allow you. I guess you have to manually hack the axes and ticks later. Also, the plot<->glyph interface doesn't allow you to expose a (q,r) <-> (x,y) coord transform function, certainly not a bidirectional one.
The default axes don't seem to have any accessors to automatically find their current extent/limits; p.yaxis.start/end are empty unless you specified them. The result from p.yaxis.major_tick_in,p.yaxis.major_tick_out is also wrong, for this plot it gives (2,6) for both x and y, seems to be clipping those to the interior multiples of 2(?). How to automatically get the axes' extent?
My current plot:
My code:
import pandas as pd
import numpy as np
from math import sqrt
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource
from bokeh.models.glyphs import HexTile
from bokeh.io import show
# Data source is a list of county abbreviations, in (q,r) coords...
counties = np.array([
['TE','DY','AM','DN', ''],
['DL','FM','MN','AH', ''],
['SO','LM','CN','LH', ''],
['MO','RN','LD','WH','MH'],
['GA','OY','KE','D', ''],
['', 'CE','LS','WW', ''],
['LC','TA','KK','CW', ''],
['KY','CR','WF','WX', ''],
])
#counties = counties[::-1] # UNUSED: flip so origin is at bottom-left
# (q,r) Coordinate system is “odd/even-r” horizontal Offset coords
r, q = np.mgrid[0:8, (4+1):0:-1]
q = q[counties!='']
r = r[counties!='']
sqrt3 = sqrt(3)
# Try to transform odd-r (q,r) offset coords -> (x,y). Per Red Blob Games' tutorial.
x = q - (r//2) # this may be slightly dubious
y = r
counties_df = pd.DataFrame({'q': q, 'r': r, 'abbrev': counties[counties!=''], 'x': x, 'y': y })
counties_ds = ColumnDataSource(ColumnDataSource.from_df(counties_df)) # ({'q': q, 'r': r, 'abbrev': counties[counties != '']})
p = figure(tools='save,crosshair') # match_aspect=True?
glyph = HexTile(orientation='pointytop', q='x', r='y', size=0.76, fill_color='#f6f699', line_color='black') # q,r,size,scale=??!?!!? size=0.76 is an empirical hack.
p.add_glyph(counties_ds, glyph)
p.xaxis.minor_tick_line_color = None
p.yaxis.minor_tick_line_color = None
print(f'Axes: x={p.xaxis.major_tick_in}:{p.xaxis.major_tick_out} y={p.yaxis.major_tick_in}:{p.yaxis.major_tick_out}')
# Now can't manage to get the right coords for text labels
p.text(q, -r, text=["(%d, %d)" % (q,r) for (q, r) in zip(q, r)], text_baseline="middle", text_align="center")
# Ideally I ultimately want to fix this and plot `abbrev` column as the text label
show(p)
There is an axial_to_cartesian function that will just compute the hex centers for you. You can then attach the labels in a variety of orientations and anchoring from these.
Bokeh does not force the origin to be anywhere. There is one axial to cartesian mapping Bokeh uses, exactly what is given by axial_to_cartesian. The position of the Hex tiles (and hence the cartesian coordinates that the axes display) follows from this. If you want different ticks, Bokeh affords lots of control points over both tick location and tick labelling.
There is more than one convention for Axial coords. Bokeh picked the one that has the r-axis tile "up an to the left", i.e. the one explicitly shown here:
https://docs.bokeh.org/en/latest/docs/user_guide/plotting.html#hex-tiles
Bokeh expects up-and-to-the-left axial coords. You will need to convert whatever coordinate system you have to that. For "squishing" you will need to set match_aspect=True to ensure the "data space" aspect ratio matches the "pixel space" aspect ratio 1-1.
Alternatively, if you don't or can't use auto-ranging you will need to set the plot size carefully and also control the border sizes with min_border_left etc to make sure the borders are always big enough to accommodate any tick labels you have (so that the inner region will not be resized)
I don't really understand this question, but you have absolute control over what ticks visually appear, regardless of the underlying tick data. Besides the built-in formatters, there is FuncTickFormatter that lets you format ticks any way you want with a snippet of JS code. [1] (And you also have control of where ticks are located, if you want that.)
[1] Please note the CoffeeScript and from_py_func options are both deprecated and being removed in then next 2.0 release.
Again, you'll want to use axial_to_cartesian to position anything other then Hex tiles. No other glyphs in Bokeh understand axial coordinates (which is why we provide the conversion function).
You misunderstood what major_tick_in and major_tick_out are for. They are literally how far the ticks visually extend inside and outside the plot frame, in pixels.
Auto-ranging (with DataRange1d) is only computed in the browser, in JavaScript, which is why the start/end are not available on the "Python" side. If you need to know the start/end, you will need to explicitly set the start/end, yourself. Note, however that match_aspect=True only function with DataRange1d. If you explicitly set start/end manually, Bokeh will assume you know what you want, and will honor what you ask for, regardless of what it does to aspect.
Below are my solution and plot. Mainly per #bigreddot's advice, but there's still some coordinate hacking needed:
Expecting users to pass input coords as axial instead of offset coords is a major limitation. I work around this. There's no point in creating a offset_to_cartesian() because we need to negate r in two out of three places:
My input is even-r offset coords. I still need to manually apply the offset: q = q + (r+1)//2
I need to manually negate r in both the axial_to_cartesian() call and the datasource creation for the glyph. (But not in the text() call).
The call needs to be: axial_to_cartesian(q, -r, size=2/3, orientation='pointytop')
Need p = figure(match_aspect=True ...) to prevent squishing
I need to manually create my x,y axes to get the range right
Solution:
import pandas as pd
import numpy as np
from math import sqrt
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource, Range1d
from bokeh.models.glyphs import HexTile
from bokeh.io import curdoc, show
from bokeh.util.hex import cartesian_to_axial, axial_to_cartesian
counties = np.array([
['DL','DY','AM','', ''],
['FM','TE','AH','DN', ''],
['SO','LM','CN','MN', ''],
['MO','RN','LD','MH','LH'],
['GA','OY','WH','D' ,'' ],
['' ,'CE','LS','KE','WW'],
['LC','TA','KK','CW','' ],
['KY','CR','WF','WX','' ]
])
counties = np.flip(counties, (0)) # Flip UD for bokeh
# (q,r) Coordinate system is “odd/even-r” horizontal Offset coords
r, q = np.mgrid[0:8, 0:(4+1)]
q = q[counties!='']
r = r[counties!='']
# Transform for odd-r offset coords; +r-axis goes up
q = q + (r+1)//2
#r = -r # cannot globally negate 'r', see comments
# Transform odd-r offset coords (q,r) -> (x,y)
x, y = axial_to_cartesian(q, -r, size=2/3, orientation='pointytop')
counties_df = pd.DataFrame({'q': q, 'r': -r, 'abbrev': counties[counties!=''], 'x': x, 'y': y })
counties_ds = ColumnDataSource(ColumnDataSource.from_df(counties_df)) # ({'q': q, 'r': r, 'abbrev': counties[counties != '']})
p = figure(match_aspect=True, tools='save,crosshair')
glyph = HexTile(orientation='pointytop', q='q', r='r', size=2/3, fill_color='#f6f699', line_color='black') # q,r,size,scale=??!?!!?
p.add_glyph(counties_ds, glyph)
p.x_range = Range1d(-2,6)
p.y_range = Range1d(-1,8)
p.xaxis.minor_tick_line_color = None
p.yaxis.minor_tick_line_color = None
p.text(x, y, text=["(%d, %d)" % (q,r) for (q, r) in zip(q, r)],
text_baseline="middle", text_align="center")
show(p)
I am trying to open a full-color image from a '.fits' file. But, when comparing it to the corresponding '.gif' image, there appears to be an error regarding its color and size.
How can I view the real-color image in its proper dimensions?
As an example, one can select the '.fits' file and corresponding '.gif' file located at the top of this webpage. My example code, which uses the APLPY module, is below.
def from_fits_to_image(color_scheme, rloc, rname='synop_Ml_0.2104', rext='.fits', cmap=None):
"""
color_scheme : 'rgb', 'grayscale', or 'false color'; color scheme of image to be shown
rloc : type <str>; location of file to be read
rname : type <str>; name of file to be read
rext : type <str>; extension of file to be read
cmap : None or type <str>; colormap
"""
rpath = rloc + rname + rext
if color_scheme == 'rgb':
pic = aplpy.FITSFigure(rpath)
# pic.show_rgb(alt_filename) # what filename is supposed to go here?
else:
pic = aplpy.FITSFigure(rpath)
if color_scheme == 'grayscale':
pic.show_grayscale()
elif color_scheme == 'false color':
if cmap is None:
pic.show_colorscale()
else:
pic.show_colorscale(cmap=cmap)
# plt.savefig(...)
plt.show()
So long as one provides the proper rloc (the location of the downloaded '.fits' file) and color_scheme, the above code will run.
Calling the function below will show an empty plot of the proper dimensions. To make it non-empty, I must supply another existing filename, though I am unclear on what exactly that should be.
from_fits_to_image(color_scheme='rgb', rloc=rloc)
Each of the function calls below reveal a plot that has been resized to small. While color_scheme='grayscale' seems to colorize the plot properly, the other methods do not properly colorize the image.
from_fits_to_image('grayscale', rloc=rloc)
from_fits_to_image('false color', rloc=rloc)
from_fits_to_image('false color', rloc=rloc, cmap='plasma')
For comparison, the '.gif' image is below. Ideally, the output will look exactly like the image below.
EDIT:
I have tried using astropy, PIL, and pyfits unsuccessfully. Any help would be appreciated.
EDIT 2:
Below is the result using fits from astropy.io.
from astropy.io import fits
def reada(rloc, rname='synop_Ml_0.1998', rext='.fits'):
""" """
rpath = rloc + rname + rext
# hdu_list = fits.open(rpath)
# hdu_list.info()
pic = fits.getdata(rpath)
plt.imshow(pic)
plt.show()
reada(rloc=rloc)
I've played with the vmin and vmax kwargs, but to no success. Also, using pyfits to open the file results in the following error, even when using pyfits.open(rpath, uint=True, do_not_scale_image_data=True):
TypeError: Image data can not convert to float
Judging from the gif, it seems that the image is a false-color one, and it's a matter of choosing a right color map. I can't say if there's an equivalent color map in python to the one you've linked to, but we can find something that is pretty close, recreating all the features in the image:
fig = aplpy.FITSFigure('synop_Ml_0.2104.fits')
fig.show_colorscale(vmin=-60, vmax=60, cmap='hot')
which shows the following (note that aplpy does not understand this coordinate system so it plots the figure in pixel coordinates):
The second part of the question is more tricky, and I can't fully answer it. Firstly, you need to convert the Carrington time into longitude, and sine latitude into degrees, and then plot the new values for the axis labels either instead of the old ones or as a parasite axes alongside the old ones (you can refer to an parasite axes example here).
Now it looks like the Carrington time is just degrees rotated since November 9, 1853, and the x-axis spans exactly 360, so I assume the conversion is simply an offset by 757079.95, the the x-axis value at the left edge. We can double-check it on the world coordinates, by looking at how the pixel span of the map corresponds to the coordinate span:
In [88]: fig._wcs.wcs_pix2world(fig._ax1.get_xlim(), fig._ax1.get_ylim(), origin=1)
Out[88]: [array([757439.95, 757079.95]), array([-1., 1.])]
and the difference in values for the xaxis edges, 757079.95 and 757439.95, is exactly 360 degrees.
So then we can use some matplotlib tricks to manually offset the coordinate values to force them to go from zero to 360 and get the xaxis to match your gif image:
# using hidden attributes is non-pythonic but aplpy does not leave us other options
ax = fig._ax1
x_range = ax.get_xlim()
new_ticklabels = np.arange(60, 361, 60)
new_tick_positions = new_ticklabels / 360. * x_range[1] + x_range[0]
ax.set_xticks(new_tick_positions)
ax.set_xticklabels(new_ticklabels)
fig.axis_labels.set_xtext('Carrington Longitude')
Keep in mind that aplpy is a library designed for plotting celestial, not solar, coordinates, so getting the axes transform to work properly could be a rather painful process. An alternative way, and perhaps a better one, would be to plot the fits file with sunpy, a python library for solar physics. However, I have never used it, and it seems to throw an error for this particular fits file. Looks like you need to modify the header of the fits file to get the coordinates properly read. Perhaps you can get in touch with sunpy community if you want to use the library?