We Keep Coding

Plotting lines in python where a z value corresponds to a specific hex color [duplicate]

I am trying to create a colored line with certain conditions. Basically I would like to have the line colored red when pointing down on the y axis, green when pointing up and blue when neither.
I played around with some similar examples I found but I have never been able to convert them to work with plot() on an axis. Just wondering how this could be done.
Here is some code that I have come up with so far:
#create x,y coordinates
x = numpy.random.choice(10,10)
y = numpy.random.choice(10,10)
#create an array of colors based on direction of line (0=r, 1=g, 2=b)
colors = []
#create an array that is one position away from original
#to determine direction of line
yCopy = list(y[1:])
for y1,y2 in zip(y,yCopy):
if y1 > y2:
colors.append(0)
elif y1 < y2:
colors.append(1)
else:
colors.append(2)
#add tenth spot to array as loop only does nine
colors.append(2)
#create a numpy array of colors
categories = numpy.array(colors)
#create a color map with the three colors
colormap = numpy.array([matplotlib.colors.colorConverter.to_rgb('r'),matplotlib.colors.colorConverter.to_rgb('g'),matplotlib.colors.colorConverter.to_rgb('b')])
#plot line
matplotlib.axes.plot(x,y,color=colormap[categories])
Not sure how to get plot() to accept an array of colors. I always get an error about the format type used as the color. Tried heximal, decimal, string and float. Works perfect with scatter().

I don't think that you can use an array of colors in plot (the documentation says that color can be any matlab color, while the scatter docs say you can use an array).
However, you could fake it by plotting each line separately:
import numpy
from matplotlib import pyplot as plt
x = range(10)
y = numpy.random.choice(10,10)
for x1, x2, y1,y2 in zip(x, x[1:], y, y[1:]):
if y1 > y2:
plt.plot([x1, x2], [y1, y2], 'r')
elif y1 < y2:
plt.plot([x1, x2], [y1, y2], 'g')
else:
plt.plot([x1, x2], [y1, y2], 'b')
plt.show()

OK. So I figured out how to do it using LineCollecion to draw the line on a axis.
import numpy as np
import pylab as pl
from matplotlib import collections as mc
segments = []
colors = np.zeros(shape=(10,4))
x = range(10)
y = np.random.choice(10,10)
i = 0
for x1, x2, y1,y2 in zip(x, x[1:], y, y[1:]):
if y1 > y2:
colors[i] = tuple([1,0,0,1])
elif y1 < y2:
colors[i] = tuple([0,1,0,1])
else:
colors[i] = tuple([0,0,1,1])
segments.append([(x1, y1), (x2, y2)])
i += 1
lc = mc.LineCollection(segments, colors=colors, linewidths=2)
fig, ax = pl.subplots()
ax.add_collection(lc)
ax.autoscale()
ax.margins(0.1)
pl.show()

There is an example on the matplotlib page showing how to use a LineCollection to plot a multicolored line.
The remaining problem is to get the colors for the line collection. So if y are the values to compare,
cm = dict(zip(range(-1,2,1),list("gbr")))
colors = list( map( cm.get , np.sign(np.diff(y)) ))
Complete code:
import numpy as np; np.random.seed(5)
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
x = np.arange(10)
y = np.random.choice(10,10)
points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
cm = dict(zip(range(-1,2,1),list("rbg")))
colors = list( map( cm.get , np.sign(np.diff(y)) ))
lc = LineCollection(segments, colors=colors, linewidths=2)
fig, ax = plt.subplots()
ax.add_collection(lc)
ax.autoscale()
ax.margins(0.1)
plt.show()

Drawing a line from origin to points on the circle

This code plots a particular row of phases given in a NumPy array on a circle of radius 2pi using the matplotlib module.
How can I draw straight lines from the origin(0,0) to these points/phases plotted on the circle?
import numpy as np
import matplotlib.pyplot as plt
def circle(theta):
circle_angle = np.linspace(0, 2*np.pi, 100)
radius = np.sqrt(1)
plt.figure()
fig, ax = plt.subplots(1)
x1 = radius*np.cos(circle_angle )
x2 = radius*np.sin(circle_angle )
plt.plot(x1, x2)
ax.set_aspect(1)
plt.grid(linestyle = '--',axis='both')
plt.plot(radius*np.cos(theta[:1]),radius*np.sin(theta[:1]),'*')
theta = np.array([[1, 2.3, 3,4,4.5], [4.2, 5, 6,3.6,4.3],[2,3,4,3.4,5.6],[0.2,3.4,4.5,6,4]])
print(theta[:,1])
circle(theta)

Looks like you were very close. Instead of plotting just the x,y coordinates of the points on the circle you can plot the lines with plt.plot([x1, x2], [y1, y2]) where x1 and y1 are 0
I've just iterated through the points in the first row of your array and done that.
for i in range(len(theta[0])):
plt.plot([0, radius*np.cos(theta[0][i])], [0, radius*np.sin(theta[0][i])])

2D plot xy-lists of different length

I have an arbitrary, large number (50-1000) of lists, representing X and Y coordinates, I'd like to plot them in one figure.
The lists are of different length, usually 100-1000 elements each. I get the lists as pairs of x and y coordinates (see example), but could easily convert them to 2xN arrays. They need to be plotted in order, from first to the last element. Each line separately.
Is there a way to pack all my lists to one (or two; x and y) object that matplotlib can read?
This example gives the wanted output but is unhandy when there is a lot of data.
I'm happy for a solution that takes advantage of numpy.
from matplotlib import pyplot as plt
fig, ax = plt.subplots(1,1)
x1 = [1,2,5] # usually much longer and a larger number of lists
y1 = [3,2,4]
x2 = [1,6,5,3]
y2 = [7,6,3,2]
x3 = [4]
y3 = [4]
for x, y, in zip([x1, x2, x3],[y1, y2, y3]):
ax.plot(x,y, 'k.-')
plt.show()
I would prefer something like this:
# f() is the function i need, to formats the data for plotting
X = f(x1, x2, x3)
Y = f(y1, y2, y3)
#... so that I can do some processing of the arrays X, and Y here.
ax.plot(X, Y, 'k.-')

You can use a LineCollection for that. Unfortunately, if you want to have markers in your lines, LineCollection does not support that, so you would need to do some trick like adding a scatter plot on top (see Adding line markers when using LineCollection).
from matplotlib import pyplot as plt
from matplotlib.collections import LineCollection
fig, ax = plt.subplots(1,1)
x1 = [1,2,5]
y1 = [3,2,4]
x2 = [1,6,5,3]
y2 = [7,6,3,2]
x3 = [4]
y3 = [4]
# Add lines
X = [x1, x2, x3]
Y = [y1, y2, y3]
lines = LineCollection((list(zip(x, y)) for x, y in zip(X, Y)),
colors='k', linestyles='-')
ax.add_collection(lines)
# Add markers
ax.scatter([x for xs in X for x in xs], [y for ys in Y for y in ys], c='k', marker='.')
# If you do not use the scatter plot you need to manually autoscale,
# as adding the line collection will not do it for you
ax.autoscale()
plt.show()
If you are working with arrays, you may also do as follows:
import numpy as np
# ...
X = [x1, x2, x3]
Y = [y1, y2, y3]
lines = LineCollection((np.stack([x, y], axis=1) for x, y in zip(X, Y)),
colors='k', linestyles='-')
ax.add_collection(lines)
ax.scatter(np.concatenate(X), np.concatenate(Y), c='k', marker='.')

How to make Matplotlib scatterplots transparent as a group?

I'm making some scatterplots using Matplotlib (python 3.4.0, matplotlib 1.4.3, running on Linux Mint 17). It's easy enough to set alpha transparency for each point individually; is there any way to set them as a group, so that two overlapping points from the same group don't change the color?
Example code:
import matplotlib.pyplot as plt
import numpy as np
def points(n=100):
x = np.random.uniform(size=n)
y = np.random.uniform(size=n)
return x, y
x1, y1 = points()
x2, y2 = points()
fig = plt.figure(figsize=(4,4))
ax = fig.add_subplot(111, title="Test scatter")
ax.scatter(x1, y1, s=100, color="blue", alpha=0.5)
ax.scatter(x2, y2, s=100, color="red", alpha=0.5)
fig.savefig("test_scatter.png")
Results in this output:
but I want something more like this one:
I can workaround by saving as SVG and manually grouping then in Inkscape, then setting transparency, but I'd really prefer something I can code. Any suggestions?

Yes, interesting question. You can get this scatterplot with Shapely. Here is the code :
import matplotlib.pyplot as plt
import matplotlib.patches as ptc
import numpy as np
from shapely.geometry import Point
from shapely.ops import cascaded_union
n = 100
size = 0.02
alpha = 0.5
def points():
x = np.random.uniform(size=n)
y = np.random.uniform(size=n)
return x, y
x1, y1 = points()
x2, y2 = points()
polygons1 = [Point(x1[i], y1[i]).buffer(size) for i in range(n)]
polygons2 = [Point(x2[i], y2[i]).buffer(size) for i in range(n)]
polygons1 = cascaded_union(polygons1)
polygons2 = cascaded_union(polygons2)
fig = plt.figure(figsize=(4,4))
ax = fig.add_subplot(111, title="Test scatter")
for polygon1 in polygons1:
polygon1 = ptc.Polygon(np.array(polygon1.exterior), facecolor="red", lw=0, alpha=alpha)
ax.add_patch(polygon1)
for polygon2 in polygons2:
polygon2 = ptc.Polygon(np.array(polygon2.exterior), facecolor="blue", lw=0, alpha=alpha)
ax.add_patch(polygon2)
ax.axis([-0.2, 1.2, -0.2, 1.2])
fig.savefig("test_scatter.png")
and the result is :

Interesting question, I think any use of transparency will result in the stacking effect you want to avoid. You could manually set a transparency type colour to get closer to the results you want,
import matplotlib.pyplot as plt
import numpy as np
def points(n=100):
x = np.random.uniform(size=n)
y = np.random.uniform(size=n)
return x, y
x1, y1 = points()
x2, y2 = points()
fig = plt.figure(figsize=(4,4))
ax = fig.add_subplot(111, title="Test scatter")
alpha = 0.5
ax.scatter(x1, y1, s=100, lw = 0, color=[1., alpha, alpha])
ax.scatter(x2, y2, s=100, lw = 0, color=[alpha, alpha, 1.])
plt.show()
The overlap between the different colours are not included in this way but you get,

This is a terrible, terrible hack, but it works.
You see while Matplotlib plots data points as separate objects that can overlap, it plots the line between them as a single object - even if that line is broken into several pieces by NaNs in the data.
With that in mind, you can do this:
import numpy as np
from matplotlib import pyplot as plt
plt.rcParams['lines.solid_capstyle'] = 'round'
def expand(x, y, gap=1e-4):
add = np.tile([0, gap, np.nan], len(x))
x1 = np.repeat(x, 3) + add
y1 = np.repeat(y, 3) + add
return x1, y1
x1, y1 = points()
x2, y2 = points()
fig = plt.figure(figsize=(4,4))
ax = fig.add_subplot(111, title="Test scatter")
ax.plot(*expand(x1, y1), lw=20, color="blue", alpha=0.5)
ax.plot(*expand(x2, y2), lw=20, color="red", alpha=0.5)
fig.savefig("test_scatter.png")
plt.show()
And each color will overlap with the other color but not with itself.
One caveat is that you have to be careful with the spacing between the two points you use to make each circle. If they're two far apart then the separation will be visible on your plot, but if they're too close together, matplotlib doesn't plot the line at all. That means that the separation needs to be chosen based on the range of your data, and if you plan to make an interactive plot then there's a risk of all the data points suddenly vanishing if you zoom out too much, and stretching if you zoom in too much.
As you can see, I found 1e-5 to be a good separation for data with a range of [0,1].

Just pass an argument saying edgecolors='none' to plt.scatter()

Here's a hack if you have more than just a few points to plot. I had to plot >500000 points, and the shapely solution does not scale well. I also wanted to plot a different shape other than a circle. I opted to instead plot each layer separately with alpha=1 and then read in the resulting image with np.frombuffer (as described here), then add the alpha to the whole image and plot overlays using plt.imshow. Note this solution forfeits access to the original fig object and attributes, so any other modifications to figure should be made before it's drawn.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure
def arr_from_fig(fig):
canvas = FigureCanvas(fig)
canvas.draw()
img = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
img = img.reshape(fig.canvas.get_width_height()[::-1] + (3,))
return img
def points(n=100):
x = np.random.uniform(size=n)
y = np.random.uniform(size=n)
return x, y
x1, y1 = points()
x2, y2 = points()
imgs = list()
figsize = (4, 4)
dpi = 200
for x, y, c in zip([x1, x2], [y1, y2], ['blue', 'red']):
fig = plt.figure(figsize=figsize, dpi=dpi, tight_layout={'pad':0})
ax = fig.add_subplot(111)
ax.scatter(x, y, s=100, color=c, alpha=1)
ax.axis([-0.2, 1.2, -0.2, 1.2])
ax.axis('off')
imgs.append(arr_from_fig(fig))
plt.close()
fig = plt.figure(figsize=figsize)
alpha = 0.5
alpha_scaled = 255*alpha
for img in imgs:
img_alpha = np.where((img == 255).all(-1), 0, alpha_scaled).reshape([*img.shape[:2], 1])
img_show = np.concatenate([img, img_alpha], axis=-1).astype(int)
plt.imshow(img_show, origin='lower')
ticklabels = ['{:03.1f}'.format(i) for i in np.linspace(-0.2, 1.2, 8, dtype=np.float16)]
plt.xticks(ticks=np.linspace(0, dpi*figsize[0], 8), labels=ticklabels)
plt.yticks(ticks=np.linspace(0, dpi*figsize[1], 8), labels=ticklabels);
plt.title('Test scatter');

I encountered the save issue recently, my case is there are too many points very close to each other, like 100 points of alpha 0.3 on top of each other, the alpha of the color in the generated image is almost 1. So instead of setting the alpha value in the cmap or scatter. I save it to a Pillow image and set the alpha channel there. My code:
import io
import os
import numpy as np
import numpy.ma as ma
import matplotlib.pyplot as plt
from matplotlib import colors
from PIL import Image
from dhi_base import DHIBase
class HeatMapPlot(DHIBase):
def __init__(self) -> None:
super().__init__()
# these 4 values are precalculated
top=75
left=95
width=1314
height=924
self.crop_box = (left, top, left+width, top+height)
# alpha 0.5, [0-255]
self.alpha = 128
def get_cmap(self):
v = [
...
]
return colors.LinearSegmentedColormap.from_list(
'water_level', v, 512)
def png3857(self):
"""Generate flooding images
"""
muids = np.load(os.path.join(self.npy_dir, 'myfilename.npy'))
cmap = self.get_cmap()
i = 0
for npyf in os.listdir(self.npy_dir):
if not npyf.startswith('flooding'):
continue
flooding_num = np.load(os.path.join(self.npy_dir, npyf))
image_file = os.path.join(self.img_dir, npyf.replace('npy', 'png'))
# if os.path.isfile(image_file):
# continue
# filter the water level value that is less than 0.001
masked_arr = ma.masked_where(flooding_num > 0.001, flooding_num)
flooding_masked = flooding_num[masked_arr.mask]
muids_masked = muids[masked_arr.mask, :]
plt.figure(figsize=(self.grid2D['numJ'] / 500, self.grid2D['numK'] / 500))
plt.axis('off')
plt.tight_layout()
plt.scatter(muids_masked[:, 0], muids_masked[:, 1], s=0.1, c=flooding_masked,
alpha=1, edgecolors='none', linewidths=0,
cmap=cmap,
vmin=0, vmax=1.5)
img_buf = io.BytesIO()
plt.savefig(img_buf, transparent=True, dpi=200, format='png')#, pad_inches=0)
plt.clf()
plt.close()
img_buf.seek(0)
img = Image.open(img_buf)
# Cropped image of above dimension
# (It will not change original image)
img = img.crop(self.crop_box)
alpha_channel = img.getchannel('A')
# Make all opaque pixels into semi-opaque
alpha_channel = alpha_channel.point(lambda i: self.alpha if i>0 else 0)
img.putalpha(alpha_channel)
img.save(image_file)
self.logger.info("PNG saved to {}".format(image_file))
i += 1
# if i > 15:
# break
if __name__ == "__main__":
hp = HeatMapPlot()
hp.png3857()

Multiple plots on same figure with DataFrame.Plot

While I can get multiple lines on a chart and multiple bars on a chart - I cannot get a line and bar on the same chart using the same PeriodIndex.
Faux code follows ...
# play data
n = 100
x = pd.period_range('2001-01-01', periods=n, freq='M')
y1 = (Series(np.random.randn(n)).diff() + 5).tolist()
y2 = (Series(np.random.randn(n)).diff()).tolist()
df = pd.DataFrame({'bar':y2, 'line':y1}, index=x)
# let's plot
plt.figure()
ax = df['bar'].plot(kind='bar', label='bar')
df['line'].plot(kind='line', ax=ax, label='line')
plt.savefig('fred.png', dpi=200)
plt.close()
Any help will be greatly appreciated ...

The problem is: bar plots don't use index values as x axis, but use range(0, n). You can use twiny() to create a second axes that share yaxis with the bar axes, and draw line curve in this second axes.
The most difficult thing is how to align x-axis ticks. Here we define the align function, which will align ax2.get_xlim()[0] with x1 in ax1 and ax2.get_xlim()[1] with x2 in ax1:
def align_xaxis(ax2, ax1, x1, x2):
"maps xlim of ax2 to x1 and x2 in ax1"
(x1, _), (x2, _) = ax2.transData.inverted().transform(ax1.transData.transform([[x1, 0], [x2, 0]]))
xs, xe = ax2.get_xlim()
k, b = np.polyfit([x1, x2], [xs, xe], 1)
ax2.set_xlim(xs*k+b, xe*k+b)
Here is the full code:
from matplotlib import pyplot as plt
import pandas as pd
from pandas import Series
import numpy as np
n = 50
x = pd.period_range('2001-01-01', periods=n, freq='M')
y1 = (Series(np.random.randn(n)) + 5).tolist()
y2 = (Series(np.random.randn(n))).tolist()
df = pd.DataFrame({'bar':y2, 'line':y1}, index=x)
# let's plot
plt.figure(figsize=(20, 4))
ax1 = df['bar'].plot(kind='bar', label='bar')
ax2 = ax1.twiny()
df['line'].plot(kind='line', label='line', ax=ax2)
ax2.grid(color="red", axis="x")
def align_xaxis(ax2, ax1, x1, x2):
"maps xlim of ax2 to x1 and x2 in ax1"
(x1, _), (x2, _) = ax2.transData.inverted().transform(ax1.transData.transform([[x1, 0], [x2, 0]]))
xs, xe = ax2.get_xlim()
k, b = np.polyfit([x1, x2], [xs, xe], 1)
ax2.set_xlim(xs*k+b, xe*k+b)
align_xaxis(ax2, ax1, 0, n-1)
and the output: