I have a long list of reward signals (-1 for loss, 0 for tie, and +1 for win). I want to average these signals in "windows" and then smooth this resulting curve to show progress. How do I do this with matplotlib/scipy?
My codes like:
#!/usr/bin/env python
import matplotlib
matplotlib.rcParams['backend'] = "Qt4Agg"
import matplotlib.pyplot as plt
import numpy as np
y = np.array([-1, 1, 0, -1, -1, -1, 1, 1, 1, 1, 0, 0, 0, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, -1, 1, 1, 0, 1, 1, 0, 1, -1, -1, 1, -1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, -1, 0, 1, 1, 1, -1, 1, 1, 1, 1, 0, -1, 0, 1, 0, 1, 1, 1, -1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, -1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1]
)
x = np.array(xrange(len(y)))
plt.plot(x,y)
plt.show()
I tried solutions from similar questions, like this, which recommending using a spline, but when applied to my data, that consumes all my memory and crashes my machine.
At some point I found this somewhere. I am having trouble finding the source, but I use it for convolving 1d ndarrays with various windows, and should solve your problem.
def smooth(x,window_len=11,window='hanning'):
if x.ndim != 1:
raise ValueError, "smooth only accepts 1 dimension arrays."
if x.size < window_len:
raise ValueError, "Input vector needs to be bigger than window size."
if window_len<3:
return x
if not window in ['flat', 'hanning', 'hamming', 'bartlett', 'blackman']:
raise ValueError, "Window is on of 'flat', 'hanning', 'hamming', 'bartlett', 'blackman'"
s=numpy.r_[x[window_len-1:0:-1],x,x[-1:-window_len:-1]]
if window == 'flat': #moving average
w=numpy.ones(window_len,'d')
else:
w=eval('numpy.'+window+'(window_len)')
y=numpy.convolve(w/w.sum(),s,mode='valid')
return y
So for example, with your data you'd just do:
plt.plot(smooth(y))
plt.show()
And you get:
The answer you linked recommends using scipy.interpolate.spline which constructs the b-spline representation using full matrices. This is why it consumes this much memory. If smoothing splines is what you're after, at the moment you're better off using scipy.interpolate.UnivariateSpline, it should have saner memory footprint.
If you need some window averages/convolutions, check out numpy.convolve and/or convolution/window functionality in scipy.signal.
Related
I'm programming a code where I use a matrix full of 0's and 1's, the idea is to represent a galaxy, so the 0's are like the void and the 1's will be solar systems (for now), later I intend to add more elements. So, I was wondering if there's a way to plot this elements sorta like a heat map (1 = red and 0 = blue). I'd appreciate any ideas or suggestions if you think there's a better way to pose the problem. Thanks in advance!
Using plt.imshow we can make heat maps you can read more about it here: https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.imshow.html
import numpy as np
import matplotlib.pyplot as plt
matrix = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])
plt.imshow(matrix, cmap='hot', interpolation='nearest')
plt.show()
output:
My go-to is Plotly for any kind of figure
import plotly.express as px
matrix = np.reshape([random.choice([0,1]) for n in range(10000)], (100, 100))
fig = px.imshow(matrix,
color_continuous_scale=['blue','red'])
fig.show()
I have a vector X of size 100x2 and the corresponding binary labels in a vector y ={1, -1} of length 100. I would like to plot the scattered data with s.t. I get the features on the axis and the color of the data point corresponds to a label e.g. red is -1, yellow is 1 for a given data point.
I've been looking into matplotlib and the fcn scatter however it accepts only a single feature vector and its label.
I would be grateful for any help.
You can do this easily using seaborn (or matplotlib as well). Below is the code.
I am creating a random array of size 100x2 and calling it X. I am creating a random array of 0s and 1s of size 100x1 and calling it Y
>> import numpy as np
>> X = np.random.randint(100, size=(100, 2))
>> Y = np.random.choice([0, 1], size=(100))
>> X
array([[11, 47],
[23, 2],
[91, 14],
[65, 32],
[81, 78],
....
>> Y
array([0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1,
0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0,
1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1,
0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1,
1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1])
Use Seaborn scatterplot
import seaborn as sns
sns.scatterplot(x=X[:,0], y=X[:,1], hue=Y)
Output sns scatterplot
I have a 1 dimensional pyTorch tensor (dtype: int32) and was wondering if there was a way to perform a Dirac Delta function on the elements in this tensor, i.e:
f = tensor[1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1]
f_after_dirac_delta = tensor[0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 1]
Thanks for any help in advance!
EDIT: as #GirishDattatrayHegde mentioned, the term Dirac-Delta was misleading. The correct term should have been a Kronecker-Delta. My apologies.
If I understand correctly, you want to compare successive elemeents of your tensor. This should work :
import torch
f = torch.tensor([1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1])
f_dirac = (f[1:] == f[:-1]).to(torch.long)
Suppose I have a time series such as:
[1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0 , 1, 1, 1, 1]
and I know there is some noise in the signal. I want to remove the noise as best I can and still output a binary signal. The above example would turn into something like:
[1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 , 1, 1, 1, 1]
I have implemented a naive rule-based approach where I iterate through the values and have some minimum amount of 1s or 0s I need to "swap" the signal.
It seems like there must be a better way to do it. A lot of the results from googling around give non-binary output. Is there some scipy function I could leverage for this?
There are two similar functions that can help you: scipy.signal.argrelmin and scipy.signal.argrelmax. There are search for local min/max in discrete arrays. You should pass your array and neighbours search radius as order. Your problem can be solved by their combination:
>>> a = np.asarray([1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0 , 1, 1, 1, 1], int)
>>> signal.argrelmin(a, order=3)
(array([4], dtype=int32),)
>>> signal.argrelmax(a, order=3)
(array([15], dtype=int32),)
Then you can just replace these elements.
I'm trying to visualize a list of 2048280 integers which are either 1's or 0's. There is a function that outputs this list from a (width=1515 height=1352) image file. The function
test_results = [(numpy.argmax(SomeFunctionReturningAnArrayForEachGivenPixel))
for y in xrange(1352) for x in range(1532)]
returns an array of size 2058280 (=1515x1352) = as expected. For each y, 1532 values of 1/0 are returned and stored in the array.
Now, when this "test_results" array is returned, I want to save it as an image. So I np.reshape() the array to size (1352,1515,1). All is fine. Logically, I should save this list as a grayscale image. I changed the ndarray data type to 'unit8' and multiplied the pixel values by 127 or 255.
But no matter what I do, the Image.fromarray() function keeps saying that either 'it cannot handle this data type' or 'too many dimensions' or simply gives an error. When I debug it into the Image functions, it looks like the Image library cannot retrieve the array's 'stride'!
All the examples on the net simply reshape the list into an array and save them as an image! Is there anything wrong with my list?
I have already tried various modes ('RGB' , 'L' , '1'). I also changed the data type of my array into uint8, int8, np.uint8(), uint32..
result=self.evaluate(test_data,box) #returns the array
re_array= np.asarray(result,dtype='uint8')
res2 = np.reshape(reray,(1352,1515,1))
res3 =(res2*255)
i = Image.fromarray(res3,'1') ## Raises the exception
i.save('me.png')
For a grayscale image, don't add the trivial third dimension to your array. Leave it as a two-dimensional array: res2 = np.reshape(reray, (1352, 1515)) (assuming reray is the one-dimensional array).
Here's a simple example that worked for me. data is a two-dimensional array with type np.uint8 containing 0s and 1s:
In [29]: data
Out[29]:
array([[0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1],
[0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0],
[1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1],
[1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0],
[1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1],
[1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0],
[0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0],
[1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0],
[1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1],
[0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0]], dtype=uint8)
Create an image from 255*data with mode 'L', and save it as a PNG file:
In [30]: img = Image.fromarray(255*data, mode='L')
In [31]: img.save('foo.png')
When I tried to create the image using mode='1', I wasn't able to get a correct PNG file. Pillow has some known problems with moving between numpy arrays and images with bit depth 1.
Another option is to use numpngw. (I'm the author numpngw.) It allows you to save the data to a PNG file with bit depth 1:
In [40]: import numpngw
In [41]: numpngw.write_png('foo.png', data, bitdepth=1)