We Keep Coding

How can I plot only particular values in xarray?

I am using data from cdasws to plot dynamic spectra. I am following the example found here https://cdaweb.gsfc.nasa.gov/WebServices/REST/jupyter/CdasWsExample.html
This is my code which I have modified to obtain a dynamic spectra for STEREO.
from cdasws import CdasWs
from cdasws.datarepresentation import DataRepresentation
import matplotlib.pyplot as plt
cdas = CdasWs()
import numpy as np
datasets = cdas.get_datasets(observatoryGroup='STEREO')
for index, dataset in enumerate(datasets):
print(dataset['Id'], dataset['Label'])
variables = cdas.get_variables('STEREO_LEVEL2_SWAVES')
for variable_1 in variables:
print(variable_1['Name'], variable_1['LongDescription'])
data = cdas.get_data('STEREO_LEVEL2_SWAVES', ['avg_intens_ahead'],
'2020-07-11T02:00:00Z', '2020-07-11T03:00:00Z',
dataRepresentation = DataRepresentation.XARRAY)[1]
print(data)
plt.figure(figsize = (15,7))
# plt.ylim(100,1000)
plt.xticks(fontsize=18)
plt.yticks(fontsize=18)
plt.yscale('log')
sorted_data.transpose().plot()
plt.xlabel("Time",size=18)
plt.ylabel("Frequency (kHz)",size=18)
plt.show()
Using this code gives a plot that looks something like this,
My question is, is there anyway of plotting this spectrum only for a particular frequency? For example, I want to plot just the intensity values at 636 kHz, is there any way I can do that?
Any help is greatly appreciated, I dont understand xarray, I have never worked with it before.
Edit -
Using the command,
data_stereo.avg_intens_ahead.loc[:,625].plot()
generates a plot that looks like,
While this is useful, what I needed is;
for the dynamic spectrum, if i choose a particular frequency like 600khz, can it display something like this (i have just added white boxes to clarify what i mean) -

If you still want the plot to be 2D, but to include a subset of your data along one of the dimensions, you can provide an array of indices or a slice object. For example:
data_stereo.avg_intens_ahead.sel(
frequency=[625]
).plot()
Or
# include a 10% band on either side
data_stereo.avg_intens_ahead.sel(
frequency=slice(625*0.9, 625*1.1)
).plot()
Alternatively, if you would actually like your plot to show white space outside this selected area, you could mask your data with where:
data_stereo.avg_intens_ahead.where(
data_stereo.frequency==625
).plot()

Plotting a large point cloud using plotly produces a blank graph

Plotting a fairly large point cloud in python using plotly produces a graph with axes (not representative of the data range) and no data points.
The code:
import pandas as pd
import plotly.express as px
import numpy as np
all_res = np.load('fullshelf4_11_2019.npy' )
all_res.shape
(3, 6742382)
np.max(all_res[2])
697.5553566696478
np.min(all_res[2])
-676.311654692491
frm = pd.DataFrame(data=np.transpose(all_res[0:, 0:]),columns=["X", "Y", "Z"])
fig = px.scatter_3d(frm, x='X', y='Y', z='Z')
fig.update_traces(marker=dict(size=4))
fig.update_layout(margin=dict(l=0, r=0, b=0, t=0))
fig.show()
Alternatively you could generate random data and follow the process through
all_res = np.random.rand(3, 6742382)
Which also produces a blank graph with a axis scales that are incorrect.
So -- what am I doing wrong, and is there a better way to plot such a moderately large data set?
Thanks for your help!

Try plotting using ipyvolume.It can handle large point cloud datasets.

It seems like that's too much data for WebGL to handle. I managed to plot 100k points, but 1M points already caused Jupyter to crash. However, a 3D scatterplot of 6.7 million points is of questionable value anyway. You probably won't be able to make any sense out of it (except for data boundaries maybe) and it will be super slow to rotate etc.
I would try to think of alternative approaches, depending on what you want to do. Maybe pick a representative subset of points and plot those.

I would suggest using pythreejs for a point cloud. It has very good performance, even for a large number of points.
import pythreejs as p3
import numpy as np
N = 1_000_000
# Positions centered around the origin
positions = np.random.normal(loc=0.0, scale=100.0, size=(N, 3)).astype('float32')
# Create a buffer geometry with random color for each point
geometry = p3.BufferGeometry(
attributes={'position': p3.BufferAttribute(array=positions),
'color': p3.BufferAttribute(
array=np.random.random((N, 3)).astype('float32'))})
# Create a points material
material = p3.PointsMaterial(vertexColors='VertexColors', size=1)
# Combine the geometry and material into a Points object
points = p3.Points(geometry=geometry, material=material)
# Create the scene and the renderer
view_width = 700
view_height = 500
camera = p3.PerspectiveCamera(position=[800.0, 0, 0], aspect=view_width/view_height)
scene = p3.Scene(children=[points, camera], background="#DDDDDD")
controller = p3.OrbitControls(controlling=camera)
renderer = p3.Renderer(camera=camera, scene=scene, controls=[controller],
width=view_width, height=view_height)
renderer

How to make Plt.plot show my parabolic line in python?

Maybe this will be duplicate question but I couldn't find any solution for this.
Normally what I coded should show me a curved line in python. But with this code I cant see it. Is there a problem with my code or pycharm ? This code only shows me an empty graphic with the correct axes.
And I did adding "ro" in plt.plot(at[i], st, "ro"). This showed me the spots on the graph but what I want to see the complete line.
at = [0,1,2,3,4,5,6]
for i in range(len(at)):
st = at[i]**2
plt.plot(at[i], st)
plt.show()

This is how you would normally do this:
import numpy as np
import matplotlib.pyplot as plt
at = np.array([0,1,2,3,4,5,6])
at2 = at ** 2
plt.plot(at,at2)
plt.show()
you can use something like plt.plot(at,at2, c='red', marker='o') to see the spots.
for detailed explanation please read the documentation.

Maybe rather calculate the to be plotted values entirely before plotting.
at = [0,1,2,3,4,5,6]
y = [xi**2 for xi in at]
plt.plot(at, y)
Or do it alternatively with a function
from math import pow
at = [0,1,2,3,4,5,6]
def parabolic(x):
return [pow(xi,2) for xi in x]
plt.plot(at, parabolic(at))
both return the following plot:

the other answers give fixes for your question, but don't tell you why your code is not working.
the reason for not "seeing anything" is that plt.plot(at[i], st) was trying to draw lines between the points you give it. but because you were only ever giving it single values it didn't have anything to draw lines between. as a result, nothing appeared on the plot
when you changed to call plt.plot(at[i], st, 'ro') you're telling it to draw single circles at points and these don't go between points so would appear
the other answers showed you how to pass multiple values to plot and hence matplotlib could draw lines between these values.
one of your comments says "its not parabolic still" and this is because matplotlib isn't a symbolic plotting library. you just give it numeric values and it draws these onto the output device. sympy is a library for doing symbolic computation and supports plotting, e.g:
from sympy import symbols, plot
x = symbols('x')
plot(x**2, (x, 0, 6))
does the right thing for me. the current release (1.4) doesn't handle discontinuities, but this will be fixed in the next release

How to overplot arrays of different shape?

I'm trying to overplot two arrays with different shapes but I'm unable to project one on the top of the other. For example:
#importing the relevant packages
import numpy as np
import matplotlib.pyplot as plt
def overplot(data1,data2):
'''
This function should make a contour plot
of data2 over the data1 plot.
'''
#creating the figure
fig = plt.figure()
#adding an axe
ax = fig.add_axes([1,1,1,1])
#making the plot for the
#first dataset
ax.imshow(data1)
#overplotting the contours
#for the second dataset
ax.contour(data2, projection = data2,
levels = [0.5,0.7])
#showing the figure
plt.show(fig)
return
if __name__ == '__main__':
'''
testing zone
'''
#creating two mock datasets
data1 = np.random.rand(3,3)
data2 = np.random.rand(9,9)
#using the overplot
overplot(data1,data2)
Currently, my output is something like:
While what I actually would like is to project the contours of the second dataset into the first one. This way, if I got images of the same object but with different resolution for the cameras I would be able to do such plots. How can I do that?
Thanks for your time and attention.

It's generally best to make the data match, and then plot it. This way you have complete control over how things are done.
In the simple example you give, you could use repeat along each axis to expand the 3x3 data to match the 9x9 data. That is, you could use, data1b = np.repeat(np.repeat(data1, 3, axis=1), 3, axis=0) to give:
But for the more interesting case of images, like you mention at the end of your question, then the axes probably won't be integer multiples and you'll be better served by a spline or other type interpolation. This difference is an example of why it's better to have control over this yourself, since there are many ways to to this type of mapping.

Artifacts in a filled contour plot on 3D axes

I have a frustrating problem that only manifests itself when plotting filled contour plots on 3D axes and only in certain situations.
Here is an example of the issue I am experiencing:
and
These are the same data at different contouring intervals.
You'll notice on the left side of the domain there is mis-filling occurring. This is a plot with the Z points squished into the Z=0 plane, via a plotting command like
ax3d.contourf(X, Y, dbz[z25,:,:], zdir='z', offset=0, levels=levels, cmap='pymeteo_radar', alpha=0.50)
The miscontouring happens regardless of alpha level or colormap used, but is sensitive to the number of levels. The use of zdir and offset do not effect the mis-contouring (the artifact just occurs on the Z surface. If I do not fill the contour, there is no mis-contouring. I can also alter the domain to sometimes make the issue better (or worse), but I have many plots to make within the same domain so that is not a fix.
This issue does not occur when the same data is plotted on 2D axes, e.g.:
This plot has some extra data on it, but you can see that the filled contouring does not have the same artifact from mis-filling the contour that occurs on the 3d axes.
Below is a script you can run to reproduce the issue.
#!/usr/bin/env python
import numpy as np
import matplotlib.pyplot as plt
import mpl_toolkits.mplot3d.axes3d as p3
data=np.array([[53.9751, 51.5681, 50.7119, 51.1049, 51.5339, 51.4977, 51.2387,50.761, 50.1732, 49.8218, 49.5442, 48.936, 47.4498, 46.6484, 45.8542, 45.136, 44.5268, 44.071, 43.7665, 43.5928, 43.5269, 43.5385, 43.6053, 45.565, 47.0071, 46.8664, 47.372, 47.8324, 48.295, 48.731, 49.0522, 49.4001, 49.7111, 49.9919, 50.2527, 50.4928, 50.7135, 50.8831, 51.0806, 51.2683 ],
[55.6671, 52.53, 50.7764, 50.5632, 51.2095, 51.5659, 51.521, 51.2143, 50.653, 50.2371, 49.989, 49.8089, 49.6058, 47.8355, 47.3124, 46.7346, 46.1616, 45.6498, 45.2462, 44.967, 44.8005, 44.7284, 44.7295, 44.7869, 46.959, 45.0194, 46.73, 48.0766, 48.9395, 49.5325, 49.8498, 50.1887, 50.4798, 50.7406, 50.9808, 51.2003, 51.4074, 51.555, 51.7429, 51.9218 ],
[56.6513, 53.5919, 51.2774, 50.3133, 50.7705, 51.533, 51.8287, 51.7083, 51.2816, 50.7933, 50.4806, 50.2671, 50.1009, 50.0096, 49.9052, 49.4698, 47.4655, 47.0717, 46.6849, 46.3583, 46.1122, 45.952, 45.8678, 45.8485, 45.8811, 45.956, 46.0634, 47.2225, 49.4363, 50.2482, 50.527, 50.8558, 51.1358, 51.3809, 51.607, 51.8179, 52.0161, 52.1454, 52.3263, 52.497 ],
[57.078, 54.3224, 52.0759, 50.4679, 50.4677, 51.297, 52.0284, 52.1594, 51.9395, 51.5518, 51.1419, 50.8765, 50.6686, 50.5101, 50.4078, 50.3473, 50.3592, 50.3813, 49.7504, 47.55, 47.324, 47.1365, 46.9978, 46.9119, 46.8743, 46.8811, 46.9257, 47.0013, 50.0148, 50.9106, 51.1133, 51.4282, 51.7064, 51.943, 52.1587, 52.3597, 52.4789, 52.6631, 52.8359, 52.9966 ],
[57.3835, 54.9025, 52.8571, 50.9842, 50.5197, 51.1494, 52.0599, 52.4732, 52.4716, 52.2656, 51.9535, 51.6068, 51.3466, 51.1513, 50.9708, 50.8321, 50.7639, 50.7944, 50.8817, 49.8122, 48.2038, 48.086, 47.9704, 47.8735, 47.8035, 47.7644, 47.7574, 47.7803, 50.8194, 51.5486, 51.6645, 51.9745, 52.2349, 52.4508, 52.6481, 52.8317, 52.9412, 53.1097, 53.2699, 53.4171 ],
[57.9157, 55.6092, 53.6306, 51.8011, 50.9372, 51.2615, 52.1406, 52.7436, 52.8528, 52.7829, 52.6322, 52.403, 52.1149, 51.866, 51.6624, 51.4773, 51.317, 51.2183, 51.2153, 51.1367, 48.5913, 48.6216, 48.6218, 48.5951, 48.5589, 48.527, 48.5081, 50.5185, 51.6998, 51.905, 52.2258, 52.4891, 52.7062, 52.8926, 53.0655, 53.2251, 53.3262, 53.4755, 53.6169, 53.7471 ],
[58.6093, 56.432, 54.307, 52.6277, 51.584, 51.6482, 52.3762, 53.0685, 53.2545, 53.217, 53.1356, 53.0351, 52.8481, 52.6154, 52.39, 52.177, 51.9977, 51.843, 51.7172, 51.4587, 48.7481, 48.7984, 48.864, 48.9291, 48.9843, 49.0228, 50.496, 51.8667, 52.3404, 52.4759, 52.6889, 52.8851, 53.0525, 53.2072, 53.354, 53.4576, 53.5925, 53.7217, 53.8432, 53.956 ],
[58.9719, 56.9885, 54.8768, 53.3526, 52.3025, 52.2089, 52.7762, 53.4444, 53.6768, 53.6706, 53.5692, 53.5162, 53.4373, 53.2886, 53.1113, 52.9065, 52.6988, 52.5193, 52.3544, 52.0384, 48.9624, 48.9653, 49.0005, 49.0574, 49.1258, 50.692, 51.9726, 52.4309, 52.699, 52.8194, 52.9845, 53.1336, 53.2669, 53.393, 53.5118, 53.6086, 53.7213, 53.8293, 53.9308, 54.026 ],
[58.5754, 56.945, 55.068, 53.7798, 52.9469, 52.854, 53.3136,53.8929, 54.1205, 54.1178, 54.0128, 53.9289, 53.8906, 53.8239,53.717, 53.5724, 53.3818, 53.1892, 53.009, 49.3078, 49.2524,49.2165, 49.2032, 49.2187, 50.463, 51.9497, 52.4487, 52.7041,52.8358, 52.9776, 53.1101, 53.2293, 53.3419, 53.4487, 53.5401,53.6365, 53.7301, 53.8205, 53.9062, 53.9869 ],
[57.623, 56.547, 55.0117, 54.0512, 53.5372, 53.5246, 53.927,54.3868, 54.5828, 54.5811, 54.4501, 54.3235, 54.2626, 54.2334,54.1802, 54.1137, 53.9897, 53.8202, 49.796, 49.6864, 49.5946,49.5216, 49.4703, 49.4432, 51.8479, 52.5574, 52.8359, 52.9722,53.0827, 53.1826, 53.2747, 53.3597, 53.4405, 53.5138, 53.5944,53.6751, 53.7536, 53.829, 53.9019, 53.9721 ],
[56.902, 56.0005, 54.9159, 54.3352, 54.123, 54.2014, 54.5659,54.8917, 55.0307, 55.0139, 54.8838, 54.7044, 54.5863, 54.5548,54.5258, 54.4957, 54.4633, 51.4821, 50.1897, 50.0758, 49.9683,49.8704, 49.7842, 51.5064, 52.7625, 53.0724, 53.1926, 53.2682,53.3404, 53.4119, 53.4831, 53.5517, 53.6169, 53.6763, 53.7383,53.8009, 53.8644, 53.9281, 53.9905, 54.0517 ],
[56.3455, 55.5524, 54.9336, 54.6836, 54.703, 54.8657, 55.1749,55.3844, 55.4521, 55.4019, 55.2622, 55.0281, 54.8981, 54.6591,54.7866, 54.7678, 54.7654, 54.0436, 54.2302, 52.2533, 50.3305,50.2276, 50.1268, 52.9617, 53.4395, 53.5504, 53.5481, 53.5524,53.5699, 53.6014, 53.644, 53.6931, 53.7445, 53.7996, 53.8548,53.9097, 53.9655, 54.0229, 54.0813, 54.1393 ],
[55.7493, 55.3019, 55.1012, 55.0906, 55.234, 55.4751, 55.7134,55.8462, 55.8461, 55.7425, 55.5725, 55.3535, 55.1612, 54.958,55.0193, 54.9584, 54.9531, 54.8886, 54.8256, 54.2211, 50.6477,50.5564, 53.0546, 53.8592, 54.08, 54.0288, 53.9509, 53.8796,53.8307, 53.8073, 53.8034, 53.8142, 53.8383, 53.8725, 53.9128,53.9558, 54.0013, 54.0497, 54.103, 54.1597 ],
[55.2575, 55.1664, 55.3165, 55.5004, 55.7345, 55.9901, 56.1852,56.2599, 56.2027, 56.0454, 55.818, 55.5754, 55.302, 55.2083,55.0224, 55.1415, 55.0656, 55.0446, 55.0263, 54.7728, 50.8924,53.4671, 54.2587, 54.5146, 54.6171, 54.519, 54.3857, 54.2497,54.1355, 54.0509, 53.9932, 53.9584, 53.941, 53.939, 53.9527,53.9798, 54.0111, 54.0465, 54.0868, 54.1339 ],
[54.8665, 55.1533, 55.5095, 55.8512, 56.1541, 56.3995, 56.5593,56.6009, 56.5079, 56.3001, 56.0178, 55.7187, 55.448, 55.063,55.2016, 55.2116, 55.1817, 55.112, 55.1099, 55.0299, 54.3358,54.6966, 54.9199, 55.0156, 55.0728, 54.975, 54.8299, 54.6609,54.493, 54.3475, 54.2349, 54.1517, 54.0928, 54.0516, 54.0245,54.013, 54.0206, 54.0404, 54.0667, 54.0989 ],
[54.2676, 55.1132, 55.6112, 56.09, 56.428, 56.6661, 56.8056,56.8374, 56.7339, 56.4923, 56.1474, 55.7977, 55.4805, 55.2341,54.8999, 55.2662, 55.2927, 55.185, 55.1237, 55.1268, 54.9772,55.1418, 55.2612, 55.3333, 55.379, 55.3244, 55.2153, 55.0629,54.881, 54.6926, 54.523, 54.3866, 54.2855, 54.2118, 54.1583,54.1191, 54.0935, 54.0834, 54.0885, 54.1057 ],
[54.1771, 55.0795, 55.7075, 56.1772, 56.5183, 56.7522, 56.8898,56.9315, 56.8427, 56.6056, 56.2317, 55.8095, 55.4436, 55.183,55.0284, 54.9504, 55.2833, 55.2563, 55.1498, 55.1342, 55.1331,55.259, 55.3705, 55.4452, 55.4955, 55.5087, 55.4697, 55.3766,55.2324, 55.049, 54.8485, 54.6578, 54.4995, 54.3822, 54.3002,54.2427, 54.2022, 54.1749, 54.1598, 54.1561 ],
[53.9112, 54.85, 55.6641, 56.0844, 56.4062, 56.6232, 56.757,56.8149, 56.7669, 56.5754, 56.2311, 55.785, 55.366, 55.0104,54.812, 54.8845, 55.1273, 55.2339, 55.1976, 55.1049, 55.0913,55.1843, 55.3048, 55.4076, 55.4709, 55.518, 55.5455, 55.5329,55.4636, 55.3349, 55.1595, 54.9529, 54.7462, 54.5681, 54.4342,54.3439, 54.2848, 54.2446, 54.2222, 54.2135 ],
[53.9368, 54.9196, 55.4408, 55.7999, 56.0652, 56.2423, 56.348,56.4106, 56.4114, 56.3028, 56.0519, 55.6779, 55.2493, 54.8836,54.6592, 54.6347, 54.8341, 55.0606, 55.1396, 55.0967, 55.0325,55.0501, 55.1451, 55.2627, 55.3559, 55.4216, 55.4789, 55.5183,55.5245, 55.4779, 55.3701, 55.2072, 55.0029, 54.7876, 54.5915,54.4378, 54.3368, 54.2787, 54.2415, 54.2271 ],
[53.9325, 54.6506, 55.0421, 55.2926, 55.4603, 55.5679, 55.6285,55.6792, 55.7234, 55.731, 55.639, 55.3923, 55.043, 54.6845,54.4188, 54.3242, 54.4606, 54.7449, 54.9548, 55.0171, 55.0047,54.9454, 54.9666, 55.0651, 55.1828, 55.2677, 55.3308, 55.3914,55.438, 55.4544, 55.4277, 55.3385, 55.1907, 54.9981, 54.7786,54.5691, 54.4013, 54.2898, 54.233, 54.1994 ] ])
fig = plt.figure()
ax = fig.add_subplot(111,projection='3d')
X,Y = np.meshgrid(np.arange(-30.0,-20.0,0.25), np.arange(20.0,25,0.25))
ax.contourf(X,Y,data,zdir='z',offset=0, levels=np.arange(0,75,1))
ax.set_zlim(0.0,2.0)
plt.savefig('testfig.png')
plt.close()
This code will produce the plot:
In all of the cases I have observed this mis-contouring the bad triangle always has a vertex near the bottom left of the domain. My data is regularly gridded and for the domain in question is uniform in X and Y. In this case the mis-filling will go away if the number of contour levels is reduced. In some other cases this does not always help or just changes the visual appearance of the error. In any case, even at very coarse contouring I still get errors in a subset of my plots.
Has anyone seen this before and found a fix for it? Am I overlooking something? I'm open to workarounds that don't involve lowering my contouring level (which does reduce the errors overall). If others are in agreement that this could be a bug in the mplot3d, I will file a bug report with them (Issue opened here). I have a feeling the problem lies with contouring very strong gradients when the levels option causes dense contours, but oddly only on 3d axes.
Relevant version information:
Python 3.4.1
matplotlib 1.4.3
numpy 1.9.0

This turned out to be a longstanding bug in matplotlib.mplot3d that ignores path information when taking 2D contourf sets and extending them into 3D. This causes, under certain circumstances, paths with holes to render improperly when a path segment intended as a "move" is instead "drawn".
I contributed a fix for this issue to matplotlib and this bug is fixed in the matplotlib 1.5.0 stable release.
The same test code as in the question produces a correct plot with matplotlib 1.5, as seen below:

The problem is most probably in matplotlib itself and you're not doing anything wrong.
By experimenting a bit I found that if you multiply the input data by 1.01 or 0.999 the plot comes out right, but 1.001 or 0.9999 is not enough to fix the issue.
Adding or subtracting a constant instead shifts the color but keeps the problem evident.
As a wild guess some internal computation falls in a singularity (even if I cannot think what formula would be in danger in this case).
You should submit a bug to their tracker.
EDIT
On a second thought may be matplotlib is trying to compute contour polygons instead of just computing a background texture on a texel-by-texel basis and this could result in annoying accuracy problems that depend on the value. Drawing contour lines is instead much easier because you can just compute the segments in a marching-square approach without worrying about rebuilding the full contour line topology (and for example if a very tiny segment is missing from the line contour plot you're not going to notice anyway).
If this is indeed the bug then may be the fix is not easy because requires a full reimplementation of the plane drawing in a completely different (even if easier) way.