I have a function that adds PlotDataItem to a specific plot widget, however, if I try to use the removeItem function on the plot widget, it doesn't really do anything. I was seeking help on how I can make remove item work for this specific scenario? Any other tips you may recommend for optimization, readability, etc. are also greatly appreciated as I am still fairly new to PyQt and even Python itself. Thank you!
This function includes the removeItem() function.
def updateGraph(self):
"""Clears and updates graph to match the toggled checkboxes.
"""
# self.graphWidget.clear()
for checkboxNumber, checkbox in enumerate(
self.scenarioWidget.findChildren(QtWidgets.QCheckBox)
):
if checkbox.isChecked():
peak = self._model.get_peak(checkboxNumber)
duration = self._model.get_duration(checkboxNumber)
self.drawLine(
name=checkbox.objectName(),
peak=peak,
color=2 * checkboxNumber,
duration=duration,
)
else:
self.graphWidget.removeItem(pg.PlotDataItem(name=checkbox.objectName()))
# TODO: Allow for removal of individual pg.PlotDataItems via self.graphWidget.removeItem()
This function is where the PlotDataItems are added to the plot widget.
def drawLine(self, name, peak, color, duration=100.0):
"""Graphs sinusoidal wave off given 'peak' and 'duration' predictions to model epidemic spread.
Arguments:
name {string} -- Name of scenario/curve
peak {float} -- Predicted peak (%) of epidemic.
color {float} -- Color of line to graph.
Keyword Arguments:
duration {float} -- Predicted duration of epidemic (in days). (default: {100.0})
"""
X = np.arange(duration)
y = peak * np.sin((np.pi / duration) * X)
self.graphWidget.addItem(
pg.PlotDataItem(X, y, name=name, pen=pg.mkPen(width=3, color=color))
)
You're creating a new object with pg.PlotDataItem(name=checkbox.objectName()), so it will not be found as it's completely new.
Untested but should work:
for item in self.graphWidget.listDataItems():
if item.name() == checkbox.objectName():
self.graphWidget.removeItem(item)
Related
Goal
I want to plot a large number of cubes (arranged in a 3D grid) with different colors and opacities.
Current State and question
I have come up with a solution using vispy, but the performance is very poor - drawing takes very long and the window is very unresponsive. Also, there seem to be some glitches in the visualization, but I could live with those.
Is there a more efficient/elegant way to implement that? I am open to using other packages (I have tried open3d but found it difficult to specify colors and opacities - the documentation is not very verbose). However, I need to use python.
What I did so far
The first problem I had to solve with vispy was that I was unable to create cubes at custom positions. I therefore wrote a subclass that can do that:
import vispy.visuals
from vispy.geometry import create_box
class PositionedCubeVisual(vispy.visuals.BoxVisual):
def __init__(self, size=1, position=(0, 0, 0), width_segments=1,
height_segments=1, depth_segments=1, planes=None,
vertex_colors=None, face_colors=None,
color=(0.5, 0.5, 1, 1), edge_color=None, **kwargs):
vertices, filled_indices, outline_indices = create_box(
size, size, size, width_segments, height_segments,
depth_segments, planes)
for column, pos in zip(vertices['position'].T, position):
column += pos
self._mesh = vispy.visuals.MeshVisual(vertices['position'], filled_indices,
vertex_colors, face_colors, color)
if edge_color:
self._border = vispy.visuals.MeshVisual(vertices['position'], outline_indices,
color=edge_color, mode='lines')
else:
self._border = vispy.visuals.MeshVisual()
vispy.visuals.CompoundVisual.__init__(self, [self._mesh, self._border], **kwargs)
self.mesh.set_gl_state(polygon_offset_fill=True,
polygon_offset=(1, 1), depth_test=True)
PositionedCube = vispy.scene.visuals.create_visual_node(PositionedCubeVisual)
I then plot the cubes as follows:
import numpy as np
import vispy.scene
def plot_grid_cubes(x, y, z, c=None, size=1, alpha=0.1, edge_color="black",
cmap="viridis", bgcolor="#FFFFFF"):
canvas = vispy.scene.SceneCanvas(keys='interactive', show=True)
view = canvas.central_widget.add_view()
view.bgcolor = bgcolor
view.camera = 'turntable'
c = get_color_array(c, alpha, min(len(x), len(y), len(z)), cmap)
for xx, yy, zz, cc in zip(x, y, z, c):
cube = PositionedCube(size, (xx, yy, zz), color=cc, edge_color=edge_color, parent=view.scene)
canvas.app.run()
def get_color_array(c, alpha, size, cmap):
if c is not None:
cmap = cm.get_cmap(cmap)
if hasattr(c, "__iter__"):
c = np.array(c, copy=True, dtype=float)
c -= c.min()
c *= 255/c.max()
return cmap(c.astype(int), alpha)
else:
color = np.ones((size, 4))
color[:, 3] = alpha
return color
This can then be applied as follows:
plot_grid_cubes([0, 1], [0, 1], [0, 1], c=[0.3, 0.5], alpha=[0.3, 0.8])
The example above works great, but it becomes poor if I plot thousands of cubes.
Regarding performance on vispy, you may want to read this:
Each Visual object in VisPy is an OpenGL Program consisting of at least a vertex shader and a fragment shader (see Modern OpenGL). In general, except for some very specific cases, OpenGL Programs can only be executed one at a time by a single OpenGL context. This means that in your VisPy visualization each Visual object you tell VisPy to draw will extend how long each update (draw) takes. When frames per second (FPS) or responsiveness are a concern, this means each Visual you add reduces the performance of your visualization.
While VisPy is constantly striving to improve performance, there are things that you can do in the mean time (depending on your particular case). The most important change that you can make is to lower the number of Visual objects you have. For a lot of Visuals it is possible to combine them into one by putting a little extra work into the data you provide them. For example, instead of creating 10 separate LineVisuals, create 1 LineVisual that draws 10 lines. While this is a simple example, the same concept applies to other types of Visuals in VisPy and more complex use cases. As a last resort for the most complex cases, a custom Visual (custom shader code) may be necessary. Before writing a custom Visual, check with VisPy maintainers by asking a question on gitter or creating a question as a GitHub issue.
Now for the BoxVisual this is a little difficult because as far as I can tell this "convenience" class doesn't allow you to make many boxes with a single BoxVisual instance. Since you are already comfortable making a Visual subclass I would recommend making the MeshVisuals yourself and providing the vertices for each box as one single position array.
As for not being able to specify position, this won't apply to your custom Visual class that will use the all-in-one array since you'll be providing each position at the beginning, but I thought I should still describe it. It is unfortunate that the BoxVisual is trying to be so convenient that it isn't helpful in this case since other Visuals allow you to pass your vertex positions on creation. In other cases or when you only want to make small modifications, typically what is done in VisPy is to use one or more "transforms" added to the Visual to shift (transform) the positions passed to the Visual. For example:
from vispy.visuals.transforms import STTransform
cube = ... create a cube visual ...
cube.transform = STTransform(scale=(1.0, 1.0, 1.0), translate=(0.0, 0.0, 0.0))
where you change the scale and translate values as needed to effect (X, Y, Z) coordinate values. After this, if you modify the cube.transform.translate = (new_x, new_y, new_z) property (or .scale or use another transform class) directly this has the benefit of only modifying that property on the GPU and not needing to recompute and resend the vertex positions (better performance).
I'm trying to do the following:
Extract the melody of me asking a question (word "Hey?" recorded to
wav) so I get a melody pattern that I can apply to any other
recorded/synthesized speech (basically how F0 changes in time).
Use polynomial interpolation (Lagrange?) so I get a function that describes the melody (approximately of course).
Apply the function to another recorded voice sample. (eg. word "Hey." so it's transformed to a question "Hey?", or transform the end of a sentence to sound like a question [eg. "Is it ok." => "Is it ok?"]). Voila, that's it.
What I have done? Where am I?
Firstly, I have dived into the math that stands behind the fft and signal processing (basics). I want to do it programatically so I decided to use python.
I performed the fft on the entire "Hey?" voice sample and got data in frequency domain (please don't mind y-axis units, I haven't normalized them)
So far so good. Then I decided to divide my signal into chunks so I get more clear frequency information - peaks and so on - this is a blind shot, me trying to grasp the idea of manipulating the frequency and analyzing the audio data. It gets me nowhere however, not in a direction I want, at least.
Now, if I took those peaks, got an interpolated function from them, and applied the function on another voice sample (a part of a voice sample, that is also ffted of course) and performed inversed fft I wouldn't get what I wanted, right?
I would only change the magnitude so it wouldn't affect the melody itself (I think so).
Then I used spec and pyin methods from librosa to extract the real F0-in-time - the melody of asking question "Hey?". And as we would expect, we can clearly see an increase in frequency value:
And a non-question statement looks like this - let's say it's moreless constant.
The same applies to a longer speech sample:
Now, I assume that I have blocks to build my algorithm/process but I still don't know how to assemble them beacause there are some blanks in my understanding of what's going on under the hood.
I consider that I need to find a way to map the F0-in-time curve from the spectrogram to the "pure" FFT data, get an interpolated function from it and then apply the function on another voice sample.
Is there any elegant (inelegant would be ok too) way to do this? I need to be pointed in a right direction beceause I can feel I'm close but I'm basically stuck.
The code that works behind the above charts is taken just from the librosa docs and other stackoverflow questions, it's just a draft/POC so please don't comment on style, if you could :)
fft in chunks:
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import wavfile
import os
file = os.path.join("dir", "hej_n_nat.wav")
fs, signal = wavfile.read(file)
CHUNK = 1024
afft = np.abs(np.fft.fft(signal[0:CHUNK]))
freqs = np.linspace(0, fs, CHUNK)[0:int(fs / 2)]
spectrogram_chunk = freqs / np.amax(freqs * 1.0)
# Plot spectral analysis
plt.plot(freqs[0:250], afft[0:250])
plt.show()
spectrogram:
import librosa.display
import numpy as np
import matplotlib.pyplot as plt
import os
file = os.path.join("/path/to/dir", "hej_n_nat.wav")
y, sr = librosa.load(file, sr=44100)
f0, voiced_flag, voiced_probs = librosa.pyin(y, fmin=librosa.note_to_hz('C2'), fmax=librosa.note_to_hz('C7'))
times = librosa.times_like(f0)
D = librosa.amplitude_to_db(np.abs(librosa.stft(y)), ref=np.max)
fig, ax = plt.subplots()
img = librosa.display.specshow(D, x_axis='time', y_axis='log', ax=ax)
ax.set(title='pYIN fundamental frequency estimation')
fig.colorbar(img, ax=ax, format="%+2.f dB")
ax.plot(times, f0, label='f0', color='cyan', linewidth=2)
ax.legend(loc='upper right')
plt.show()
Hints, questions and comments much appreciated.
The problem was that I didn't know how to modify the fundamental frequency (F0). By modifying it I mean modify F0 and its harmonics, as well.
The spectrograms in question show frequencies at certain points in time with power (dB) of certain frequency point.
Since I know which time bin holds which frequency from the melody (green line below) ...
....I need to compute a function that represents that green line so I can apply it to other speech samples.
So I need to use some interpolation method which takes as parameters the sample F0 function points.
One need to remember that degree of the polynomial should equal to the number of points. The example doesn't have that unfortunately, but the effect is somehow ok as for the prototype.
def _get_bin_nr(val, bins):
the_bin_no = np.nan
for b in range(0, bins.size - 1):
if bins[b] <= val < bins[b + 1]:
the_bin_no = b
elif val > bins[bins.size - 1]:
the_bin_no = bins.size - 1
return the_bin_no
def calculate_pattern_poly_coeff(file_name):
y_source, sr_source = librosa.load(os.path.join(ROOT_DIR, file_name), sr=sr)
f0_source, voiced_flag, voiced_probs = librosa.pyin(y_source, fmin=librosa.note_to_hz('C2'),
fmax=librosa.note_to_hz('C7'), pad_mode='constant',
center=True, frame_length=4096, hop_length=512, sr=sr_source)
all_freq_bins = librosa.core.fft_frequencies(sr=sr, n_fft=n_fft)
f0_freq_bins = list(filter(lambda x: np.isfinite(x), map(lambda val: _get_bin_nr(val, all_freq_bins), f0_source)))
return np.polynomial.polynomial.polyfit(np.arange(0, len(f0_freq_bins), 1), f0_freq_bins, 3)
def calculate_pattern_poly_func(coefficients):
return np.poly1d(coefficients)
Method calculate_pattern_poly_coeff calculates polynomial coefficients.
Using pythons poly1d lib I can compute function which can modify the speech. How to do that?
I just need to move up or down all values vertically at certain point in time.
for instance I want to move all frequencies at time bin 0,75 seconds up 3 times -> it means that frequency will be increased and the melody at that point will sound higher.
Code:
def transform(sentence_audio_sample, mode=None, show_spectrograms=False, frames_from_end_to_transform=12):
# cutting out silence
y_trimmed, idx = librosa.effects.trim(sentence_audio_sample, top_db=60, frame_length=256, hop_length=64)
stft_original = librosa.stft(y_trimmed, hop_length=hop_length, pad_mode='constant', center=True)
stft_original_roll = stft_original.copy()
rolled = stft_original_roll.copy()
source_frames_count = np.shape(stft_original_roll)[1]
sentence_ending_first_frame = source_frames_count - frames_from_end_to_transform
sentence_len = np.shape(stft_original_roll)[1]
for i in range(sentence_ending_first_frame + 1, sentence_len):
if mode == 'question':
by = int(_question_pattern(i) / 500)
elif mode == 'exclamation':
by = int(_exclamation_pattern(i) / 500)
else:
by = 0
rolled = _roll_column(rolled, i, by)
transformed_data = librosa.istft(rolled, hop_length=hop_length, center=True)
def _roll_column(two_d_array, column, shift):
two_d_array[:, column] = np.roll(two_d_array[:, column], shift)
return two_d_array
In this case I am simply rolling up or down frequencies referencing certain time bin.
This needs to be polished as it doesn't take into consideration an actual state of the transformed sample. It just rolls it up/down according to the factor calculated using the polynomial function computer earlier.
You can check full code of my project at github, "audio" package contains pattern calculator and audio transform algorithm described above.
Feel free to ask if something's unclear :)
I would like to set the legend on a self defined, custom position.
My final goal would be to get the settings of an already existing chart and use the same settings for a new chart.
I read in the docs it's possible to set the legend like this:
(http://python-pptx.readthedocs.io/en/latest/api/enum/XlLegendPosition.html#xllegendposition)
from pptx.enum.chart import XL_LEGEND_POSITION
chart.has_legend = True
chart.legend.position = XL_LEGEND_POSITION.CUSTOM
But I get a ValueError:
ValueError: CUSTOM (-4161) not a member of XL_LEGEND_POSITION enumeration
Did I miss anything or how can I set the legend on a custom position?
The .CUSTOM member of the XL_LEGEND_POSITION is a reporting member only (roughly like "read-only"). It is intended as the value of the Legend.position property when the legend has been manually adjusted (dragged and dropped with the mouse using the UI). Unlike the other members of that enumeration, it is not "assignable" and could not by itself of course set the position to where you wanted it.
Custom placement of the legend is not yet supported by the python-pptx API. If you wanted to do it you'd have to manipulate the underlying XML with low-level lxml calls. You'd need to understand the relevant XML schema and semantics to know what to do with that XML to produce the result you were after. This sort of thing is commonly called a "workaround function" in python-pptx and python-docx (they work very similarly being based on the same architecture). A Google search on "python-pptx" OR "python-docx" workaround function will find you some examples used for other purposes that you may find helpful if you decide to take that approach.
I couldn't find a fully formed answer to this, so I thought it would be worth posting the workaround that I used:
from pptx.oxml.xmlchemy import OxmlElement
def SubElement(parent, tagname, **kwargs):
element = OxmlElement(tagname)
element.attrib.update(kwargs)
parent.append(element)
return element
def manuallySetLegendPosition(
chart,
x,
y,
w,
h
):
## Inside layout, add manualLayout
L = chart.legend._element.get_or_add_layout()
mL = L.get_or_add_manualLayout()
## Add xMode and yMode and set vals to edge
xM = SubElement(mL, 'c:xMode', val="edge")
xY = SubElement(mL, 'c:yMode', val="edge")
## Add x, value is between -1 and 1 as a proportion of the chart width
## point of reference on the legend is its centre, not top left
xE = SubElement(mL, 'c:x', val=str(x))
## Add y, same concept as above
yE = SubElement(mL, 'c:y', val=str(y))
## Add w, legend height as a proportion of chart height
wE = SubElement(mL, 'c:w', val=str(w))
## Add h, same concept as above
hE = SubElement(mL, 'c:h', val=str(h))
This is not a duplicate of this or this, as the answer there was not at all satisfactory to my problem, I don't want to deal with this per label. This is also not a duplicate of this as it doesn't deal with my specific problem.
I want to set the angular axis labels of polar plots, not one by one, but by a single time initialization method. This must be possible, as there appear to be ways to similar things with other axes types.
I knew how to do this before hand, but handn't seen the exact same question here and good solutions were also not found here. While I'm not sure if this is the best method, it is certainly better than setting the format per label!
So the solution I've found is using the FunctionFormatter. The definition is short, so i'll just paste it here.
class FuncFormatter(Formatter):
"""
Use a user-defined function for formatting.
The function should take in two inputs (a tick value ``x`` and a
position ``pos``), and return a string containing the corresponding
tick label.
"""
def __init__(self, func):
self.func = func
def __call__(self, x, pos=None):
"""
Return the value of the user defined function.
`x` and `pos` are passed through as-is.
"""
return self.func(x, pos)
This formatter class will allow us to create a function, pass it as an argument, and the output of that function will be the format of our plot angular labels.
You can then use PolarAxis.xaxis.set_major_formatter(formatter) to use your newly create formatter and only the angular axis labels will be changed. The same thing can be done with the yaxis attribute instead, and will cause the inner radial labels to change as well.
Here is what our function looks like that we will pass:
def radian_function(x, pos =None):
# the function formatter sends
rad_x = x/math.pi
return "{}π".format(str(rad_x if rad_x % 1 else int(rad_x)))
it uses standard python formatting strings as an output, getting rid of unnecessary decimals and appending the pi symbol to the end of the string to keep it in terms of pi.
The full program looks like this:
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
import math
def radian_function(x, pos =None):
# the function formatter sends
rad_x = x/math.pi
return "{}π".format(str(rad_x if rad_x % 1 else int(rad_x)))
ax = plt.subplot(111, projection='polar')
ax.set_rmax(4)
ax.set_rticks([1, 2, 3, 4])
ax.grid(True)
ax.set_title("Polar axis label example", va='bottom')
# sets the formatter for the entire set of angular axis labels
ax.xaxis.set_major_formatter(ticker.FuncFormatter(radian_function))
# sets the formatter for the radius inner labels.
#ax.yaxis.set_major_formatter(ticker.FuncFormatter(radian_function))
plt.show()
which outputs
You could further improve the formatter to check for one (so that 1π is simply shown as π) or check for 0 in a similar fashion. You can even use the position variable (which I left out since it was unnecessary) to further improve visual formatting.
such a function might look like this:
def radian_function(x, pos =None):
# the function formatter sends
rad_x = x/math.pi
if rad_x == 0:
return "0"
elif rad_x == 1:
return "π"
return "{}π".format(str(rad_x if rad_x % 1 else int(rad_x)))
I am relatively new to Bokeh and have written a function that allows a user to choose which data to plot using tabs. The function make_plot() below is relatively slow because the dataset being plotted is large, and I have 30 tabs so I would like to only create the plot when a user clicks on a tab (not pre-load all 30 plots). I don't have experience with javascript, is there a way I can do this in Python?
Here is my function:
def plot_all_outputs(sa_dict, min_val=0.01, top=100, stacked=True,
error_bars=True, log_axis=True,
highlighted_parameters=[]):
"""
This function calls make_plot() for all the sensitivity
analysis output files and lets you choose which output to view
using tabs
Parameters:
-----------
sa_dict : a dictionary with all the sensitivity analysis
results
min_val : a float indicating the minimum sensitivity value
to be shown
top : integer indicating the number of parameters to
display (highest sensitivity values)
stacked1 : Boolean indicating in bars should be stacked for
each parameter.
error_bars : Booelan indicating if error bars are shown (True)
or are omitted (False)
log_axis : Boolean indicating if log axis should be used
(True) or if a linear axis should be used (False).
highlighted_parameters : List of strings indicating which parameter wedges
will be highlighted
Returns:
--------
p : a bokeh plot generated with plotting.make_plot() that includes tabs
for all the possible outputs.
"""
tabs_dictionary = {}
outcomes_array = []
for files in sa_dict.keys():
outcomes_array.append(sa_dict[files][0])
for i in range(len(sa_dict)):
p = make_plot(outcomes_array[i],
top=top,
minvalues=min_val,
stacked=stacked,
errorbar=error_bars,
lgaxis=log_axis,
highlight=highlighted_parameters
)
tabs_dictionary[i] = Panel(child=p, title=sa_dict.keys()[i])
tabs = Tabs(tabs=tabs_dictionary.values())
p = show(tabs)
return p
In order to plot on tab selection, you could add your code for plotting to the on_change property of your tabs:
tabs = Tabs(tabs=[tab_01,tab_02])
def tabs_on_change(attr, old, new):
print("the active panel is " + str(tabs.active))
plot_tab_function(tabs.active) #<--your plotting code here
tabs.on_change('active', tabs_on_change)
Here, tabs.active is the index of the selected tab.