We Keep Coding

How to draw a line in VR in HARFANG?

I am using HARFANG for a scientific visualization project in VR, with the Python API. I based my work on the tutorial given here: https://github.com/harfang3d/tutorials-hg2/blob/master/scene_vr.py
But there is one thing I can't do :(
Is it possible to display vertices and lines in the VR view?
To do this in the render pipeline I figured out from the tutorials that the line vid = hg.GetSceneForwardPipelinePassViewId(passId, hg.SFPP_Opaque) would let me get the exact render pass into which I could inject my line draws.
However, I can't get it to work in a VR code. The best I've been able to do so far is to desync the view of the two eyes...

What happens here is that your views are overwritten by the drawing pass of your lines.
You have to figure out how BGFX works: it uses a (very simple) system of views, indexed by IDs going from 0 to 255.
As detailed in the manual :
Each call to a drawing function such as DrawLines or DrawModel is queued as multiple draw commands on the specified view. When Frame is called all views are processed in order, from the lowest to the highest id.
What you need to do is to increment the view id before using it for your specific drawing commands:
vid = 0 # keep track of the next free view id
passId = hg.SceneForwardPipelinePassViewId()
# Prepare view-independent render data once
vid, passId = hg.PrepareSceneForwardPipelineCommonRenderData(vid, scene, render_data, pipeline, res, passId)
vr_eye_rect = hg.IntRect(0, 0, vr_state.width, vr_state.height)
# Prepare the left eye render data then draw to its framebuffer
vid, passId = hg.PrepareSceneForwardPipelineViewDependentRenderData(vid, left, scene, render_data, pipeline, res, passId)
vid, passId = hg.SubmitSceneToForwardPipeline(vid, scene, vr_eye_rect, left, pipeline, render_data, res, vr_left_fb.GetHandle())
# Prepare the right eye render data then draw to its framebuffer
vid, passId = hg.PrepareSceneForwardPipelineViewDependentRenderData(vid, right, scene, render_data, pipeline, res, passId)
vid, passId = hg.SubmitSceneToForwardPipeline(vid, scene, vr_eye_rect, right, pipeline, render_data, res, vr_right_fb.GetHandle())
# Display lines:
hg.SetViewFrameBuffer(vid, vr_left_fb.GetHandle())
hg.SetViewRect(vid, 0, 0, vr_state.width, vr_state.height)
hg.SetViewClear(vid, 0, 0, 1.0, 0)
hg.SetViewTransform(vid, left.view, left.proj)
draw_line(vid, hg.Vec3(-2, 0.5, 0), hg.Vec3(2, 0.5, 0), hg.Color.Red, hg.Color.Blue)
vid += 1
hg.SetViewFrameBuffer(vid, vr_right_fb.GetHandle())
hg.SetViewRect(vid, 0, 0, vr_state.width, vr_state.height)
hg.SetViewClear(vid, 0, 0, 1.0, 0)
hg.SetViewTransform(vid, right.view, right.proj)
draw_line(vid, hg.Vec3(-2, 0.5, 0), hg.Vec3(2, 0.5, 0), hg.Color.Red, hg.Color.Blue)
vid += 1
And from here you can go on with the rest of the code snippet you mentioned
As told in the manual, the lines will be drawn after the scene (thanks to the view id being incremented), so you can either clear the DepthBuffer or keep it so that the lines will go through your objects.
Please also note that you have to increment the view id because you are doing custom rendering operations. Most of the time, the API will do it for you (as PrepareSceneForwardPipelineViewDependentRenderData or SubmitSceneToForwardPipeline for example)

vPython / GlowScript Sensor Position

I'm showing some strange discrepancies between my declared camera position (scene.camera.pos), and the actual camera position. I can't believe this feature is just broken, am I missing something here?
Here's the code, and the output shown below
GlowScript 3.1 VPython
cube = box(pos=vector(0, 0, 0), size=vector(1,1,1), color=color.red, texture=textures.rough)
scene.lights = [distant_light(direction=vector(0.4226, 0, -0.9063),color=color.gray(0.7)),distant_light(direction=vector(0.4226, 0, -0.9063),color=color.gray(0.7))]
scene.background = color.gray(0.8)
scene.camera.pos = vector(3,3,-3)
scene.camera.axis = cube.pos - scene.camera.pos
#scene.forward=cube.pos
#scene.camera.center=cube.pos
#scene.camera.fov = (pi/180)*10
#scene.camera.axis = vector(0, 0, 0)
#scene.up = vector(0,1,0)
while True:
rate(0.5)
scene.append_to_title(scene.camera.pos)
#scene.camera.rotate(angle=0.05, axis=vec(0,0,1), origin=vec(0,10,0))
#scene.capture("woah")

I think I see the problem. There is a conflict between you manipulating the camera and the default scene.autoscale = True. If you set scene.autoscale = False before manipulating the camera, I think you'll find that the program behaves as expected. At the very least, this implies a need for the camera documentation to point out this conflict.

Finally checked back in on this and it's working properly now.

q table with gym (using box observation space)

I'm trying to run a q-learning algorithm with this observation space:
self.observation_space = spaces.Box(low=np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), high=np.array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1]), dtype=np.flo
when im trying to access the q table, like this:
q_value = q_table[state][action]
I'm getting this error:
IndexError: arrays used as indices must be of integer (or boolean) type
So my question is: how am i supposed to access the q table when my observation space is definded using space.box?
If thats needed, this is how the q_table is defined (its a code i took from the internet, trying to adjust it to my project):
num_box = tuple((env.observation_space.high + np.ones(env.observation_space.shape)).astype(int))
q_table = np.zeros(num_box + (env.action_space.n,))

You're not saying of what type q_table is. I will assume it's an numpy array defined as in OpenAI Gym and Python set up for Q-learning:
action_space_size = env.action_space.n
state_space_size = env.observation_space.n
q_table = np.zeros((state_space_size, action_space_size))
You're getting this error because you're not indexing the elements of the numpy array with an integer. Again, I haven't seen your code, but I believe you are trying to get a specific row of the Q table using a tuple.
Regardless, you should not use a Box observation space when using Q-learning, but rather a Discrete one. When using Q-learning, you need to know the number of states in advance, to initialize the Q-table.
Box spaces are for real values, and the number of dimensions of the space does not define the number of states. For example, if you create a Box space like this:
spaces.Box(low=0, high=1, shape=(2, 2), dtype=np.float16)
you won't have 4 states, but potentially infinite states. The parameters low=0 and high=1 indicate the minimum and maximum value of the four variables in the Box space, but there can be may several values between 0 and 1 (0.1, 0.2, etc.). For this reason, you cannot estimate the number of states beforehand.
If you use np.uint8 (or any integer type) as dtype, you could potentially count the number of states, but it would still be a stretch to use Box spaces instead of Discrete spaces. Moreover, even using integer values the following will not work:
num_box = tuple((env.observation_space.high + np.ones(env.observation_space.shape)).astype(int))
q_table = np.zeros(num_box + (env.action_space.n,))

PyQt5 QGridLayout sizing incorrect

I'm having some issues with QGridLayout in pyqt5. I'm trying to make a GUI that has a stack of buttons on one side, a table on the other side, and a plot that occupies the entire bottom of the window. This is the first program I've ever made, so I might have more issues than I know.
I've arranged the buttons within a QTableWidget, and the main QTableWidget contains several fields where users can enter data. I'd like the data entry table to be larger in size than the button table, but resizing it as in this answer doesn't seem to do anything. The button table is larger no matter the columnSpan entry I put in. What am I doing wrong?
Here are the relevant bits of code:
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
self.setGeometry(50, 50, 700, 1000)
self.home()
def home(self):
self.central_widget = QWidget()
self.setCentralWidget(self.central_widget)
self.button_table = QTableWidget(self)
self.layer_add = QPushButton("Add layer", self)
self.plotter = QPushButton("plot transmission", self)
self.layer_table = QTableWidget(self)
self.graphWidget = pg.PlotWidget(self)
self.grid = QGridLayout()
self.grid.setSpacing(10)
self.grid.addWidget(self.button_table, 0, 0, 1, 1)
self.grid.addWidget(self.layer_table, 0, 1, 1, 3)
self.grid.addWidget(self.graphWidget, 1, 0, 1, 4)
self.centralWidget().setLayout(self.grid)
self.show()
I doodled in what I'd ideally like to have happen... here's a picture of what it looks like with the above code
and in red what I'd like to have happen.
Edit: I don't understand why, if I set the QGridLayout columnSpan to be 1 for the table on the left and 3 for the table on the right, the left-hand table is still significantly wider. I am open to either learning how to fix that, understanding how to make the left-hand table auto-shrink to the size of the buttons within it, or an alternative layout suggestion. Thanks for any help!

While you answered your own question, it seems that you changed the behavior by removing the first table (moreover, changing the resize mode of the first column stretch doesn't have much to do with your issue). So I'm answering to your [edited] question, even if it's missing the part in which you added the buttons to the first table.
The main problem was that you were setting a column span too big for the second table:
self.grid.addWidget(self.button_table, 0, 0, 1, 1)
self.grid.addWidget(self.layer_table, 0, 1, 1, 3) # <- 3 columns!
self.grid.addWidget(self.graphWidget, 1, 0, 1, 4)
In the code above, you're telling the layout that the layer_table will have a column span of 3 columns. Even if you are not actually using three columns, by doing this the layout thinks that the second table will (probably) occupy more space than the first.
Normally, a QGridLayout will use the columnStretch property for that, but since by default the stretch is 0 for all columns and rows, it will use the span as a reference.
In fact, using the following:
self.grid.addWidget(self.button_table, 0, 0, 1, 1)
self.grid.addWidget(self.layer_table, 0, 1, 1, 1) # <- 1 column!
self.grid.addWidget(self.graphWidget, 1, 0, 1, 2)
is the same as this:
self.grid.addWidget(self.button_table, 0, 0, 1, 1)
self.grid.addWidget(self.layer_table, 0, 1, 1, 3) # <- 3 columns!
self.grid.addWidget(self.graphWidget, 1, 0, 1, 4)
self.grid.setColumnStretch(0, 1)
self.grid.setColumnStretch(1, 1)
In the first case, the column span is 1 (one widget, one column), and, since the two widgets are of the same type, they will use half of the available horizontal space. In the second, the column span of the right table is 3 (as in your code), but the stretch is 1 for both the first and second column, and 0 for the third and fourth, meaning that a widget that occupies the second, third and fourth column will have the same available space than a widget that occupies the first, thus obtaining the horizontal space equally divided between those two widgets.
col1 | col2 | col3 | col4
1 | 1 | 0 | 0
Since the second table occupies columns 2 to 4, it will have a stretch of 1 (1 + 0 + 0). Stretches are used by layouts to equally divide the space between widgets (considering their size hints, their minimum size hints, or their minimum/maximum size whenever they're set): the stretches are summed integer values, and then the layout uses the proportions between the sum and those values to resize widgets.
To ensure that the first table uses only the minimum space required to show its contents, you need to do the following:
set the sizeAdjustPolicy (which is a property of every QAbstractScrollArea descendant, including every item view) to AdjustToContents; this will make the table "tell" the layout that its size hint is based on its minimum contents;
set the resize mode of the horizontal header to adjust all of its sections (as in columns) to their contents;
set the horizontal size policy of the table to Maximum; the term "maximum" might be counterintuitive, but it means that the widget's size cannot be larger than its size hint; still, it could be shrunk if any other widget requires space (but not less than the minimumSizeHint) so, alternatively, you could use Fixed (meaning that it cannot even shrink), but it's usually better to allow widgets to be shrunk anyway if the layout is too crowded and the user requires to make the window smaller than it is;
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
self.setGeometry(50, 50, 700, 1000)
self.home()
def home(self):
self.central_widget = QWidget()
self.setCentralWidget(self.central_widget)
self.button_table = QTableWidget(self)
self.button_table.setRowCount(3)
self.button_table.setColumnCount(1)
# set the sizeHint of the table view (actually, its ancestor class,
# QAbstractScrollArea) to the minimum size required to show its contents
self.button_table.setSizeAdjustPolicy(self.button_table.AdjustToContents)
# set all sections of the horizontal headers to adjust themselves to
# their contents
self.button_table.horizontalHeader().setSectionResizeMode(
QHeaderView.ResizeToContents)
# get the current sizePolicy and set it to Maximum, meaning that it will
# use its sizeHint as "maximum": it can expand, but there's no need for
# that, so if any other sibling widget requires more space, it can use it
policy = self.button_table.sizePolicy()
policy.setHorizontalPolicy(policy.Maximum)
# apply the changed policy
self.button_table.setSizePolicy(policy)
self.layer_add = QPushButton("Add layer", self)
self.plotter = QPushButton("plot transmission", self)
# I restored the following lines, which were missing in your edit
self.button_table.setCellWidget(0, 0, self.layer_add)
self.button_table.setCellWidget(1, 0, self.plotter)
self.layer_table = QTableWidget(self)
self.graphWidget = pg.PlotWidget(self)
self.grid = QGridLayout()
self.grid.setSpacing(10)
self.grid.addWidget(self.button_table, 0, 0, 1, 1)
self.grid.addWidget(self.layer_table, 0, 1, 1, 1)
self.grid.addWidget(self.graphWidget, 1, 0, 1, 2)
self.centralWidget().setLayout(self.grid)
As you can see, now the left table only uses the minimum required width, based on the horizontal header width (plus the vertical header width), which in turn is based on the sum of the maximum width of each column.

Ahh turns out that using setColumnStretch on column 1 fixed this problem. I also changed the left table to a QVBoxLayout and put it in using QGridLayout.addLayout, so everything looks better now
I still don't quite understand why the two tables were unequal widths on QGridLayout though, regardless of the number of columns selected.

Neural Network Data Sparsity

I am using PyBrain to train a network on music. The input is two notes, and the output is the next two notes.
Each note is represented by an integer mapped to a note (E.G C# = 11, F = 7), the octave, and the duration. So I was using a dataset as such:
ds = SupervisedDataSet(6, 6)
Which would look like ([note1, octave1, duration1, note2, octave2, duration2], [note1, octave1, duration1, note2, octave2, duration2])
However, I ran into a problem with chords (I.E more than one note played at once). To solve this, I got rid of the first integer representing a note and replaced it with 22 integers, set to either one or zero, to indicate which notes are being played. I still have this followed by octave and duration.
So for example, the following
[0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 4, 0.5]
represents a chord of C#, E and A, with an octave of 4 and duration of 0.5.
PyBrain always gives me an output of all zeros after training and testing. I understand why it's doing this but I don't know how to fix it.
Is there a better way to represent the notes/chords so that PyBrain won't have this problem?
EDIT: I have since converted the bit vector into a decimal number, and while the network isn't just giving zeros anymore it's still pretty clear it's not learning the patterns correctly.
I am using a network like this:
net = buildNetwork(6, 24, 6, bias=True, hiddenclass=LSTMLayer, recurrent=True)
and a trainer like this:
trainer = BackpropTrainer(net, ds, verbose = True)
when I train I am getting a huge error, something like ten or a hundred thousand.

Your problem is not so clear for me, I think it needs more detailed explanation, but depended what I understood I suppose that you don't need reccurence in your network, also try to use another activation function in hidden layer, for example Softmax. I tested it on some data set of samples with 6 nodes input and 6 - output and it is being trained properly, so I there I suggest you my version:
from pybrain.tools.shortcuts import buildNetwork
from pybrain.datasets import SupervisedDataSet
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.structure.modules import SoftmaxLayer
ds = SupervisedDataSet(6, 6)
#
# fill dataset
#
net = buildNetwork(6, 24, 6, bias=True, hiddenclass=SoftmaxLayer)
trainer = BackpropTrainer(net, ds)
train:
error = 10
while error > 0.00001: #choose error like you want
error = trainer.train()
print error #just for logging
#and activate
print net.activate([*,*,*,*,*,*])