I am trying to write a module in python which will draw a numpy array of color data (rgb) to screen. At the moment I am currently using a 3 dimensional color array like this:
numpy.ones((10,10,3),dtype=np.float32,order='F') # (for 10x10 pure white tiles)
binding it to a buffer and using a glVertexAttribArray to broadcast the data to an array of tiles (point sprites) (in this case a 10x10 array) and this works fine for a static image.
But I want to be able to change the data in the array and have buffer reflect this change without having to rebuild it from scratch.
Currently I've built the buffer with:
glBufferData(GL_ARRAY_BUFFER, buffer_data.nbytes, buffer_data, GL_DYNAMIC_DRAW)
where buffer_data is the numpy array. What (if anything) could I pass instead (some pointer into memory perhaps?)
If you want to quickly render a rapidly changing numpy array, you might consider taking a look at glumpy. If you do go with a pure pyopengl solution I'd also be curious to see how it works.
Edit: see my answer here for an example of how to use Glumpy to view a constantly updating numpy array
glBufferData is for updating the entire buffer as it will create a new buffer each time.
What you want is either:
glMapBuffer / glUnmapBuffer.
glMapBuffer copies the buffer to client memory and alter the values locally, then push the changes back to the GPU with glUnmapBuffer.
glBufferSubData
This allows you to update small sections of a buffer, instead of the entire thing.
It sounds like you also want some class that automatically picks up these changes.
I cannot confirm if this is a good idea, but you could wrap or extend numpy.array and over-ride the built in method setitem.
Related
I'm interested in migrating from psychtoolbox to shady for my stimulus presentation. I looked through the online docs, but it is not very clear to me how to replicate what I'm currently doing in matlab in shady.
What I do is actually very simple. For each trial,
I load from disk a single image (I do luminance linearization off-line), which contains all the frames I plan to display in that trial (the stimulus is 1000x1000 px, and I present 25 frames, hence the image is 5000x5000px. I only use BW images, so I have a single int8 value per pixel).
I transfer the entire image from the CPU to the GPU
At some point (externally controlled) I copy the first frame to the video buffer and present it
At some other point (externally controlled) I trigger the presentation of the
remaining 24 frames (copying the relevant part of the image to video buffer for each video frame, and then calling flip()).
The external control happens by having another machine communicate with the stimulus presentation code over TCP/IP. After the control PC sends a command to the presentation PC and this is executed, the presentation PC needs to send back an acknowledgement message to the control PC. I need to send three ACK messages, one when the first frame appears on screen, one when the 2nd frame appears on screen, and one when the 25th frame appears on screen (this way the control PC can easily verify if a frame has been dropped).
In matlab I do this by calling the blocking method flip() to present a frame, and when it returns I send the ACK to the control PC.
That's it. How would I do that in shady? Is there an example that I should look at?
The places to look for this information are the docstrings of Shady.Stimulus and Shady.Stimulus.LoadTexture, as well as the included example script animated-textures.py.
Like most things Python, there are multiple ways to do what you want. Here's how I would do it:
w = Shady.World()
s = w.Stimulus( [frame00, frame01, frame02, ...], multipage=True )
where each frameNN is a 1000x1000-pixel numpy array (either floating-point or uint8).
Alternatively you can ask Shady to load directly from disk:
s = w.Stimulus('trial01/*.png', multipage=True)
where directory trial01 contains twenty-five 1000x1000-pixel image files, named (say) 00.png through 24.png so that they get sorted correctly. Or you could supply an explicit list of filenames.
Either way, whether you loaded from memory or from disk, the frames are all transferred to the graphics card in that call. You can then (time-critically) switch between them with:
s.page = 0 # or any number up to 24 in your case
Note that, due to our use of the multipage option, we're using the "page" animation mechanism (create one OpenGL texture per frame) instead of the default "frame" mechanism (create one 1000x25000 OpenGL texture) because the latter would exceed the maximum allowable dimensions for a single texture on many graphics cards. The distinction between these mechanisms is discussed in the docstring for the Shady.Stimulus class as well as in the aforementioned interactive demo:
python -m Shady demo animated-textures
To prepare the next trial, you might use .LoadPages() (new in Shady version 1.8.7). This loops through the existing "pages" loading new textures into the previously-used graphics-card texture buffers, and adds further pages as necessary:
s.LoadPages('trial02/*.png')
Now, you mention that your established workflow is to concatenate the frames as a single 5000x5000-pixel image. My solutions above assume that you have done the work of cutting it up again into 1000x1000-pixel frames, presumably using numpy calls (sounds like you might be doing the equivalent in Matlab at the moment). If you're going to keep saving as 5000x5000, the best way of staying in control of things might indeed be to maintain your own code for cutting it up. But it's worth mentioning that you could take the entirely different strategy of transferring it all in one go:
s = w.Stimulus('trial01_5000x5000.png', size=1000)
This loads the entire pre-prepared 5000x5000 image from disk (or again from memory, if you want to pass a 5000x5000 numpy array instead of a filename) into a single texture in the graphics card's memory. However, because of the size specification, the Stimulus will only show the lower-left 1000x1000-pixel portion of the array. You can then switch "frames" by shifting the carrier relative to the envelope. For example, if you were to say:
s.carrierTranslation = [-1000, -2000]
then you would be looking at the frame located one "column" across and two "rows" up in your 5x5 array.
As a final note, remember that you could take advantage of Shady's on-the-fly gamma-correction and dithering–they're happening anyway unless you explicitly disable them, though of course they have no physical effect if you leave the stimulus .gamma at 1.0 and use integer pixel values. So you could generate your stimuli as separate 1000x1000 arrays, each containing unlinearized floating-point values in the range [0.0,1.0], and let Shady worry about everything beyond that.
Just trying to understand how a piece of Python code works. The code in question modifies the dimensions of a numpy array using cv2.resize. I would like to know the way that opencv modifies the array in order to populate the new dimensions of the array. I'm trying to translate this into C#, so any psuedocode in an explanation would also be appreciated.
print(last_heatmap.shape) --> [32,32,21]
print(type(last_heatmap)) --> numpy.ndarray
last_heatmap = cv2.resize(last_heatmap, (256,256))
print(last_heatmap.shape) --> [256,256,21]
print(type(last_heatmap)) --> numpy.ndarray
I'm looking for a understanding of how this particular function works, as a bonus suggestions about how to replicate this in other languages (for c#, perhaps an 3d-array of floats?). Thanks.
OpenCV’s resize does not actually modify the array, it creates a completely new one. The documentation for the function can be found here.
For each pixel in the destination array it will scale its coordinates to match the size of the source array. The pixel at the mapped coordinate will be looked up in the source array and written to the destination. By default it does a bilinear interpolation, considering averaged sub pixel values.
Image scaling is a common operation. You will not have to reimplement this in C# and can instead rely on libraries.
I have seen an example of loading a TIFF sequence of say a 3D tiff stack or animation. However I cannot figure out how to do the opposite, taking say, a 3D+ numpy array and saving it as a TIFF sequence. Are there are any examples of encoding this? I can read a 2D array using PIL.fromarray. It would be nice if this method had some way of loading a multi-dimensional array but a naive method call will throw an unsupported-type exception.
Presumably if one did write such a sequence they might also want to add some headers to dictate channels, time and so forth. My particular bias is being able to open such images in applications like ImageJ/FIJI or converting from TIFF to other formats. Maybe there are better ways to go about this in the first place.
I know this is very old, but in case anyone comes here looking for the answer, this does seem to have been solved nicely.
e.g.
im.save("filename.tiff", format="TIFF", save_all=True)
https://pillow.readthedocs.io/en/latest/releasenotes/3.4.0.html
I am trying to create image database compatible with cPickle. Initially list is empty. Data of each image in the directory is added as new row to the list. Images are 224x224. Size of images is on average 8KB. After loading around 10000 images my pc hangs. There are no mouse movement. Nothing happens. needs restart. Below is code snippet which does this..
cr=csv.reader(open(csv_file,"rb"))
for row in cr:
print row[0], row[1]
try:
image=Image.open(row[0]+'.jpg').convert('LA')
pixels=[]
pixels=[f[0] for f in list(image.getdata())]
#pix=np.array(image)
dataset.append(pixels)
#dataset.append(pix)
labels.append(row[1])
del image
except:
print("image not found")
I tried reducing size of images to 28X28 and it works. But i don't want to reduce the size of the images. I am using python 64 bit executable. RAM is 4GB. Ubuntu 14.04. I suspect this is happening due to limited stack space, and list is taking more than available stack space. If so, how do i create this huge list? is there any workaround for this issue? My end goal is to create an numpy array with pixel data as its rows. Currently i am converting list into numpy array.. Is there a solution for this problem??
If the data will eventually be numpy array, maybe try using numpy.memmap. It works like "normal" numpy arrays, the difference is that the data is actually stored on the disk in binary. Only the requested chunks of array are put in RAM, thus may get rid of your problem.
If the size of data array is determined, you just need to set up the correct dimension when creating a memmap object. If not, check out
numpy.memmap.resize, and you should be able to create it anyways.
Oh, and there are other solutions such as PyTables.
Good luck!
I am trying to change the gamma value of of a video widget (Phonon.VideoWidget) displayed in a QGraphicsView via a proxy. I noticed the QGraphicsEffect from QtGui works on it via proxy but there are only defaults for blur, single color overlay, and drop shadow. Phonon.VideoWidget itself has options for brightness, contrast, and even hue that work great but no options for gamma correction strangely. QGraphicsEffects are not very fast but they definitely works with the videowidget playing media via graphicsproxy. I decided I would start by creating my own QGraphicsEffect but at the current state it does not seem possible.
I started with something simpler, a single QPixmap (I take the QImage from it. Since QGraphicsEffect seems to work primarily with QPixmaps in its virtual draw function I figure this is a good place to start) and gamma correcting that. It worked. However, it is currently far too slow even with numpy (setPixel for QImage is even slower), taking 5 seconds or more to convert one 1200x900, 400kb jpg image.
The way I do it is I create a list of gamma converted values from 0-255 based on the gamma value entered. Then using numpy, with the QImage from the pixmap, I create an array that points to the image data and finally edit each pixel with it's corresponding value on the table (so no extra calculations are made for each pixel). The meat of the code is as follows
gammaTable = []
for i in xrange(256):
gammaPixel = CorrectGamma( i, gammaValue ) #returns numpy.int8 or int, works properly
gammmaTable.append(gammaPixel)
qimage = myPixmap.toImage()
bytes = qimage.bits() #qimage.constBits()
imageBytes = numpy.asarray(bytes).reshape(qimage.width(), qimage.height(), 4)
#Change each pixel
for x in xrange(qimage.width()):
for y in xrange(qimage.height()):
imageBytes[x,y,0] = gammmaTable[imageBytes[x,y,0]] #gammaDictionary[imageBytes[x,y,0]]
imageBytes[x,y,1] = gammmaTable[imageBytes[x,y,1]] #gammaDictionary[imageBytes[x,y,1]]
imageBytes[x,y,2] = gammmaTable[imageBytes[x,y,2]] #gammaDictionary[imageBytes[x,y,2]]
imageBytes[x,y,3] = gammmaTable[imageBytes[x,y,3]] #gammaDictionary[imageBytes[x,y,3]]
return qimage
I only run it once at the start of the program, and it works, but it is far too slow. I also tried using QImage.scanLine() but I honestly am not sure how to use it and there is no setScanLine() function to work with either. constScanLine returns an uneditable array. Another thing in the defaults I looked into was QImage.colorTable(). However, they are always empty for me so I wasn't able to work with them.
I am considering trying to OpenCV for python but I am not sure if it will suit my needs but I did see a youtube video earlier of some person claiming to use Qt and OpenCV to create what seemed like an overlay (looked to be a noisefilter and nightfilter) over a video stream in a widget, and that is more or less what I need. It did not say anything about it however and could have been C++ Qt (I am working Python and Qt4, via PySide).
I have a good feeling that if I did not do something wrong that maybe Python is too slow for this. Currently the bottleneck seems to be that it takes a long time to iterate and change each value of the numpy imageBytes array that points to the QImage's data array. I don't know how to work with C++ and compiling for PySide/Python and I'm not sure if even if I could that I can even translate this to C++ especially when it comes to the idea of dealing with pointers and sharing between Python and C++.
I also have a feeling I am missing out on a solution somewhere but I have not discovered any other options to think about or try. I was perhaps thinking that maybe I could put some kind of overlay over the graphics items but I realize it makes little sense and there seems to be no such thing (closest is QGraphicsEffect).