I am new to PsychoPy, having previously worked with Pygame for several months (I switched to enable stimuli to be presented on multiple screens).
I am trying to figure out how to use PsychoPy to display an animation created using a sequence of images. I previously achieved this in Pygame by saving the entire sequence of images in a single large png file (a spritesheet) and then flipping only a fraction of that image (eg. 480 x 480 pixels) per frame, while moving onto the next equally sized section of the image in the next frame. This is roughly what my code looked like in Pygame. I would be really keen to hear if there is an equivalent way of generating animations in PsychoPy by selecting only parts of an image to be displayed with each frame. So far, googling this has not provided any answers!
gameDisplay=pygame.display.set_mode((800, 480))
sequence=pygame.image.load('C:\Users\...\image_sequence.png')
#This image contains 10 images in a row which I cycle through to get an animation
image_width=480
image_height=480
start=time.time()
frame_count=0
refresh=0
while time.time()<=start+15:
gameDisplay.blit(sequence,(160,0),(frame_count*image_width,0,image_width,image_height))
if time.time()>= start+(refresh*0.25): #Flip a new image say every 250 msec
pygame.display.update()
frame_count+=1
refresh+=1
if frame_count ==10:
frame_count=0
You could use a square aperture to restrict what's visible and then move the image. So something like this (untested, but could give you some ideas):
from psychopy import visual
win = visual.Window(units='pix') # easiest to use pixels as unit
aperture = visual.Aperture(win, shape='rect', size=(480, 480))
image = visual.ImageStim('C:\Users\...\image_sequence.png')
# Move through x positions
for x in range(10):
image.pos = [(-10.0/2*+0.5+x)*480, 0] # not sure this is right, but it should move through the x-positions
image.draw()
win.flip()
If you have the original images, I think that it would be simpler to just display the original images in sequence.
import glob
from psychopy import visual
image_names = glob.glob('C:\Users\...\*.png')
# Create psychopy objects
win = visual.Window()
image_stims = [visual.ImageStim(win, image) for image in image_names]
# Display images one by one
for image in image_stims:
image.draw()
win.flip()
# add more flips here if you want a lower frame rate
Perhaps it is even fast enough to load them during runtime without dropping frames, which would simplify the code and load memory less:
# Imports, glob, and win here
# Create an ImageStim and update the image each frame
stim = visual.ImageStim(win)
for name in image_names:
stim.image = name
stim.draw()
win.flip()
Actually, given a spritesheet you might be able to do something funky and more efficient using the GratingStim. This loads an image as a texture and then allows you to set the spatial frequncy (sf) and phase of that texture. If 1.0/sf (in both dimensions) is less than the width of the stimulus (in both dimensions) only a fraction of the texture will be shown and the phase determines which fraction that will be. It isn't designed for this purpose - it's usually used to create more than one cycle of texture not less than one - but I think it will work.
Related
I am trying to create a pipeline in which I first render an image using the blender python API (I am using Blender 2.90) and then perform some image processing in python. I want to fetch the image directly from blender without first writing the rendered image to disk and then loading it again. I ran the following code within the blender GUI to do so:
import bpy
import numpy as np
import PIL.Image as Image
from skimage.util import img_as_ubyte
resolution_x = 512
resolution_y = 512
# render settings
scene = bpy.context.scene
scene.render.engine = 'BLENDER_EEVEE'
scene.render.resolution_x = resolution_x
scene.render.resolution_y = resolution_y
scene.render.image_settings.file_format = 'PNG'
scene.render.filepath = "path/to/good_image.png"
# create Viewer Layer in Compositor
scene.use_nodes = True
tree = scene.node_tree
nodes = tree.nodes
links = tree.links
for node in nodes:
nodes.remove(node)
render_layer_node = nodes.new('CompositorNodeRLayers')
viewer_node = nodes.new('CompositorNodeViewer')
links.new(viewer_node.inputs[0], render_layer_node.outputs[0])
# render scene and get pixels from Viewer Node
bpy.ops.render.render(write_still=True)
pixels = bpy.data.images['Viewer Node'].pixels
# do some processing and save
img = np.flip(img_as_ubyte(np.array(pixels[:]).reshape((resolution_y, resolution_x, 4))), axis=0)
Image.fromarray(img).save("path/to/bad_image.png")
Problem: The image I get from the Viewer Node is much darker (bad image) than the image saved in the conventional way (good image). Does anyone have an idea why this happens and how to fix it? Does blender maybe treat pixel values differently than I expect?
Some additional information:
Before conversion to uint8, the values of the alpha channel within the dark image are 1.0 (as they actually should be). Background values in the dark image are not 0.0 or negative (as one might guess from appearance), but 0.05...
What I tried:
I thought that pixels might be scaled within range -1 to 1, so I rescaled the pixels to range 0 to 1 before transforming to uint8... Did not lead to the correct image either :(
It's because the image that you get from the Viewer Node is the one "straight from compositing" before color management takes place. You can have a look at the documentation here: this image is still in the linear space.
Your good_image.png on the other hand is obtained after transformation into the "Display Space" (see diagram in the doc). Hence it was transformed into a log-space, maybe gamma-corrected, etc.
Finally, you can get an image that is close to (but slightly different though) to the good image from the viewer node by calling bpy.data.images['Viewer Node'].save_render(filepath) instead, but there is no way to directly extract the color-managed version without rendering to a file first. You can probably do it yourself by adding PyOpenColorIO to your script and applying the color management from this module.
I'm looking for a library that enables to "create pictures" (or even videos) with the following functions:
Accepting picture inputs
Resizing said inputs to fit given template / scheme
Positioning the pictures in pre-set up layers or coordinates
A rather schematic approach to look at this:
whereas the red spots are supposed to represent e.g. text, picture (or if possible video) elements.
The end goal would be to give the .py script multiple input pictures and the .py creating a finished version like mentioned above.
Solutions I tried were looking into Python PIL, but I wasn't able to find what I was looking for.
Yes, it is possible to do this with Python.
The library you are looking for is OpenCV([https://opencv.org][1]/).
Some basic OpenCV python tutorials (https://docs.opencv.org/master/d9/df8/tutorial_root.html).
1) You can use imread() function to read images from files.
2) You can use resize() function to resize the images.
3) You can create a empty master numpy array matching the size and depth(color depth) of the black rectangle in the figure you have shown, resize your image and copy the contents into the empty array starting from the position you want.
Below is a sample code which does something close to what you might need, you can modify this to suit your actual needs. (Since your requirements are not clear I have written the code like this so that it can at least guide you.)
import numpy as np
import cv2
import matplotlib.pyplot as plt
# You can store most of these values in another file and load them.
# You can modify this to set the dimensions of the background image.
BG_IMAGE_WIDTH = 100
BG_IMAGE_HEIGHT = 100
BG_IMAGE_COLOR_DEPTH = 3
# This will act as the black bounding box you have shown in your figure.
# You can also load another image instead of creating empty background image.
empty_background_image = np.zeros(
(BG_IMAGE_HEIGHT, BG_IMAGE_WIDTH, BG_IMAGE_COLOR_DEPTH),
dtype=np.int
)
# Loading an image.
# This will be copied later into one of those red boxes you have shown.
IMAGE_PATH = "./image1.jpg"
foreground_image = cv2.imread(IMAGE_PATH)
# Setting the resize target and top left position with respect to bg image.
X_POS = 4
Y_POS = 10
RESIZE_TARGET_WIDTH = 30
RESIZE_TARGET_HEIGHT = 30
# Resizing
foreground_image= cv2.resize(
src=foreground_image,
dsize=(RESIZE_TARGET_WIDTH, RESIZE_TARGET_HEIGHT),
)
# Copying this into background image
empty_background_image[
Y_POS: Y_POS + RESIZE_TARGET_HEIGHT,
X_POS: X_POS + RESIZE_TARGET_WIDTH
] = foreground_image
plt.imshow(empty_background_image)
plt.show()
I'm writing a code that part of it is reading an image source and displaying it on the screen for the user to interact with. I also need the sharpened image data. I use the following to read the data and display it in pyGame
def image_and_sharpen_array(file_name):
#read the image data and return it, with the sharpened image
image = misc.imread(file_name)
blurred = ndimage.gaussian_filter(image,3)
edge = ndimage.gaussian_filter(blurred,1)
alpha = 20
out = blurred + alpha*(blurred - edge)
return image,out
#get image data
scan,sharpen = image_and_sharpen_array('foo.jpg')
w,h,c = scan.shape
#setting up pygame
pygame.init()
screen = pygame.display.set_mode((w,h))
pygame.surfarray.blit_array(screen,scan)
pygame.display.update()
And the image is displayed on the screen only rotated and inverted. Is this due to differences between misc.imread and pyGame? Or is this due to something wrong in my code?
Is there other way to do this? The majority of solution I read involved saving the figure and then reading it with ``pyGame''.
I often use the numpy swapaxes() method:
In this case we only need to invert x and y axis (axis number 0 and 1) before displaying our array :
return image.swapaxes(0,1),out
I thought technico provided a good solution - just a little lean on info. Assuming get_arr() is a function that returns the pixel array:
pixl_arr = get_arr()
pixl_arr = numpy.swapaxes(pixl_arr, 0, 1)
new_surf = pygame.pixelcopy.make_surface(pixl_arr)
screen.blit(new_surf, (dest_x, dest_y))
Alternatively, if you know that the image will always be of the same dimensions (as in iterating through frames of a video or gif file), it would be more efficient to reuse the same surface:
pixl_arr = get_arr()
pixl_arr = numpy.swapaxes(pixl_arr, 0, 1)
pygame.pixelcopy.array_to_surface(old_surf, pixl_arr)
screen.blit(old_surf, (dest_x, dest_y))
YMMV, but so far this is working well for me.
Every lib has its own way of interpreting image arrays. By 'rotated' I suppose you mean transposed. That's the way PyGame shows up numpy arrays. There are many ways to make it look 'correct'. Actually there are many ways even to show up the array, which gives you full control over channel representation and so on. In pygame version 1.9.2, this is the fastest array rendering that I could ever achieve. (Note for earlier version this will not work!).
This function will fill the surface with array:
def put_array(surface, myarr): # put array into surface
bv = surface.get_view("0")
bv.write(myarr.tostring())
If that is not working, use this, should work everywhere:
# put array data into a pygame surface
def put_arr(surface, myarr):
bv = surface.get_buffer()
bv.write(myarr.tostring(), 0)
You probably still get not what you want, so it is transposed or have swapped color channels. The idea is, manage your arrays in that form, which suites this surface buffer. To find out what is correct channel order and axes order, use openCV library (cv2.imread(filename)). With openCV you open images in BGR order as standard, and it has a lot of conversion functions. If I remember correctly, when writing directly to surface buffer, BGR is the correct order for 24 bit and BGRA for a 32 bit surface. So you can try to put the image array which you get out of file with this function and blit to the screen.
There are other ways to draw arrays e.g. here is whole set of helper functions http://www.pygame.org/docs/ref/surfarray.html
But I would not recommend using it, since surfaces are not for direct pixel manipulating, you will probably get lost in references.
Small tip: To do 'signalling test' use a picture, like this. So you will immediately see if something is wrong, just load as array and try to render.
My suggestion is to use the pygame.transform module. There are the flip and rotate methods, which you can use to however your transformation is. Look up the docs on this.
My recommendation is to save the output image to a new Surface, and then apply the transformations, and blit to the display.
temp_surf = pygame.Surface((w,h))
pygame.surfarray.blit(temp_surf, scan)
'''transform temp_surf'''
screen.blit(temp_surf, (0,0))
I have no idea why this is. It is probably something to do with the order in which the axes are transferred from a 2d array to a pygame Surface.
I am currently working on a project to capture and process photos on a raspberry Pi.
The photos are 6000X4000 about 2 mb, from a nikon D5200 camera.
Everything is working fine, i have made a proof of concept in Java and want to transform this to python or C depending on which language is faster on the raspberry.
No the problem is that the images need to be cropped and re-sized, this takes a very long time in the raspberry. In java the whole process of reading the image, cropping and writing the new image takes about 2 minutes.
I have also tried ImageMagick but in command-line this even takes up to 3 minutes.
With a small python script i made this is reduces to 20 seconds, but this is still a bit to long for my project.
Currently i am installing OpenCV to check if this is faster, this process takes around 4 hours so i thought in the meantime i can ask a question here.
Does anybody have any good idea's or libraries to speed up the process of cropping and re-sizing the images.
Following is the python code i used
import Image
def crop_image(input_image, output_image, start_x, start_y, width, height):
"""Pass input name image, output name image, x coordinate to start croping, y coordinate to start croping, width to crop, height to crop """
input_img = Image.open(input_image)
box = (start_x, start_y, start_x + width, start_y + height)
output_img = input_img.crop(box)
output_img.save(output_image +".jpg")
def main():
crop_image("test.jpg","output", 1000, 0, 4000, 4000)
if __name__ == '__main__': main()
First approach (without sprites)
import pyglet
#from pyglet.gl import *
image = pyglet.resource.image('test.jpg')
texture = image.get_texture()
## -- In case you plan on rendering the image, use the following gl set:
#gl.glTexParameteri(gl.GL_TEXTURE_2D, gl.GL_TEXTURE_MAG_FILTER, gl.GL_NEAREST)
texture.width = 1024
texture.height = 768
texture.get_region(256, 192,771, 576)
texture.save('wham.png') # <- To save as JPG again, install PIL
Second attempt (with sprites, unfinished)
import pyglet, time
start = time.time() #DEBUG
texture = pyglet.image.load('test.jpg')
print('Loaded image in',time.time()-start,'sec') #DEBUG
sprite = pyglet.sprite.Sprite(texture)
print('Converted to sprite in',time.time()-start,'sec') #DEBUG
print(sprite.width) #DEBUG
# Gives: 6000
sprite.scale = 0.5
print('Rescaled image in',time.time()-start,'sec') #DEBUG
print(sprite.width) #DEBUG
# Gives: 3000
Both solutions end up around 3-5 seconds on an extremely slow PC with a shitty mechanical disk running under Windows XP with.. i can't even count the number of applications running including active virus scans etc.. But note that I can't remember how to save a sprite to disk, you need to access to AbstractImage data container within the sprite to get it out.
You will be heavily limited to your disk/memory-card I/O.
My image was 16MB 6000x4000 pixels.. Which i was suprised it whent as fast as 3 seconds to load.
Have you tried jpegtran. It provides for lossless cropping of jpeg. It should be in the libjpeg-progs package. I suspect that decoding the image to crop it, then re-encoding it is too much for the SD card to take.
I am working on stream generator for my video mapping set, but I am not able to get the image steady.
I open a v4l2loopback device with python-v4l2 and generate a video stream through it based on png, so can generate live video's in my vj set and still video map them and apply effects.
Test case:
1) load v4l2loopback module
2) run python:
import fcntl, numpy
from v4l2 import *
from PIL import Image
height = 600
width = 634
device = open('/dev/video4', 'wr')
print(device)
capability = v4l2_capability()
print(fcntl.ioctl(device, VIDIOC_QUERYCAP, capability))
print("v4l2 driver: " + capability.driver)
format = v4l2_format()
format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT
format.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB32
format.fmt.pix.width = width
format.fmt.pix.height = height
format.fmt.pix.field = V4L2_FIELD_NONE
format.fmt.pix.bytesperline = format.fmt.pix.width * 4
format.fmt.pix.sizeimage = format.fmt.pix.width * format.fmt.pix.height * 4
format.fmt.pix.colorspace = V4L2_COLORSPACE_SRGB
print(fcntl.ioctl(device, VIDIOC_S_FMT, format))
img = Image.open('img/0.png')
img = img.convert('RGBA')
while True:
device.write(numpy.array(img))
3) run Cheese or other v4l2 stream viewer.
The result is a proper colored and sized image, but it jumps every frame from left to right and always a little more to the left so you get a sliding and jumpy video result.
What am I doing wrong?
Best regards,
Harriebo
ps: if you woul like to see the results check: link So far the LiVES, puredate, gem video mapping setup is working greath with the v4l2 streams.
So I got it a sort of working, but not sure if it's the right way. What I need to do for a stable video stream:
1) don't use custom resolutions, they get messy.
2) send every frame twice. I think this has to do with interlacing / top / bottom frame.
3) for 640x480 shift all pixels 260 spaces to the left in the array, other wise the image is not straight, not for 1024x768 doh... not sure why this is.
4) play is at a slightly lower frame rate as the program can generate.
After all that it is a 99% stable every 10 sec. or so there is one buggy frame. I think it has to do that the framerate the program generates is not 100% stable.
Suggestions on why or how I can do this better are still welcome.
For updates see: https://github.com/umlaeute/v4l2loopback/issues/32