I have made a morse code translator and I want it to be able to record a flashing light and make it into morse code. I think I will need OpenCV or a light sensor, but I don't know how to use either of them. I haven't got any code for it yet, as I couldn't find any solutions anywhere else.
The following is just a concept of what you could try. Yes, you could also train a neural network for this but if your setup is simple enough, some engineering will do.
We first create a "toy-video" to work with:
import numpy as np
import matplotlib.pyplot as plt
# Create a toy "video"
image = np.asarray([
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 1, 2, 2, 1],
[0, 0, 2, 4, 4, 2],
[0, 0, 2, 4, 4, 2],
[0, 0, 1, 2, 2, 1],
])
signal = np.asarray([0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0])
x = list(range(len(signal)))
signal = np.interp(np.linspace(0, len(signal), 100), x, signal)[..., None]
frames = np.einsum('tk,xy->txyk', signal, image)[..., 0]
Plot a few frames:
fig, axes = plt.subplots(1, 12, sharex='all', sharey='all')
for i, ax in enumerate(axes):
ax.matshow(frames[i], vmin=0, vmax=1)
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
ax.set_title(i)
plt.show()
Now that you have this kind of toy video, it's pretty straight forward to convert it back to some sort of binary signal. You'd simply compute the average brightness of each frame:
reconstructed = frames.mean(1).mean(1)
reconstructed_bin = reconstructed > 0.5
plt.plot(reconstructed, label='original')
plt.plot(reconstructed_bin, label='binary')
plt.title('Reconstructed Signal')
plt.legend()
plt.show()
From here we only have to determine the length of each flash.
# This is ugly, I know. Just for understanding though:
# 1. Splits the binary signal on zero-values
# 2. Filters out the garbage (accept only lists where len(e) > 1)
# 3. Gets the length of the remaining list == the duration of each flash
tmp = np.split(reconstructed_bin, np.where(reconstructed_bin == 0)[0][1:])
flashes = list(map(len, filter(lambda e: len(e) > 1, tmp)))
We can now take a look at how long flashes take:
print(flashes)
gives us
[5, 5, 5, 10, 9, 9, 5, 5, 5]
So.. "short" flashes seem to take 5 frames, "long" around 10. With this we can classify each flash as either being "long" or "short" by defining a sensible threshold of 7 like so:
# Classify each flash-duration
flashes_classified = list(map(lambda f: 'long' if f > 7 else 'short', flashes))
And let's repeat for pauses
# Repeat for pauses
tmp = np.split(reconstructed_bin, np.where(reconstructed_bin != False)[0][1:])
pauses = list(map(len, filter(lambda e: len(e) > 1, tmp)))
pauses_classified = np.asarray(list(map(lambda f: 'w' if f > 6 else 'c', pauses)))
pauses_indices, = np.where(np.asarray(pauses_classified) == 'w')
Now we can visualize the results.
fig = plt.figure()
ax = fig.gca()
ax.bar(range(len(flashes)), flashes, label='Flash duration')
ax.set_xticks(list(range(len(flashes_classified))))
ax.set_xticklabels(flashes_classified)
[ax.axvline(idx-0.5, ls='--', c='r', label='Pause' if i == 0 else None) for i, idx in enumerate(pauses_indices)]
plt.legend()
plt.show()
It somewhat depends on your environment. You might try inexpensively with a Raspberry Pi Zero (£9) or even a Pico (£4) or Arduino and an attached LDR - Light Dependent Resistor for £1 rather than a £100 USB camera.
Your program would then come down to repeatedly measuring the resistance (which depends on the light intensity) and making it into long and short pulses.
This has the benefit of being cheap and not requiring you to learn OpenCV, but Stefan's idea is far more fun and has my vote!
Related
I am trying to figure out how to do this with numpy, so I can then convert it to c++ from scratch. I have figured out how to do it when the mode is constant. The way that is done is shown below.
import numpy as np
from scipy import signal
a = np.array([[1, 2, 0, 0], [5, 3, 0, 4], [0, 0, 0, 7], [9, 3, 0, 0]])
k = np.array([[1,0,0],[0,1,0],[0,0,0]])
a = np.pad(a, 1)
k = np.flip(k)
output = signal.convolve(a, k, 'valid')
Which then comes out to the same output as scipy.ndimage.filters.convolve(a, k, mode='constant) So I thought that when the mode was reflect it would work the same way. Except, that the line a = np.pad(a, 1) would be changed to a = np.pad(a, 1, mode='reflect'). However, that does not seem to be the case. Could someone explain how it would work from scratch using numpy and scipy.signal.convolve? Thank you.
What I want to do is crop a volume to remove all irrelevant data. For example, say I have a 100x100x100 volume filled with zeros, except for a 50x50x50 volume within that is filled with ones.
How do I obtain the cropped 50x50x50 volume from the original ?
Here's the naive method I came up with.
import numpy as np
import tensorflow as tf
test=np.zeros((100,100,100)) # create an empty 100x100x100 volume
rand=np.random.rand(66,25,34) # create a 66x25x34 filled volume
test[10:76, 20:45, 30:64] = rand # partially fill the empty volume
# initialize the cropping coordinates
minx=miny=minz=0
maxx=maxy=maxz=0
maxx,maxy,maxz=np.subtract(test.shape,1)
# compute the optimal cropping coordinates
dimensions=test.shape
while(tf.reduce_max(test[minx,:,:]) == 0): # check for empty slices along the x axis
minx+=1
while(tf.reduce_max(test[:,miny,:]) == 0): # check for empty slices along the y axis
miny+=1
while(tf.reduce_max(test[:,:,minz]) == 0): # check for empty slices along the z axis
minz+=1
while(tf.reduce_max(test[maxx,:,:]) == 0):
maxx-=1
while(tf.reduce_max(test[:,maxy,:]) == 0):
maxy-=1
while(tf.reduce_max(test[:,:,maxz]) == 0):
maxz-=1
maxx,maxy,maxz=np.add((maxx,maxy,maxz),1)
crop = test[minx:maxx,miny:maxy,minz:maxz]
print(minx,miny,minz,maxx,maxy,maxz)
print(rand.shape)
print(crop.shape)
This prints:
10 20 30 76 45 64
(66, 25, 34)
(66, 25, 34)
, which is correct. However, it takes too long and is probably suboptimal. I'm looking for better ways to achieve the same thing.
NB:
The subvolume wouldn't necessarily be a cuboid, it could be any shape.
I want to keep gaps within the subvolume, only remove what's "outside" the shape to be cropped.
(Edit)
Oops, I hadn't seen the comment about keeping the so-called "gaps" between elements! This should be the one, finally.
def get_nonzero_sub(arr):
arr_slices = tuple(np.s_[curr_arr.min():curr_arr.max() + 1] for curr_arr in arr.nonzero())
return arr[arr_slices]
While you wait for a sensible response (I would guess this is a builtin function in an image processing library somewhere), here's a way
y, x = np.where(np.any(test, 0))
z, _ = np.where(np.any(test, 1))
test[min(z):max(z)+1, min(y):max(y)+1, min(x):max(x)+1]
I think leaving tf out of this should up your performance.
Explanation (based on 2D array)
test = np.array([
[0, 0, 0, 0, 0, ],
[0, 0, 1, 2, 0, ],
[0, 0, 3, 0, 0, ],
[0, 0, 0, 0, 0, ],
[0, 0, 0, 0, 0, ],
])
We want to crop it to get
[[1, 2]
[3, 0]]
np.any(..., 0) this will 'iterate' over axis 0 and return True if any of the elements in the slice are truthy. I show the result of this in the comments here:
np.array([
[0, 0, 0, 0, 0, ], # False
[0, 0, 1, 2, 0, ], # True
[0, 0, 3, 0, 0, ], # True
[0, 0, 0, 0, 0, ], # False
[0, 0, 0, 0, 0, ], # False
])
i.e. it returns np.array([False, True, True, False, False])
np.any(..., 1) does the same as step 2 but over axis 1 instead of axis zero i.e.
np.array([
[0, 0, 0, 0, 0, ],
[0, 0, 1, 2, 0, ],
[0, 0, 3, 0, 0, ],
[0, 0, 0, 0, 0, ],
[0, 0, 0, 0, 0, ],
# False False True True False
])
Note that in the case of a 3D array, these steps return 2D arrays
(x,) = np.where(...) this returns the index values of the truthy values in an array. So np.where([False, True, True, False, False]) returns (array([1, 2]),). Note that this is a tuple so in the 2D case we would need to call (x,) = ... so x is just the array array([1, 2]). The syntax is nicer in the 2D case as we can use tuple-unpacking i.e x, y = ...
Note that in the 3D case, np.where can give us the value for 2 axes at a time. I chose to do x-y in one go and then z-? in the second go. The ? is either x or y, I can't be bothered to work out which and since we don't need it I throw it away in a variable named _ which by convention is a reasonable place to store junk output you don't actually want. Note I need to do z, _ = as I want the tuple-unpacking and not just z = otherwise z become the tuple with both arrays.
Well, this step is pretty much the same as what you did at the end of your answer so I assume you understand it. Simple slicing in each dimension from the first element with a value in that dimension to the last. You need the + 1 because slicing in python are not inclusive of the index after the :.
Hopefully that's clear?
I'm trying to interpolate a 2d unstructured grid using scipy.interpolate.SmoothBivariateSpline. I'm afraid I have not understood how it is supposed to work.
I've tried with a very simple example:
from scipy import interpolate
x = [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]
y = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]
z = [-0.07453796, -0.10857792, -0.07307213, 0.01813757, -0.31634891, -0.47235507, -0.33198942, -0.28530956, -0.26995915, -0.40618327, -0.0950876, -0.18871505]
xy_func = interpolate.SmoothBivariateSpline(x, y, z, kx=1, ky=1, s=0)
print(xy_func.ev(0, 1), xy_func.ev(1, 0), xy_func.ev(1, 3))
I've visualized the result and it is obvious that it is incorrect. I also evaluated the result on some of the data points since it should be clear what the output should be. From the print I expected to get the output "-0.10857792, -0.31634891, -0.28530956", but I got the result "-0.1390947215 -0.272092075 -0.16190767".
Where am I off?
I think there are two problems:
If you allow a curvier shape by increasing i.e. kx=2 and ky=3, you already get a much better fit of your predictions.
However, because SmoothBivariateSpline doesnt like the vertical nature of your test data, you will not get very good results anyway. If you change x so that it increments more evenly i.e. (x=range(len(x)), it looks much better.
I am using Vapory which is a wrapper Python library for Povray. It allows using Python functions to manipulate typical Povray operations.
I want to superimpose 3D models in every frame of my video stream. The way to do this in Vapory is the following:
from vapory import *
from moviepy.video.io.ffmpeg_writer import ffmpeg_write_image
light = LightSource([10, 15, -20], [1.3, 1.3, 1.3])
wall = Plane([0, 0, 1], 20, Texture(Pigment('color', [1, 1, 1])))
ground = Plane( [0, 1, 0], 0,
Texture( Pigment( 'color', [1, 1, 1]),
Finish( 'phong', 0.1,
'reflection',0.4,
'metallic', 0.3)))
sphere1 = Sphere([-4, 2, 2], 2.0, Pigment('color', [0, 0, 1]),
Finish('phong', 0.8,
'reflection', 0.5))
sphere2 =Sphere([4, 1, 0], 1.0, Texture('T_Ruby_Glass'),
Interior('ior',2))
scene = Scene( Camera("location", [0, 5, -10], "look_at", [1, 3, 0]),
objects = [ ground, wall, sphere1, sphere2, light],
included=["glass.inc"] )
def embed_in_scene(image):
ffmpeg_write_image("__temp__.png", image)
image_ratio = 1.0*image.shape[1]/image.shape[0]
screen = Box([0, 0, 0], [1, 1, 0], Texture(
Pigment( ImageMap('png', '"__temp__.png"', 'once')),
Finish('ambient', 1.2) ),
'scale', [10, 10/image_ratio,1],
'rotate', [0, 20, 0],
'translate', [-3, 1, 3])
new_scene = scene.add_objects([screen])
return new_scene.render(width=800, height=480, antialiasing=0.001)
clip = (VideoFileClip("bunny.mp4") # File containing the original video
.subclip(23, 47) # cut between t=23 and 47 seconds
.fl_image(embed_in_scene) # <= The magic happens
.fadein(1).fadeout(1)
.audio_fadein(1).audio_fadeout(1))
clip.write_videofile("bunny2.mp4",bitrate='8000k')
which results with a video stream as follows:
What I want, however, is that movie box being the whole scene, and spheres to remain where they are. The first thought was to remove the rotation function from the code and it did work, however I still cannot stretch the movie frame to the end corners of the actual scene.
Any thoughts?
EDIT: So I was able to move the camera, get the object to the center. However I still could not get the movie full screen. This is because the camera object is told to look towards the coordinates, and I don't know what coordinates the camera should be directed at, in order to get the picture in full screen. See:
Say you have an image in the form of a numpy.array:
vals=numpy.array([[3,24,25,6,2],[8,7,6,3,2],[1,4,23,23,1],[45,4,6,7,8],[17,11,2,86,84]])
And you want to compute how many cells are inside each object, given a threshold value of 17 (example):
from scipy import ndimage
from skimage.measure import regionprops
blobs = numpy.where(vals>17, 1, 0)
labels, no_objects = ndimage.label(blobs)
props = regionprops(blobs)
If you check, this gives an image with 4 distinct objects over the threshold:
In[1]: blobs
Out[1]:
array([[0, 1, 1, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 1, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 1, 1]])
In fact:
In[2]: no_objects
Out[2]: 4
I want to compute the number of cells (or area) of each object. The intended outcome is a dictionary with the object ID: number of cells format:
size={0:2,1:2,2:1,3:2}
My attempt:
size={}
for label in props:
size[label]=props[label].area
Returns an error:
Traceback (most recent call last):
File "<ipython-input-76-e7744547aa17>", line 3, in <module>
size[label]=props[label].area
TypeError: list indices must be integers, not _RegionProperties
I understand I am using label incorrectly, but the intent is to iterate over the objects. How to do this?
A bit of testing and research sometimes goes a long way.
The problem is both with blobs, because it is not carrying the different labels but only 0,1 values, and label, which needs to be replaced by an iterator looping over range(0,no_objects).
This solution seems to be working:
import skimage.measure as measure
import numpy
from scipy import ndimage
from skimage.measure import regionprops
vals=numpy.array([[3,24,25,6,2],[8,7,6,3,2],[1,4,23,23,1],[45,4,6,7,8],[17,11,2,86,84]])
blobs = numpy.where(vals>17, 1, 0)
labels, no_objects = ndimage.label(blobs)
#blobs is not in an amicable type to be processed right now, so:
labelled=ndimage.label(blobs)
resh_labelled=labelled[0].reshape((vals.shape[0],vals.shape[1])) #labelled is a tuple: only the first element matters
#here come the props
props=measure.regionprops(resh_labelled)
#here come the sought-after areas
size={i:props[i].area for i in range (0, no_objects)}
Result:
In[1]: size
Out[1]: {0: 2, 1: 2, 2: 1, 3: 2}
And if anyone wants to check for the labels:
In[2]: labels
Out[2]:
array([[0, 1, 1, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 2, 2, 0],
[3, 0, 0, 0, 0],
[0, 0, 0, 4, 4]])
And if anyone wants to plot the 4 objects found:
import matplotlib.pyplot as plt
plt.set_cmap('OrRd')
plt.imshow(labels,origin='upper')
To answer the original question:
You have to apply regionprops to the labeled image: props = regionprops(labels)
You can then construct the dictionary using:
size = {r.label: r.area for r in props}
which yields
{1: 2, 2: 2, 3: 1, 4: 2}
That regionprops will generate a lot more information than just the area of each blob. So, if you are just looking to get the count of pixels for the blobs, as an alternative and with focus on performance, we can use np.bincount on labels obtained with ndimage.label, like so -
np.bincount(labels.ravel())[1:]
Thus, for the given sample -
In [53]: labeled_areas = np.bincount(labels.ravel())[1:]
In [54]: labeled_areas
Out[54]: array([2, 2, 1, 2])
To have these results in a dictionary, one additional step would be -
In [55]: dict(zip(range(no_objects), labeled_areas))
Out[55]: {0: 2, 1: 2, 2: 1, 3: 2}