I want to filter out image data as per the text message attached to it.
Hub Code
import imagezmq
import cv2
hub = imagezmq.ImageHub()
print('Listening')
while True:
rpi_name, image = hub.recv_jpg()
print('Received from ' + rpi_name)
hub.send_reply(b'OK')
Server 1
import imagezmq
import cv2
import simplejpeg
hub = imagezmq.ImageSender()
feed = cv2.VideoCapture(url)
while True:
ret, frame = feed.read()
image = simplejpeg.encode_jpeg(frame, quality=60, colorspace='BGR')
hub.send_jpg('1',image)
Server 2
import imagezmq
import cv2
import simplejpeg
hub = imagezmq.ImageSender()
feed = cv2.VideoCapture(url)
while True:
ret, frame = feed.read()
image = simplejpeg.encode_jpeg(frame, quality=60, colorspace='BGR')
hub.send_jpg('2',image)
In the hub I receive data form both the servers simultaneously. What i want is to filter the data out. Like i just want to receive data which have message(this is used to differentiate servers) 1 only.
Or what would be the fasted way to get data from a server if the hub is receiving data from many servers.
For each (text, image) pair received, the text message contained in rpi_name
contains the '1' or '2' depending on the sending server. To filter images by sending server, you need to
use the text to differentiate what action is done based on the text portion of each message.
Here is one example of how to do that. I have added an example function
do_something() that uses an if statement to take different actions depending
on the source of the image.
Hub Code (with added function to take different actions based on sending server)
# Here is example Hub code showing how to Filter or take other
# action based on the source of rpi_name, image
#
import imagezmq
import cv2
def do_something(rpi_name, image):
# parameter rpi_name contains the "text message" which differs by server
# parameter image contains the image that arrived from that server
if rpi_name == '1':
pass # or do something like save the image or process it somehow
elif rpi_name == '2':
pass # or do something like save the image or process it somehow
else:
pass # what do you want to do if server is not one of the 2 above?
hub = imagezmq.ImageHub()
print('Listening')
while True:
rpi_name, image = hub.recv_jpg()
print('Received from ' + rpi_name)
do_something(rpi_name, image)
hub.send_reply(b'OK')
FYI, I am the author of imageZMQ.
I've trained a yolov4 neural network to detect pens. Everything has gone currectly, but when I try to test my model, its just not working. Here's what it's showing-
Here's the command-
!./darknet detector test data/obj.data cfg/yolov4-custom.cfg /mydrive/YOLOV4/training/yolov4-custom_last.weights /mydrive/image1.jpeg
imShow('predictions.jpg')
I dont know if this is relevant, but here's the imshow function-
def imShow(path):
import cv2
import matplotlib.pyplot as plt
%matplotlib inline
image = cv2.imread(path)
height, width = image.shape[:2]
resized_image = cv2.resize(image,(3*width, 3*height), interpolation = cv2.INTER_CUBIC)
fig = plt.gcf()
fig.set_size_inches(18, 10)
plt.axis("off")
plt.imshow(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))
plt.show()
Someone please help me fix this issue. Thanks a lot!
I was trying to send a notification with an image after a bit of googling I found this code in stack overflow
even though the code below gives a notification with space for the image, the image doesn't load
What I have tried
I tried adding time.sleep() between few lines to give it some time to load the image
tried changing the interpreter to python 3.9
tried changing image source to another image
executing the code from another computer
none of these worked
import winrt.windows.ui.notifications as notifications
import winrt.windows.data.xml.dom as dom
import time
app = r'C:\Users\Sandramohan\AppData\Local\Programs\Python\Python38\python.exe'
nManager = notifications.ToastNotificationManager
notifier = nManager.create_toast_notifier(app)
tString = """
<toast>
<visual>
<binding template='ToastGeneric'>
<text>Another Message from Tim!</text>
<text>Hi there!</text>
<image placement="appLogoOverride" HintCrop="circle" src="https://www.decotaime.fr/decoration/images/178/Tableau-design-plexi-Pixel-Art-Marylin-Blue-50x50_L26640.jpg"/>
</binding>
</visual>
</toast>
"""
xDoc = dom.XmlDocument()
time.sleep(5)
xDoc.load_xml(tString)
notification = notifications.ToastNotification(xDoc)
#display notification
notifier.show(notification)
I tried using some local image from my computer as src and it worked.
If it is not required for the image source to be online, simply download it and use it locally.
tString = """
<toast>
<visual>
<binding template='ToastGeneric'>
<text>Another Message from Tim!</text>
<text>Hi there!</text>
<image placement="appLogoOverride" HintCrop="circle" src="C:/.../main.ico"/>
</binding>
</visual>
</toast>
"""
Result
After running the code, I got this result:
I used random .ico from my PC just for test.
I was practicing OpenCV on google colaboratory becasuse I don't know how to use OpenCV on GPU, when I run OpenCV on my hardware, It takes a lot of CPU, so I went to Google colaboratory.
The link to my notebook is here.
If you don't want to watch it, then here is the code:
import cv2
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
cap = cv2.VideoCapture(0)
while True:
_, img = cap.read()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.1, 4)
for (x, y, w, h) in faces:
cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)
cv2.imshow('img', img)
k = cv2.waitKey(30) & 0xff
if k==27:
break
cap.release()
The same code worked fine on my PC, but not on Google Colaboratory. The error is:
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-5-0d9472926d8c> in <module>()
6 while True:
7 _, img = cap.read()
----> 8 gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
9 faces = face_cascade.detectMultiScale(gray, 1.1, 4)
10 for (x, y, w, h) in faces:
error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
PS~I have the haarcascade file inside the same directory of my notebook in Google Colaboratory
How to deal with it? If not then is there any "concrete" solution to run OpenCV on my CUDA enabled GPU instead of CPU? Thanks in advance!
_src.empty() means that it had problem to get frame from camera and img is None and when it tries cvtColor(None, ...) then it gives _src.empty().
You should check if img is not None: because cv2 doesn't raise error when it can't get frame from camera or read image from file. And sometimes camera needs time to "warm up" and it can gives few empty frames (None).
VideoCapture(0) reads frame from camera directly connected to computer which runs this code - and when you run code on server Google Colaboratory then it means camera connected directly to server Google Colaboratory (not your local camera) but this server doesn't have camera so VideoCapture(0) can't work on Google Colaboratory.
cv2 can't get image from your local camera when it runs on server. Your web browser may have access to your camera but it needs JavaScript to get frame and send to server - but server needs code to get this frame
I checked in Google if Google Colaboratory can access local webcam and it seems they created script for this - Camera Capture - in first cell is function take_photo() which uses JavaScript to access your camera and display in browser, and in second cell this function is used to display image from local camera and to take screenshot.
You should use this function instead of VideoCapture(0) to work on server with your local camera.
BTW: Belove take_photo() there is also information about cv2.im_show() because it also works only with monitor directly connected to computer which runs this code (and this computer has to run GUI like Windows on Windows , X11 on Linux) - and when you run it on server then it want to display on monitor directly connected to server - but server usually works without monitor (and without GUI)
Google Colaboratory has special replacement which displays in web browser
from google.colab.patches import cv2_imshow
BTW: If you will have problem with loading haarcascades .xml then you may need folder to filename. cv2 has special variable for this cv2.data.haarcascades
path = os.path.join(cv2.data.haarcascades, 'haarcascade_frontalface_default.xml')
cv2.CascadeClassifier( path )
You can also see what is in this folder
import os
filenames = os.listdir(cv2.data.haarcascades)
filenames = sorted(filenames)
print('\n'.join(filenames))
EDIT:
I created code which can get from local webcam frame by frame without using button and without saving in file. Problem is that it is slow - because it still have to send frame from local web browser to google colab server and later back to local web browser
Python code with JavaScript functions
#
# based on: https://colab.research.google.com/notebooks/snippets/advanced_outputs.ipynb#scrollTo=2viqYx97hPMi
#
from IPython.display import display, Javascript
from google.colab.output import eval_js
from base64 import b64decode, b64encode
import numpy as np
def init_camera():
"""Create objects and functions in HTML/JavaScript to access local web camera"""
js = Javascript('''
// global variables to use in both functions
var div = null;
var video = null; // <video> to display stream from local webcam
var stream = null; // stream from local webcam
var canvas = null; // <canvas> for single frame from <video> and convert frame to JPG
var img = null; // <img> to display JPG after processing with `cv2`
async function initCamera() {
// place for video (and eventually buttons)
div = document.createElement('div');
document.body.appendChild(div);
// <video> to display video
video = document.createElement('video');
video.style.display = 'block';
div.appendChild(video);
// get webcam stream and assing to <video>
stream = await navigator.mediaDevices.getUserMedia({video: true});
video.srcObject = stream;
// start playing stream from webcam in <video>
await video.play();
// Resize the output to fit the video element.
google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);
// <canvas> for frame from <video>
canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
//div.appendChild(input_canvas); // there is no need to display to get image (but you can display it for test)
// <img> for image after processing with `cv2`
img = document.createElement('img');
img.width = video.videoWidth;
img.height = video.videoHeight;
div.appendChild(img);
}
async function takeImage(quality) {
// draw frame from <video> on <canvas>
canvas.getContext('2d').drawImage(video, 0, 0);
// stop webcam stream
//stream.getVideoTracks()[0].stop();
// get data from <canvas> as JPG image decoded base64 and with header "data:image/jpg;base64,"
return canvas.toDataURL('image/jpeg', quality);
//return canvas.toDataURL('image/png', quality);
}
async function showImage(image) {
// it needs string "data:image/jpg;base64,JPG-DATA-ENCODED-BASE64"
// it will replace previous image in `<img src="">`
img.src = image;
// TODO: create <img> if doesn't exists,
// TODO: use `id` to use different `<img>` for different image - like `name` in `cv2.imshow(name, image)`
}
''')
display(js)
eval_js('initCamera()')
def take_frame(quality=0.8):
"""Get frame from web camera"""
data = eval_js('takeImage({})'.format(quality)) # run JavaScript code to get image (JPG as string base64) from <canvas>
header, data = data.split(',') # split header ("data:image/jpg;base64,") and base64 data (JPG)
data = b64decode(data) # decode base64
data = np.frombuffer(data, dtype=np.uint8) # create numpy array with JPG data
img = cv2.imdecode(data, cv2.IMREAD_UNCHANGED) # uncompress JPG data to array of pixels
return img
def show_frame(img, quality=0.8):
"""Put frame as <img src="data:image/jpg;base64,...."> """
ret, data = cv2.imencode('.jpg', img) # compress array of pixels to JPG data
data = b64encode(data) # encode base64
data = data.decode() # convert bytes to string
data = 'data:image/jpg;base64,' + data # join header ("data:image/jpg;base64,") and base64 data (JPG)
eval_js('showImage("{}")'.format(data)) # run JavaScript code to put image (JPG as string base64) in <img>
# argument in `showImage` needs `" "`
And code which uses it in loop
#
# based on: https://colab.research.google.com/notebooks/snippets/advanced_outputs.ipynb#scrollTo=zo9YYDL4SYZr
#
#from google.colab.patches import cv2_imshow # I don't use it but own function `show_frame()`
import cv2
import os
face_cascade = cv2.CascadeClassifier(os.path.join(cv2.data.haarcascades, 'haarcascade_frontalface_default.xml'))
# init JavaScript code
init_camera()
while True:
try:
img = take_frame()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#cv2_imshow(gray) # it creates new image for every frame (it doesn't replace previous image) so it is useless
#show_frame(gray) # it replace previous image
faces = face_cascade.detectMultiScale(gray, 1.1, 4)
for (x, y, w, h) in faces:
cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)
#cv2_imshow(img) # it creates new image for every frame (it doesn't replace previous image) so it is useless
show_frame(img) # it replace previous image
except Exception as err:
print('Exception:', err)
I don't use from google.colab.patches import cv2_imshow because it always add new image on page instead of replacing existing image.
The same code as Notebook on Google Colab:
https://colab.research.google.com/drive/1j7HTapCLx7BQUBp3USiQPZkA0zBKgLM0?usp=sharing
The possible problem in the code is, you need to give full-path as the directory when using Haar-like features.
face_cascade = cv2.CascadeClassifier('/User/path/to/opencv/data/haarcascades/haarcascade_frontalface_default.xml')
The colab issue with opencv has been known for quite some time, also the same question asked here
As stated here, you can use the cv2_imshow to display the image, but you want to process Camera frames.
from google.colab.patches import cv2_imshow
img = cv2.imread('logo.png', cv2.IMREAD_UNCHANGED)
cv2_imshow(img)
One possible answer:
Insert Camera Capture snippet, the method take_photobut you need to modify the method.
face_cascade = cv2.CascadeClassifier('/opencv/data/haarcascades/haarcascade_frontalface_default.xml')
try:
filename = take_photo()
img = Image(filename)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.1, 4)
for (x, y, w, h) in faces:
cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)
cv2_imshow("img", img)
except Exception as err:
print(str(err))
The above code requires editing since there is no direct way to use VideoCapture you have to modify take_photo
I'm using Basler camera and python to record some video. I can successfully capture individual frames, but I don't know how to record a video.
Following is my code:
import os
import pypylon
from imageio import imwrite
import time
start=time.time()
print('Sampling rate (Hz):')
fsamp = input()
fsamp = float(fsamp)
time_exposure = 1000000*(1/fsamp)
available_cameras = pypylon.factory.find_devices()
cam = pypylon.factory.create_device(available_cameras[0])
cam.open()
#cam.properties['AcquisitionFrameRateEnable'] = True
#cam.properties['AcquisitionFrameRate'] = 1000
cam.properties['ExposureTime'] = time_exposure
buffer = tuple(cam.grab_images(2000))
for count, image in enumerate(buffer):
filename = str('I:/Example/{}.png'.format(count))
imwrite(filename, image)
del buffer
I haven't found a way to record a video using pypylon; it seems to be a pretty light wrapper around Pylon. However, I have found a way to save a video using imageio:
from imageio import get_writer
with get_writer('I:/output-filename.mp4', fps=fps) as writer:
# Some stuff with the frames
The above can be used with .mov, .avi, .mpg, .mpeg, .mp4, .mkv or .wmv, so long as the FFmpeg program is available. How you will install this program depends on your operating system. See this link for details on the parameters you can use.
Then, simply replace the call to imwrite with:
writer.append_data(image)
ensuring that this occurs in the with block.
An example implementation:
import os
import pypylon
from imageio import get_writer
while True:
try:
fsamp = float(input('Sampling rate (Hz): '))
break
except ValueError:
print('Invalid input.')
time_exposure = 1000000 / fsamp
available_cameras = pypylon.factory.find_devices()
cam = pypylon.factory.create_device(available_cameras[0])
cam.open()
cam.properties['ExposureTime'] = time_exposure
buffer = tuple(cam.grab_images(2000))
with get_writer(
'I:/output-filename.mkv', # mkv players often support H.264
fps=fsamp, # FPS is in units Hz; should be real-time.
codec='libx264', # When used properly, this is basically
# "PNG for video" (i.e. lossless)
quality=None, # disables variable compression
pixelformat='rgb24', # keep it as RGB colours
ffmpeg_params=[ # compatibility with older library versions
'-preset', # set to faster, veryfast, superfast, ultrafast
'fast', # for higher speed but worse compression
'-crf', # quality; set to 0 for lossless, but keep in mind
'11' # that the camera probably adds static anyway
]
) as writer:
for image in buffer:
writer.append_data(image)
del buffer