I'm currently trying to code something that will let websites view my webcam. I'm roughly following the tutorial linked on this website, except using Python and pygame instead of Processing.
At the moment, my code is grabbing a pygame image (which was originally a SimpleCV image), attempting to convert it into jpg format, and send it over websockets to the client where it will display it inside an img tag. However, I can't seem to figure out how to convert a pygame image into jpg and get it to display properly on the web browser.
This is my code for the server, which uses Flask and gevent:
#!/usr/bin/env python
import base64
import cStringIO
import time
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from flask import Flask, request, render_template
import pygame
pygame.init()
import SimpleCV as scv
app = Flask(__name__)
cam = scv.Camera(0)
#app.route('/')
def index():
return render_template('index.html')
#app.route('/camera')
def camera():
if request.environ.get('wsgi.websocket'):
ws = request.environ['wsgi.websocket']
while True:
image = cam.getImage().flipHorizontal().getPGSurface()
data = cStringIO.StringIO()
pygame.image.save(image, data)
ws.send(base64.b64encode(data.getvalue()))
time.sleep(0.5)
if __name__ == '__main__':
http_server = WSGIServer(('',5000), app, handler_class=WebSocketHandler)
http_server.serve_forever()
This is my HTML file:
<!DOCTYPE HTML>
<html>
<head>
<title>Flask/Gevent WebSocket Test</title>
<script src="http://code.jquery.com/jquery-1.10.1.min.js"></script>
<script type="text/javascript" charset="utf-8">
$(document).ready(function(){
if ("WebSocket" in window) {
cam = new WebSocket("ws://" + document.domain + ":5000/camera");
cam.onmessage = function (msg) {
$("#cam").attr('src', 'data:image/jpg;base64,' + msg.data);
};
cam.onerror = function(e) {
console.log(e);
}
} else {
alert("WebSocket not supported");
}
});
</script>
</head>
<body>
<img id="cam" src="" width="640" height="480" />
</body>
</html>
These are the specific lines that I think I'm having trouble with:
while True:
image = cam.getImage().flipHorizontal().getPGSurface()
data = cStringIO.StringIO()
pygame.image.save(image, data)
ws.send(base64.b64encode(data.getvalue()))
time.sleep(0.5)
Currently, if I try and run my code, going to localhost:5000 will display an invalid jpg image. It also becomes really laggy if I try running it on Firefox, but that may be an unrelated issue that I can debug later.
I've checked and made sure that the pygame image is a valid one, since I'm converting it from another library, and also checked that I was using websockets correctly by sending text data back and forth.
I've also tried calling pygame.image.to_string to try and convert the pygame surface into RGB format, but that also doesn't work.
What am I doing wrong?
Using the underlying PIL image, we can write to a file-like object, read back and base-64 encode it:
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from flask import Flask, request
from time import sleep
from cStringIO import StringIO
import pygame
pygame.init()
import SimpleCV as scv
app = Flask(__name__)
cam = scv.Camera(0)
#app.route('/camera')
def camera():
if request.environ.get('wsgi.websocket'):
ws = request.environ['wsgi.websocket']
while True:
fp = StringIO()
image = cam.getImage().flipHorizontal().getPIL()
image.save(fp, 'JPEG')
ws.send(fp.getvalue().encode("base64"))
#fp.close() << benchmark and memory tests needed
sleep(0.5)
if __name__ == '__main__':
http_server = WSGIServer(('',5000), app, handler_class=WebSocketHandler)
http_server.serve_forever()
I'm fighting with the same issue and the problem is a double codification. In the Python file, you have to remove the line "ws.send(base64.b64encode(data.getvalue()))" and send the image without encoded. Then in the js file, your script will make the codification and that all.
Related
I'm currently trying to code something that will let websites view my webcam. I'm roughly following the tutorial linked on this website, except using Python and pygame instead of Processing.
At the moment, my code is grabbing a pygame image (which was originally a SimpleCV image), attempting to convert it into jpg format, and send it over websockets to the client where it will display it inside an img tag. However, I can't seem to figure out how to convert a pygame image into jpg and get it to display properly on the web browser.
This is my code for the server, which uses Flask and gevent:
#!/usr/bin/env python
import base64
import cStringIO
import time
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from flask import Flask, request, render_template
import pygame
pygame.init()
import SimpleCV as scv
app = Flask(__name__)
cam = scv.Camera(0)
#app.route('/')
def index():
return render_template('index.html')
#app.route('/camera')
def camera():
if request.environ.get('wsgi.websocket'):
ws = request.environ['wsgi.websocket']
while True:
image = cam.getImage().flipHorizontal().getPGSurface()
data = cStringIO.StringIO()
pygame.image.save(image, data)
ws.send(base64.b64encode(data.getvalue()))
time.sleep(0.5)
if __name__ == '__main__':
http_server = WSGIServer(('',5000), app, handler_class=WebSocketHandler)
http_server.serve_forever()
This is my HTML file:
<!DOCTYPE HTML>
<html>
<head>
<title>Flask/Gevent WebSocket Test</title>
<script src="http://code.jquery.com/jquery-1.10.1.min.js"></script>
<script type="text/javascript" charset="utf-8">
$(document).ready(function(){
if ("WebSocket" in window) {
cam = new WebSocket("ws://" + document.domain + ":5000/camera");
cam.onmessage = function (msg) {
$("#cam").attr('src', 'data:image/jpg;base64,' + msg.data);
};
cam.onerror = function(e) {
console.log(e);
}
} else {
alert("WebSocket not supported");
}
});
</script>
</head>
<body>
<img id="cam" src="" width="640" height="480" />
</body>
</html>
These are the specific lines that I think I'm having trouble with:
while True:
image = cam.getImage().flipHorizontal().getPGSurface()
data = cStringIO.StringIO()
pygame.image.save(image, data)
ws.send(base64.b64encode(data.getvalue()))
time.sleep(0.5)
Currently, if I try and run my code, going to localhost:5000 will display an invalid jpg image. It also becomes really laggy if I try running it on Firefox, but that may be an unrelated issue that I can debug later.
I've checked and made sure that the pygame image is a valid one, since I'm converting it from another library, and also checked that I was using websockets correctly by sending text data back and forth.
I've also tried calling pygame.image.to_string to try and convert the pygame surface into RGB format, but that also doesn't work.
What am I doing wrong?
Using the underlying PIL image, we can write to a file-like object, read back and base-64 encode it:
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from flask import Flask, request
from time import sleep
from cStringIO import StringIO
import pygame
pygame.init()
import SimpleCV as scv
app = Flask(__name__)
cam = scv.Camera(0)
#app.route('/camera')
def camera():
if request.environ.get('wsgi.websocket'):
ws = request.environ['wsgi.websocket']
while True:
fp = StringIO()
image = cam.getImage().flipHorizontal().getPIL()
image.save(fp, 'JPEG')
ws.send(fp.getvalue().encode("base64"))
#fp.close() << benchmark and memory tests needed
sleep(0.5)
if __name__ == '__main__':
http_server = WSGIServer(('',5000), app, handler_class=WebSocketHandler)
http_server.serve_forever()
I'm fighting with the same issue and the problem is a double codification. In the Python file, you have to remove the line "ws.send(base64.b64encode(data.getvalue()))" and send the image without encoded. Then in the js file, your script will make the codification and that all.
I have created a face recognition model using keras and tensorflow, and now I am trying to convert it as a web application using flask and python. My requirement is that, I need a live webcam displayed on the webpage and by clicking on a button it should take the picture and save it to a specified directory, and using that picture the application should recognize the person. if the person is not found in the dataset then a message should be displayed over the webpage that unknown identity is been found. To do this job I have started learning flask and after that when it comes to the requirement it was very difficult for me. somebody help me out to solve this situation.
What you want to do is streaming with Flask by using the webcam Stream and handle it with Machine Learning. Your main script for the web server in flask will allow you to load your index.html file and then Stream each frame through the /video_feed path:
from flask import Flask, render_template, Response, jsonify
from camera import VideoCamera
import cv2
app = Flask(__name__)
video_stream = VideoCamera()
#app.route('/')
def index():
return render_template('index.html')
def gen(camera):
while True:
frame = camera.get_frame()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
#app.route('/video_feed')
def video_feed():
return Response(gen(video_stream),
mimetype='multipart/x-mixed-replace; boundary=frame')
if __name__ == '__main__':
app.run(host='127.0.0.1', debug=True,port="5000")
Then you need the VideoCamera class in wich you will handle each frame and where you can make every prediction or processing you want on the frames. The camera.py file :
class VideoCamera(object):
def __init__(self):
self.video = cv2.VideoCapture(0)
def __del__(self):
self.video.release()
def get_frame(self):
ret, frame = self.video.read()
# DO WHAT YOU WANT WITH TENSORFLOW / KERAS AND OPENCV
ret, jpeg = cv2.imencode('.jpg', frame)
return jpeg.tobytes()
And finally the page showing the video Stream in the html file index.html (in the templates/ folder, if not exist generate it) :
<!DOCTYPE html>
<html lang="en">
<head>
<title>Video Stream</title>
</head>
<body>
<img src="{{ url_for('video_feed') }}" />
</body>
</html>
from flask import Flask,request,jsonify
import numpy as np
import cv2
import tensorflow as tf
import base64
app = Flask(__name__)
graph = tf.get_default_graph()
#app.route('/')
def hello_world():
return """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<video id="video" width="640" height="480" autoplay></video>
<button id="snap">Snap Photo</button>
<canvas id="canvas" width="640" height="480"></canvas>
</body>
<script>
var video = document.getElementById('video');
if(navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({ video: true }).then(function(stream) {
//video.src = window.URL.createObjectURL(stream);
video.srcObject = stream;
video.play();
});
}
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
var video = document.getElementById('video');
// Trigger photo take
document.getElementById("snap").addEventListener("click", function() {
context.drawImage(video, 0, 0, 640, 480);
var request = new XMLHttpRequest();
request.open('POST', '/submit?image=' + video.toString('base64'), true);
request.send();
});
</script>
</html>
"""
# HtmlVideoElement
#app.route('/test',methods=['GET'])
def test():
return "hello world!"
#app.route('/submit',methods=['POST'])
def submit():
image = request.args.get('image')
print(type(image))
return ""`
i have done like this, but the problem is that, when calling the API /submit in decorator, i get my image stored as HTMLVideoElement when print the type of image variable, I dont know how to convert it into Jpeg format and use it for further purpose.
So I've got a setup of flask + jinja and I'm trying to display an image in an html file being rendered by jinja.
In Python I'm converting it into base64 and then sending it to the template. I'm then using an image tag to display the image.
<img src="data:image/jpeg;base64,{{ myimage }}">
I've confirmed that the Python encoding is correct, it displays as it should when I simply write an html file with the base64 embedded into it. Where it seems to fail is from the template modifying the output a little bit. In particular:
<img src=3D"data:;base64,/9j/4QAYR
...
baW4WqWj/2Q=3D=3D"/>
Jinja seems to be screwing around by adding the text 3D in a couple places where it looks like it shouldn't be. I haven't specified anything differently, and when I printed out myimage just as text, it came up the way I expected it to, starting with /9j and ending with /2Q==
I'm not sure if there's something with the way I'm interpreting it in Jijna or what, but it just doesn't load. I see the image src tag in the email source, but there's just nothing where I expect the image to be loaded.
Markup the variable myimage as safe:
<img src="data:image/jpeg;base64,{{ myimage | safe }}">
Simple single file app (uses requests library):
from flask import Flask, render_template_string
import base64
import requests
app = Flask(__name__)
global _base64_encoded_image
#app.route('/')
def index():
_html_template = '''
<p><img src="data:image/jpeg;base64,{{ myimage | safe }}"><p>
<p><img src="data:image/jpeg;base64,{{ myimage | e }}"><p>
<p><img src="data:image/jpeg;base64,{{ myimage }}"><p>
'''
global _base64_encoded_image
return render_template_string(_html_template, myimage=_base64_encoded_image)
#app.before_first_request
def before_first_request():
global _base64_encoded_image
_url = "http://via.placeholder.com/200?text=Flask/Jinja2"
_r = requests.get(_url)
_base64_encoded_image = base64.b64encode(_r.content)
print _base64_encoded_image
if __name__ == '__main__':
app.run()
This looks like a URL encoding problem as = is a character used in that base64 alphabet and its URL encoding is %3D. Try doing a urllib.quote on the base64 data before rendering it with Jinja.
I am trying to create a simple web server with twisted in python. I am having trouble serving an m4a audio file though.
In the current program, when I load http://localhost:8880/mp3.html, it works fine. It shows the audio player and the mp3 plays. In addition, the program prints both "/mp3.html" and "/test.mp3".
However, when I load http://localhost:8880/m4a.html, it doesn't work. It shows the audio player, but the m4a doesn't play. In addition, the program prints only "/m4a.html" and not "/test.m4a".
My current code is below.
import urlparse
import os
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.resource import Resource
from twisted.web.static import File
import time
import subprocess
import mimetypes
class playM4A(Resource):
isLeaf = True
def render_GET(self, request):
this=urlparse.urlparse(request.path)#scheme,netloc,path,query
root,ext=os.path.splitext(this.path)
filename=os.path.basename(request.path)
fileFolder=request.path.replace(filename,"")
self.serverRoot=os.getcwd()
print request.path
if ext==".m4a":
thisFile=File(self.serverRoot+request.path)
return File.render_GET(thisFile,request)
elif ext==".mp3":
thisFile=File(self.serverRoot+request.path)
return File.render_GET(thisFile,request)
elif filename=="m4a.html":
return """
<html>
<audio controls>
<source src="http://localhost:8880/test.m4a" type="audio/mp4a-latm">
Your browser does not support the audio element.
</audio>
not m4a </html>"""
elif filename=="mp3.html":
return """
<html>
<audio controls>
<source src="http://localhost:8880/test.mp3" type="audio/mp3">
Your browser does not support the audio element.
</audio>
not m4a </html>"""
resource = playM4A()
factory = Site(resource)
reactor.listenTCP(8880, factory)
reactor.run()
The code works if you change audio/mp4a-latm to audio/mp4
Really simple, I want to create a temporary html page that I display with the usual webbrowser.
Why does the following code produce an empty page?
import tempfile
import webbrowser
import time
with tempfile.NamedTemporaryFile('r+', suffix = '.html') as f:
f.write('<html><body><h1>Test</h1></body></html>')
webbrowser.open('file://' + f.name)
time.sleep(1) # to prevent the file from dying before displayed
Because your file doesn't exist on the disk and sits entirely in memory. That's why the browser starts but opens nothing since no code has been provided.
Try this:
#!/usr/bin/python
import tempfile
import webbrowser
tmp=tempfile.NamedTemporaryFile(delete=False)
path=tmp.name+'.html'
f=open(path, 'w')
f.write("<html><body><h1>Test</h1></body></html>")
f.close()
webbrowser.open('file://' + path)
Tested on Python 3.4.
import subprocess
import webbrowser
from http.server import BaseHTTPRequestHandler, HTTPServer
PORT = 7000
HOST = '127.0.0.1'
SERVER_ADDRESS = '{host}:{port}'.format(host=HOST, port=PORT)
FULL_SERVER_ADDRESS = 'http://' + SERVER_ADDRESS
def TemproraryHttpServer(page_content_type, raw_data):
"""
A simpe, temprorary http web server on the pure Python 3.
It has features for processing pages with a XML or HTML content.
"""
class HTTPServerRequestHandler(BaseHTTPRequestHandler):
"""
An handler of request for the server, hosting XML-pages.
"""
def do_GET(self):
"""Handle GET requests"""
# response from page
self.send_response(200)
# set up headers for pages
content_type = 'text/{0}'.format(page_content_type)
self.send_header('Content-type', content_type)
self.end_headers()
# writing data on a page
self.wfile.write(bytes(raw_data, encoding='utf'))
return
if page_content_type not in ['html', 'xml']:
raise ValueError('This server can serve only HTML or XML pages.')
page_content_type = page_content_type
# kill a process, hosted on a localhost:PORT
subprocess.call(['fuser', '-k', '{0}/tcp'.format(PORT)])
# Started creating a temprorary http server.
httpd = HTTPServer((HOST, PORT), HTTPServerRequestHandler)
# run a temprorary http server
httpd.serve_forever()
if __name__ == '__main__':
def run_xml_server():
xml_data = """
<note>
<to>Tove</to>
<from>Jani</from>
<heading>Reminder</heading>
<body>Don't forget me this weekend!</body>
</note>
"""
# open in a browser URL and see a result
webbrowser.open(FULL_SERVER_ADDRESS)
# run server
TemproraryHttpServer('xml', xml_data)
def run_html_server():
html_data = """
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>
<h1>This is a Heading</h1>
<p>This is a paragraph.</p>
</body>
</html>
"""
# open in a browser URL and see a result
webbrowser.open(FULL_SERVER_ADDRESS)
# run server
TemproraryHttpServer('html', html_data)
# choice needed server:
# run_xml_server()
# run_html_server()
Just change the file's current position.
import tempfile
import webbrowser
import time
with tempfile.NamedTemporaryFile('r+', suffix = '.html') as f:
f.write('<html><body><h1>Test</h1></body></html>')
webbrowser.open('file://' + f.name)
f.seek(0)
time.sleep(1) # to prevent the file from dying before displayed
This piece of code is an edition to the previews one and continuation of the discussion with the OP. It shows time.sleep() after webbrowser.open(). I don't think it's actually needed because the /tmp directory is emptied automatically on a regular bases by the OS but the OP commented that if he deletes the temp file via Python it is deleted before it gets fully loaded by the browser. Most likely it happens because the "browser" process is detached from this scrip which is its parent and Python doesn't wait for the process completion before executing the next statement. I though it would be clear for everyone without explanation but obviously you, guys, don't read comments.
import os, time
# ...
webbrowser.open('file://' + path)
time.sleep(1)
if os.path.exists(path):
os.remove(path)