Greeting,
I was working on a drone project, I wanted to take stream from my drone process it on my laptop and give a command based on processing, I was using the flask framework from the same.
Currently, as the first step I want to take the stream from drone and PUT it to the flask server and view it on the flask website, not doing the processing part right now.
I PUT the video to server after compressing it into jpg and using base 64 to encode it and then finally use json.dumps() and then requests.put() it.
On the server side in flask server program I get its using request.json, use json.loads(), but I am not clear what to do next.
I am not experienced enough with flask, web development and with limited experience and knowledge made the programs, but it returns error 405 on the flask program.
Here are the programs
flask server
import base64
import json
from flask import Flask, make_response, render_template, request
app = Flask(__name__)
def getFrames(img):
pass
#app.route('/video', methods=['POST', 'GET'])
def video():
if request.method == 'PUT':
load = json.loads(request.json)
imdata = base64.b64decode(load['image'])
respose = make_response(imdata.tobytes())
return respose
#app.route('/')
def index():
return render_template('index.html')
#app.route('/cmd')
def cmd():
pass
if __name__ == "__main__":
app.run(debug=True)
index.html
<!DOCTYPE html>
<html>
<head>
<title>Video Stream</title>
</head>
<body>
<h1>
Live Stream
</h1>
<div>
<img src="{{ url_for('video') }}" width="50%">
</div>
</body>
</html>
drone program
import base64
import json
import requests
import cv2
cap = cv2.VideoCapture(1)
ip = '' #url returned by the flask program
while True:
success, img = cap.read()
cv2.imshow("OUTPUT", img)
_, imdata = cv2.imencode('.JPG', img)
jStr = json.dumps({"image": base64.b64encode(imdata).decode('ascii')})
requests.put(url=(ip + '/video'), data=jStr)
if cv2.waitKey(1) == 27:
break
Any help is highly appreciated!!!
You don't have to convert to base64 and use JSON. it can be simpler and faster to send JPG directly as raw bytes
And to make it simpler I would use /upload to send image from drone to server, and /video to send image to users.
import requests
import cv2
cap = cv2.VideoCapture(0)
while True:
success, img = cap.read()
if success:
cv2.imshow("OUTPUT", img)
_, imdata = cv2.imencode('.JPG', img)
print('.', end='', flush=True)
requests.put('http://127.0.0.1:5000/upload', data=imdata.tobytes())
# 40ms = 25 frames per second (1000ms/40ms),
# 1000ms = 1 frame per second (1000ms/1000ms)
# but this will work only when `imshow()` is used.
# Without `imshow()` it will need `time.sleep(0.04)` or `time.sleep(1)`
if cv2.waitKey(40) == 27: # 40ms = 25 frames per second (1000ms/40ms)
break
cv2.destroyAllWindows()
cap.release()
Now flask. This part is not complete.
It gets image from drone and keep in global variable. And when user open page then it loads single image from /video
from flask import Flask, make_response, render_template, request
app = Flask(__name__)
frame = None # global variable to keep single JPG
#app.route('/upload', methods=['PUT'])
def upload():
global frame
# keep jpg data in global variable
frame = request.data
return "OK"
#app.route('/video')
def video():
if frame:
return make_response(frame)
else:
return ""
#app.route('/')
def index():
return 'image:<br><img src="/video">'
if __name__ == "__main__":
app.run(debug=True)
At this moment it can display only one static image. It needs to send it as motion-jpeg
EDIT:
Version which sends motion-jpeg so you see video.
It works correctly with Chrome, Microsoft Edge and Brave (all use chrome engine).
Problem makes Firefox. It hangs and tries to load image all time. I don't know what is the real problem but if I add time.sleep() then it can solve problem.
from flask import Flask, Response, render_template_string, request
import time
app = Flask(__name__)
frame = None # global variable to keep single JPG,
# at start you could assign bytes from empty JPG
#app.route('/upload', methods=['PUT'])
def upload():
global frame
# keep jpg data in global variable
frame = request.data
return "OK"
def gen():
while True:
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n'
b'\r\n' + frame + b'\r\n')
time.sleep(0.04) # my Firefox needs some time to display image / Chrome displays image without it
# 0.04s = 40ms = 25 frames per second
#app.route('/video')
def video():
if frame:
# if you use `boundary=other_name` then you have to yield `b--other_name\r\n`
return Response(gen(), mimetype='multipart/x-mixed-replace; boundary=frame')
else:
return ""
#app.route('/')
def index():
return 'image:<br><img src="/video">'
#return render_template_string('image:<br><img src="{{ url_for("video") }}">')
if __name__ == "__main__":
app.run(debug=True)#, use_reloader=False)
Server may runs users in separated threads on processes and sometimes it may not share frame between users. If I use use_reloader=False then I can stop sending to /upload and this stops video in browser, and later I can start again sending to /upload and browser again displays stream (without reloading page). Without use_reloader=False browser doesn't restart video and it needs to reload page. Maybe it will need to use flask.g to keep frame. Or /upload will have to save frame in file or database and /video will have to read frame from file or database.
Related
I am currently trying to use OpenCV with Python to load a video from a url onto a localhost webpage. The loaded video is a little choppy but the main problem is that it stops reading the video frames after a while and displays the following error message.
[h264 # 0955e140] error while decoding MB 87 29, bytestream -5
[h264 # 0955e500] left block unavailable for requested intra4x4 mode -1
[h264 # 0955e500] error while decoding MB 0 44, bytestream 126
Debugging middleware caught exception in streamed response at a point where response headers were already sent.
Traceback (most recent call last):
File "C:\Users\\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\werkzeug\wsgi.py", line 506, in __next__
return self._next()
File "C:\Users\\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\werkzeug\wrappers\base_response.py", line 45, in _iter_encoded
for item in iterable:
File "C:\Users\\Downloads\VideoStreamingFlask\main.py", line 12, in gen
frame = camera.get_frame()
File "C:\Users\\Downloads\VideoStreamingFlask\camera.py", line 13, in get_frame
ret, jpeg = cv2.imencode('.jpg', image)
cv2.error: OpenCV(4.3.0) C:\projects\opencv-python\opencv\modules\imgcodecs\src\loadsave.cpp:919: error: (-215:Assertion failed) !image.empty() in function 'cv::imencode'
Code
main.py
from flask import Flask, render_template, Response
from camera import VideoCamera
app = Flask(__name__)
#app.route('/')
def index():
return render_template('index.html')
def gen(camera):
while True:
frame = camera.get_frame()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
#app.route('/video_feed')
def video_feed():
return Response(gen(VideoCamera()),
mimetype='multipart/x-mixed-replace; boundary=frame')
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
camera.py
import cv2
class VideoCamera(object):
def __init__(self):
self.video = cv2.VideoCapture(*url*)
def __del__(self):
self.video.release()
def get_frame(self):
success, image = self.video.read()
ret, jpeg = cv2.imencode('.jpg', image)
return jpeg.tobytes()
Questions
What might be causing the problem here?
How do I make the video less choppy?
The crash in your python happened because video.read() failed. Therefore, image can not be passed to cv2.imencode(). You should check the success value in get_frame(self) and be prepared that sometimes camera.get_frame() will not return a valid Jpeg.
Now, let's understand why video.read() failed in this case. This could happen if the connection to the camera was not good enough and some packets got lost. But more likely, your VideoCapture was not fast enough to handle the video stream.
This could be improved if you reduce the work that the video capture thread is doing. As suggested in another discussion, offload processing to a separate thread.
Currently, your flask server listens to camera stream, converts it to a series of Jpegs, and sends them to the client over HTTP. If you have a thread dedicated to camera stream, you may find that your server cannot pass every video frame, because the encoder and HTTP transport are too slow. So, some frames will be skipped.
Here is a detailed article about video streaming with flask: https://blog.miguelgrinberg.com/post/flask-video-streaming-revisited. You can find some other open-source projects that stream video to browser, not necessarily with opencv and python.
I am able to send the frame through web API using below script:
import cv2
import requests
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
cv2.imshow('frame', frame)
#frame_im = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
print(type(frame))
frame_in = cv2.imencode('.jpg', frame)
headers = {'Content-Type': 'image/jpg'}
files = {'form': frame_in}
#img_files = urlopen(frame_in)
response = requests.post(
url="http://127.0.0.1:5000/test",
data=files,
headers=headers
)
if cv2.waitKey(1) and 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
I am sending the data to API using mentioned below Script and trying to recieve every frame and save it in some folder
from flask import Flask, redirect, url_for, request, Response, render_template, send_file,jsonify
import base64
app = Flask(__name__)
#app.route('/test', methods=['POST'])
def test():
if request.method == 'POST':
body = request.data
#print(request.form)
print(body)
print("Test Revealed")
return jsonify({"hi":"hi"})
if __name__ == '__main__':
#app.secret_key = os.urandom(12)
app.run(host='127.0.0.1', port=5000, threaded= True)
And got Results with huge line of as mentioned below :
=%5B39%5D&form=%5B13%5D&form=%5B146%5D&form=%5B192%5D&form=%5B117%5D&form=%5B24%5D&form=%5B206%5D&form=%5B70%5D&form=%5B106%5D&form=%5B121%5D&form=%5B235%5D&form=%5B41%5D&form=%5B114%5D&form=%5B180%5D&form=%5B215%5D&form=%5B98%5D&form=%5B84%5D&form=%5B212%5D&form=%5B149%5D&form=%5B154%5D&form=%5B63%5D&form=%5B255%5D&form=%5B217%5D'
Test Revealed
127.0.0.1 - - [07/Jan/2020 10:09:42] "POST /test HTTP/1.1" 200 -
I want to convert the format which I am receiving but I am still able to understand how to convert the data and save it as an image file.
Suggestion will be realy helpful here
I want to send and receive images in bytes in my flask API. I also want to send some json alongside the image. How can I achieve this?
Below is my current solution that does not work
flask:
#app.route('/add_face', methods=['GET', 'POST'])
def add_face():
if request.method == 'POST':
# print(request.json)
nparr = np.fromstring(request.form['img'], np.uint8)
print(request.form['img'])
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
cv2.imshow("frame", img)
cv2.waitKey(1)
return "list of names & faces"
client:
def save_encoding(img_file):
URL = "http://localhost:5000/add_face"
img = open(img_file, 'rb').read()
response = requests.post(URL, data={"name":"obama", "img":str(img)})
print(response.content)
produced error:
cv2.imshow("frame", img)
cv2.error: OpenCV(3.4.3) /io/opencv/modules/highgui/src/window.cpp:356: error: (-215:Assertion failed) size.width>0 && size.height>0 in function 'imshow'
The following has worked for me.
I don't have the client code but I have a curl request. That should do the trick,
SERVER
from flask import request
from PIL import Image
import io
#app.route("/add_face", methods=["POST"])
def predict():
image = request.files["image"]
image_bytes = Image.open(io.BytesIO(image.read()))
CLIENT SIDE
curl -X POST -F image=#PATH/TO/FILE 'http://localhost:5000/add_face'
It is easier to send images in base64 format, by doing that you get rid of problems about sending/receiving binary data since you just work with a string. Also it is more convenient in web stuff. Tested code below:
Server side:
from flask import Flask, render_template, request
import pandas as pd
import cv2
import numpy as np
import base64
app = Flask(__name__)
#app.route('/add_face', methods=['GET', 'POST'])
def add_face():
if request.method == 'POST':
# read encoded image
imageString = base64.b64decode(request.form['img'])
# convert binary data to numpy array
nparr = np.fromstring(imageString, np.uint8)
# let opencv decode image to correct format
img = cv2.imdecode(nparr, cv2.IMREAD_ANYCOLOR);
cv2.imshow("frame", img)
cv2.waitKey(0)
return "list of names & faces"
if __name__ == '__main__':
app.run(debug=True, port=5000)
Client side:
import requests
import base64
URL = "http://localhost:5000/add_face"
# first, encode our image with base64
with open("block.png", "rb") as imageFile:
img = base64.b64encode(imageFile.read())
response = requests.post(URL, data={"name":"obama", "img":str(img)})
print(response.content)
You can use COLOR instead of ANYCOLOR if you are sure about your input images.
I have a webpage where the user uploads an image and my python code will make some adjustments to it and do some analysis. I want the newly generated image to be displayed to the user on the webpage as soon as it's generated, and then continue doing the analysis that needs to be done, and then update the webpage with that information. However, I am not sure how to communicate from flask to the webpage halfway through the function (once the new image is generated) that the website can display the newly generated image, as using render_template can only be done at the end of the function.
My python (flask) code is as follows:
#app.route('/uploaded', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
filename = secure_filename(file.filename)
f = request.files['file']
f.save(secure_filename(f.filename))
image = cv2.imread(filename)
im = Image.fromarray(image)
# make some adjustments to the image (not shown here) and then save it...
im.save(os.path.join(app.config['UPLOAD_FOLDER'], 'your_file.jpg'))
# after the new image is generated, display it on the website at {{ place_for_generated_image }}
# do some more analysis, then finally:
return render_template('index.html', analysis = analysis)
HTML is straightforward:
<form action = "http://localhost/uploaded" method = "POST"
enctype = "multipart/form-data">
<input type = "file" name = "file" class="form-control-file">
<input type = "submit" class="btn btn-info" value="Upload Image" button id="uploadfile" onclick="uploadfile()">
</form>
{{ place_for_generated_image }}
{{ analysis }}
You need to use multiple ajax calls to achieve that. Something like the below:
First make a route to handle the image upload
#app.route('/uploaded', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
filename = secure_filename(file.filename)
f = request.files['file']
f.save(secure_filename(f.filename))
image = cv2.imread(filename)
im = Image.fromarray(image)
# make some adjustments to the image (not shown here) and then save it...
im.save(os.path.join(app.config['UPLOAD_FOLDER'], 'your_file.jpg'))
# Here you need to convert your image to a base64 string and return that string to your ajax call
# do some more analysis, then finally:
return 'YOUR BASE64 ENCODED IMAGE STRING'
This route will handle your analysis. Your second nested ajax call will communicate with this route
from flask import Response
#app.route('/analyze', methods=['GET', 'POST'])
def analyze():
# run some analysis
return Response(status=200, response="Anlysis Done")
This what your javascript code should look like. You can place it in a script tag in your template. If you place it in a separate JS file, make sure to look at my comment for the url_for in the second ajax call
$('form').submit(function(e){
e.preventDefault();
$.ajax({
url: $(this).attr('action'),
type: 'POST',
data: $(this).serialize()
}).done(function(res){
// Here you write code that will display the returned base64 image
// from the upload_file route Just update your img tag src to the
// base64 string you received as the response
// SECOND AJAX CALL TO RUN THE ANALYTICS
$.ajax({
url: "{{url_for('analysis')}}", // if this JS code in a separate JS file you will have to declare a URL variable in a script tag in your template as {{url_for}} is only accessible from your template
type: 'POST',
data: 'any data you want to send can be in JSON format'
}).done(function(res){
// analysis is done
})
})
})
I'm trying to listen for a file being sent over post with the key 'image', then return it back to the caller.
from flask import Flask, request, make_response
app = Flask(__name__)
#app.route('/', methods = ['POST', 'GET'])
def img():
if request.method == 'GET':
return "yo, you forgot to post"
else:
img = request.files['image']
resp = make_response(img)
resp.status_code = 200
return resp
if __name__ == "__main__":
app.debug = True
app.run(host='0.0.0.0')
This fails with the message:
TypeError: 'FileStorage' object is not callable
I'm not massively familiar with Python, but I thought by doing the above that I was passing the byte[] straight through memory?
How do you achieve this without it failing? I want to take it the image blob from the request, manipulate it (not currently), and return it straight back out. In memory, without saving it to the file system.
You can serve out the image again with the Flask.send_image() function; it accepts a file-like object as well as filenames. You'll have to copy the file data over to an in-memory file object (the request.files file objects are closed too early to be suitable):
from io import BytesIO
return send_file(BytesIO(request.files['image'].read())
You may want to add additional headers, such as the mime-type, to the response as the send_file() function has less information to go on here than with 'regural' file objects. The sending browser probably included a content type for example:
import mimetypes
img = request.files['image']
mimetype = img.mimetype or mimetypes.guess_type(img.filename)[0]
return send_file(BytesIO(img.read()), mimetype=mimetype)
This does require you to hold the whole image in memory.