I am trying to upload a video file from an Unity client to a python server, but when I have tried to do it with UnityWebRequest in Unity and http module in python, but the server receives invalid video file.
My question is how to upload a video file from Unity to python over Http?
This is my code in Unity:
IEnumerator StartUploadCoroutine()
{
// Show a load file dialog and wait for a response from user
yield return FileBrowser.WaitForLoadDialog(false, null, "Load File", "Load");
isOpen = false;
// Upload File to movie server
if (FileBrowser.Success)
{
StreamReader reader = new StreamReader(FileBrowser.Result);
StartCoroutine(UploadCoroutine(reader)); // upload file to server;
}
}
/* Upload the chosen video file to the movie server */
IEnumerator UploadCoroutine(StreamReader reader)
{
UnityWebRequest www = UnityWebRequest.Post(videoPlayerManager.GetServerIp() + ":" + port.ToString(), reader.ReadToEnd());
yield return www.SendWebRequest();
if (www.isNetworkError || www.isHttpError)
{
Debug.Log(www.error);
}
else
{
Debug.Log("Form upload complete!");
}
}
And this is my Python code:
from http.server import BaseHTTPRequestHandler, HTTPServer
class HandleRequests(BaseHTTPRequestHandler):
def do_POST(self):
'''Reads post request body'''
content_length = int(self.headers['Content-Length'])
body = self.rfile.read(content_length)
self.send_response(200)
self.end_headers()
outF = open("myOutFile.mp4", "wb")
outF.write(body)
host = "127.0.0.1"
port = 9999
HTTPServer((host, port), HandleRequests).serve_forever()
I think in uploading a file (any extension) to a server, Encodings are not important.
Look for two things:
Check that your local video file and uploaded video file have the same Size. if they don't match, it means that you have a problem with uploading progress. (Check here)
You are using POST method to upload the video. It means that the whole of the file will upload to your server and then your python script can use it on 'body' variable. I files size match, I can suggest you use a framework like Flask as server-side python script.
I figured it out thanks to > MohammadReza Arashiyan
My problem was sending the file from Unity as string instead of a bytes array.
I have changed my python code to Flask server so I could easily get the file from the post request, and I ended up with these codes:
This is my code in Unity:
IEnumerator StartUploadCoroutine()
{
// Show a load file dialog and wait for a response from user
yield return FileBrowser.WaitForLoadDialog(false, null, "Load File", "Load");
isOpen = false;
// Upload File to movie server
if (FileBrowser.Success)
{
StartCoroutine(UploadCoroutine(FileBrowser.Result)); // upload file to server;
}
}
/* Upload the chosen video file to the movie server */
IEnumerator UploadCoroutine(string filePath)
{
WWWForm form = new WWWForm();
form.AddBinaryData("vidFile", File.ReadAllBytes(filePath));
UnityWebRequest www = UnityWebRequest.Post(videoPlayerManager.GetServerUrl(), form);
yield return www.SendWebRequest();
if (www.isNetworkError || www.isHttpError)
{
Debug.Log(www.error);
}
else
{
Debug.Log("Form upload complete!");
}
}
And this is my Python code:
import os
from flask import Flask, request, send_from_directory
from gevent.pywsgi import WSGIServer
IP = "127.0.0.1"
PORT = 9999
# set the project root directory as the static folder
app = Flask(__name__)
#app.route('/', methods=['POST'])
def DownloadFile():
# request.form to get form parameter
vidFile = request.files["vidFile"].read()
outF = open("myOutFile.mp4", "wb")
outF.write(vidFile)
return ''
if __name__ == "__main__":
http_server = WSGIServer((IP, PORT), app)
http_server.serve_forever()
Related
I want to upload a .wav file from react frontend to node server and then send the file to a python file for speech recognition.
I used multer to get the file in the post route. The file I get looks like this.
{
"fieldname": "file",
"originalname": "00f0204f_nohash_0.wav",
"encoding": "7bit",
"mimetype": "audio/wave",
"destination": "./public/",
"filename": "IMAGE-1635358708022.wav",
"path": "public\\IMAGE-1635358708022.wav",
"size": 32044
}
Now I want to fork a child process for python from index.js and want to sent the file for ASR.
The python file code looks like this:
import speech_recognition as sr
def read_in():
lines = sys.stdin.readlines()
return json.loads(lines[0])
def main():
a = read_in()
r = sr.Recognizer()
with sr.AudioFile(a) as source:
audio_text = r.listen(source)
try:
text = r.recognize_google(audio_text)
print('Conversion Started')
print(text)
except:
print('Error Occurred - Try Again')
How should I send the uploaded file from node to this python file for computation? I am new at this, so I am really confused.
if you are post processing the file, then get the file content with REST API:
# Postman generated
import http.client
host = "ur ip"
path = "server path:port" # port match as the express server
def webhooktriggered(path): # i haven't done any project about py web before, so goodluck
conn = http.client.HTTPSConnection(host)
payload = ''
headers = {}
conn.request("GET", path, payload, headers)
res = conn.getresponse()
data = res.read()
return (data.decode("utf-8"))
before, stream your media with express:
// Original: https://www.codegrepper.com/code-examples/javascript/node+js+express+sendfile+fs+filestream
var http = require('http'),
fileSystem = require('fs'),
path = require('path');
http.createServer(function(request, response) {
var filePath = path.join(__dirname, 'pathto/yourfilehere.wav');
var stat = fileSystem.statSync(filePath);
response.writeHead(200, {
'Content-Type': 'audio/x-wav', // change to whatever u want
'Content-Length': stat.size
});
var readStream = fileSystem.createReadStream(filePath);
// We replaced all the event handlers with a simple call to readStream.pipe()
readStream.pipe(response);
})
.listen(2000); //as soon we will use this port in rest client
Trigger when process, using axios
const axios = require('axios')
function uploadSuccess(path){ // must be the streamed express media path, so python rest can download it then
axios.get('http://pythonbackserverip/?path=' + encodeURI(path))
.then({
console.log("upload successfully!")
})
}
note: my code is not accurate, modify it before use
I am trying to download a file stored in Azure Blob Storage with my react app using SAS and am running into issues. I have a working version that relies on having the flask app download the file, then sending the blob to the react app to be downloaded again (obviously not ideal). Here's the current implementation:
flask endpoint:
from azure.storage.blob import BlobServiceClient
blobService = BlobServiceClient(account_url="https://<account_name>.blob.core.windows.net/", credential=<blob_key>)
#app.route('/download')
def downloadFile():
filename = request.args.get('filename')
blob_client = blobService.get_blob_client(container='<container_name>', blob=filename)
blobObject = blob_client.download_blob()
fileObject = io.BytesIO(blobObject.readall())
return send_file(fileObject, attachment_filename=filename, as_attachment=True)
get request in react:
const getFile = (e) => {
e.preventDefault();
const filename = e.currentTarget.getAttribute('name');
axios({
url: `${serverUrl}/download?filename=${filename}`,
method: 'GET',
responseType: 'blob',
})
.then(({ data }) => {
const link = document.createElement('a');
const url = URL.createObjectURL(new Blob([data]));
link.href = url;
link.setAttribute('download', filename);
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
})
.catch(() => {
_isMounted.current && setDisplayError(true);
});
};
I would like to be able to just have my react app download the file direct from blob storage, but am running into authentication issues, along with another issue where clicking the download button navigates the browser to the url rather than just downloading the file at the location while staying on the current page. Here is the new code with the issues.
new flask endpoint:
from azure.storage.blob._shared_access_signature import BlobSharedAccessSignature
signatureService = BlobSharedAccessSignature('<account_name>', account_key='<azure_key>')
#app.route('/download')
def downloadFile():
filename = request.args.get('filename')
expiration = datetime.datetime.today() + datetime.timedelta(minutes=5)
container = '<container_name>'
key = signatureService.generate_blob(container, filename, permission='read', expiry=expiration)
data = {
'container': container,
'key': key
}
return app.response_class(
response=jsonifyWithNumPy(data),
status=200,
mimetype='application/json'
)
get request in react:
const getFile = (e) => {
e.preventDefault();
const filename = e.currentTarget.getAttribute('name');
axios
.get(`${serverUrl}/download?filename=${filename}`)
.then(({data}) => {
const url = `https://<account_name>.blob.core.windows.net/${data.container}/${filename}?${data.key}`
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', filename);
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
})
.catch(() => {
_isMounted.current && setDisplayError(true);
});
};
This is the error I get when I follow the URL generated by the above react code:
<Error>
<Code>AuthenticationFailed</Code>
<Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:48c49456-001e-008b-263c-3625fd000000 Time:2021-04-20T23:23:26.1909093Z</Message>
<AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail>
</Error>
I tried your code to create a blob SAS key and get the same error, just try the code below that works for me to create a blob SAS key and access a blob successfully:
from azure.storage.blob import generate_blob_sas,BlobSasPermissions
from datetime import datetime, timedelta
account = ''
container = ''
blob = ''
account_key = ''
permission=BlobSasPermissions(read=True)
exp = datetime.utcnow() + timedelta(hours=1)
key = generate_blob_sas(account_name=account,container_name=container,blob_name=blob,account_key=account_key,permission=permission,expiry=exp)
This question already has answers here:
Configure Flask dev server to be visible across the network
(17 answers)
Closed 1 year ago.
I've created a simple flutter app which will get data from an api in json format sent from the backend (Python flask) and print out the data on click of a button.
When I click on the button, I get the error
[ERROR:flutter/lib/ui/ui_dart_state.cc(177)]
Unhandled Exception: SocketException: OS Error: Connection refused, errno = 111, address = 127.0.0.1, port = 36820
I'm very new to flutter, please tell me where I did wrong and help me figure it out.
Flutter Dart code:
import 'dart:async';
import 'dart:convert';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
void main() {
runApp(new MaterialApp(
home: new HomePage(),
));
}
class HomePage extends StatefulWidget {
#override
HomePageState createState() => new HomePageState();
}
class HomePageState extends State<HomePage> {
List data;
Future<String> getData() async {
var response = await http.get(
Uri.encodeFull("http://127.0.0.1:5000/ "),
headers: {
"Accept": "application/json"
}
);
data = json.decode(response.body);
print(data);
return "Success!";
}
#override
Widget build(BuildContext context) {
return new Scaffold(
body: new Center(
child: new RaisedButton(
child: new Text("Get data"),
onPressed: getData,
),
),
);
}
}
Python code:
from flask import Flask, jsonify
app = Flask(__name__)
#app.route('/')
def hello_world():
return jsonify({"about": "Hello World!"})
if __name__ == '__main__':
app.run()
Looks like you are running your back-end from localhost. In this case, 127.0.0.1 will only work if you run the application on your PC (and not another phone/device. Not sure about simulators though).
Trying to access 127.0.0.1 from a phone is like trying to access a server hosted on your phone (which evidently will not be found).
So, to run the app on your phone, you need to find your PC's (private) IP address, using e.g. ifconfig from your terminal. Once you get this IP address, use the URL {your-priv-IP}:5000 instead of 127.0.0.1:5000 in your code. Make sure though that your PC and phone are connected to the same Wifi network for this to work.
I'm trying to send a group of .mp4 files first from a web browser client, then to express, then to flask. This code might look simple, but it has taken literally days to get this far, because nobody on the internet has posted an example of how to do this with large files (.mp4 video files), and to be honest, there are a lot of inconsistencies in using these libraries.
I think it might be an encoding error from express to Flask, however. The actual data seems to be being transferred, but ffmpeg doesn't recognize the received .mp4 files as a valid video file.
From the client, I send an XHR request like this:
var formData = new FormData();
for(var x=0; x< datas.videoDatas.length; x++){
// data.videoDatas[x] is a blob of a .mp4 file. it can be played
formData.append(x+"_video.mp4",datas.videoDatas[x]);
}
var xhr = new XMLHttpRequest();
xhr.open("POST", 'http://localhost:3000/uploadVid', true);
xhr.onreadystatechange = function() {
if(this.readyState == XMLHttpRequest.DONE && this.status == 200) {
console.log(xhr.response);
}
}
xhr.onload=console.log;
xhr.send(formData);
The express server at port 3000 receives the post request using the "multer" library. Data is stored into memory as a buffer.
var express = require('express');
var multer=require("multer");
var request=require("request")
var app = express();
var storage = multer.memoryStorage();
const upload = multer({
storage: storage
}).any();
app.post('/uploadVid',function(req,res){
var originalResponse=res;
console.log("begin upload");
upload(req, res, function(err) {
console.log("begin upload to flask server");
var requestStruct={
url: 'http://localhost:3001/receiveUpload2?numFiles='+req.files.length,
method: 'POST',
form: {
}
}
for(var x=0; x < req.files.length; x++){
var testBuffer=req.files[x].buffer; // should be a buffer of the .mp4 file
//testBuffer = new Buffer(10);
//testBuffer=testBuffer.toString();
requestStruct.form[x+'_customBufferFile']= {
value: testBuffer,
options: {
filename: req.files[x].fieldname
}
}
}
request(requestStruct, function(err,res,body){
originalResponse.send(body)
});
});
});
http.createServer(app).listen(3000)
In the python flask server, the .mp4 files are received as a buffer and written to a .mp4 file. However, the .mp4 file is unplayable by ffmpeg "Unknown EBML doctype '(none)'0/0 0_video.mp4: End of file"
from flask import Flask
from flask import request;
app = Flask(__name__);
#app.route('/receiveUpload2', methods=["POST"])
def receiveUpload2():
print("uploaded received....");
numFiles=int(request.args.get("numFiles").encode('ascii', 'ignore'));
print("numFiles",int(numFiles));
for x in range(0,numFiles):
name=request.form.getlist(str(x)+'_customBufferFile[options][filename]');
name=name[0];
print("writing",name);
filebytes=request.form.getlist(str(x)+'_customBufferFile[value]');
filebytes=filebytes[0];
#print(filebytes);
newFileByteArray = bytearray(filebytes,'utf8')
with open("./uploads/"+name,"wb") as output:
output.write(newFileByteArray);
app.run(host="localhost", port=3001);
I am trying nodejs for the first time. I am using it with python shell. I am trying to transfer a file from one PC to another using Post request
app.js (Server PC)
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.post('/mytestapp', function(req, res) {
console.log(req)
var command = req.body.command;
var parameter = req.body.parameter;
console.log(command + "|" + parameter)
pyshell.send(command + "|" + parameter);
res.send("POST Handler for /create")
});
python file send file from (Client PC)
f = open(filePath, 'rb')
try:
response = requests.post(serverURL, data={'command':'savefile'}, files={os.path.basename(filePath): f})
I use fiddler and the request seems to contain the file on Client PC, but I can't seem to get the file on Server PC. How can I extract and save the file? Is it because I am missing headers? what should I use? thanks
I'm going to guess and say you're using Express based on the syntax in your question. Express doesn't ship with out of the box support for file uploading.
You can use the multer or busboy middleware packages to add multipart upload support.
Its actually pretty easy to do this, here is a sample with multer
const express = require('express')
const bodyParser = require('body-parser')
const multer = require('multer')
const server = express()
const port = process.env.PORT || 1337
// Create a multer upload directory called 'tmp' within your __dirname
const upload = multer({dest: 'tmp'})
server.use(bodyParser.json())
server.use(bodyParser.urlencoded({extended: true}))
// For this route, use the upload.array() middleware function to
// parse the multipart upload and add the files to a req.files array
server.port('/mytestapp', upload.array('files') (req, res) => {
// req.files will now contain an array of files uploaded
console.log(req.files)
})
server.listen(port, () => {
console.log(`Listening on ${port}`)
})