I want to store a file from a Flask service to a Minio Bucket without storing it in a directory first. What I can do so far is have the user upload a file using a Flask service, store it in a directory, and then use the put_object function from Minio to upload it.
Is something like that possible?
Below you can see the upload_file flask method and the upload_object method that utilizes the put_object function from Minio:
UPLOAD_FOLDER = 'uploads'
ALLOWED_EXTENSIONS = {'txt', 'pdf', 'png', 'jpg', 'jpeg', 'gif', '7z', 'iso'}
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/', methods=['GET','POST'])
def upload_file():
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return 'No file found'
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return 'msg error'
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save('uploads/'+filename)
print(file.filename)
# upload file to minio service
upload_object(name=str(file.filename))
return 'object stored in bucket'
return '''
<!doctype html>
<title>Upload IPS File</title>
<h1>Upload IPS File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
And the upload_object():
def upload_object(name):
try:
path = 'uploads/'+str(name)
print(path)
file_data = open(path, 'rb')
print(file_data)
file_stat = os.stat('uploads/'+str(name))
print(minioClient.put_object('miniotestbucket', str(name), file_data, file_stat.st_size))
except ResponseError as e:
print(e)
except FileNotFoundError as e:
print(e)
I already checked this Question but I did not quite understand what was needed to do.
It should absolutely be possible since you just read the file content after saving it. This post tells a bit more about FileStorage. Since FileStorage extends stream you have many of the same features while reading the file from the request as when reading it from a file on disk. Since you cannot stat the file I used another way to tell the size of the file found here.
You can try something like this:
#app.route("/", methods=["GET", "POST"])
def upload_file():
if request.method == "POST":
uploaded_file = request.files["file"]
if uploaded_file:
bucket_name = "miniotestbucket"
size = os.fstat(uploaded_file.fileno()).st_size
minioClient.put_object(
bucket_name, uploaded_file.filename, uploaded_file, size
)
return "object stored in bucket"
return """
<!doctype html>
<title>Upload IPS File</title>
<h1>Upload IPS File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
"""
I don't have access to a Minio bucket but I did some testing by uploading a YAML file and reading the contents with yaml.load() which worked just fine.
Related
Im posting the following data to my flask image upload server. Problem is an image does not get saved. There are no errors in console.
I have tried decoding the image bytes and it seems to be a properly formed multipart request.
So why isn't the image uploading properly?
import os
from flask import Flask, flash, request, redirect, url_for, send_from_directory
import sys
UPLOAD_FOLDER = '/Users/admin/Desktop/uploads'
ALLOWED_EXTENSIONS = {'txt', 'pdf', 'png', 'jpg', 'jpeg', 'gif'}
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
print(request.get_data(), file=sys.stdout)
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
file.save(os.path.join(app.config['UPLOAD_FOLDER'], file.filename))
return redirect(url_for('uploaded_file',
filename=file.filename))
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/uploads/<filename>')
def uploaded_file(filename):
return send_from_directory(app.config['UPLOAD_FOLDER'], filename)
if __name__ == "__main__":
# Quick test configuration. Please use proper Flask configuration options
# in production settings, and use a separate file or environment variables
# to manage the secret key!
app.secret_key = 'super secret key'
app.config['SESSION_TYPE'] = 'filesystem'
# sess.init_app(app)
app.debug = True
app.run()
Data im posting: (I captured it using print(request.get_data(), file=sys.stdout) located in code)
https://pastecode.io/s/cpev4ny4
could you please verify if your apache or whatever you use has permission to write ( save an image) in that directory?
I started learning Flask by myself today, and while I was toying around with it I made a small program to download a file, and it worked great.
#app.route('/download')
def download():
path = "Path_to_my_file"
return send_file(path, as_attachment=True)
Now I was changing my program a bit and I removed the 'send_file' function and added some other commands. However, when I tried running the program, the download still happened. I have no idea why though.
My Code:
from flask import *
from flask_sqlalchemy import SQLAlchemy
from werkzeug.utils import secure_filename
import os
usera = ""
UPLOAD_FOLDER = 'I:/Python'
ALLOWED_EXTENSIONS = {'txt', 'pdf', 'png', 'jpg', 'jpeg', 'gif'}
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
app.config ['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///downloads.sqlite3'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
user = db.Column(db.String)
download = db.column(db.String)
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/upload', methods=['GET', 'POST'])
def upload_file():
global usera, db
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
db.session.add(User(user = usera, download = filename))
return render_template('lol.html')
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/login',methods = ['POST', 'GET'])
def login():
global db, user
if request.method == 'POST':
usera = request.form['user']
return render_template('lol.html')
#app.route('/download')
def download():
path = "I:/Python/"
users = User.query.all()
return users
if __name__ == '__main__':
app.run(host='0.0.0.0')
I had faced same issue where flask was returning old files (in your case it is still sending files even if code is changed). This issue is due to the fact that file is cached in browser. Clear the browser cache and your api will start working.
I am having trouble uploading a file to my Flask server using the Requests module for Python.
import os
from flask import Flask, request, redirect, url_for
from werkzeug import secure_filename
UPLOAD_FOLDER = '/Upload/'
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
#app.route("/", methods=['GET', 'POST'])
def index():
if request.method == 'POST':
file = request.files['file']
if file:
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('index'))
return """
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form action="" method=post enctype=multipart/form-data>
<p><input type=file name=file>
<input type=submit value=Upload>
</form>
<p>%s</p>
""" % "<br>".join(os.listdir(app.config['UPLOAD_FOLDER'],))
if __name__ == "__main__":
app.run(host='0.0.0.0', debug=True)
I am able to upload a file via web page, but I wanted to upload file with the requests module like this:
import requests
r = requests.post('http://127.0.0.1:5000', files={'random.txt': open('random.txt', 'rb')})
It keeps returning 400 and saying that "The browser (or proxy) sent a request that this server could not understand"
I feel like I am missing something simple, but I cannot figure it out.
Because you have <input> with name=file so you need
files={'file': ('random.txt', open('random.txt', 'rb'))}
Examples in requests doc: POST a Multipart-Encoded File
You upload the file as the random.txt field:
files={'random.txt': open('random.txt', 'rb')}
# ^^^^^^^^^^^^ this is the field name
but look for a field named file instead:
file = request.files['file']
# ^^^^^^ the field name
Make those two match; using file for the files dictionary, for example:
files={'file': open('random.txt', 'rb')}
Note that requests will automatically detect the filename for that open fileobject and include it in the part headers.
I'm following this tutorial on how to set up file uploading webservice using Flask. Below is a snipped of my code. I can see that the def upload_file(): gets the correct filename and allows it through, however, the line that actually saves the file does not get executed and I receive "Internal server error (500)" response. It seems that the issue is with actually storing files on disk.
What else do I need to do to set up my web service to save received files using Flask?
UPDATED - the code below works now
from flask import Flask
from flask import Response, jsonify, request, redirect, url_for
#persistent storage in dictionary
import shelve
#printing ip of host
import socket
#file uploading
import os
from werkzeug.utils import secure_filename
#serving files that were uploaded
from flask import send_from_directory
#mac-specific relative path to the script's location
UPLOAD_FOLDER = './audio'
ALLOWED_EXTENSIONS = set(['txt', 'm4a', 'mp4'])
#configuration
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1] in ALLOWED_EXTENSIONS
#app.route('/', methods=['GET', 'POST'])
def upload_file():
print 'upload file'
try:
os.stat(app.config['UPLOAD_FOLDER'])
except:
os.mkdir(app.config['UPLOAD_FOLDER'])
if request.method == 'POST':
file = request.files['file']
print 'filename: ' + file.filename
if file and allowed_file(file.filename):
print 'allowing file'
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('uploaded_file',filename=filename))
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form action="" method=post enctype=multipart/form-data>
<p><input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/uploads/<filename>')
def uploaded_file(filename):
return send_from_directory(app.config['UPLOAD_FOLDER'],
filename)
if __name__ == '__main__':
print(socket.gethostbyname(socket.gethostname()))
app.run(host='0.0.0.0')
I'm trying to make upload site that takes request and stores the files in 'static' folder in flask. I always get bad request when I try sending "POST" request on in.The only reason that I need it is to test my drag and drop javascript uploader. Can someone point me in the right direction?
import os
from flask import Flask, request, redirect, url_for
from werkzeug import secure_filename
UPLOAD_FOLDER = 'static'
ALLOWED_EXTENSIONS = set(['txt', 'pdf', 'png', 'jpg', 'jpeg', 'gif'])
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1] in ALLOWED_EXTENSIONS
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
file = request.files['xhr2upload'] # [0]
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('uploaded_file',
filename=filename))
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form action="" method=post enctype=multipart/form-data>
<p><input type=file name=file>
<input type=submit value=Upload>
</form>
'''
if __name__ == '__main__':
app.run()
This is a common problem to have with Flask early on - when you attempt to access request.args, request.form, request.values and request.files they will throw a KeyError if the key does not exist in them - just as an ordinary dictionary would in Python (try {}["test"] in any Python interpreter). Flask adds a bit of sugar to the request dictionaries - its KeyError is actually a subclass of HTTPException that raises a 400 Bad Request if it is not caught. (See this part of the quickstart documentation)
The line that is causing this issue is request.files['xhr2upload']. You do not include any JavaScript on the page and the only <input type="file"> you have on the page has the name of "file", not "xhr2upload". Either change request.files['xhr2upload'] to request.files['file'] or load an ajax uploader into your page and have it post your files using the xhr2upload name.
If you have a form that may have a field and you want to check if a field is defined without raising a KeyError then you can use the .get method:
request.form["not_here"] # KeyError -> 400 Bad Request
request.form.get("might_be_here") # Value if provided, else None
you must add "multiple" to file input tag, like so:
<input type="file" name=file multiple>
and made an for bucle instead "file" on flask backend
files = ...
for file in files
file.save(os.path.join...