Created a web app and have managed to build a function to clean up csv files from Google my Business exports. However when I run the function with the code I have written I get the following error message:
Not Found
The requested URL was not found on the server. If you entered the URL
manually please check your spelling and try again.
Not sure where I am going wrong
mport os
import pandas as pd
from flask import Flask, request, redirect, url_for
from flask import Flask, make_response
from werkzeug.utils import secure_filename
UPLOAD_FOLDER = './Downloads/gmbreports'
if not os.path.exists(UPLOAD_FOLDER):
os.makedirs(UPLOAD_FOLDER)
ALLOWED_EXTENSIONS = 'csv'
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
def transform(text_file_contents):
disc = open('clean.csv')
disc2 = open('clean_two.csv','w')
#cleaning up csv
for row in disc:
row = row.strip()
row = row[1:-1]
row = row.replace('""','"')
disc2.write(row+'\n')
disc2.close()
disc.close()
discovery = pd.read_csv('clean_two.csv')
discovery_clean = discovery.iloc[1:]
cols = list(discovery_clean.columns[4:])
discovery_clean[cols] = discovery_clean[cols].apply(pd.to_numeric,errors='coerce')
return discovery_clean
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
if file.filename == '':
flash('You need to upload a csv file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('uploaded_file',
filename=filename))
return '''
<!doctype html>
<title>Google My Business Discovery Report Builder</title>
<h1>Upload GMB Discovery csv</h1>
<form action="\transform" method="post" enctype="multipart/form-data">
<p><input type="file" name="file">
<input type="submit" value=Upload>
</form>
'''
#app.route('/transform',methods=["POST"])
def transform_view():
request_file=request.files['file']
request_file.save('clean.csv')
if not request_file:
return "No file"
result = transform()
print(result)
response = make_response(result)
response.headers["Content-Disposition"] ="attachment; filename=result.csv"
return response
if __name__=='__main__':
app.run()
Note: I get this error message after running the script and uploading a csv. The desired outcome is uploading a csv, and showing it on my screen as a cleaned up data table
There are multiple problems here. First, Wondercricket is correct about the form action. It needs to be changed to "/transform".
Once that's done, you're still getting an Internal Server Error. In this case, your transform function is defined as taking a single parameter, but it's not being called with one. Simply change
def transform(text_file_contents):
to
def transform():
That should allow the upload to happen, and the transform will now run. However, there's still one last problem. transform returns a pandas DataFrame, which Flask can't use as a response. Change
response = make_response(result)
to
response = make_response(result.to_csv())
and you should be good to go.
Related
Im posting the following data to my flask image upload server. Problem is an image does not get saved. There are no errors in console.
I have tried decoding the image bytes and it seems to be a properly formed multipart request.
So why isn't the image uploading properly?
import os
from flask import Flask, flash, request, redirect, url_for, send_from_directory
import sys
UPLOAD_FOLDER = '/Users/admin/Desktop/uploads'
ALLOWED_EXTENSIONS = {'txt', 'pdf', 'png', 'jpg', 'jpeg', 'gif'}
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
print(request.get_data(), file=sys.stdout)
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
file.save(os.path.join(app.config['UPLOAD_FOLDER'], file.filename))
return redirect(url_for('uploaded_file',
filename=file.filename))
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/uploads/<filename>')
def uploaded_file(filename):
return send_from_directory(app.config['UPLOAD_FOLDER'], filename)
if __name__ == "__main__":
# Quick test configuration. Please use proper Flask configuration options
# in production settings, and use a separate file or environment variables
# to manage the secret key!
app.secret_key = 'super secret key'
app.config['SESSION_TYPE'] = 'filesystem'
# sess.init_app(app)
app.debug = True
app.run()
Data im posting: (I captured it using print(request.get_data(), file=sys.stdout) located in code)
https://pastecode.io/s/cpev4ny4
could you please verify if your apache or whatever you use has permission to write ( save an image) in that directory?
I have a page with a search bar that is connected to my database and I am creating a button where the user will be able to update the database by uploading .xlsx file.
I wrote scripts and functions but when I click Submit button I get this error:
werkzeug.exceptions.BadRequestKeyError: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand. KeyError: 'product'
Everything works ok when I run python function from PyCharm or when I delete search_bar function in main.py and then run the function but it doesn't work if I have search_bar function at all.
Any idea why it isn't working?
main.py
from flask import Flask, render_template, request, redirect, url_for, session, send_file, send_from_directory, flash
from werkzeug.utils import secure_filename
from flask_mysqldb import MySQL
import MySQLdb.cursors
import os
from compare import upload_excel_to_sql
app = Flask(__name__)
#app.route('/content)
def content():
# Check if user is loggedin
if 'loggedin' in session:
# User is loggedin show them the home page
return render_template('content.html', username=session['username'])
# User is not loggedin redirect to login page
return redirect(url_for('login'))
# SEARCH IN DATABASE
#app.route('/content', methods=['GET', 'POST'])
def search_bar():
if 'loggedin' in session:
if request.method == "POST":
cursor = connection.cursor(buffered=True)
product = request.form['product']
# search by project name or title or brand
cursor.execute("USE content_repository")
cursor.execute("""SELECT upload_date,
product_code,
Project_name,
Country,
Brand,
Category,
Title,
Bullet_Points,
Description,
version
FROM product WHERE Project_name LIKE %s OR Title LIKE %s OR Brand LIKE %s""",
(product, product, product))
connection.commit()
data = cursor.fetchall()
# all in the search box will return all the tuples
if len(data) == 0 and product == 'all':
cursor.execute("""SELECT upload_date,
product_code,
Project_name,
Country,
Brand,
Category,
Title,
Bullet_Points,
Description,
version
FROM product""")
connection.commit()
data = cursor.fetchall()
return render_template('content.html', data=data)
return render_template('content.html')
return redirect((url_for('content')))
# UPLOADING AND PROCESSING FILE
UPLOAD_FOLDER = os.path.dirname(os.path.abspath(__file__)) + '/uploads/'
DOWNLOAD_FOLDER = os.path.dirname(os.path.abspath(__file__)) + '/downloads/'
# Allow user to upload only xlsx files'
ALLOWED_EXTENSIONS = {'xlsx'}
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
app.config['DOWNLOAD_FOLDER'] = DOWNLOAD_FOLDER
# Set up the maximum size of the file, in this case it is 10 mb
app.config['MAX_CONTENT_LENGTH'] = 10 * 1024 * 1024
# Functions that check if an extension is valid
def allowed_file(filename):
return '.' in filename and filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/content_repository', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(UPLOAD_FOLDER, filename))
# we call process_file function and pass the uploaded file path as an argument
filename = upload_excel_to_sql(os.path.join(UPLOAD_FOLDER, filename), UPLOAD_FOLDER)
return redirect(url_for('content', filename=filename))
return render_template('content.html', username=session['username'])
if __name__ == '__main__':
app.run(debug=True)
compare.py
from sqlalchemy import create_engine
import pymysql
import pandas as pd
from aifc import Error
import os
def upload_excel_to_sql(file_path, upload_folder):
"""
Load data from excel to sql database
:param path: path to excel with dataset (str)
:return: True if everything is correct
"""
output_path = os.path.join(upload_folder, file_path)
df_scrapped_data = pd.read_excel(output_path)
for column in df_scrapped_data.columns:
if column.startswith("Unnamed"):
df_scrapped_data.drop([column], inplace=True, axis=1)
engine = create_engine("mysql+pymysql://{user}:{psw}#localhost/{db}".format(user="root", psw="root", db="content"))
df_scrapped_data.to_sql('product', con=engine, if_exists='append', index=False)
return engine
content.html
form element with search bar and with upload and submit elements.
<form method="POST" action="">
<input type="search" placeholder="Search" aria-label="Search" name="product">
<button type="submit">Search</button>
</form>
<form method="POST" action="" enctype="multipart/form-data">
<p><input type="file" name="file"></p>
<p><input type="submit" value="Submit"></p>
</form>
I started learning Flask by myself today, and while I was toying around with it I made a small program to download a file, and it worked great.
#app.route('/download')
def download():
path = "Path_to_my_file"
return send_file(path, as_attachment=True)
Now I was changing my program a bit and I removed the 'send_file' function and added some other commands. However, when I tried running the program, the download still happened. I have no idea why though.
My Code:
from flask import *
from flask_sqlalchemy import SQLAlchemy
from werkzeug.utils import secure_filename
import os
usera = ""
UPLOAD_FOLDER = 'I:/Python'
ALLOWED_EXTENSIONS = {'txt', 'pdf', 'png', 'jpg', 'jpeg', 'gif'}
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
app.config ['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///downloads.sqlite3'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
user = db.Column(db.String)
download = db.column(db.String)
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/upload', methods=['GET', 'POST'])
def upload_file():
global usera, db
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
db.session.add(User(user = usera, download = filename))
return render_template('lol.html')
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/login',methods = ['POST', 'GET'])
def login():
global db, user
if request.method == 'POST':
usera = request.form['user']
return render_template('lol.html')
#app.route('/download')
def download():
path = "I:/Python/"
users = User.query.all()
return users
if __name__ == '__main__':
app.run(host='0.0.0.0')
I had faced same issue where flask was returning old files (in your case it is still sending files even if code is changed). This issue is due to the fact that file is cached in browser. Clear the browser cache and your api will start working.
I am trying to make an application that allows users to upload images from their computer. This is as part of my CS50x final project, so I'm working in the CS50 IDE. Here is the code for my application:
application.py
import os
from flask import *
app = Flask(__name__)
UPLOAD_FOLDER = 'uploads'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
#app.route('/')
def myindex():
return render_template("file_upload_form.html")
#app.route('/upload', methods = ['POST'])
def upload():
if request.method == 'POST':
file = request.files['file']
file.save(os.path.join(app.config['UPLOAD_FOLDER'], "test.jpg"))
return redirect("/")
if __name__ == "__main__":
app.run(debug=True)
file_upload_form.html
<html>
<head>
<title>upload</title>
</head>
<body>
<form action = "/upload" method = "post" enctype="multipart/form-data">
<input type="file" name="file" />
<input type = "submit" value="Upload">
</form>
</body>
</html>
For the vast majority of files, when I submit the form, I get a 500 Internal Server Error with no traceback or explanation. Curiously though, some files it seems to work with. I have found two images that work: both JPEGs and both relatively small. All other images I have tried have caused the error.
I cannot work out what is causing the Server Error, or why some images work and others don't. Can anyone see any reason why it wouldn't work for other images?
Thanks
I tested out your code and it worked fine on my side, but there is a couple of additions i would make, just to make it more robust.
You are calling all your uploads test.jpg and there is no mechanism in place to ensure that the file that gets uploaded is a jpg file. So technically, i could upload a pdf and it would be renamed in to a test.jpg.
A better method is to just give the file - the same name it had, including the extension. If you want to re-name the file, rather strip the extension first and add it to the new filename like so:
ext = file.filename.rsplit('.', 1)[1].lower()
newFilename = 'newname.{}'.format(ext)
Also there is no MAX_CONTENT_LENGTH - I would add that as well.
So i re-wrote your python code to look like this:
import os
from flask import *
from werkzeug.utils import secure_filename
app = Flask(__name__)
UPLOAD_FOLDER = './uploads'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
app.config['ALLOWED_IMAGES'] = set(['png', 'jpg', 'jpeg'])
app.config['MAX_CONTENT_LENGTH'] = 16 * 1024 * 1024
def allowed_image(filename):
return '.' in filename and filename.rsplit('.', 1)[1].lower() in app.config['ALLOWED_IMAGES']
#app.route('/')
def myindex():
return render_template("index.html")
#app.route('/upload', methods = ['POST'])
def upload():
if request.method == 'POST':
file = request.files['file']
if file and allowed_image(file.filename):
file.save(os.path.join(app.config['UPLOAD_FOLDER'], secure_filename(file.filename)))
return redirect(request.url)
if __name__ == "__main__":
app.run(debug=True)
You are sending the post to: <form action = "/success"...>
Where is your /success handler?
UPDATE:
Try:
if request.method == 'POST':
print(request.files)
print(request.form)
print(request.POST)
You need to check the logs to see what there error is.
I have a locally hosted Flask server and a small React application. I'm trying to use FilePond as an easy solution for image uploading. FilePond takes care of sending each image to the server.
So the problem I'm obviously having is with the backend code. I've set up my server like this, per Flask's docs
UPLOAD_FOLDER='/images'
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg', 'gif'])
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/', methods=['GET', 'POST'])
def upload_file():
print(request.files)
if request.method == "POST":
# check if the post request has the file part
if 'file' not in request.files:
print('No file part')
return make_response("No File Part", 400)
file = request.files["file"]
# if user does not select file, browser also submit an empty part
# without filename
if file.filename == '':
print('No selected file')
return make_response("No Selected File", 400)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename) # filenames can be dangerous!
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return make_response("Success", 201)
However, when put a
print(request.files)
statement at the beginning of the upload_file() function, it seems that this is the request the server is receiving:
ImmutableMultiDict([('images', <FileStorage: 'imageName.jpg' ('image/jpeg')>)])
and I have no idea how to deal with that. I don't know if 'images' is where 'file' is supposed to be. I'm just not really sure where to go from here or what to do with the data being received. I don't know what is, I've never seen that <> syntax being used.
Can anyone help?
Here's a pastebin for the whole server-side code
Thank you!
Here's a proof of concept app that lets you upload multiple images using a form:
Note the enctype="multipart/form-data" attribute of the form, without this you cannot upload files.
Also note the multiple attribute of the file input. This allows client to select multiple files. You need to use request.files.getlist() to obtain the list of all uploaded files.
Once you generate the path to save the file, saving a werkzeug.FileStorage object is is just calling its .save(path_to_save) method.
from flask import Flask, request, render_template_string, redirect, abort
from werkzeug import secure_filename
from pathlib import Path
UPLOAD_DIR: Path = Path(__file__).parent / 'uploads'
UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
app = Flask(__name__)
def is_valid_upload(upload) -> bool:
# some validation logic
return Path(upload.filename).suffix.lower() in ['.jpg', '.jpeg']
#app.route('/', methods=['GET', 'POST'])
def upload():
html = '''
<form action='/' method='POST' enctype="multipart/form-data">
<input type="file" name='images' multiple>
<button>Upload</button>
</form>
'''
if request.method == 'GET':
return html
uploaded_files = request.files.getlist('images')
if not uploaded_files or not uploaded_files[0].filename:
return redirect('/')
valid_uploads = list(filter(is_valid_upload, uploaded_files))
if not valid_uploads:
return 'invalid image(s)', 400
for upload in valid_uploads:
filename = secure_filename(upload.filename)
save_path = str(UPLOAD_DIR / filename)
upload.save(save_path)
return 'uploaded'
if __name__ == "__main__":
app.run()