We Keep Coding

How do you pass a url as a REST GET parameter in FLASK

So this is my code.
from flask import Flask, request
from flask_restful import Resource, Api
app = Flask(__name__)
app.config['DEBUG'] = True
api = Api(app)
# Make the WSGI interface available at the top level so wfastcgi can get it.
wsgi_app = app.wsgi_app
class Default(Resource):
def get(self, name):
"""Renders a sample page."""
return "Hello " + name
class LiveStats(Resource):
def get(self, url):
return "Trying to get " + url
# data = request.get(url)
# return data
api.add_resource(Default, '/default/<string:name>') # Route_1
api.add_resource(LiveStats, '/liveStats/<path:url>') # Route_2
if __name__ == '__main__':
import os
HOST = os.environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(os.environ.get('SERVER_PORT', '5555'))
except ValueError:
PORT = 5555
app.run(HOST, PORT)
Now firstly this post helped a lot. how-to-pass-urls-as-parameters-in-a-get-request-within-python-flask-restplus
Changing what I originally had.
api.add_resource(LiveStats, '/liveStats/<string:url>') # Route_2
to this
api.add_resource(LiveStats, '/liveStats/<path:url>') # Route_2
got rid of 404 errors that I had but now I am noticing that it's not passing all of the url.
Example if I try this
localhost:60933/liveStats/http://address/Statistics?NoLogo=1%26KSLive=1
I get this
Trying to get http://address/Statistics
so it has taken off ?NoLogo=1%26KSLive=1
How do you prevent this?

All characters after the ? are considered parameters. From the docs:
To access parameters submitted in the URL (?key=value) you can use the args attribute:
searchword = request.args.get('key', '')
We recommend accessing URL
parameters with get or by catching the KeyError because users might
change the URL and presenting them a 400 bad request page in that case
is not user friendly.
For a full list of methods and attributes of the request object, head
over to the Request documentation.
Maybe you could encode the query string in a way that you can retrieve it as a single parameter on the back end, but not sure it would be useful.
If you don't want to access the args individually, you can access the full query string:
request.query_string
Putting this all together I think this will work for you:
class LiveStats(Resource):
def get(self, url):
return "Trying to get " + url + request.query_string

python requests post file return null

I need help with python requests. I want to post a file to a server. Here is my example code for posting the file
import requests
with open('try.txt', 'rb') as myfile:
url = <myurl>
files = {'value':myfile}
response = requests.post(url, files=files)
try.txt is not a blank file. It contains some sentences
On the server side, here is the snippet for receiving the file.
from flask_restful import Api, Resource, reqparse
class MyThing(Resource):
def get(self):
pass
def post(self):
parser = reqparse.RequestParser()
parser.add_argument('value')
args = parser.parse_args()
value = args['value']
if value.endswith(".txt") or value.endswith(".csv"):
## do something
else:
## do something
However when I run the server and try to post the file to the server, it return an error saying that
if value.endswith(".txt") or value.endswith(".csv"):
AttributeError: 'NoneType' object has no attribute 'endswith'
I check the value by print files and it return like this
{'value': <open file 'try.txt', mode 'rb' at 0x7f041db8e420>}
Does this mean value is null? or did I missed something on my server side code? Thank you for the help.

Bottle/python error handling with default_error_handler

With bottle/python I'm trying to get a more detailed error handling. There is a page describing a method
How to return error messages in JSON with Bottle HTTPError?, but can't implement it with my project.
ara.hayrabedian's answer on the mentioned page works, but with the hope to get more details for error situations Michael's code has some charm. Only any variation I tested fails. Basically I have (out of a longer coding):
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from bottle import Bottle, run, static_file, view, template, \
get, post, request, debug
from bottle import route, response, error
import json
app = Bottle()
#class JSONErrorBottle(bottle.Bottle): ### just an not working alternative!?
class JSONErrorBottle(Bottle):
def default_error_handler(app, res):
bottle.response.content_type = 'application/json'
print("XXXXXXX " + json.dumps(dict(error=res.body, status_code=res.status_code)))
return json.dumps(dict(error=res.body, status_code=res.status_code))
app.install(JSONErrorBottle)
def main():
app.run(host = prefs['server'], port = prefs['port'], reloader=False)
if __name__ == '__main__':
rcode = main()
Calling an invalid page that 'default_error_handler' isn't called, just the standard bottle html error page with "Error: 404 Not Found"

Solution for micro-services design
def handle_404(error):
return "404 Error Page not Found"
app = bottle.Bottle()
app.error_handler = {
404: handle_404
}
bottle.run(app)

The Michael's way is indeed the most "right" way.
That worked for me as expected (at least with python-3.6.6 and bottle-0.12.13):
from bottle import Bottle, run, abort
import bottle, json
class JSONErrorBottle(Bottle):
def default_error_handler(self, res):
bottle.response.content_type = 'application/json'
return json.dumps(dict(error = res.body, status_code = res.status_code))
app = JSONErrorBottle()
#app.route('/hello')
def hello():
return dict(message = "Hello World!")
#app.route('/err')
def err():
abort(401, 'My Err')
run(app, host='0.0.0.0', port=8080, debug=True)
Now every error() and abort() is json

Content-length error in google cloud endpoints testing

I get the following error whenever I want to test a 404 HTTP error path in my code:
AssertionError: Content-Length is different from actual app_iter length (512!=60)
I have created a minimal sample that triggers this behavior:
import unittest
import endpoints
from protorpc import remote
from protorpc.message_types import VoidMessage
import webtest
#endpoints.api(name='test', version='v1')
class HelloWorld(remote.Service):
#endpoints.method(VoidMessage, VoidMessage,
path='test_path', http_method='POST',
name='test_name')
def test(self, request):
raise endpoints.NotFoundException("Not found")
class AppTest(unittest.TestCase):
def setUp(self):
app = endpoints.api_server([HelloWorld])
self.testapp = webtest.TestApp(app)
# Test the handler.
def testHelloWorldHandler(self):
response = self.testapp.post('/_ah/spi/HelloWorld.test', extra_environ={
'SERVER_SOFTWARE': 'Development/X', 'CONTENT_TYPE': 'application/json'})
So what am I doing wrong?

This is a known error with App Engine.
Endpoints does not set the correct Content-Length header when you raise an exception:
https://code.google.com/p/googleappengine/issues/detail?id=10544
To fix it there is a diff file included in the link above, or follow my instructions to temporarily patch it by yourself.
1. Open apiserving.py
On a mac you can find the file at:
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/endpoints-1.0/endpoints
2. Locate the following section (line 467):
It should look like this:
headers_dict = dict([(k.lower(), v) for k, v in headers])
if self.__is_json_error(status, headers_dict):
status, body = self.protorpc_to_endpoints_error(status, body)
3. Change it to this:
headers_dict = dict([(k.lower(), v) for k, v in headers])
if self.__is_json_error(status, headers_dict):
pre_body_length = len(body)
status, body = self.protorpc_to_endpoints_error(status, body)
post_body_length = len(body)
if pre_body_length != post_body_length:
for index, header in enumerate(headers):
header_key, _header_value = header
if header_key == 'content-length':
headers[index] = (header_key, str(post_body_length))
break
4. All done!
Endpoints will return the correct Content-Length, WebOb will be happy and your API tests will be working :)

Proper way to modify a response in Flask with eg process_response

Given a simple Flask application, I'm just curious about whether there is a proper way to modify a Response in the hooks such as process_response?
e.g. Given:
from flask import Flask, Response
class MyFlask(Flask):
def process_response(self, response):
# edit response data, eg. add "... MORE!", but
# keep eg mimetype, status_code
response.data += "... This is added" # but should I modify `data`?
return response
# or should I:
# return Response(response.data + "... this is also added",
# mimetype=response.mimetype, etc)
app = MyFlask(__name__)
#app.route('/')
def root():
return "abddef"
if __name__ == '__main__':
app.run()
Is it proper to just create a new response each time, or is it canonical to just edit in-place the response parameter and return that modified response?
This may be purely stylistic, but I'm curious – and I haven't noticed anything in my reading that would indicate the preferred way to do this (even though it's probably quite common).
Thanks for reading.

From the Flask.process_response docs:
Can be overridden in order to modify the response object before it's sent to the WSGI server.
The response object is created on flask dispacher mechanism (Flask.full_dispatch_request). So if you want to create response objects under your own way, override Flask.make_reponse. Use Flask.process_response only when the desired modifications can be made using the created response object parameter.

Actually, you can use Flask.process_response to intercept and modify the response this way:
from flask import Flask
import json
import ast
appVersion = 'v1.0.0'
class LocalFlask(Flask):
def process_response(self, response):
#Every response will be processed here first
response.headers['App-Version'] = appVersion
success = True if response.status_code in [ 200, 201, 204 ] else False
message = 'Ok' if success else 'Error'
dict_str = response.data.decode("UTF-8")
dataDict = ast.literal_eval(dict_str)
standard_response_data = {
'success': success,
'message': message,
'result': dataDict
}
response.data = json.dumps(standard_response_data)
super(LocalFlask, self).process_response(response)
return response