We Keep Coding

Faster way of redirecting all requests to sanic?

I'm attempting to redirect all requests in sanic web server. So, for example, if someone went to localhost:5595/example/hi it would forward to a website. I know how to normally do that in sanic, but it would be to slow to redirect 100 urls. Is there any faster way of doing this?

I am not 100% sure if this is the answer you are looking for. But, if you are asking about making a super crude proxy server in Sanic to redirect all requests, something like this would do (see server1.py).
# server1.py
from sanic import Sanic
from sanic.response import redirect
app = Sanic("server1")
#app.route("/<path:path>")
async def proxy(request, path):
return redirect(f"http://localhost:9992/{path}")
app.run(port=9991)
And so we have a place for traffic to go:
from sanic import Sanic
from sanic.response import text
app = Sanic("server1")
#app.route("/<foo>/<bar>")
async def endpoint(request, foo, bar):
return text(f"Did you know {foo=} and {bar=}?")
app.run(port=9992)
Now, let's test it out:
$ curl localhost:9991/hello/world -Li
HTTP/1.1 302 Found
Location: http://localhost:9992/hello/world
content-length: 0
connection: keep-alive
content-type: text/html; charset=utf-8
HTTP/1.1 200 OK
content-length: 35
connection: keep-alive
content-type: text/plain; charset=utf-8
Did you know foo='hello' and bar='world'?

Having trouble calling a get request using Falcon framework

I'm following Falcon tutorial for Python.
Everything worked fine until this part:
The response I'm getting when trying this command http localhost:8000/images is:
HTTP/1.1 500 Internal Server Error
Content-Length: 110
Content-Type: text/plain
Date: Sat, 01 Dec 2018 15:50:26 GMT
Server: waitress
Internal Server Error
The server encountered an unexpected internal server error
(generated by waitress)
I read it's a problem in the code but I can't find it, it's exactly as in the tutorial, app.py file:
import falcon
from images import Resource
api = application = falcon.API()
images = Resource()
api.add_route('/images', images)`
images.py:
import json
import falcon
class Resource(object):
def on_get(self, req, resp):
doc = {
'images': [
{
'href': '/images/1eaf6ef1-7f2d-4ecc-a8d5-6e8adba7cc0e.png'
}
]
}
# Create a JSON representation of the resource
resp.body = json.dumps(doc, ensure_ascii=False)
# The following line can be omitted because 200 is the default
# status returned by the framework, but it is included here to
# illustrate how this may be overridden as needed.
resp.status = falcon.HTTP_200
Also, I have an empty file named __init__.py and all the files are in the same folder, C:\look\look.
P.S.
I tried to add an HTTP requests scratch file (using PyCharm IDE) but there is no option to add that kind of a file (after I press Ctrl + Shift + Alt + Insert). I couldn't find how to fix this anywhere.

I see that the question is pretty old, but I found a solution. Just run the server with command:
waitress-serve --port=8000 --call look.app:get_app
since we get the app from calling the function get_app()

Python requests module using TLSv1.2 even though I am specifying TLSv1.1

Our payment processing partner can only support TLSv1.1 for the time being. Switching processors is not an option right now.
In Python 2.7.10, I have a script which requests a GET from my dev web server, but at runtime it uses TLSv1.2 instead of TLSv1.1 like I am telling it to.
import requests
from requests.packages.urllib3.poolmanager import PoolManager
import ssl
class MyAdapter(requests.adapters.HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(
num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1_1,
)
print("poolmanager set")
def do_it():
with requests.Session() as s:
s.mount('https://', MyAdapter())
url = 'https://my.server.com/api/3.0/order/?page=1&limit=10'
response = requests.get(url, verify=True)
print("status code: %r" % response.status_code)
if __name__ == '__main__':
print do_it()
The output is:
poolmanager set
status code: 200
My nginx access log, however, says that the protocol used was TLSv1.2:
127.0.0.1 - [15/Dec/2015:13:38:19 -0600] TLSv1.2/ECDHE-RSA-AES128-GCM-SHA256 "GET /api/3.0/order/?page=1&limit=10 HTTP/1.1" 200 176 "-" "python-requests/2.9.0"
I have poured over the skint documentation around implementing and mounting HTTPAdapters with the requests library, and I cannot find anything that would indicate why my client is speaking TLSv1.2 instead of TLSv1.1 like I am telling it to.

The solution is to use s.get (the Session with the mounted HTTPAdapter) instead of requests.get.

POST to Flask from espduino timing out

I'm playing with an esp8266 providing WiFi to an Arduino with this library. I have it set up properly to POST to Pushover as well as requestbin, and between the debug output and the interface from these tools, I can verify that the request data is being POSTed properly, and the esp8266 / arduino unit's response status code properly shows 200.
I wanted to test the reliability of the setup, so I figured I'd turn to Flask. I put in a for loop to POST 100 requests from the espduino to a Flask app running at 0.0.0.0:5000. The POST contains a test string (trying to roughly gauge simple data integrity, make sure the string comes through uncorrupted) as well as the number of the loop that is being sent (e.g. 0 on the first loop ... 99 on the last). Flask keeps track of these requests and updates its output to show which requests came through properly and which did not.
The setup is working great except for one thing: the module is timing out after each request. Each POST appears to work, Flask updates the output appropriately, but the espduino takes its full 5 seconds (default timeout) for each request and says it got a response code of 0.
So to reiterate the situation: my espduino setup properly POSTs and gets a 200 response from requestb.in and pushover.net, but with my local Flask server POSTs and then times out.
WHYT:
Make sure all form data is read by Flask -> no difference
Serve with gunicorn instead of the built-in Flask server -> no difference
Change response content type explicitly to "text/html" -> no difference
Change response content type to "application/json" -> no difference
Add ; charset=utf-8 to content type -> no difference
Change Flask api endpoint from index / to /api -> no difference
Change port -> no difference
Verify Firewall is off (OS X 10.10.5 hosting the Flask app) -> no difference
nc -l 5000 and check the incoming post from the module (looks good)
Post from Postman and curl to the Flask app to inspect response (and compare with responses from Requestb.in / sites that work)
Removed special chars from the test_string
Change WSGIRequestHandler.protocol_version to HTTP/1.1 -> no difference
I've spent a couple days working on this and not made much progress. The biggest breakthrough I had today was that I can successfully post and get 200 responses if I put my gunicorn / Flask setup behind nginx, with no changes to the espduino code, so I'm sure there's something that Flask is or isn't doing (I was concerned it might be the espduino's processing of an IP address vs domain name, but I think this rules that out).
Summary of the setups I've tried:
POST to requestb.in -> POST works, 200 response, no timeout
POST to pushover.net -> POST works, 200 response, no timeout
POST to local Flask server -> POST works, no response, times out
GET to local Flask server -> no response, times out (have not tested if GETs data)
POST to local gunicorn serving Flask app -> POST works, no response, times out
POST to local nginx serving gunicorn / Flask -> POST works, 200 response, no timeout
POST to local gunicorn serving httpbin -> no response, times out
POST to local cherrypy server -> no response, times out
Current code:
from flask import Flask, request, make_response
from datetime import datetime
app = Flask(__name__)
ESPDUINO_IP = 'XXX.XXX.XXX.XXX'
INCOMING_TEST_STRING = 'This is my espduino test string!'
incoming_requests = []
with open('results.txt', 'w') as f:
f.write("Test run start: {}\n".format(datetime.now()))
#app.route('/api', methods=['GET', 'POST'])
def count_requests():
if request.method == 'POST':
form = request.form
incoming_ip = request.remote_addr
if incoming_ip == ESPDUINO_IP and form['test_string'] == INCOMING_TEST_STRING:
test_num = int(form['test_num'])
incoming_requests.append(test_num)
msg = "All is peachy!"
with open('results.txt', 'a') as f:
f.write("{:02d}: {}\n".format(test_num, datetime.now()))
else:
msg = "Hey, you're not the espduino!<br>"
msg += str(len(incoming_requests)) + " requests so far.<br>"
missing = set(range(100)) - set(incoming_requests)
msg += "Missing: {}<br>".format(', '.join(map(str, missing)) if missing else "None!")
msg += '<br>'.join(map(str, incoming_requests))
resp = make_response('{"this": "that"}')
resp.headers['Content-Type'] = "application/json"
return resp
# return "<html><body>{}</body></html>".format(msg)
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
Here's what the POST from the espduino looks like:
$ nc -l 5000
POST /api HTTP/1.1
Host: XXX.XXX.XXX.XXX
Content-Length: 55
Connection: close
Content-Type: application/x-www-form-urlencoded
User-Agent: ESPDRUINO#tuanpm
test_string=This is my espduino test string!&test_num=0
As compared to curl -X POST -d 'test_string=This is my espduino test string!&test_num=0' localhost:5000/api:
$ nc -l 5000
POST /api HTTP/1.1
Host: localhost:5000
User-Agent: curl/7.43.0
Accept: */*
Content-Length: 55
Content-Type: application/x-www-form-urlencoded
test_string=This is my espduino test string!&test_num=0
Would love to hear any ideas on what may be going on. I'm wondering if this might be a WSGI problem?
Update Aug 31, 2015:
I still haven't figured this out, but it's definitely not a problem specific to Flask. As I noted above, I've also replicated the timeout with CherryPy, as well as with python3 -m http.server --bind 0.0.0.0 5000 (and changing the espduino code to GET /), as well as with ruby -run -e httpd. I still don't understand why nginx, requestbin, etc. serve it with no problems.
In response to #Miguel's comment about the HOST header not having the port, I'm working on forking and building the firmware to change this, but in the meantime I hard-coded the client host and port into a little HTTP server script with no luck.
from http.server import BaseHTTPRequestHandler, HTTPServer
class MyServer(BaseHTTPRequestHandler):
# protocol_version = 'HTTP/1.1'
# close_connection = True
def _set_headers(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
def do_GET(self):
self._set_headers()
self.wfile.write(b"<html><body><h1>hi!</h1></body></html>")
def do_HEAD(self):
self._set_headers()
def do_POST(self):
self.client_address = ("192.168.0.4", 5000)
self._set_headers()
self.wfile.write(b"<html><body><h1>POST!</h1></body></html>")
# import pdb; pdb.set_trace()
def run(server_class=HTTPServer, handler_class=MyServer, port=5000):
server_address = ('0.0.0.0', port)
httpd = server_class(server_address, handler_class)
print('Starting httpd...')
httpd.serve_forever()
if __name__ == "__main__":
run()
Looking through tcpdump to see if I can find any difference between the working (nginx) and nonworking network data. Haven't found anything so far, but I'm also new to the tool.
Update Sep 08, 2015
Still haven't figured this out, but it looks like the tcpdump is significantly different between the nginx and Python servers. Here is an example POST and response -- I have replaced the IPs with ESPDUINO_IP and OSX_IP for clarity, and cleaned up the surrounding ACK calls and such. I need to look into why the Python response is getting interrupted by that odd line -- I examined 10+ consecutive POST / Response pairs, and every one of the Python responses was interrupted like that (in exactly the same place), and none of the nginx responses were, so I wonder if that might be the problem. (Also, as you can see, during this round of testing I had changed the response body to text instead of JSON -- no change in results.)
nginx (works)
POST /api HTTP/1.1
Host: OSX_IP
Content-Length: 29
Connection: close
Content-Type: application/x-www-form-urlencoded; charset=utf-8
User-Agent: espduino#n8henrie
test_string=simple&test_num=0
09:16:04.079291 IP OSX_IP.commplex-main > ESPDUINO_IP.49146: Flags [P.], seq 1:183, ack 211, win 65535, length 182
HTTP/1.1 200 OK
Server: nginx/1.8.0
Date: Mon, 31 Aug 2015 15:16:04 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 26
Connection: close
<html><body></body></html>
Flask (times out)
POST /api HTTP/1.1
Host: OSX_IP
Content-Length: 29
Connection: close
Content-Type: application/x-www-form-urlencoded; charset=utf-8
User-Agent: espduino#n8henrie
test_string=simple&test_num=3
09:00:19.424086 IP OSX_IP.commplex-main > ESPDUINO_IP.48931: Flags [P.], seq 1:18, ack 211, win 65535, length 17
HTTP/1.0 200 OK
09:00:36.382125 IP OSX_IP.commplex-main > ESPDUINO_IP.48931: Flags [FP.], seq 18:181, ack 211, win 65535, length 163
E....F#.#..,...e.......#...k..S.P.......Content-Type: text/html; charset=utf-8
Content-Length: 26
Server: Werkzeug/0.10.4 Python/3.4.3
Date: Mon, 31 Aug 2015 15:00:36 GMT
<html><body></body></html>
It looks to me like Python is breaking the response into two halves for some reason, e.g. one part of length 17 and another of length 163, as compared to nginx's single response of length 182.
Update Sep 10, 2015
Interestingly, everything works as expected if I run it through mitmproxy -- even directly to the Flask app without nginx or gunicorn. As soon as I remove mitmproxy, it's back to the timeouts as per above.

Still haven't fixed the problem, but I think I may have figured out what's causing it. Not a Flask problem after all.
Unfortunately this seems to be a bug with the esp_bridge library whose firmware espduino uses in the esp8266. Pardon the likely incorrect terminology, but from what I can tell for some reason it doesn't seem to be joining TCP packets. The servers that produce an HTTP response that is split into separate TCP packets (e.g. Flask) are failing, while tcpdump can verify that nginx and mitmproxy are joining the split TCP packets and returning a response in a single packet, which is why they are working.
https://github.com/tuanpmt/esp_bridge/issues/10
Update 20160128
I revisited this issue today and found a workaround. While the ideal solution would be to fix esp_bridge to reassemble multi-packet reponses, as long as the response is quite small one can force Flask to write the responses in a single packet.
from werkzeug.serving import WSGIRequestHandler
# Your Flask code here...
if __name__ == "__main__":
WSGIRequestHandler.wbufsize = -1
app.run()

Internal Server Error when using Flask session

I want to save an ID between requests, using Flask session cookie, but I'm getting an Internal Server Error as result, when I perform a request.
I prototyped a simple Flask app for demonstrating my problem:
#!/usr/bin/env python
from flask import Flask, session
app = Flask(__name__)
#app.route('/')
def run():
session['tmp'] = 43
return '43'
if __name__ == '__main__':
app.run()
Why I can't store the session cookie with the following value when I perform the request?

According to Flask sessions documentation:
...
What this means is that the user could look at the contents of your
cookie but not modify it, unless they know the secret key used for
signing.
In order to use sessions you have to set a secret key.
Set secret key. And you should return string, not int.
#!/usr/bin/env python
from flask import Flask, session
app = Flask(__name__)
#app.route('/')
def run():
session['tmp'] = 43
return '43'
if __name__ == '__main__':
app.secret_key = 'A0Zr98j/3yX R~XHH!jmN]LWX/,?RT'
app.run()

As #falsetru mentioned, you have to set a secret key.
Before sending the session cookie to the user's browser, Flask signs the cookies cryptographically, and that doesn't mean that you cannot decode the cookie. I presume that Flask keeps track of the signed cookies, so it can perform it's own 'magic', in order to determine if the cookie that was sent along with the request (request headers), is a valid cookie or not.
Some methods that you may use, all related with Flask class instance, generally defined as app:
defining the secret_key variable for app object
app.secret_key = b'6hc/_gsh,./;2ZZx3c6_s,1//'
using the config() method
app.config['SECRET_KEY'] = b'6hc/_gsh,./;2ZZx3c6_s,1//'
using an external configuration file for the entire Flask application
$ grep pyfile app.py
app.config.from_pyfile('flask_settings.cfg')
$ cat flask_settings.py
SECRET_KEY = b'6hc/_gsh,./;2ZZx3c6_s,1//'
Here's an example (an adaptation from this article), focused on providing a more clearer picture of Flask session cookie, considering the participation of both Client and Server sides:
from flask import Flask, request, session
import os
app = Flask(__name__)
#app.route('/')
def f_index():
# Request Headers, sent on every request
print("\n\n\n[Client-side]\n", request.headers)
if 'visits' in session:
# getting value from session dict (Server-side) and incrementing by 1
session['visits'] = session.get('visits') + 1
else:
# first visit, generates the key/value pair {"visits":1}
session['visits'] = 1
# 'session' cookie tracked from every request sent
print("[Server-side]\n", session)
return "Total visits:{0}".format(session.get('visits'))
if __name__ == "__main__":
app.secret_key = os.urandom(24)
app.run()
Here's the output:
$ python3 sessions.py
* Serving Flask app "sessions" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[Client-side]
Upgrade-Insecure-Requests: 1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Connection: keep-alive
Host: 127.0.0.1:5000
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
[Server-side]
<SecureCookieSession {'visits': 1}>
127.0.0.1 - - [12/Oct/2018 14:27:05] "GET / HTTP/1.1" 200 -
[Client-side]
Upgrade-Insecure-Requests: 1
Cookie: session=eyJ2aXNpdHMiOjF9.DqKHCQ.MSZ7J-Zicehb6rr8qw43dCVXVNA # <--- session cookie
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Connection: keep-alive
Host: 127.0.0.1:5000
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
[Server-side]
<SecureCookieSession {'visits': 2}>
127.0.0.1 - - [12/Oct/2018 14:27:14] "GET / HTTP/1.1" 200 -
You may have noticed that in the example above, I'm using the os lib and the urandom() function, in order to generate Flask's secret key, right?
From the official doc:
How to generate good secret keys
A secret key should be as random as possible. Your operating system has ways to generate pretty random data based on a cryptographic random generator. Use the following command to quickly generate a value for Flask.secret_key (or SECRET_KEY):
$ python -c 'import os; print(os.urandom(16))'
b'_5#y2L"F4Q8z\n\xec]/'
PLUS NOTE
As you can see, the creators of Flask support the practice of using os.urandom() for building the Flask secret key, from older versions of the tool to its latest version. So: why #joshlsullivan's answer received downvotes (deserves an upvote) and why #MikhailKashkin writes that, using os.urandom() is terrible idea, are mysteries.

Under app = Flask(__name__) place this: app.secret_key = os.urandom(24).