I built a simple flask server in order to automate python scripts based on an HTTP POST Request coming in from an outside service. This service sends these requests in response to events that we have created based on data conditions. All the flask server needs to do is parse the requests, and take the numerical value stored in the request and use it to run the corresponding python script. At one point, this system was working consistently. Fast forward a few months, the server was no longer working when tried again. No changes were made to the code. According to wireshark, The computer hosting the flask server is receiving the request to the correct port, but the request is now timing out. The host system is failing to respond to the request. Any Idea what is going on here? The firewall has temporarily been turned off.
Alternatively, is there another better package to achieve this goal with?
from flask import Flask, request
import threading
import runpy
app = Flask(__name__)
#app.route('/', methods=['POST'])
def PostHandler():
directory = {}
with open(r"M:\redacted") as f:
for line in f:
(key,val) = line.split()
directory[int(key)] = val
print(directory)
path = r"M:\redacted"
content = request.json
content = content['value']
print(content)
sel_script = int(content)
print(directory[sel_script])
runpy.run_path(path_name=path + directory[sel_script])
return
app.run(host="10.244.xx.xx", port=8080, threaded=True)
Related
I have a python library and need to provide services for others to use. Since the local installation is not a simple 'pip install ...' to complete, I want to set up a http service in Flask so users don't need to locally install it.
If the http server is very close to users' machines, I have a couple of questions to ask, since I am new to http service:
Is http call significantly slower than local call?
In local calls, my api can process texts (large amount of plaint text) in batches to speed up, instead of one by one processing. But the typical http service receives one text, process it and return it. Is that would be slower? Or the http client needs to write concurrent code to make http calls and the http service doesn't need to worry about any concurrency issue.
Is there any general method to speed up http service call?
My http service code would look like:
from flask import Flask
from flask import request
from flask import jsonify
api = Flask(__name__)
#api.route('/process', methods=['GET'])
def process():
text, stopwords = _parse_parameters()
pos = str(wlu.process(text, stopwords=stopwords))
return pos
def _parse_parameters():
text = request.args.get('text')
stopwords = request.args.get('stopwords')
return (text, stopwords)
if __name__ == '__main__':
#api.run(host='0.0.0.0', port='5000')
from waitress import serve
serve(api, host="0.0.0.0", port=5000)
EDIT:
'Local call' means users install the python library locally and do the typical importing of dependencies:
import my_library
So many web applications these days run on their own microservers, it can be hard to implement them on shared hosting platforms. The apps listen on a dedicated port you can customize or reverse proxy, but shared hosting usually only has 80 and 443 open.
Just as an example, the handy web-based editor ICEcoder is a PHP application, so you just drop the files in a directory and away you go. However, the Cloud9 editor runs its own server. You can customize the port, but again, you cant run the reverse proxy.
I had the idea of using a PHP or Python CGI script as an intermediary. Something like:
www.mydomain/mydirectory/middleman.py
from BaseHTTPServer import BaseHTTPRequestHandler
import urlparse, json
# hpyothetical apache api
import apache
parsed_path = urlparse.urlparse(self.path)
response = apache(url=parsed_path, port=8080)
sendStuffBack(response)
Would this be possible with Apache? How would I implement it?
Edit:
Here is what I did based on #grawity's answer.
helloflask.py
#!/usr/bin/env python
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
middle.py
#!/usr/bin/env python
print ("Content-Type: text/html")
print()
import requests
#response = requests.get("http://localhost:5000")
response = requests.get("http://localhost:8888/token=8a387fe88d662e2568f9b8ec2398191452492e7184536670")
print(response.text)
Your Python project is a reverse proxy, and the API you're looking for is just ordinary HTTP. (After all, that's how web browsers interact with Apache already...)
To make HTTP requests, you need a client like urllib or requests:
import requests
response = requests.get("http://" + apache_host + ":8080/" + parsed_path)
By default, all your apps and microservers will think that all clients come from localhost. If that's a problem, see if your apps accept the X-Forwarded-For header. (If they do, include it in all your requests.)
I am creating a REST API using python flask. The API is ready and works on port number 8000 of my localhost. Now I intend to give this REST API a user friendly interface for which I decided to go with python - restplus. I thought of calling this service (running on 8000) internally from swagger application running on 5000
I was able to create the basic structure of the API (Swagger). The code for which looks like this:
import flask
from flask import Flask, request
from flask_restplus import Resource, Api
app = Flask(__name__)
api = Api(app)
#api.route('/HybridComparator/<string:url2>/<string:url1>')
class HybridComparator(Resource):
def get(self, url1, url2):
print(url1)
print(url2)
return url1 + ' ' + url2
if __name__ == '__main__':
app.run(debug=True)
The application as a whole runs seamlessly (with random strings as parameters) on port 5000. But when the URLs I pass are actual links, the application returns a response of 404 - Not found. Further to my investigation I realized the culprit being '/' embedded within the links I try to provide. Is there a way to handle URLs in particular?
Should I encode them before sending a request. (This will make my parameters look ugly). Is there something I am missing?
This is an entirely old question and I am sure you solved your problem by now.
But for new searchers, this may come in handy;
replace <string:url2>/<string:url1> with <path:url2>/<path:url1>
it seems that :
#api.route('/HybridComparator/<path:url2>/<path:url1>')
should fix it ,it fixes the 404 but i am getting only "http:/" part of the param
I am running a webserver based on Flask, which serves a resource being versioned (e.g. installation file of some versioned program). I want to serve my HTTP client with new resource only in case, it already does not have the current version available. If there is new version, I want the client to download the resource and install it.
my Flask server looks like this
import json
import redis
import math
import requests
from flask import Flask,render_template,request
app=Flask(__name__)
#app.route('/version', methods=['GET','POST'])
def getversion():
r_server=redis.Redis("127.0.0.1")
if request.method == 'POST':
jsonobj_recieve=request.data
data=json.loads(jsonobj)
currentversion=r_server.hget('version')
if data == currentversion:
#code to return a 'ok'
else:
#code to return 'not ok' also should send the updated file to the client
else:
return r_server.hget('version')
if __name__ == '__main__':
app.run(
debug=True,
host="127.0.0.1",
port=80
)
my client is very basic:
import sys
import json
import requests
url="http://127.0.0.1/version"
jsonobj=json.dumps(str(sys.argv[1]))
print jsonobj
r=requests.post(url,data=jsonobj)
I will likely have to recode the entire client, this is not a problem but I really have no idea where to start....
Requirements Review
have web app, serving a versioned resource. It can be e.g. file with an applications.
have client, which allows fetching the resource only in case, the version of resource on the server and what client has locally already available differ
the client is aware of version string of the resource
allow client to learn new version string if new version is available
HTTP like design of your solution
If you want to allow downloading an application only in case, the client does not have it already, following design could be used:
use etag header. This usually contains some string describing unique status of resource you want to get from that url. In your case it could be current version number of your application.
in your request, use header "if-none-match", providing version number of your application present at client. This will result in HTTP Status code 306 - Not Modified in case, your client and server share the same version of resource. In case it differs, you would simply provide the content of the resource and use it. Your resource shall also denote in etag current version of the resource and your client shall take note of it, or find new version name from other sources (like from the downloaded file).
This design follows HTTP principles.
Flask serving resource with declaring version in etag
This is focusing on showing the principle, you shall elaborate on providing real content of the resource.
from flask import Flask, Response, request
import werkzeug.exceptions
app = Flask(__name__)
class NotModified(werkzeug.exceptions.HTTPException):
code = 304
def get_response(self, environment):
return Response(status=304)
#app.route('/download/app')
def downloadapp():
currver = "1.0"
if request.if_none_match and currver in request.if_none_match:
raise NotModified
def generate():
yield "app_file_part 1"
yield "app_file_part 2"
yield "app_file_part 3"
return Response(generate(), headers={"etag": currver})
if __name__ == '__main__':
app.run(debug=True)
Client getting resource only, if it is new
import requests
ver = "1.0"
url = "http://localhost:5000/download/app"
req = requests.get(url, headers={"If-None-Match": ver})
if req.status_code == 200:
print "new content of resource", req.content
new_ver = req.headers["etag"]
else:
print "resource did not change since last time"
Alternative solution of web part using web server (e.g. NGINX)
Assuming the resource is static file, which updates only sometime, you shall be able configuring your web server, e.g. NGINX, to serve that resource and declaring in your configuration explicit value for etag header to the version string.
Note, that as it was not requested, this alternative solution is not elaborated here (and was not tested).
Client implementation would not be modified by that (here it pays back the design is following HTTP concepts).
There are multiple ways of achieving this but as this is a Flask app, here's one using HTTP.
If the version is OK, just return a relevant status code, like a 200 OK. You can add a JSON response in the body if that's necessary. If you return a string with flask, the status code will be 200 OK and you can inspect that in your client.
If the version differs, return the URL where the file is located. The client will have to
download the file. That's pretty simple using requests. Here's a typical example for downloading file by streaming requests:
def get(url, chunk_size=1024):
""" Download a file in chunks of n bytes """
fn = url.split("/")[-1] # if you're url is complicated, use urlparse.
stream = requests.get(url, stream=True)
with open(fn, "wb") as local:
for chunk in stream.iter_content(chunk_size=chunk_size):
if chunk:
f.write(chunk)
return fn
This is very simplified. If your file is not static and cannot live on the server (like software update patches probably shouldn't) then you'll have to figure out a way to get the file from a database or generate it on the fly.
I have created a web service in django and its hosted on a shared server.The django web service respond to request from a game made in unity. But whenever game tries to request a django Web service url the server send empty resonse.Response is always:
WWW Error: server return empty string
The Unity webplayer expects a http served policy file named "crossdomain.xml" to be available on the domain you want to access with the WWW class, (although this is not needed if it is the same domain that is hosting the unity3d file).So I placed a file "crossdomain.xml" at the root of my domain ,but still i am getting same empty reply.Help plz...
EDIT:
I tried it through browser my service works fine and reply with proper response.And you know what My game can communicate to django web service when both are running on local machine.But now the django project is hosted on actual server and when game tried accessing service it never get response :(
url.py
urlpatterns = patterns('',
url(r'^crossdomain.xml$',views.CrossDomain),
url(r'^ReadFile/$',views.ReadFile),
)
views.py
def CrossDomain(request):
f = open(settings.MEDIA_ROOT+'jsondata/crossdomain.xml', 'r')
data = f.read()
f.close()
return HttpResponse(data, mimetype="application/xml")
def ReadFile(request):
f = open(settings.MEDIA_ROOT+'jsondata/some_file.json', 'r')
data = f.read()
f.close()
return HttpResponse(data, mimetype="application/javascript")
def Test(request):
return HttpResponse("Hello", mimetype="text/plain")
As I said using django for this is slight overkill because you could just serve them. Point aside though. If your serving on a different server it could be
A) Connection problems mean that your response is lost
B) Firewall issues mean that the request mean something
C) The server isn't setup correctly and therefore it justs get an error.
You need to test the response on the server. so is you access the page on the server through your browser. If so then make the game make a request and check the server error and access logs. In the apache access log you should see something like
GET "/url" 200 each time a request is made.
If you don't see any request getting through then either the request isn't made or its been lost.
If you do then the problem is in the code somewhere.