i need a little help here, since i am new to python, i am trying to do a nice app that can tell me if my website is down or not, then send it to twitter.
class Tweet(webapp.RequestHandler):
def get(self):
import oauth
client = oauth.TwitterClient(TWITTER_CONSUMER_KEY,
TWITTER_CONSUMER_SECRET,
None
)
webstatus = {"status": "this is where the site status need's to be",
"lat": 44.42765100,
"long":26.103172
}
client.make_request('http://twitter.com/statuses/update.json',
token=TWITTER_ACCESS_TOKEN,
secret=TWITTER_ACCESS_TOKEN_SECRET,
additional_params=webstatus,
protected=True,
method='POST'
)
self.response.out.write(webstatus)
def main():
application = webapp.WSGIApplication([('/', Tweet)])
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
now the check website part is missing, so i am extremely new to python and i need a little bit of help
any idea of a function/class that can check a specific url and the answer/error code can be send to twitter using the upper script
and i need a little bit of help at implementing url check in the script above, this is my first time interacting with python.
if you are wondering, upper class uses https://github.com/mikeknapp/AppEngine-OAuth-Library lib
cheers
PS: the url check functionality need's to be based on urlfetch class, more safe for google appengine
You could use Google App Engine URL Fetch API.
The fetch() function returns a Response object containing the HTTP status_code.
Just fetch the url and check the status with something like this:
from google.appengine.api import urlfetch
def is_down(url):
result = urlfetch.fetch(url, method = urlfetch.HEAD)
return result.status_code != 200
Checking if a website exists:
import httplib
from httplib import HTTP
from urlparse import urlparse
def checkUrl(url):
p = urlparse(url)
h = HTTP(p[1])
h.putrequest('HEAD', p[2])
h.endheaders()
return h.getreply()[0] == httplib.OK
We only get the header of a given URL and check the response code of the web server.
Update: The last line is modified according to the remark of Daenyth.
Related
So I have a Python script that sends a get request to an API and returns information. For educational purposes, I wanted to create an API that I could pass parameters to which would return results from that script. I couldn't really find a similar example on stackoverflow or on the official flask documentation for an example that meets what I'm looking for. Everything I'm seeing is for returning data that you already have existing in a database or in a json file.
Here's an example of the script that I have right now.
script.py
api_url = www.statetaxes.com
parameters = {"taxinfo": "state:Texas county:soaker_county zip_code:78331"}
token = "abcd123456"
response= request.get(api_url, params=parameters, headers={"authentication":token})
print(response.json()) >----- [{tax rate: 1.5%, education_quality:great, crime_rate:0}]
The user would set the parameters in a config file somewhere like so:
[taxinfo]
county: some_county
zip_code: 12345
I'd want to read that text file, grab those parameters and push it to an API endpoint. I can figure out the part on how to read those parameters and send it to the API, but I'm not sure how I'm supposed to be configuring the rest of my FLASK app. I'd assume that I'd have to take those parameters which I'm reading from the text file as an argument. Am I on the right path here?
#app.route('/taxes/<taxinfo>', methods = ['GET', 'POST', 'PUT' ])
def state_taxes(taxinfo):
def api_call():
api_url = www.statetaxes.com
parameters=taxinfo
token = "abcd123456"
response= request.get(api_url, params=parameters, headers={"authentication":token})
return api_call
if __name__== "__main__":
app.run(debug=True)
I'm setting up a small Python service to act as an REST API reverse proxy, but hoping there's some libraries available to help speed this process up.
Need to be able to run a function to calculate a variable to inject as a request header when the request is proxied through to the backend.
As it stands I have a simpler script to do the function to get the variable and inject it into a Nginx config file and then force a Nginx hot reload via signals, but trying to remove this dependency for what should be a fairly simple task.
Would a good approach be to use falcon as the listener and combine it with another approach to inject and forward requests?
Thanks for reading.
Edit: Been reading https://aiohttp.readthedocs.io/en/stable/ as it seems to be the right direction.
Thanks to someone over at falcon, this is now the accepted answer!
import io
import falcon
import requests
class Proxy(object):
UPSTREAM = 'https://httpbin.org'
def __init__(self):
self.session = requests.Session()
def handle(self, req, resp):
headers = dict(req.headers, Via='Falcon')
for name in ('HOST', 'CONNECTION', 'REFERER'):
headers.pop(name, None)
request = requests.Request(req.method, self.UPSTREAM + req.path,
data=req.bounded_stream.read(),
headers=headers)
prepared = request.prepare()
from_upstream = self.session.send(prepared, stream=True)
resp.content_type = from_upstream.headers.get('Content-Type',
falcon.MEDIA_HTML)
resp.status = falcon.get_http_status(from_upstream.status_code)
resp.stream = from_upstream.iter_content(io.DEFAULT_BUFFER_SIZE)
api = falcon.API()
api.add_sink(Proxy().handle)
I'm trying to get Flask to handle cross-site scripting properly. I've taken the crossdomain decorator snippet from here:
http://flask.pocoo.org/snippets/56/
In the code below, I've put the decorator snippet and the basic flask server.
I'm calling the decorator with headers='Content-Type' because otherwise I was getting "Request header field Content-Type is not allowed by Access-Control-Allow-Headers." in the browser.
So here is my question:
As-is, the code below works. But when I want to restrict to only a specific server like so:
#crossdomain(origin='myserver.com', headers='Content-Type')
I get the browser error
"Origin http://myserver.com is not allowed by Access-Control-Allow-Origin."
I can't get it working for anything other than origin='*'.
Does anybody have any ideas?
Here is the complete code:
from datetime import timedelta
from flask import make_response, request, current_app, Flask, jsonify
from functools import update_wrapper
def crossdomain(origin=None, methods=None, headers=None,
max_age=21600, attach_to_all=True,
automatic_options=True):
if methods is not None:
methods = ', '.join(sorted(x.upper() for x in methods))
if headers is not None and not isinstance(headers, basestring):
headers = ', '.join(x.upper() for x in headers)
if not isinstance(origin, basestring):
origin = ', '.join(origin)
if isinstance(max_age, timedelta):
max_age = max_age.total_seconds()
def get_methods():
if methods is not None:
return methods
options_resp = current_app.make_default_options_response()
return options_resp.headers['allow']
def decorator(f):
def wrapped_function(*args, **kwargs):
if automatic_options and request.method == 'OPTIONS':
resp = current_app.make_default_options_response()
else:
resp = make_response(f(*args, **kwargs))
if not attach_to_all and request.method != 'OPTIONS':
return resp
h = resp.headers
h['Access-Control-Allow-Origin'] = origin
h['Access-Control-Allow-Methods'] = get_methods()
h['Access-Control-Max-Age'] = str(max_age)
if headers is not None:
h['Access-Control-Allow-Headers'] = headers
return resp
f.provide_automatic_options = False
return update_wrapper(wrapped_function, f)
return decorator
app = Flask(__name__)
#app.route('/my_service', methods=['POST', 'OPTIONS'])
#crossdomain(origin='*', headers='Content-Type')
def my_service():
return jsonify(foo='cross domain ftw')
if __name__ == '__main__':
app.run(host="0.0.0.0", port=8080, debug=True)
For reference my python version is 2.7.2
Flask version is 0.7.2
I just tried the same code with python version 2.7.3 and Flask version 0.8.
With these versions, it fails with
#crossdomain(origin='myserver.com', headers='Content-Type')
but it works with
#crossdomain(origin='http://myserver.com', headers='Content-Type')
Perhaps it just doesn't work with Flask 0.7.2? (despite what it says on the snippet page).
EDIT:
After playing with this a lot more (and upgrading to Flask 0.9) it seems that the real problem (or yet another problem) might be related to having multiple allowed origins in a list. In other words, using the above code like this:
#crossdomain(origin=['http://myserver.com', 'http://myserver2.com'], headers='Content-Type')
doesn't work.
To fix this problem I tweaked the decorator. See code here:
http://chopapp.com/#351l7gc3
This code returns only the domain of the requesting site if it is in the list. Kinda quirky, but at least for me, problem solved :)
Python is trying hard to prevent you from exposing yourself to cross site scripting attacks.
One fix is by giving in, and having your requests hit the same server the flask script is running on. Fetching JSON from far away servers defined in strings is risky business anyway.
I was able to fix it by letting the browser keep itself on the same server, like this:
$('a#calculate').bind('click', function() {
$.getJSON('/_add_numbers', {
a: $('input[name="a"]').val(),
b: $('input[name="b"]').val()
}, function(data) {
$("#result").text(data.request);
});
return false;
});
Notice how getJSON method is passed a /_add_numbers. That communicates to the browser to stay on the same host and look for that page. Then the browser is happy and secure we are staying on the same host, and you never get the error:
Origin http://myserver.com is not allowed by Access-Control-Allow-Origin
I am a newcomer to the Python and Twisted game so excuse the ignorance I will likely be asking this question with. As a sort of first program, I am trying to write a basic HTTP server using twisted.web.sever which would simply print to screen the HTTP request, and then print to screen the HTTP response. I am trying to print the entire message. Here is what I have so far:
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.resource import Resource
import time
class TestPage(Resource):
isLeaf = True
def render_GET(self, request):
response = "Success"
print "You're request was %s" % request
print "The sever's response was %s" % response
return response
resource = TestPage()
factory = Site(resource)
reactor.listenTCP(8000, factory)
reactor.run()
So far, I am having success printing the request. What I want to know is where I can access the raw response data, not just the textual message. Also, if I wanted to start parsing the request/response for information, what would be the best way to go about doing that?
Edit: I'm also new to stackoverflow, how do I get this code to display properly?
Take a look at the Request and IRequest API docs to get an idea of what that request parameter offers you. You should be able to find just about everything in the request there.
I'm not sure what you mean by raw response data though. The response is up to you to generate.
I have a Google App Engine app - http://mylovelyapp.appspot.com/
It has a page - mylovelypage
For the moment, the page just does self.response.out.write('OK')
If I run the following Python at my computer:
import urllib2
f = urllib2.urlopen("http://mylovelyapp.appspot.com/mylovelypage")
s = f.read()
print s
f.close()
it prints "OK"
the problem is if I add login:required to this page in the app's yaml
then this prints out the HTML of the Google Accounts login page
I've tried "normal" authentication approaches. e.g.
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(None,
uri='http://mylovelyapp.appspot.com/mylovelypage',
user='billy.bob#gmail.com',
passwd='billybobspasswd')
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
But it makes no difference - I still get the login page's HTML back.
I've tried Google's ClientLogin auth API, but I can't get it to work.
h = httplib2.Http()
auth_uri = 'https://www.google.com/accounts/ClientLogin'
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
myrequest = "Email=%s&Passwd=%s&service=ah&source=DALELANE-0.0" % ("billy.bob#gmail.com", "billybobspassword")
response, content = h.request(auth_uri, 'POST', body=myrequest, headers=headers)
if response['status'] == '200':
authtok = re.search('Auth=(\S*)', content).group(1)
headers = {}
headers['Authorization'] = 'GoogleLogin auth=%s' % authtok.strip()
headers['Content-Length'] = '0'
response, content = h.request("http://mylovelyapp.appspot.com/mylovelypage",
'POST',
body="",
headers=headers)
while response['status'] == "302":
response, content = h.request(response['location'], 'POST', body="", headers=headers)
print content
I do seem to be able to get some token correctly, but attempts to use it in the header when I call 'mylovelypage' still just return me the login page's HTML. :-(
Can anyone help, please?
Could I use the GData client library to do this sort of thing? From
what I've read, I think it should be able to access App Engine apps,
but I haven't been any more successful at getting the authentication working for App Engine stuff there either
Any pointers to samples, articles, or even just keywords I should be
searching for to get me started, would be very much appreciated.
Thanks!
appcfg.py, the tool that uploads data to App Engine has to do exactly this to authenticate itself with the App Engine server. The relevant functionality is abstracted into appengine_rpc.py. In a nutshell, the solution is:
Use the Google ClientLogin API to obtain an authentication token. appengine_rpc.py does this in _GetAuthToken
Send the auth token to a special URL on your App Engine app. That page then returns a cookie and a 302 redirect. Ignore the redirect and store the cookie. appcfg.py does this in _GetAuthCookie
Use the returned cookie in all future requests.
You may also want to look at _Authenticate, to see how appcfg handles the various return codes from ClientLogin, and _GetOpener, to see how appcfg creates a urllib2 OpenerDirector that doesn't follow HTTP redirects. Or you could, in fact, just use the AbstractRpcServer and HttpRpcServer classes wholesale, since they do pretty much everything you need.
thanks to Arachnid for the answer - it worked as suggested
here is a simplified copy of the code, in case it is helpful to the next person to try!
import os
import urllib
import urllib2
import cookielib
users_email_address = "billy.bob#gmail.com"
users_password = "billybobspassword"
target_authenticated_google_app_engine_uri = 'http://mylovelyapp.appspot.com/mylovelypage'
my_app_name = "yay-1.0"
# we use a cookie to authenticate with Google App Engine
# by registering a cookie handler here, this will automatically store the
# cookie returned when we use urllib2 to open http://currentcost.appspot.com/_ah/login
cookiejar = cookielib.LWPCookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar))
urllib2.install_opener(opener)
#
# get an AuthToken from Google accounts
#
auth_uri = 'https://www.google.com/accounts/ClientLogin'
authreq_data = urllib.urlencode({ "Email": users_email_address,
"Passwd": users_password,
"service": "ah",
"source": my_app_name,
"accountType": "HOSTED_OR_GOOGLE" })
auth_req = urllib2.Request(auth_uri, data=authreq_data)
auth_resp = urllib2.urlopen(auth_req)
auth_resp_body = auth_resp.read()
# auth response includes several fields - we're interested in
# the bit after Auth=
auth_resp_dict = dict(x.split("=")
for x in auth_resp_body.split("\n") if x)
authtoken = auth_resp_dict["Auth"]
#
# get a cookie
#
# the call to request a cookie will also automatically redirect us to the page
# that we want to go to
# the cookie jar will automatically provide the cookie when we reach the
# redirected location
# this is where I actually want to go to
serv_uri = target_authenticated_google_app_engine_uri
serv_args = {}
serv_args['continue'] = serv_uri
serv_args['auth'] = authtoken
full_serv_uri = "http://mylovelyapp.appspot.com/_ah/login?%s" % (urllib.urlencode(serv_args))
serv_req = urllib2.Request(full_serv_uri)
serv_resp = urllib2.urlopen(serv_req)
serv_resp_body = serv_resp.read()
# serv_resp_body should contain the contents of the
# target_authenticated_google_app_engine_uri page - as we will have been
# redirected to that page automatically
#
# to prove this, I'm just gonna print it out
print serv_resp_body
for those who can't get ClientLogin to work, try app engine's OAuth support.
Im not too familiar with AppEngine, or Googles web apis, but for a brute force approach you could write a script with something like mechanize (http://wwwsearch.sourceforge.net/mechanize/) to simply walk through the login process before you begin doing the real work of the client.
I'm not a python expert or a app engine expert. But did you try following the sample appl at http://code.google.com/appengine/docs/gettingstarted/usingusers.html. I created one at http://quizengine.appspot.com, it seemed to work fine with Google authentication and everything.
Just a suggestion, but look in to the getting started guide. Take it easy if the suggestion sounds naive. :)
Thanks.