So I'm really out of my element with RESTful stuff. I'm trying to make a thing on that subscribes to an action/webhook(?) on twitch.tv so that if someone goes live, it knows.
I want to use this webhook here:
https://dev.twitch.tv/docs/api/webhooks-reference/#topic-stream-changed
I made a Flask server in server.py:
app = Flask(__name__)
#app.route('/', methods=['POST'])
def result():
print(request.data)
return 'Received'
and the POST I make is over here in notify.py:
import requests
r = requests.post("https://api.twitch.tv/helix/webhooks/hub", headers={'Client-ID': client_id}, data={"hub.callback": "http://127.0.0.1:5000/","hub.mode":"subscribe",'hub.topic':"https://api.twitch.tv/helix/streams?user_id=XXXXXX"})
Running my code shows nothing on the server, not even Received so I guess I'm doing something wrong.
if I do a GET, on
request = requests.get('https://api.twitch.tv/helix/streams?user_id=xxxxxx', headers={'Client-ID': client_id})
the result is b ' '
and I have no idea what that means
in notifiy.py, putting a print(r) returns <Response [202]> but I think I want a [200]
I assume my server needs to be reachable by Twitch to see it but I'm not sure.
Any help is appreciated!
Final EDIT... I have created a proof of concept
You're POSTing "hub.callback": "http://127.0.0.1:5000/". That URL is only accessible on your machine.
This is supposed to be a URL which is accessible from the Twitch infrastructure. If you can't register a domain, then you could use something like ngrok to get a valid URI which routes back to your Flask development server for testing.
Once you have sent the POST request, you can get the webhook subscriptions to confirm the post request worked. This is also possible using curl, with the commands included to the right of this documentation.
Assuming you see valid subscriptions there, then the endpoint you provide as hub.callback should receive a hit from Twitch...
when a stream changes; e.g., stream goes online or offline, the stream title changes, or the game changes.
Within the route you'd then do some logic to deal with the result of that request.
Update re comments
You may wish to try updating hub.lease_seconds: (here)
Number of seconds until the subscription expires. Default: 0. Maximum: 864000.
The default (0) allows you to test the subscription-creation workflow without creating any subscriptions (since they expire immediately). After testing, to actually create subscriptions, you must specify a larger value.
This belongs in the dictionary passed as the data argument to requests.post in notify.py:
Related
So essentially, what I'm trying to do is remove a user's session id whenever they leave the website/close their tab because I want them to re-login every time they access the site. I've heard from other Stack Overflow questions that I should try out the Flask-SocketIO extension, and use the disconnect event to detect when they leave the website and then pop their ids from the session. So that's exactly what I did, however, whenever I pop the session, it doesn't actually register. Here's the full code I used to try implement that.
# Socket IO Events
#socket_io.on('connect')
def on_connect():
app.logger.info("Connected!")
app.logger.info("SESSION INFO: " + str(session))
#socket_io.on('disconnect')
def on_disconnect():
app.logger.info("Client disconnected!")
app.logger.info("SESSION INFO: " + str(session))
if 'id' in session:
session.pop('id')
So as you can tell, whenever I sign up and head to the home page, I receive a session id, and when this disconnect event fires, my session id gets popped. However, take a look at this output.
[2021-07-30 14:20:39,765] INFO in app: Connected!
[2021-07-30 14:20:39,766] INFO in app: SESSION INFO: {'id': 1}
yjI1iUG5o3YmBgRlAAAA: Sending packet PING data None
9jo8DD7RQ5mEcUWZAAAI: Upgrade to websocket successful
yjI1iUG5o3YmBgRlAAAA: Received packet PONG data
HYTS2jEcmhqp9Vq5AAAE: Sending packet PING data None
KQznMBiop36XZLcTAAAC: Client is gone, closing socket
[2021-07-30 14:20:53,854] INFO in app: Client disconnected!
[2021-07-30 14:20:53,854] INFO in app: SESSION INFO: {}
[2021-07-30 14:21:35,164] INFO in app: Connected!
hBzSZoZ-W7_nesEBAAAK: Received request to upgrade to websocket
[2021-07-30 14:21:35,168] INFO in app: SESSION INFO: {'id': 1}
After connecting to the page, it gives me a session id. Then, when I disconnect, it removes my session id and it clearly shows over there, that it removed my session id as there isn't anything in the session dict. However, when I reconnect, it like automatically gives me my session id.
Now based on what I've read from this other question on removing session ids, I cannot use socketio to alter the client's cookies, that's at least what I understood from it. He also said that it'd be better to store the sessions on the server side. But I find that a little troublesome and I don't want to just give up on this. Is there any way that I may store client sessions using Flask's built-in sessions system(that store cookies on the client's side), but still allow me to alter them from a socket-io perspective? I'm just very lost on this. Hope someone can explain how socket-io works or just provide a good and comprehensive article on it. Thanks in advance :) But the main problem is, Flask-SocketIO isn't popping the session id when I tell it to and I'm not sure why.
I'm sorry but you have been misled. Using Flask-SocketIO just so that you can be notified when a person leaves your site is extremely overkill, and as you've seen, it doesn't even work for your purposes.
If you want to know why you can't make changes to the user session from a Socket.IO event handler the reason is that Flask-SocketIO uses WebSocket. The user session is maintained in a cookie, which can only be modified in a server response to an HTTP request initiated by the client. WebSocket has no ability to modify cookies.
So I would forget about using Flask-SocketIO for this. it's really not one of its use cases.
Instead, I suggest you look into adding a beforeunload event in your page, where you can delete the session cookie directly in the client. For this to work you will also need to set the SESSION_COOKIE_HTTPONLY configuration option to Flask, so that JavaScript in your page can see, modify and delete the cookie. Note that this may have security implications, so think about it very carefully before doing it.
I am a bit confused as to how these callbacks work.
I have seen some REST APIs ask for callback URLs mainly for authentication(Oauth) and if I understand correctly, the API calls an endpoint implemented at our backend once they're done. But if this is a website and the API calls a backend how will it call the particular session that an user triggered that API call from?
Do we generate some unique ID and use that to identify each session?
Also is it possible to implement something like this for uses other than authentication?
Excuse me if this seems trivial but I've been trying to figure it out for hours now.
Thanks in advance! :)
P.S: I'm trying it out in python using FastAPI.
Yes, you usually include a key that identifies the "transaction". Either one that is pre-shared (for webhooks where an API makes callbacks at specific times when an event happens and the request isn't a response to something initiated by you), or one that you generate to identify the session itself after redirecting a user outside of your own application.
remote server makes a request to /webhooks?key=foo
.. or what usually happens, includes a signature as an HTTP header that signs the message based on an already exchanged key.
See for example Stripe's webhook documentation for an example of how the request is signed.
However, in the case where the user is redirected back to your application (which is the flow for many authentication schemes), the redirect back is given to the user's client - and their session will still be active. In that case the redirect will include a session identifier from the authentication system, which you can then use in your backend to verify that the user actually was authenticated and with what autorization.
/foo -> redirect -> https://remote/auth
(enters authentication information)
https://remote/auth -> redirect -> /authenticated?secret_key=foobar
I have a web application that uses GitHub's OAuth API in order to allow the app's users to interact with GitHub. However, I'm seeing some very odd behaviour with regards to the session cookie.
As a bit of background, I am using peewee to interface with Heroku's Postgres server, and have a User model like so:
class User(peewee.Model):
login = peewee.TextField(unique=False)
token = peewee.TextField()
I am using the web application flow described in the GitHub OAuth documentation, and am successfully getting called back with an access token, which I store in the database, and also in the session [1]:
#app.route('/callback')
def finishlogin():
# I've verified that `token` and `login` are both valid at this point
user = User.create(login=login, token=token)
session['token'] = token
return redirect(url_for('home'))
My route for home is as follows:
#app.route('/')
def home():
if 'token' in session:
user = User.get(token=session.get('token'))
return 'Your login is {}'.format(user.login)
else:
# ...
So far, so good, and this works correctly. However, I am experiencing instances of users logging in, refreshing the page and finding that they are suddenly logged in as someone else. Logging the requests to the app shows that on the second request the session cookie itself has sent the wrong value (i.e. session.get('token') in home() returns a valid, but incorrect value. Clearly the user's browser can't know any other session value, so it seems that there is some "leakage" in setting the session between different clients and requests.
I'm not sure what the problem might be. My database is stored on the Flask g object as described in the peewee docs and has before_request and teardown_request hooks set up to open and close the database connection, and from all the documentation and example code I have read (and I've read a lot!), I seem to be using the session object correctly. I have set up a working secret_key for the session store.
I'm wondering if this could be something going on with Heroku and their routing mesh? But then, how would one user suddenly send another user's session?
Any hints or advice would be appreciated—I've been staring at this for a long time and am at my wits' end.
[1] I'm aware that storing the token directly is a bad design choice. The application is non-public and this will be fixed, but for now I want to describe the problem as it exists, even though it's not ideal.
Answering my own question for future reference.
It seems that this was being caused by Flask's default session cookie behaviour, which is to send a Set-Cookie header with every single request, even for static assets. Our local Squid proxy was therefore gladly caching those requests and re-issuing Set-Cookie headers for every user.
Setting Cache-Control headers for the whole app seems to fix the issue.
I have and API service that make this:
session[parent].pop(child)
But, when I call this service more than one time and at the same time in Browser, this seems too fast for Flask (or Python, or I don't know). Just one 'child' is removed. I tried to use
del session[parent][child]
but the problem was the same. I can get a list of values in my API Service to resolve that, but, I want to understand why this is happening.
I don't know if it is a Flask problem, a Python problem, a 'Web Stuff' problem...
It's a 'Web Stuff' problem.
What happens is that the browser stores the last version it received. But if it receives responses out of order, or you abort a request before it is completed, the browser won't be storing that version.
Flask stores the data for session entirely in a cookie. There is nothing stored on the server side other than the server-side secret used to encrypt and decrypt the contents.
A response with that cookie is sent to the browser, and the browser stores the cookie. This is an entirely opaque piece of data to the browser, it cannot do anything with it as it is compressed and cryptographically signed.
The browser will then send that cookie as is back to the Flask server every time a request is made. If the response to that request contains a new version of the cookie, then that'll be stored in the browser cookie storage. Any new request started after storing will then use the new cookie.
If however, you start a request before a response has been fully processed, or did not complete handling the response, then an older cookie could be used and your server decodes that older version with the changes not made.
Just set session.modified = True each time you modify your session.
This tells flask to update session after request.
Originally, I tried to post an ajax request from my client side to a third party url, but it seems that the browser have security issues with that. I thought about sending an ajax to the server side, from there to send a GET request to the third party, get the response and send it back to the client side. How can I do that with flask?
Install the requests module (much nicer than using urllib2) and then define a route which makes the necessary request - something like:
import requests
from flask import Flask
app = Flask(__name__)
#app.route('/some-url')
def get_data():
return requests.get('http://example.com').content
Depending on your set up though, it'd be better to configure your webserver to reverse proxy to the target site under a certain URL.
Flask alone does not have this capability, but it is a simple matter to write a request handler that makes a request to another server using an HTTP client library and then return that response.
# third-party HTTP client library
import requests
# assume that "app" below is your flask app, and that
# "Response" is imported from flask.
#app.route("/proxy-example")
def proxy_example():
r = requests.get("http://example.com/other-endpoint")
return Response(
r.text,
status=r.status_code,
content_type=r.headers['content-type'],
)
However, this will not achieve exactly the same result as you might expect from a client-side request. Since your server cannot "see" any cookies that the client browser has stored for the target site, your proxied request will be effectively anonymous and so, depending on the target site, may fail or give you a different response than you'd get requesting that resource in the browser.
If you have a relationship with the third-party URL (that is, if you control it or are able to work with the people who do) they can give access for cross-domain requests in the browser using CORS (which is only supported in modern browsers) or JSON-P (an older workaround that predates CORS).
The third-party provider could also give you access to the data you want at an endpoint that is designed to accept requests from other servers and that provides a mechanism for you to authenticate your app. The most popular protocol for this is OAuth.
As the other answers have stated using the requests module for python would be the best way to tackle this from the coding perspective. However as the comments mentioned (and the reason I came to this question) this can give an error that the request was denied. This error is likely cause by SELinux.
To check if this is the issue first make sure SELinux is enabled with this command:
sestatus
If 'Current Mode' is 'enforcing' then SELinux is enabled.
Next get the current bool values that apply directly to apache with this command:
getsebool -a | grep httpd
Look for the setting 'httpd_can_network_connect' this determines if apache is allowed to make TCP requests out to the network. If it is on then all apache TCP requests will be allowed. To turn it on run the following as root:
setsebool -P httpd_can_network_connect 1
If you only need database access (I had this problem before which is why I suspected SELinux here) then it would probably be better to only turn on 'httpd_cna_network_connect'.
The purpose of this policy is that if a hacker was to hijack your apache server they would not be able to get out through the server to the rest of your internal network.
This probably would've been better as a comment but I don't have enough rep..
Sources:
https://tag1consulting.com/blog/stop-disabling-selinux
https://wiki.centos.org/TipsAndTricks/SelinuxBooleans