We Keep Coding

Using Twilio with Python to make a person-to-person call

I have setup a trial Programmable Voice account with Twilio. I am using a Zoiper softphone endpoint. I am attempting to have my client's server initiate a call through Twilio to a live person. I need to actually speak with the person called.
However, in using the tutorial code, Twilio's "url" parameter intercepts the call with it's own voice message:
from twilio.rest import Client
account_sid = 'Axxxx'
auth_token = 'xxxxx'
client = Client(account_sid, auth_token)
call = client.calls.create(
url='http://demo.twilio.com/docs/classic.mp3',
to='+15553334929',
from_='+18334447682'
)
print(call.sid)
The call recipient hears the the message, but we cannot speak together.
Twilio's own tutorial materials are on how to use either TwilML or the "url" parameter to have the computer automatically work with the call. However, I do not need that. I need to have two live people speak to each other once the server initiates the call through Twilio. Right now, the "url" / TwilML is just standing between the live people.

This blog shows an example that makes an outgoing call and then joins the caller into the call. See the "Start a two-user call from your App" section.
You might also want to take a look at this answer which shows another way to do this with the JS API (the principals should be easily transferable to the python API). Basically it makes two calls and joins them in a Conference.
Disclaimer: I'm not a lawyer and this is not legal advice, but I'm assuming you are familiar with TCPA and have reviewed regulations/requirements related to that. It's possible that your code initiating the outgoing call in an automated fashion (instead of an agent clicking a button manually to initiate a call) changes your level of regulatory exposure.

This is called, Call Forwarding. The article below provides all the different ways to accomplish this task
Setting Up Call Forwarding

Twilio developer evangelist here.
Welcome to StackOverflow!
You can do this with Twilio Studio. There in your dashboard, click the + button to make a new flow and call it whatever you wish.
It comes with the Trigger widget, which will initiate your flow when the trigger (in this case, an Incoming Call) is fired.
You only need one widget: the Connect Call To widget. Drag it onto the canvas and connect the dot from that initial Incoming Call trigger to the dot in the upper left corner of that new Connect Call widget. Select Single Number in the right sidebar dropdown and enter whatever number you'd like to forward calls to.
Lastly, you'll need a Twilio phone number. Purchase one in the Twilio Phone numbers section of your console.
Scroll down to the Voice & Fax section and select Webhooks, TwiML Bins, Functions, Studio, or Proxy from the initial dropdown. Next to “A Call Comes In” select Studio Flow and choose the flow you just made to connect/relate it to the number. Lastly click Save and tada! If someone calls your Twilio number you just purchased, they will route you to the number you specified in the Twilio Studio flow.
Let me know if this helps at all! :D

Is there a way to limit the number of concurrent requests from one IP with Gunicorn?

Basically I'm running a Flask web server that crunches a bunch of data and sends it back to the user. We aren't expecting many users ~60, but I've noticed what could be an issue with concurrency. Right now, if I open a tab and send a request to have some data crunched, it takes about 30s, for our application that's ok.
If I open another tab and send the same request at the same time, unicorn will do it concurrently, this is great if we have two seperate users making two seperate requests. But what happens if I have one user open 4 or 8 tabs and send the same request? It backs up the server for everyone else, is there a way I can tell Gunicorn to only accept 1 request at a time from the same IP?

A better solution to the answer by #jon would be limiting the access by your web server instead of the application server. A good way would always be to have separation between the responsibilities to be carried out by the different layers of your application. Ideally, the application server, flask should not have any configuration for the limiting or anything to do with from where the requests are coming. The responsibility of the web server, in this case nginx is to route the request based on certain parameters to the right client. The limiting should be done at this layer.
Now, coming to the limiting, you could do it by using the limit_req_zone directive in the http block config of nginx
http {
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
...
server {
...
location / {
limit_req zone=one burst=5;
proxy_pass ...
}
where, binary_remote_addris the IP of the client and not more than 1 request per second at an average is allowed, with bursts not exceeding 5 requests.
Pro-tip: Since the subsequent requests from the same IP would be held in a queue, there is a good chance of nginx timing out. Hence, it would be advisable to have a better proxy_read_timeout and if the reports take longer then also adjusting the timeout of gunicorn
Documentation of limit_req_zone
A blog post by nginx on rate limiting can be found here

This is probably NOT best handled at the flask level. But if you had to do it there, then it turns out someone else already designed a flask plugin to do just this:
https://flask-limiter.readthedocs.io/en/stable/
If a request takes at least 30s then make your limit by address for one request every 30s. This will solve the issue of impatient users obsessively clicking instead of waiting for a very long process to finish.
This isn't exactly what you requested, since it means that longer/shorter requests may overlap and allow multiple requests at the same time, which doesn't fully exclude the behavior you describe of multiple tabs, etc. That said, if you are able to tell your users to wait 30 seconds for anything, it sounds like you are in the drivers seat for setting UX expectations. Probably a good wait/progress message will help too if you can build an asynchronous server interaction.

Python POST request gives unknown result

So I'm really out of my element with RESTful stuff. I'm trying to make a thing on that subscribes to an action/webhook(?) on twitch.tv so that if someone goes live, it knows.
I want to use this webhook here:
https://dev.twitch.tv/docs/api/webhooks-reference/#topic-stream-changed
I made a Flask server in server.py:
app = Flask(__name__)
#app.route('/', methods=['POST'])
def result():
print(request.data)
return 'Received'
and the POST I make is over here in notify.py:
import requests
r = requests.post("https://api.twitch.tv/helix/webhooks/hub", headers={'Client-ID': client_id}, data={"hub.callback": "http://127.0.0.1:5000/","hub.mode":"subscribe",'hub.topic':"https://api.twitch.tv/helix/streams?user_id=XXXXXX"})
Running my code shows nothing on the server, not even Received so I guess I'm doing something wrong.
if I do a GET, on
request = requests.get('https://api.twitch.tv/helix/streams?user_id=xxxxxx', headers={'Client-ID': client_id})
the result is b ' '
and I have no idea what that means
in notifiy.py, putting a print(r) returns <Response [202]> but I think I want a [200]
I assume my server needs to be reachable by Twitch to see it but I'm not sure.
Any help is appreciated!

Final EDIT... I have created a proof of concept
You're POSTing "hub.callback": "http://127.0.0.1:5000/". That URL is only accessible on your machine.
This is supposed to be a URL which is accessible from the Twitch infrastructure. If you can't register a domain, then you could use something like ngrok to get a valid URI which routes back to your Flask development server for testing.
Once you have sent the POST request, you can get the webhook subscriptions to confirm the post request worked. This is also possible using curl, with the commands included to the right of this documentation.
Assuming you see valid subscriptions there, then the endpoint you provide as hub.callback should receive a hit from Twitch...
when a stream changes; e.g., stream goes online or offline, the stream title changes, or the game changes.
Within the route you'd then do some logic to deal with the result of that request.
Update re comments
You may wish to try updating hub.lease_seconds: (here)
Number of seconds until the subscription expires. Default: 0. Maximum: 864000.
The default (0) allows you to test the subscription-creation workflow without creating any subscriptions (since they expire immediately). After testing, to actually create subscriptions, you must specify a larger value.
This belongs in the dictionary passed as the data argument to requests.post in notify.py:

WebSocket Safety

I decided to write a websocket chat that he supported text messaging [unlimited number of characters, not like Twitter :)] and file transfer.
Ask this question. How to make banal identification data? That is, when the connection is open, the server is just a set of bytes, and even if there is json, it can be easy to substitute: mark as "message" and send confusing file(since the number of characters is not limited, it will sent to all people users).
That is, briefly, how to discern what comes from the user?
Thanks in advance!
P.S. Transfer files via jquery is not accepted, as the websocket is not protected.
UPD
Any wrote that did not understand a question essence. Explain: 1. There is a chat on tornado WS. 2. In chat can send as text messages and files. 3. I control it with javascript BUT if someone will make a connection and send a file for example, as a message? Clients receives a huge number of "unknown byte". The situation is like this - {'msg': 'Hello, world!!1'}; {'file': a file}. All is well, but it would be a shame if {'msg': a file}. It should be clear ;)

Welcome to SO.
Before I answer your question, allow me to clarify a few points about this site and our community. This is your first question, so maybe you didn't know:
SO (StackOverflow) is not a forum, it's more like both a chat and a library. If you ask a question, you are expected to stay on line for at least an hour or two to check in and give more information if somebody asks you.
If somebody asks for more information, edit the question - don't answer inside the comments (you can add a comment saying that you updated the question, but the question should be edited).
If you are not here to clarify your question, it WILL be closed because there are many people wanting answers and we give priority to the people who are here to respond.
It is important that your questions include information about what you already tried. It is better if your questions include some of the code you wrote when you tried your own solution.
Otherwise, it feels like you are trying to outsource your work to the community - which is a very big No No.
It is important that your questions are clear and that you write in the best English that you can manage.
Please read the comments to see what I mean about the community's expectations.
As for Websocket security:
I do not know what language you work with, so I will write in Ruby because it's easier for me and I think the code will be easier to read.
Websockets start as an HTTP connection which is then "upgraded" to a websocket connection.
Since most applications already have HTTP security / authentication logic that they wrote for the website, it is best to use this same security / authentication logic for websockets.
This is usually done BEFORE the connection is upgraded.
It is true that sometimes people write a new security / authentication logic layer for websockets, but that is not the most effective way to deal with this issue (mainly because messages could be sent to the websocket while authentication is still being processed, which starts a new world of issues and considerations).
Here is an example, using the Plezi framework, that requires authentication before the upgrade. test this example by installing the plezi gem (terminal: $ [sudo] gem install plezi) and running the following code in irb (terminal : $ irb):
require 'plezi'
class WebsocketSecDemo
# pre_connect is called before the upgrade, while still in HTTP mode.
def pre_connect
# return false (refuse the connection) unless the auth logic is okay
return false unless auth_logic
true
end
def on_message data
puts "got #{data}"
response << data
broadcast :ws_write, data
end
def index
"Check this using http://www.websocket.org/echo.html
\r\nConnect to the websocket at ws://localhost:3000/
\r\nTo authenticate, visit http://localhost:3000/login
\r\nTo un-authenticate visit http://localhost:3000/logout"
end
def login
cookies[:my_auth] = true
redirect_to :index
end
def logout
cookies[:my_auth] = nil
redirect_to :index
end
# protected methods aren't available to the HTTP router.
protected
def ws_write data
response << data
end
# The worst auth logic in the history of man kind...
def auth_logic
cookies[:my_auth] == true
end
end
Plezi.listen
Plezi.route '/', WebsocketSecDemo
# exit the terminal to start the demo
exit
Now visit:
localhost:3000 - for instructions
localhost:3000/login - to authenticate
localhost:3000/logout - to remove authentication
Also use www.websocket.org/echo.html a few times to try and connect to the websocket at ws://localhost:3000/ - try this:
before authentication;
after authentication (login); and
after you 'logout'...
Good Luck!

How to make sure that my AJAX requests are originating from the same server in Python

I have already asked a question about IP Authentication here: TastyPie Authentication from the same server
However, I need something more! An IP address could be very easily spoofed.
Scenario: My API (TastyPie) and Client App (in javascript) are on the same server/site/domain. My users don't login. I want to consume my API in my javascript client side.
Question: How can I make sure (authentication) that my AJAX requests are originating from the same server?
I'm using Tatypie. I need to authentication that the requests from the client are being made on the same server/domain etc. I cannot use 'logged in sessions' as my users don't login.
I have looked at private keys and generating a signature but they can viewed in the javascript making that method insecure. If I do it in a way to request a signature form the server (hiding the private key in some python code) anyone can make the same http request to get_signature that my javascript makes, thus defeating the point.
I also tried to have the Django view put the signature in the view eliminating the need to make the get_signature call. This is safe, but means that I have to now refresh the page every time to get a new signature. From a users point of view only the first call to the API would work, after which they need to refresh, again pointless.
I cannot believe I'm the only person with this requirement. This is a common scenario I'm sure. Please help :) An example using custom authentication in Tastypie would be welcome too.
Thanks
Added:

Depending on your infrastructure #dragonx's answer might interest you most.
my 2c
You want to make sure that only if a client visits your website can use the api? Hmm does the bot, robot, crawler fall in the same category with the client then? Or am I wrong? This can be easily exploited in case you really want to secure it really.
I cannot believe I'm the only person with this requirement.
Maybe not, but as you can see you are prone to several attacks to your API and that can be a reason for someone not sharing your design and making security stricter with auth.
EDIT
Since we are talking about AJAX requests what does the IP part has to do with this? The IP will always be the Client's IP! So probably, you want a public API...
I would Go with the tokens/session/cookie part.
I 'd go with a generated token that lasts a little while and a flow described below.
I'd go with a limiter per some time, like Github does. Eg 60 requests per hour per ip or more for registered users
To overcome the problem with the refreshing token I would just do this:
Client visits the site
-> server generates API TOKEN INIT
-> Client gets API TOKEN INIT which is valid only for starting 1 request.
Client makes AJAX Request to API
-> Client uses API TOKEN INIT
-> Server checks against API TOKEN INIT and limits
-> Server accepts request
-> Server passes back API TOKEN
-> Client consumes response data and stores API TOKEN for further usage (Will be stored in browser memory via JS)
Client Starts Comm with the API for a limited amount of time or requests. Notice that you know also the init token date so you can use it to check against the 1st visit on the page.
The 1st token is generated via the server when the client visits.
Then the client uses that token in order to obtain a real one, that lasts for some time or something else as of limitation.
This makes someone actually visit the webpage and then he can access the API for a limit amount of time, requests perhaps etc.
This way you don't need refreshing.
Of course the above scenario could be simplified with only one token and a time limit as mentioned above.
Of course the above scenario is prone to advanced crawlers, etc since you have no authentication.
Of course a clever attacker can grab tokens from server and repeat the steps but, then you already had that that problem from start.
Some extra points
As the comments provided please close writes to the API. You don't want to be a victim of DOS attacks with writes if you have doubts about your implementation(if not use auth) or for extra security
The token scenario as described above can also become more complicated eg by constantly exchanging tokens
Just for reference GAE Cloud storage uses signed_urls for kind of the same purpose.
Hope it helps.
PS. regarding IP spoofing and Defense against spoofing attacks wikipedia says so packet's won't be returned to the attacker:
Some upper layer protocols provide their own defense against IP
spoofing attacks. For example, Transmission Control Protocol (TCP)
uses sequence numbers negotiated with the remote machine to ensure
that arriving packets are part of an established connection. Since the
attacker normally can't see any reply packets, the sequence number
must be guessed in order to hijack the connection. The poor
implementation in many older operating systems and network devices,
however, means that TCP sequence numbers can be predicted.

If it's purely the same server, you can verify requests against 127.0.0.1 or localhost.
Otherwise the solution is probably at the network level, to have a separate private subnet that you can check against. It should be difficult for an attacker to spoof your subnet without being on your subnet.

I guess you're a bit confused (or I am, please correct me). That your JS code is published on the same server as your API does not mean AJAX requests will come from your server. The clients download the JS from your server and execute it, which results in requests to your API sent from the clients, not from the same server.
Now if the above scenario correctly describes your case, what you are probably trying to do is to protect your API from bot scraping. The easiest protection is CAPTCHA, and you can find some more ideas on the Wiki page.
If you are concerned that other sites may make AJAX calls to your API to copy your site functionality, you shouldn't be--AJAX requests can only be sent to the same server as the page the JS is running on, unless it is JSONP.

Short answer: It is not possible to prevent a dedicated attacker.
You have no method of identifying a client other than with the information that they give you. For instance, username/password authentication works under the assumption that only a valid client would be able to provide valid credentials. When someone logs in, all you know is that some person provided those credentials -- you assume that this means that this means that they are a legitimate user.
Let's take a look at your scenario here, as I understand it. The only method you have of authenticating a client is IP Address, a very weak form of authentication. As you stated, this can be easily spoofed, and in with some effort your server's response can be received back to the attacker's original IP address. If this happens, you can't do anything about it. The fact is, if you assume someone from a valid IP address is a valid user, then spoofers and legitimate users are indistinguishable. This is just like if someone steals your password and tries to log in to StackOverflow. To StackOverflow, the attacker and you are indistinguishable, since all they have to go on is the username and password.
You can do fancy things with the client as mentioned in other answers, such as tokens, time limits, etc., but an dedicated attacker would be able to mimic the actions of a legitimate client, and you wouldn't be able to tell them apart because they would both appear to be from valid IP addresses. For instance, in your last example, if I was an attacker looking to make API calls, I would spoof a legitimate IP address, get the signature, and use it to make an API call, just as a legitimate client would.
If your application is critical enough to deem this level of thought into security, you should at least think of implementing something like API tokens, public key encryption, or other authentication methods that are more secure than IP addresses to tell your clients apart from any attackers. Authentication by IP address (or other easily forged tokens like hostname or headers) simply won't cut it.

may be you could achieve this by using Same-origin policy
refer http://en.wikipedia.org/wiki/Same_origin_policy

As suggested by Venkatesh Bachu, Same Origin Policy and http://en.wikipedia.org/wiki/Cross-Origin_Resource_Sharing (CORS) could be used as a solution.
In your API, you can check Origin header and respond accordingly.
Need to check if Origin header can be modified by using extensions like tamper data.
A determined hacker can still snoop by pointing browser to a local proxy server.

If this app server is running on an ordinary web server that has configurable listening IP address, set it to 127.0.0.1. With the TCPServer module, it's like
SocketServer.TCPServer(("127.0.0.1", 12345), TheHandlerClass)
Use netstat command to verify the listening address is correct as "127.0.0.1"
tcp4 0 0 127.0.0.1.12345 *.* LISTEN
This will effectively making any connection originated outside the same host impossible on the TCP level.

There are two general solution types: in-band solutions using normal web server/client mechanisms, that are easy to implement but have limitations; and out-of-band solutions that rely on you to configure something externally, that take a little more work but don't have the same limitations as in-band.
If you prefer an in-band solution, then the typical approach used to prevent cross-site request forgery (XSRF) would work well. Server issues a token with a limited life span; client uses the token in requests; privacy of token is (sort of) assured by using an HTTPS connection. This approach is used widely, and works well unless you are worried about man-in-the-middle attacks that could intercept the token, or buggy browsers that could leak data to other client-side code that's being naughty.
You can eliminate those limitations, if you're motivated, by introducing client certificates. These are kind of the flip side to the SSL certificates we all use on web servers -- they operate the same way, but are used to identify the client rather than the server. Because the certificate itself never goes over the wire (you install it locally in the browser or other client), you don't have the same threats from man-in-the-middle and browser leakage. This solution isn't used much in the wild because it's confusing to set up (very confusing for the typical user), but if you have a limited number of clients and they are under your control, then it could be feasible to deploy and manage this limited number of client certificates. The certificate operations are handled by the browser, not in client code (i.e. not in JavaScript) so your concern about key data being visible in JavaScript would not apply in this scenario.
Lastly, if you want to skip over the client configuration nonsense, use the ultimate out-of-band solution -- iptables or a similar tool to create an application-level firewall that only allows sessions that originate from network interfaces (like local loopback) that you know for certain can't be accessed off the box.