We Keep Coding

Flask: Some clients don't like responses

I have a Flask app that generates video stream links. It connects to a server using login credentials and grabs a one time use link (that expires when a new link is generated using the same credentials). Using a list of credentials I am able to stream to as many devices as I like, so long as I have enough accounts.
The issue I am having is that one of the clients doesn't like the way the stream is returned.
#app.route("/play", methods=["GET"])
def play():
def streamData():
try:
useAccount(<credentials>)
with requests.get(link, stream=True) as r:
for chunk in r.iter_content(chunk_size=1024):
yield chunk
except:
pass
finally:
freeAccount(<credentials>)
...
# return redirect(link)
return Response(streamData())
If I return a redirect then there are no playback issues at all. The problem with a redirect is I don't have a way of marking the credentials as in use, then freeing them after.
The problem client is TVHeadend. I am able to get it to work by enabling the additional avlib inside of TVHeadend... But I shouldn't have to do that. I don't have to when I return a redirect.
What could be the cause of this?
Is it possible to make my app respond in the same way as the links server does?
My guess is that TVHeadend is very strict on if something complies to whatever standards... and I am guessing my app doesn't?

Cannot connect to Binance websocket. I get: WebSocketBadStatusException: Handshake status 400 Bad Request

I am trying to connect to the Binance websocket stream.
Following their documentation I use the below code to establish the connection:
from websocket import create_connection
ws = create_connection('wss://fstream.binance.com/')
When running it though, I get the following error:
WebSocketBadStatusException: Handshake status 400 Bad Request
I could not find any info on the web about this error.
Does anyone know how to fix this?

This Point is somewhat unclear in the Binance API-Docs.
The base urls for the futures are:
wss://fstream.binance.com
wss://fstream3.binance.com
But if you just connect to these base urls you get the mentioned exception.
You should complement the url-strings to
wss://fstream.binance.com/ws
wss://fstream3.binance.com/ws
This is the same for spot markets and all the other websockets. Always put a "/ws" at the end.
You could also start subscribing with the connection-url then it looks like this spot-market example:
wss://stream.binance.com:9443/ws/btcusdt#aggTrade
(But i think connecting just with "/ws" and then live subscribing/unsubscribing as explained in the docs to streams is the better way.)

I gotta say it took me a long time to figure the solution out but here it goes.
Binance API documentation should be edited because it is missing the port for the fstream.binance.com
The port is 443.
So you should use
"fstream.binance.com:443" instead of
"fstream.binance.com".
Hope it helps. (Hell yeah it does!)

You could install python-binance and use the BinanceSocketManager
python -m pip install python-binance
Use the following code I found here
import time
from binance.client import Client # Import the Binance Client
from binance.websockets import BinanceSocketManager # Import the Binance Socket Manager
# Although fine for tutorial purposes, your API Keys should never be placed directly in the script like below.
# You should use a config file (cfg or yaml) to store them and reference when needed.
PUBLIC = '<YOUR-PUBLIC-KEY>'
SECRET = '<YOUR-SECRET-KEY>'
# Instantiate a Client
client = Client(api_key=PUBLIC, api_secret=SECRET)
# Instantiate a BinanceSocketManager, passing in the client that you instantiated
bm = BinanceSocketManager(client)
# This is our callback function. For now, it just prints messages as they come.
def handle_message(msg):
print(msg)
# Start trade socket with 'ETHBTC' and use handle_message to.. handle the message.
conn_key = bm.start_trade_socket('ETHBTC', handle_message)
# then start the socket manager
bm.start()
# let some data flow..
time.sleep(10)
# stop the socket manager
bm.stop_socket(conn_key)

You are missing the path on websocket connect!
Take a look to the binance api docs:
https://binance-docs.github.io/apidocs/futures/en/#websocket-market-streams
python-binance does not support websocket to binance futures endpoints, so you can use unicorn-binance-websocket-api instead, here is an example for future endpoints:
https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/blob/master/example_binance_futures.py

Bottle-WebSocket: How to ensure an HTTP request is from the same session as ws connection?

I built an web application using Python Bottle framework.
I used bottle-websocket plugin for WebSocket communication with clients.
Here is a part of my code.
from bottle import Bottle, request, run
from bottle.ext.websocket import GeventWebSocketServer, websocket
class MyHandler():
...
class MyServer(Bottle):
...
def _serve_websocket(self, ws):
handler = MyHandler()
some_data = request.cookies.get('some_key') # READ SOME DATA FROM HTTP REQUEST
while True:
msg = ws.receive()
handler.do_sth_on(msg, some_data) # USE THE DATA FROM HTTP REQUEST
ws.send(msg)
del(handler)
if __name__ == '__main__':
run(app=MyServer(), server=GeventWebSocketServer, host=HOST, port=PORT)
As the code shows, I need to read some data from the browser (cookies or anything in the HTTP request headers) and use it for WebSocket message processing.
How can I ensure the request is from the same browser session as the one where WebSocket connection comes?
NOTE
As I do not have much knowledge of HTTP and WebSocket, I'd love to here detailed answere as much as possible.

How can I ensure the request is from the same browser session as the one where WebSocket connection comes?
Browser session is a bit abstract since HTTP does not have a concept of sessions. HTTP and RESTful APIs is designed to be stateless, but there is options.
Usually, what you usually want to know is what user the request comes from. This is usually solved by authentication e.g. by using OpenID Connect and let the user send his JWT-token in the Authorization: header, this works for all HTTP requests, including when setting up a Websocket connection.
bottle-oauthlib seem to be a library for authenticating end-users using OAuth2 / OpenID Connect.
Another option is to identify the "browser session" using cookies but this depends on a state somewhere on the server side and is harder to implement on cloud native platforms like e.g. Kubernetes that prefer stateless workloads.

Making asynchronous HTTP requests from a flask service

I have a couple different needs for asynchrony in my Python 3.6 Flask RESTful web service running under Gunicorn.
1) I'd like for one of my service's routes to be able to send an HTTP request to another HTTP service and, without waiting for the response, send a response back to the client that called my service.
Some example code:
#route
def fire_and_forget():
# Send request to other server without waiting
# for it to send a response.
# Return my own response.
2) I'd like for another one of my service's routes to be able to send 2 or more asynchronous HTTP requests to other HTTP services and wait for them all to reply before my service sends a response.
Some example code:
#route
def combine_results():
# Send request to service A
# Send request to service B
# Wait for both to return.
# Do something with both responses
# Return my own response.
Thanks in advance.
EDIT: I am trying to avoid the additional complexity of using a queue (e.g. celery).

You can use eventlets for the the second use case. It's pretty easy to do:
import eventlet
providers = [EventfulPump(), MeetupPump()]
try:
pool = eventlet.GreenPool()
pile = eventlet.GreenPile(pool)
for each in providers:
pile.spawn(each.get, [], 5, loc) # call the interface method
except (PumpFailure, PumpOverride):
return abort(503)
results = []
for res in pile:
results += res
You can wrap each of your api endpoints in a class that implements a "common interface" (in the above it is the get method) and you can make the calls in parallel. I just place them all in a list.
Your other use case is harder to accomplish in straight python. At least a few years ago you would be forced to introduce some sort of worker process like celery to get something like that done. This question seems to cover all the issues:
Making an asynchronous task in Flask
Perhaps things have changed in flask land?

WCF and Python

Is there any example code of a cpython (not IronPython) client which can call Windows Communication Foundation (WCF) service?

I used suds.
from suds.client import Client
print "Connecting to Service..."
wsdl = "http://serviceurl.com/service.svc?WSDL"
client = Client(wsdl)
result = client.service.Method(variable1, variable2)
print result
That should get you started. I'm able to connect to exposed services from WCF and a RESTful layer. There needs to be some data massaging to help do what you need, especially if you need to bind to several namespaces.

TL;DR: For wsHttpBinding (SOAP 1.2) use zeep
In case someone is having trouble using suds (or suds-jurko for that matter) with WCF and wsHttpBinding (which is SOAP 1.2):
suds is pretty much dead (can't even pip install it on python 3)
suds-jurko seems kind-of dead. The 0.6 release has a very annoying infinite recursion bug (at least on the WSDL exposed by our service) which is fixed in the tip but that's not released and it's been 1.5years (at time of this writing in Feb'17) since the last commit.
It works on python 3 but doesn't support SOAP 1.2. Sovetnikov's answer is an attempt to get it working with 1.2 but I haven't managed to make it work for me.
zeep seems to be the current way to go and worked out of the box (I'm not affiliated with zeep, it just works for me and I spent several hours banging my head against a brick wall trying to make suds work). For zeep to work, the WCF service host configuration must include <security mode="None"/> under the wsHttpBinding node Actually zeep seems to support username and signature (x509) based WS-SE but I haven't tried that so can't speak to any problems around it.

WCF needs to expose functionality through a communication protocol. I think the most commonly used protocol is probably SOAP over HTTP. Let's assume that's
what you're using then.
Take a look at this chapter in Dive Into Python. It will show you how to
make SOAP calls.
I know of no unified way of calling a WCF service in Python, regardless of communication
protocol.

Just to help someone to access WCF SOAP 1.2 service with WS-Addressing using suds.
Main problem is to inject action name in every message.
This example for python 3 and suds port https://bitbucket.org/jurko/suds.
Example uses custom authentification based on HTTP headers, i leave it as is.
TODO: Automatically get api_direct_url from WSDL (at now it is hard coded).
from suds.plugin import MessagePlugin
from suds.sax.text import Text
from suds.wsse import Security, UsernameToken
from suds.sax.element import Element
from suds.sax.attribute import Attribute
from suds.xsd.sxbasic import Import
api_username = 'some'
api_password = 'none'
class api(object):
api_direct_url = 'some/mex'
api_url = 'some.svc?singleWsdl|Wsdl'
NS_WSA = ('wsa', 'http://www.w3.org/2005/08/addressing')
_client_instance = None
#property
def client(self):
if self._client_instance:
return self._client_instance
from suds.bindings import binding
binding.envns = ('SOAP-ENV', 'http://www.w3.org/2003/05/soap-envelope')
api_inst = self
class _WSAPlugin(MessagePlugin):
def marshalled(self, context):
api_inst._marshalled_message(context)
self._client_instance = Client(self.api_url,
plugins=[_WSAPlugin()],
headers={'Content-Type': 'application/soap+xml',
'login':api_username,
'password': api_password}
)
headers = []
headers.append(Element('To', ns=self.NS_WSA).setText(self.api_direct_url))
headers.append(Element('Action', ns=self.NS_WSA).setText('Blank'))
self._client_instance.set_options(soapheaders=headers)
cache = self._client_instance.options.cache
cache.setduration(days=10)
return self._client_instance
def _marshalled_message(self, context):
def _children(r):
if hasattr(r, 'children'):
for c in r.children:
yield from _children(c)
yield c
for el in _children(context.envelope):
if el.name == 'Action':
el.text = Text(self._current_action)
return
_current_action = None
def _invoke(self, method, *args):
try:
self._current_action = method.method.soap.action.strip('"')
return method(*args)
finally:
self._current_action = None
def GetRequestTypes(self):
return self._invoke(self.client.service.GetRequestTypes)[0]
def GetTemplateByRequestType(self, request_type_id):
js = self._invoke(self.client.service.GetTemplateByRequestType, request_type_id)
return json.loads(js)
def GetRequestStatus(self, request_guid):
return self._invoke(self.client.service.GetRequestStatus, request_guid)
def SendRequest(self, request_type_id, request_json):
r = json.dumps(request_json, ensure_ascii=False)
return self._invoke(self.client.service.SendRequest, request_type_id, r)

I do not know of any direct examples, but if the WCF service is REST enabled you could access it through POX (Plain Old XML) via the REST methods/etc (if the service has any). If you are in control of the service you could expose endpoints via REST as well.

if you need binary serialized communication over tcp then consider implementing solution like Thrift.

Even if there is not a specific example of calling WCF from Python, you should be able to make a fully SOAP compliant service with WCF. Then all you have to do is find some examples of how to call a normal SOAP service from Python.
The simplest thing will be to use the BasicHttpBinding in WCF and then you can support your own sessions by passing a session token with each request and response.