I'm trying to receive websocket messages in a greenlet, but it doent seem to be working. I have this code:
import gevent
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from geventwebsocket import WebSocketServer, WebSocketApplication, Resource
def recvWs(ws):
gevent.sleep(0)
recvedData = ws.receive()
rData = json.loads(recvedData)
print(rData)
def app(environ, start_response):
websocket = environ['wsgi.websocket']
while True:
gevent.spawn(recvWs,websocket)
gevent.sleep(0)
if __name__ == '__main__':
server = WSGIServer(("0.0.0.0", 80), app,handler_class=WebSocketHandler)
server.serve_forever()
And when running, it returns this error:
<Greenlet "Greenlet-0" at 0x23fa4306148:
recvWs(<geventwebsocket.websocket.WebSocket object at 0x0)> failed with
RuntimeError
As well as:
line 197, in read_frame
header = Header.decode_header(self.stream)
How do I fix this?
Here is an example of what I have done that works very well.
Python with bottle and gevent:
from gevent import sleep as gsleep, Timeout
from geventwebsocket import WebSocketError
from bottle import Bottle, get, post, route, request, response, template, redirect, abort
#route('/ws/app')
def handle_websocket():
wsock = request.environ.get('wsgi.websocket')
if not wsock:
abort(400, 'Expected WebSocket request.')
# Send initial data here
wsock.send(json.dumps(data))
while 1:
try:
#Process incoming message. 2 second timeout to not block
message = {}
with Timeout(2, False) as timeout:
message = wsock.receive()
if message:
message = json.loads(message)
if isinstance(message, dict):
# Do something with data and return
wsock.send(json.dumps(result))
# Add an additional second just for sanity. Not necessarily needed
gsleep(1)
except WebSocketError:
break
except Exception as exc:
traceback.print_exc()
gsleep(2)
Then in your javascript you open a websocket connection and send and recieve the data as you would normally.
Related
Context: I have a websocket 'server.py' and a script that executes some tasks: 'worker.py'. I want to use the functions in server.py to send the results from that task in worker.py. But the thing is, when a client requests the worker to send results, i need to use a function from worker.py. How can I avoid circular dependies in this situation?
Server.py:
import eventlet
#eventlet.monkey_patch()
import socketio
from flask import Flask
from flask_cors import CORS
#The Worker file
import worker
sio = socketio.Server(cors_allowed_origins='http://localhost:8100')
#sio.on('connect')
def connectHandler(sid, environ):
print('[INFO] Incoming connection from: ' + environ['REMOTE_ADDR'])
sio.emit('response', {'data' : 'Connection established, you can now request classifications by firing the "requestClassification" event.'})
#sio.on('disconnect')
def disconnectHandler(sid):
print('disconnect ', sid)
#sio.on('requestClassification')
def requestHandler(data):
print('[INFO] recieved request to classify')
print(data)
#using a function in the worker module
worker.someFunc()
eventlet.wsgi.server(eventlet.listen(('127.0.0.1', 8080)), app)
worker.py:
import server
def work():
while True:
result = doSomeTask()
print(f'sending result: {str(i)}')
server.sio.emit('result', {'data' : result})
server.sio.sleep(1)
How can i properly use the imports without defining sio (the server) twice for example?
below is the flask app module:
#app.route('/service', methods=['POST'])
def service():
postData = request.data
try:
mgmt = src.connect();
src.execute(mgmt);
return "request completed"
except Exception, e:
return ("Exception occured" + str(e));
if __name__ == "__main__":
app.run(port=5050, debug=True, use_reloader=False)
#app.run(port=5050);
Below is the module making connection:
def connect():
try:
config = ConfigParser.ConfigParser()
config.readfp(open(r'./config.ini'))
cipher_suite = Fernet(config.get('connection','token'))
decoded_pass = cipher_suite.decrypt(config.get('connection','hash'))
# Connect to the BIG-IP device.
mgmt = ManagementRoot(config.get('connection','ipaddr'),config.get('connection','user'),decoded_pass)
return mgmt;
except Exception as e:
l
log.error(e)
raise e;
I am not starting a new thread but, it throws ValueError: signal only works in main thread while connecting to BIG IP in the below step as shown in the above
mgmt = ManagementRoot(config.get('connection','ipaddr'),config.get('connection','user'),decoded_pass)
I am using the f5-sdk in which signal has been used as below:
if HAS_SIGNAL:
signal.signal(SIGALRM, timeout_handler) ## this line throws the error
signal.alarm(int(self.args['timeout']))
response = connect.get(base_uri)
signal.alarm(0)
else:
response = connect.get(base_uri)
Yup, UNIX signals can't be set up from threads other than the main thread.
Looks like the F5 SDK is using signals for connection timeouts (which is a bit silly, but so it is).
You can work around this by explicitly telling the SDK you don't have signals available by setting the flag early in your app:
from f5 import bigip
bigip.HAS_SIGNAL = False # Override autodetection
Naturally the other workaround is to connect from the main thread, but I assume that's not something you want to do for whichever reason :)
I have added the below piece of code in my flask app module instead of the commented one to make it work.
# if __name__ == "__main__":
# app.run(port=5050, debug=True, use_reloader=False)
if __name__ == "__main__":
from tornado.wsgi import WSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
http_server = HTTPServer(WSGIContainer(app))
http_server.listen(5050)
IOLoop.instance().start()
I'm working on a websocket client listening to a tornado server.
Once the client receives a message from server, the client is exiting silently.
Following is the code I've implemented.
#!/usr/bin/python
import tornado.websocket
from tornado import gen
import requests
#gen.coroutine
def test_ws():
client = yield tornado.websocket.websocket_connect("ws://localhost:8888/subscribe/ports")
msg = yield client.read_message()
print(msg)
if __name__ == "__main__":
loop = tornado.ioloop.IOLoop()
loop.run_sync(test_ws)
The client is running until it receives the first message from server. But I want to run indefinitely.
Am I missing something?
Use a loop:
#gen.coroutine
def test_ws():
client = yield tornado.websocket.websocket_connect("ws://localhost:8888/subscribe/ports")
while True:
msg = yield client.read_message()
print(msg)
I have stitched together a tornado websocket client code and using it in my python unit test case. This is my first time use of tornado websocket and not very familiar with its unit test API. Looking for some help to understand the use of tornado websocket asynchronous unit test code and the below case working.
Client class code:
import logging
import logging.config
import ssl
import time
import traceback
from tornado.concurrent import Future
from tornado import gen
from tornado.httpclient import HTTPError, HTTPRequest
from tornado.log import gen_log, app_log
from tornado.web import Application, RequestHandler
class TorWebSocketClient():
def __init__(self, ip_addr, port, cookie):
self.ip_addr = ip_addr
self.port = port
self.cookie = cookie
self.sockConnected = False
self.logger = logging.getLogger(__name__)
def Connect(self):
# Creating the websocket client for each test case.
url = "ws://{0}:{1}/socket{2}".format(str(self.ip_addr), str(self.port), self.cookie)
self.logger.debug('Websocket URL: ' + url)
sslopt={"cert_reqs": ssl.CERT_NONE,
"check_hostname": False,
"ssl_version": ssl.PROTOCOL_TLSv1}
self.logger.debug('New web socket connection is being establshed by the client')
self.ws = websocket.websocket_connect(HTTPRequest(url, headers=headers, ssl_options=sslopt), io_loop=self.io_loop)
# Start the websocket client thread. A wait is added till connection is established.
self.sockConnected = True
def send(self, data):
# Wait till websocket is connected.
if not self.ws.sock.connected:
self.logger.debug('Send failed; Websocket connection is not yet established')
return
self.logger.info('Sending data to the server: ' + data)
self.ws.write_message(data)
def recv(self, expValues):
# Read data from the response message.
resp = yield self.ws.read_message()
print '>>>> Response: ', resp
def stop(self):
self.logger.debug('Client closing the websocket connection with the server')
self.ws.close()
Unit test function is below:
import functools
import json
import logging
import logging.config
import time
# These are couple of custom classes.
import TorWebSocketClient
from infra.serverbase import Server
from tornado.testing import AsyncHTTPTestCase, gen_test, bind_unused_port, ExpectLog
class TornadoTest(AsyncHTTPTestCase):
def get_app(self):
app = tornado.web.Application([('/', EchoWebSocketHandler)])
return app
#gen_test
def testToradoWSConection(self):
# Login to the server to get the cookie.
logger = logging.getLogger(__name__)
server = Server(self.ipaddr, self.port, self.username, self.password)
result = server.Login()
self.assertEqual(result, True, 'login failed')
webSocClient = yield TorWebSocketClient(self.ipaddr, self.port, server.GetCookie())
result = webSocClient.Connect()
self.assertEqual(result, True, 'Websocket connection failed')
Error I am getting:
Traceback (most recent call last):
File "/users/usr1/pyvenv/venv/lib/python2.7/site-packages/tornado/testing.py", line 527, in post_coroutine
return self.io_loop.run_sync(
AttributeError: TornadoTest instance has no attribute 'io_loop'
----------------------------------------------------------------------
Ran 1 tests in 0.002s
FAILED (errors=1)
Did you have your own setUp function?
The io_loop is created under AsyncTestCase's setUp function, I think you need to call super's setUp function.
I'm using Autobahn to connect to a websocket like this.
class MyComponent(ApplicationSession):
#inlineCallbacks
def onJoin(self, details):
print("session ready")
def oncounter(*args, **args2):
print("event received: args: {} args2: {}".format(args, args2))
try:
yield self.subscribe(oncounter, u'topic')
print("subscribed to topic")
except Exception as e:
print("could not subscribe to topic: {0}".format(e))
if __name__ == '__main__':
addr = u"wss://mywebsocketaddress.com"
runner = ApplicationRunner(url=addr, realm=u"realm1", debug=False, debug_app=False)
runner.run(MyComponent)
This works great, and I am able to receive messages. However, after around 3-4 hours, sometimes much sooner, the messages abruptly stop coming. It appears that the websocket times out (does that happen?), possibly due to connection issues.
How can I auto-reconnect with autobahn when that happens?
Here's my attempt, but the reconnecting code is never called.
class MyClientFactory(ReconnectingClientFactory, WampWebSocketClientFactory):
maxDelay = 10
maxRetries = 5
def startedConnecting(self, connector):
print('Started to connect.')
def clientConnectionLost(self, connector, reason):
print('Lost connection. Reason: {}'.format(reason))
ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
def clientConnectionFailed(self, connector, reason):
print('Connection failed. Reason: {}'.format(reason))
ReconnectingClientFactory.clientConnectionFailed(self, connector, reason)
class MyApplicationRunner(object):
log = txaio.make_logger()
def __init__(self, url, realm, extra=None, serializers=None,
debug=False, debug_app=False,
ssl=None, proxy=None):
assert(type(url) == six.text_type)
assert(realm is None or type(realm) == six.text_type)
assert(extra is None or type(extra) == dict)
assert(proxy is None or type(proxy) == dict)
self.url = url
self.realm = realm
self.extra = extra or dict()
self.serializers = serializers
self.debug = debug
self.debug_app = debug_app
self.ssl = ssl
self.proxy = proxy
def run(self, make, start_reactor=True):
if start_reactor:
# only select framework, set loop and start logging when we are asked
# start the reactor - otherwise we are running in a program that likely
# already tool care of all this.
from twisted.internet import reactor
txaio.use_twisted()
txaio.config.loop = reactor
if self.debug or self.debug_app:
txaio.start_logging(level='debug')
else:
txaio.start_logging(level='info')
isSecure, host, port, resource, path, params = parseWsUrl(self.url)
# factory for use ApplicationSession
def create():
cfg = ComponentConfig(self.realm, self.extra)
try:
session = make(cfg)
except Exception as e:
if start_reactor:
# the app component could not be created .. fatal
self.log.error(str(e))
reactor.stop()
else:
# if we didn't start the reactor, it's up to the
# caller to deal with errors
raise
else:
session.debug_app = self.debug_app
return session
# create a WAMP-over-WebSocket transport client factory
transport_factory = MyClientFactory(create, url=self.url, serializers=self.serializers,
proxy=self.proxy, debug=self.debug)
# supress pointless log noise like
# "Starting factory <autobahn.twisted.websocket.WampWebSocketClientFactory object at 0x2b737b480e10>""
transport_factory.noisy = False
# if user passed ssl= but isn't using isSecure, we'll never
# use the ssl argument which makes no sense.
context_factory = None
if self.ssl is not None:
if not isSecure:
raise RuntimeError(
'ssl= argument value passed to %s conflicts with the "ws:" '
'prefix of the url argument. Did you mean to use "wss:"?' %
self.__class__.__name__)
context_factory = self.ssl
elif isSecure:
from twisted.internet.ssl import optionsForClientTLS
context_factory = optionsForClientTLS(host)
from twisted.internet import reactor
if self.proxy is not None:
from twisted.internet.endpoints import TCP4ClientEndpoint
client = TCP4ClientEndpoint(reactor, self.proxy['host'], self.proxy['port'])
transport_factory.contextFactory = context_factory
elif isSecure:
from twisted.internet.endpoints import SSL4ClientEndpoint
assert context_factory is not None
client = SSL4ClientEndpoint(reactor, host, port, context_factory)
else:
from twisted.internet.endpoints import TCP4ClientEndpoint
client = TCP4ClientEndpoint(reactor, host, port)
d = client.connect(transport_factory)
# as the reactor shuts down, we wish to wait until we've sent
# out our "Goodbye" message; leave() returns a Deferred that
# fires when the transport gets to STATE_CLOSED
def cleanup(proto):
if hasattr(proto, '_session') and proto._session is not None:
if proto._session.is_attached():
return proto._session.leave()
elif proto._session.is_connected():
return proto._session.disconnect()
# when our proto was created and connected, make sure it's cleaned
# up properly later on when the reactor shuts down for whatever reason
def init_proto(proto):
reactor.addSystemEventTrigger('before', 'shutdown', cleanup, proto)
return proto
# if we connect successfully, the arg is a WampWebSocketClientProtocol
d.addCallback(init_proto)
# if the user didn't ask us to start the reactor, then they
# get to deal with any connect errors themselves.
if start_reactor:
# if an error happens in the connect(), we save the underlying
# exception so that after the event-loop exits we can re-raise
# it to the caller.
class ErrorCollector(object):
exception = None
def __call__(self, failure):
self.exception = failure.value
reactor.stop()
connect_error = ErrorCollector()
d.addErrback(connect_error)
# now enter the Twisted reactor loop
reactor.run()
# if we exited due to a connection error, raise that to the
# caller
if connect_error.exception:
raise connect_error.exception
else:
# let the caller handle any errors
return d
The error I'm getting:
2016-10-09T21:00:40+0100 Connection to/from tcp4:xxx.xx.xx.xx:xxx was lost in a non-clean fashion: Connection lost
2016-10-09T21:00:40+0100 _connectionLost: [Failure instance: Traceback (failure with no frames): : Connection to the other side was lost in a non-clean fashion: Connection l
ost.
]
2016-10-09T21:00:40+0100 WAMP-over-WebSocket transport lost: wasClean=False, code=1006, reason="connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)"
2016-10-09T21:10:39+0100 EXCEPTION: no messages received
2016-10-09T21:10:39+0100 Traceback (most recent call last):
You can use a ReconnectingClientFactory if you're using Twisted. There's a simple example by the autobahn developer on github. Unfortunately there doesn't seem to be an implementation of an ApplicationRunner that comes pre-built with this functionality but it doesn't look too difficult to implement on your own. Here's an asyncio variant which should be straight forward to Twisted. You could follow this issue as it seems there is a desire by the dev team to incorporate reconnecting client.
Have you tried pip3 install autobahn-autoreconnect ?
# from autobahn.asyncio.wamp import ApplicationRunner
from autobahn_autoreconnect import ApplicationRunner
Here is an example of an automatically reconnecting ApplicationRunner. The important line to enable auto-reconnect is:
runner.run(session, auto_reconnect=True)
You will also want to activate automatic WebSocket ping/pong (eg in Crossbar.io), to a) minimize connection drops because of timeouts, and b) to allow fast detection of lost connections.