So I'm doing a unittest of a tornado server I programmed which has a websocketHandler. This is my testing code:
def test_unauthorized_websocket(self):
message = "Hey bro"
ws = websocket.create_connection('ws://localhost:8888/ws', header=['Authorization: false'])
send_dict = {'command': test_command, 'message': message}
serialized_dict = json.dumps(send_dict)
ws.send(serialized_dict)
response = ws.recv()
#test response with assert
My goal with this test was to prove that my tornado server correctly refuses and closes this websocket connection because of wrong authentication header.
This is my tornado websocketHandler code:
class WebSocketHandler(tornado.websocket.WebSocketHandler):
def open(self):
#some code
headers = self.request.headers
try:
auth = headers['Authorization']
except KeyError:
self.close(code = 1002, reason = "Unauthorized websocket")
print("IT'S CLOSED")
return
if auth == "true":
print("Authorized Websocket!")
#some code
else:
print("Unauthorized Websocket... :-(")
self.close(code = 1002, reason = "Unauthorized websocket")
So when the authentication is wrong, self.close() is called (not sure I need code and reason). This should close the websocket. But this doesn't actually happens in the "client" side. After I call create_connection() in the "client" the ws.connected variable is still True, and when I do ws.send() the on_message method of the websocketHandler is still called and when it tries to make a response with self.write_message() it raises a WebSocketClosedError. And only then the "client" actually closes its side of the websocket, ws.recv() returns nothing and ws.connected turns to False after that.
Is there a way I can communicate to the client side (through handshake headers or something) that the websocket is meant to be closed earlier on its side?
You can override prepare() and raise a tornado.web.HTTPError, instead of overriding open() and calling self.close(). This will reject the connection at the first opportunity and this will be reported on the client side as a part of create_connection() instead of on a later read.
Related
I am attempting to setup a websocket using the websocket-client library using python 3.7. The documentation from the API provider states that basic auth is required.
Below is the code I am using to try subscribing to their test channel. The test channel should send responses back nonstop until we close the socket.
email = b'myemail#domain.com'
pw = b'mypassword'
_str = (base64.b64encode(email + b':' + pw)).decode('ascii')
headerValue = 'Authorization: Basic {0}'.format(_str)
def on_message(ws, msg):
global msg_received
print("Message received: ", msg)
msg_received += 1
if msg_received > 10:
ws.send(json.dumps({"unsubscribe": "/test"}))
ws.close()
def on_error(ws, error):
print("ERROR: ", error)
def on_close(ws):
print("Closing websocket...")
def on_open(ws):
print("Opening...")
ws.send(json.dumps({'subscribe': '/test'}))
time.sleep(1)
if __name__ == '__main__':
websocket.enableTrace(True)
ws = websocket.WebSocketApp("wss://api-ws.myurl.com/",
header=[headerValue],
on_message=on_message,
on_error=on_error,
on_close=on_close))
ws.on_open = on_open
ws.run_forever()
When I run this, I am able to see my request headers, and their response headers which show that the connection was upgraded to a websocket and I am assigned a Sec-Websocket-Accept. Then, the websocket immediately closes without any responses coming through.
I have tried first sending a post request to the login api and generating a sessionID and csrftoken, and then passing those as cookies in the websocketapp object. It didn't work. I have tried passing the headers as an actual dict but that doesn't work either. I've tried way too many variations of the b64 encoding and none of them work.
Any advice would be appreciated.
I have my Tornado client continuously listening to my Tornado server in a loop, as it is mentioned here - http://tornadoweb.org/en/stable/websocket.html#client-side-support. It looks like this:
import tornado.websocket
from tornado import gen
#gen.coroutine
def test():
client = yield tornado.websocket.websocket_connect("ws://localhost:9999/ws")
client.write_message("Hello")
while True:
msg = yield client.read_message()
if msg is None:
break
print msg
client.close()
if __name__ == "__main__":
tornado.ioloop.IOLoop.instance().run_sync(test)
I'm not able to get multiple instances of clients to connect to the server. The second client always waits for the first client process to end before it connects to the server. The server is set up as follows, with reference from Websockets with Tornado: Get access from the "outside" to send messages to clients and Tornado - Listen to multiple clients simultaneously over websockets.
class WSHandler(tornado.websocket.WebSocketHandler):
clients = set()
def open(self):
print 'new connection'
WSHandler.clients.add(self)
def on_message(self, message):
print 'message received %s' % message
# process received message
# pass it to a thread which updates a variable
while True:
output = updated_variable
self.write_message(output)
def on_close(self):
print 'connection closed'
WSHandler.clients.remove(self)
application = tornado.web.Application([(r'/ws', WSHandler),])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(9999)
tornado.ioloop.IOLoop.instance().start()
But this has not worked - for some reason even after I have made a successful first connection, the second connection just fails to connect i.e. it does not even get added to the clients set.
I initially thought the while True would not block the server from receiving and handling more clients, but it does as without it multiple clients are able to connect. How can I send back continuously updated information from my internal thread without using the while True?
Any help would be greatly appreciated!
To write messages to client in a while loop, you can use the yield None inside the loop. This will pause the while loop and then Tornado's IOLoop will be free to accept new connections.
Here's an example:
#gen.coroutine
def on_message(self):
while True:
self.write_message("Hello")
yield None
Thanks for your answer #xyres! I was able to get it to work by starting a thread in the on_message method that handed processing and the while True to a function outside the WSHandler class. I believe this allowed for the method to run outside of Tornado's IOLoop, unblocking new connections.
This is how my server looks now:
def on_message(self, message):
print 'message received %s' % message
sendThread = threading.Thread(target=send, args=(self, message))
sendThread.start()
def send(client, msg):
# process received msg
# pass it to a thread which updates a variable
while True:
output = updated_variable
client.write_message(output)
Where send is a function defined outside the class which does the required computation for me and writes back to client inside thewhile True.
I have an autobahn Websocket Server with the typical onX functions in it's protocol. My problem is that I can't find a way to exit onX, while keep doing the various stuff that I wanted to do when the specific message arrived. More specifically in my onMessage function, I sometimes perform an HTTP request to an API which is very slow. As a result, the client that sent the websocket message is being blocked by the server's onMessage finalization. Even if I do self.sendMessage or reactor.callFromThread(<http request here>), or self.transport.loseConnection() from the server side, in the onMessage block, the onMessage is still executing the HTTP request and my client waits.
This is my client's code:
#asyncio.coroutine
def send(command,userPath,token):
websocket = yield from websockets.connect('wss://127.0.0.1:7000',ssl=ssl.SSLContext(protocol=ssl.PROTOCOL_TLSv1_2))
data = json.dumps({"api_command":"session","body":command,"headers": {'X-User-Path': userPath, 'X-User-Token': token}})
response = {}
try:
yield from websocket.send(data)
finally:
yield from websocket.close()
if 'command' in response:
if response['command'] == 'ACK_SESSION_COMMAND' or response['command'] == 'ACK_INITIALIZATION':
return ('OK',200)
else:
return('',400)
I even tried to just websocket.send(data), from the client, but for some reason it doesn't send the data (I don't see them arriving in the server). I don't understand how can I return from the onMessage block and keep doing my HTTP request.
And to explain my situation, I just want to sent 1 ssl websocket message to my server and immediately close the connection. Anything that can do that, suits me.
Using reactor.callInThread instead of reactor.callFromThread, causes the application flow to release and the HTTP request is performed independently in a thread. As described in the twisted documentation : http://twistedmatrix.com/documents/13.2.0/core/howto/threading.html
sorry if I got the title wrong, I'm new to Twisted and couldn't really describe my problem that well.
So the problem is, I have an IRC bot based off ircLogBot.py(http://twistedmatrix.com/documents/current/words/examples/ircLogBot.py) which relays messages between IRC and a MySQL database through a PHP page.
It has to load a PHP page every 1 second, parse the content(JSON), loop through it, then post every item to IRC. I have all of that sorted except posting it to IRC.
The reason it's hard is because the loop runs inside another thread (it has to to work) and I don't know how to call msg() from that thread.
This description has probably been really confusing, so take a look at my code. I commented the bit where I want to send my message:
from twisted.words.protocols import irc
from twisted.internet import reactor, protocol, threads
from twisted.python import log
# system imports
import time, sys, json, urllib, urllib2, threading
url = 'http://86.14.76.169/root/scripts/ircbot.php'
urltwo = 'http://86.14.76.169/root/scripts/returnchat.php'
class LogBot(irc.IRCClient):
try:
"""A logging IRC bot."""
nickname = "WorldConflictBot"
def connectionMade(self):
irc.IRCClient.connectionMade(self)
def connectionLost(self, reason):
irc.IRCClient.connectionLost(self, reason)
# callbacks for events
def signedOn(self):
"""Called when bot has succesfully signed on to server."""
self.join(self.factory.channel)
def joined(self, channel):
"""This will get called when the bot joins the channel."""
self.action('JackBot', self.factory.channel, 'Joined')
def privmsg(self, user, channel, msg):
"""This will get called when the bot receives a message."""
user = user.split('!', 1)[0]
values = {}
values = {'type' : 'message',
'message' : msg,
'username' : user,
}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
# Check to see if they're sending me a private message
if channel == self.nickname:
msg = "It isn't nice to whisper! Play nice with the group."
self.msg(user, msg)
return
# Otherwise check to see if it is a message directed at me
if msg.startswith(self.nickname + ":"):
msg = "%s: Hey :)" % user
self.msg(channel, msg)
def action(self, user, channel, msg):
"""This will get called when the bot sees someone do an action."""
user = user.split('!', 1)[0]
# irc callbacks
def irc_NICK(self, prefix, params):
"""Called when an IRC user changes their nickname."""
old_nick = prefix.split('!')[0]
new_nick = params[0]
values = {}
values = {'type' : 'nick',
'from' : old_nick,
'to' : new_nick,
}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
# For fun, override the method that determines how a nickname is changed on
# collisions. The default method appends an underscore.
def alterCollidedNick(self, nickname):
"""
Generate an altered version of a nickname that caused a collision in an
effort to create an unused related name for subsequent registration.
"""
return nickname + '^'
except KeyboardInterrupt:
LogBotLooper.exit()
sys.exit()
class LogBotFactory(protocol.ClientFactory):
"""A factory for LogBots.
A new protocol instance will be created each time we connect to the server.
"""
def __init__(self):
self.channel = 'worldconflict'
def buildProtocol(self, addr):
p = LogBot()
p.factory = self
return p
l = LogBotLooper()
l.factory = self
return l
def clientConnectionLost(self, connector, reason):
"""If we get disconnected, reconnect to server."""
connector.connect()
def clientConnectionFailed(self, connector, reason):
print "connection failed:", reason
reactor.stop()
class LogBotLooper(irc.IRCClient):
def __init__(self):
i = 0
lastid = 0
while 1:
time.sleep(1)
if(i == 0):
values = {'justlastid': 'true'}
else:
values = {'lastid' : lastid}
data = urllib.urlencode(values)
req = urllib2.Request(urltwo, data)
response = urllib2.urlopen(req)
the_page = response.read()
if(i == 0):
lastid = the_page
i += 1
else:
if(the_page != 'error'):
jsonpage = json.loads(the_page)
for message in jsonpage['messages']:
#Need to send the variable `message` to IRC.
lastid = jsonpage['highestid']
def exit(self):
sys.exit()
if __name__ == '__main__':
try:
# initialize logging
log.startLogging(sys.stdout)
# create factory protocol and application
f = LogBotFactory()
# connect factory to this host and port
reactor.connectTCP("irc.skyirc.net", 6667, f)
reactor.callInThread(LogBotLooper)
# run bot
reactor.run()
except KeyboardInterrupt:
LogBotLooper.exit()
sys.exit()
You probably knew this already, but your protocol class should really just stick to event handling and other abstractions over the transport itself. That way you preserve separation of concerns and you have a maintainable framework. In the MVC paradigm, your protocol class is a controller, or maybe even a view, but definitely not a model. Making PHP web service calls probably belongs in the model.
As far as transferring work to other threads (which you will definitely need to do for any blocking I/O such as web service calls), you need:
from twisted.internet import threads, reactor
From the main reactor thread, call threads.deferToThread(mycallable, *args, **kwargs) to have mycallable called from the next available worker thread.
From any worker thread, call reactor.callFromThread(mycallable, *args, **kwargs) to have mycallable called from the main reactor thread.
To transfer work from one worker thread to another, combine the two techniques: reactor.callFromThread(threads.deferToThread, mycallable, *args, **kwargs).
I believe both of these calls return a Deferred object (I know deferToThread does). If you add callbacks to the deferred, those callbacks will execute in the same thread as the original callable. To delegate callback execution to worker threads, use the above techniques within the callbacks. (They don't call it "Twisted" for nothing.)
If I didn't get wrong message from your post, I have same problem like yours.
http://twistedmatrix.com/documents/10.1.0/core/howto/threading.html
threads.blockingCallFromThread is another answer for this question.
Just replace
#Need to send the variable `message` to IRC.
with
threads.blockingCallFromThread(reactor, irc.send_message, message)
#I assume you call irc.send_message here
I've got a Python class I'm using to emulate an IMAP server for a unit test. When the handle() callback happens, I try to use recv() to get the data that just arrived but I'm blocking on the recv() call. I thought recv() would return with the data that was received but instead, it just sits there.
In case it matters, I can see that the server and main thread are different in the log output.
I'm assuming that this has something to do with the fact that the socket was just opened but no data was sent through it. Do I need to check to see if that's the case before trying to call recv()?
I've created the server using:
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
...
# inside another class, we've got:
def startUp(self):
server = ThreadedTCPServer(('localhost', 5593), FakeImapServerHandler)
server.allow_reuse_address = True
server_thread = threading.Thread(target=server.serve_forever)
server_thread.setDaemon(True)
server_thread.start()
Then inside some other code in our unit test, we try to connect to the fake server:
print("about to connect")
self.__imapCon = imaplib.IMAP4("localhost", 5593)
print("about to login") # we never get here
self.__imapCon.login(Config.config["imapUser"], Config.config["imapPassw"])
The server is implemented like:
class FakeImapServerHandler(SocketServer.BaseRequestHandler):
def handle(self):
print("handling request")
incoming = self.request.recv(1024)
print("request was: " + incoming ) # never get here
(Posted as an answer this time) Per the IMAP protocol, servers speak first, sending an OK greeting to the client. Both your server and client seem to be trying to recv() from each other at the same time:
atlas% telnet chaos 143
Trying 192.168.1.5...
Connected to chaos
Escape character is '^]'.
* OK chaos Cyrus IMAP4 v2.2.13-Debian-2.2.13-19 server ready