I am attempting to setup a websocket using the websocket-client library using python 3.7. The documentation from the API provider states that basic auth is required.
Below is the code I am using to try subscribing to their test channel. The test channel should send responses back nonstop until we close the socket.
email = b'myemail#domain.com'
pw = b'mypassword'
_str = (base64.b64encode(email + b':' + pw)).decode('ascii')
headerValue = 'Authorization: Basic {0}'.format(_str)
def on_message(ws, msg):
global msg_received
print("Message received: ", msg)
msg_received += 1
if msg_received > 10:
ws.send(json.dumps({"unsubscribe": "/test"}))
ws.close()
def on_error(ws, error):
print("ERROR: ", error)
def on_close(ws):
print("Closing websocket...")
def on_open(ws):
print("Opening...")
ws.send(json.dumps({'subscribe': '/test'}))
time.sleep(1)
if __name__ == '__main__':
websocket.enableTrace(True)
ws = websocket.WebSocketApp("wss://api-ws.myurl.com/",
header=[headerValue],
on_message=on_message,
on_error=on_error,
on_close=on_close))
ws.on_open = on_open
ws.run_forever()
When I run this, I am able to see my request headers, and their response headers which show that the connection was upgraded to a websocket and I am assigned a Sec-Websocket-Accept. Then, the websocket immediately closes without any responses coming through.
I have tried first sending a post request to the login api and generating a sessionID and csrftoken, and then passing those as cookies in the websocketapp object. It didn't work. I have tried passing the headers as an actual dict but that doesn't work either. I've tried way too many variations of the b64 encoding and none of them work.
Any advice would be appreciated.
Related
I was working on a prototype for security in MQTT based systems using a Raspberry Pi as my client and Keycloak as my authentication server (AS). I have managed to publish and subscribe to messages via Mosquitto broker without the authentication mechanism in place and have also successfully obtained the access token and token validation output from the keycloak validation endpoint.
However, when I implemented above sequence of tasks via a python script, to authenticate the mqtt client so it can publish/subscribe, I noticed that it is able to establish a connection but at the client end the connection closes. I suspect that this is due to the token not getting passed either due to wrong/invalid parameters or there is an issue with my token validation function
I would be really grateful if members can look at my python script and perhaps point out any errors or provide suggestions to improve my code. TIA!
import paho.mqtt.client as paho
import time
import sys
import json
import requests
from requests.auth import HTTPBasicAuth
broker="broker address"
port= 1884
sub_topic="test"
#-------------------------------------------------Function to get access token-----------------------------------------------------#
def get_token():
url = "https://token endpoint url"
payload = 'grant_type=client_credentials'
headers = {
'Content-Type':'application/x-www-form-urlencoded'
}
response = requests.post(url, headers=headers, data=payload, auth=HTTPBasicAuth("Client_ID","client_secret"))
token = json.loads(response.text)
return token["access_token"]
mytoken = get_token()
print(mytoken)
#-----------------------------------------------------------Callback functions------------------------------------------------------#
def on_connect(client,userdata,flags, rc):
print("Connected with code " +str(rc))
client.subscribe(sub_topic)
def on_subscribe(client, userdata, mid, granted_qos): #create function for callback
print("subscribed with qos",granted_qos, "\n")
pass
def on_message(client, userdata, message):
print("message received " ,str(message.payload.decode("utf-8")))
def on_publish(client,userdata,mid): #create function for callback
print("data published mid=",mid, "\n")
pass
def on_disconnect(client, userdata, rc):
print("client disconnected")
#------------------------------------------------------------Connect to broker------------------------------------------------------#
client= paho.Client("Client",transport="websockets") #create client object
#client.username_pw_set(username= "mytoken", password= "")
client.on_connect = on_connect
client.on_subscribe = on_subscribe #assign function to callback
client.on_publish = on_publish #assign function to callback
client.on_message = on_message #assign function to callback
client.on_disconnect = on_disconnect
print("connecting to broker ",broker,"on port",port)
#client.connect("ws://broker.lailaafreen.com:1884")
client.connect(broker,port,60) #establish connection
#--------------------------------Token validation-----------------------------------#
def get_valid(token):
url = "https://validation endpoint url"
payload = {}
headers = {'content-type': "multipart/form-data", 'Authorization': 'Bearer ' +mytoken}
response = requests.post(url, headers=headers)
valid = json.loads(response.text)
return valid
print(get_valid(get_token()))
#--------------------------------Publish/Subscribe----------------------------------#
client.loop_start()
print("subscribing to ",sub_topic)
#client.subscribe(sub_topic)
time.sleep(3)
client.publish("test","message") #publish
time.sleep(4)
client.disconnect()
UPDATE: Upon obtaining the access token and after attempting to connect to mosquitto broker, using the command:
sudo mosquitto_pub -h broker_addr -p 1883 -i <acces_token> -t test -m "Hi"
I get not authorised message:
Connection error: Connection Refused: not authorised.
Error: The connection was refused.
Below is my mosquitto configuration file:
persistence false
persistence_location /var/lib/mosquitto/
# Mosquitto Log File Destination
log_dest file /var/log/mosquitto/mosquitto.log
log_type all
# Adding Guman Readable Time Stamsps to /var/log/mosquitto/mosquitto.log
log_timestamp true
log_timestamp_format %Y-%m-%dT%H:%M:%S
# =================================================================
# Security
# =================================================================
allow_anonymous false
# =================================================================
# Listeners
# =================================================================
listener 1884
protocol websockets
socket_domain ipv4
listener 1883
protocol mqtt
include_dir /etc/mosquitto/conf.d
I have a game or for that matter any remote user interface with a server and multiple clients which should communicate via network.
Both client and server should be able to send updates asynchronously.
This seems to be a very natural service definition, which let's grpc manage sessions.
syntax = "proto3";
package mygame;
service Game {
rpc participate(stream ClientRequest) returns (ServerResponse);
}
message ClientRequest {
// Fields for the initial request and further updates
}
message ServerResponse {
// Game updates
}
Implementing the client is trivial (although the following code is obviously incomplete and simplified).
class Client:
def __init__(self):
self.channel = grpc.insecure_channel("localhost:50051")
self.stub = game_pb2_grpc.GameStub(channel)
self.output_queue = queue.Queue()
def output_iter(self):
while True:
client_output_msg = self.output_queue.get()
self.output_queue.task_done()
yield client_output_msg
def do_work(self):
for response in self.stub.participate(self.output_iter()):
print(response) # handle update
with grpc.insecure_channel("localhost:50051") as channel:
client = Client()
client.do_work()
What seems hard is implementing the server without blocking.
class Game(game_pb2_grpc.GameServicer):
def __init__(self):
self.pending_events = queue.Queue()
def participate(self, request_iter, context):
for client_update in request_iter:
print(client_update)
# !!!
# The next bit won't happen if the client has no updates
# !!!
try:
while True:
server_update = self.pending_events.get_nowait()
yield server_update
except queue.Empty:
pass
server = grpc.server(ThreadPoolExecutor(max_workers=100))
game_pb2_grpc.add_GameServicer_to_server(Game(), server)
server.add_insecure_port("[::]:50051")
server.start()
server.wait_for_termination()
As commented in the code, the client won't receive updates if it doesn't constantly send requests.
Maybe a async approach would be better, which might also solve other problems in this design.
PS: This issue has been solved with grpc in go here, however i don't see how to translate this to pythons grpc implementations.
I would be very happy about any help!
I was finally able to get it working using the python asynio api.
The basic idea is to decouple read and write into two coroutines using asyncio.create_task.
For anybody interested, here is a solution.
class Game(game_pb2_grpc.GameServicer):
async def read_client_requests(self, request_iter):
async for client_update in request_iter:
print("Recieved message from client:", client_update, end="")
async def write_server_responses(self, context):
for i in range(15):
await context.write(game_pb2.ServerResponse(dummy_value=str(i)))
await asyncio.sleep(0.5)
async def participate(self, request_iter, context):
read_task = asyncio.create_task(self.read_client_requests(request_iter))
write_task = asyncio.create_task(self.write_server_responses(context))
await read_task
await write_task
async def serve():
server = grpc.aio.server()
game_pb2_grpc.add_GameServicer_to_server(Game(), server)
server.add_insecure_port("[::]:50051")
await server.start()
await server.wait_for_termination()
if __name__ == "__main__":
asyncio.run(serve())
Note that instead of the write coroutine, a yield would also be sufficient.
So I'm doing a unittest of a tornado server I programmed which has a websocketHandler. This is my testing code:
def test_unauthorized_websocket(self):
message = "Hey bro"
ws = websocket.create_connection('ws://localhost:8888/ws', header=['Authorization: false'])
send_dict = {'command': test_command, 'message': message}
serialized_dict = json.dumps(send_dict)
ws.send(serialized_dict)
response = ws.recv()
#test response with assert
My goal with this test was to prove that my tornado server correctly refuses and closes this websocket connection because of wrong authentication header.
This is my tornado websocketHandler code:
class WebSocketHandler(tornado.websocket.WebSocketHandler):
def open(self):
#some code
headers = self.request.headers
try:
auth = headers['Authorization']
except KeyError:
self.close(code = 1002, reason = "Unauthorized websocket")
print("IT'S CLOSED")
return
if auth == "true":
print("Authorized Websocket!")
#some code
else:
print("Unauthorized Websocket... :-(")
self.close(code = 1002, reason = "Unauthorized websocket")
So when the authentication is wrong, self.close() is called (not sure I need code and reason). This should close the websocket. But this doesn't actually happens in the "client" side. After I call create_connection() in the "client" the ws.connected variable is still True, and when I do ws.send() the on_message method of the websocketHandler is still called and when it tries to make a response with self.write_message() it raises a WebSocketClosedError. And only then the "client" actually closes its side of the websocket, ws.recv() returns nothing and ws.connected turns to False after that.
Is there a way I can communicate to the client side (through handshake headers or something) that the websocket is meant to be closed earlier on its side?
You can override prepare() and raise a tornado.web.HTTPError, instead of overriding open() and calling self.close(). This will reject the connection at the first opportunity and this will be reported on the client side as a part of create_connection() instead of on a later read.
I use websocket to make a long-live connection with the target wss-url successfully. But after receiving one message, the code caught an error named "rsv is not implemented, yet" and closed the connection.
It seems that few people have met this problem, which described as "rsv is not implemented, yet". And the API doc of websocket never mention this issue.
The main piece of my code:
def on_message(ws, message):
print(message)
def on_error(ws, error):
print("!!!find error!!!")
print(error)
def on_close(ws):
print("### why closed ???###")
websocket.enableTrace(True)
ws = websocket.WebSocketApp(url,
on_message = on_message,
on_error = on_error,
on_close = on_close,
header = header,
cookie = cookie,
)
ws.run_forever(origin = 'https://matters.news', skip_utf8_validation = True)
It will give me only one message, and then show that:
!!!find error!!!
rsv is not implemented, yet
send: b'\x88\x82\xd9\xe2\xcc\x8c\xda\n'
### why closed ???###
I received the same error and fixed it by removing:
'Sec-WebSocket-Extensions': 'permessage-deflate'
from my headers.
I'm trying to implement a REST client in python that reacts to messages received from the server received through an opened websocket with the concerned server.
Here is the scenario:
client opens a websocket with the server
from time to time, the server sends a message to the client
when the client receives the messages, it gets some information from
the server
The current client I have is able to open the websocket and to receive the message from the server. However, as soon as it receives the messages, it gets the information from the server then terminates while I'd like to keep it listening for other messages that will make it get a new content from the server.
Here is the piece of code I have:
def openWs(serverIp, serverPort):
##ws url setting
wsUrl = "ws://"+serverIp+":"+serverPort+"/websocket"
##open ws
ws = create_connection(wsUrl)
##send user id
print "Sending User ID..."
ws.send("user_1")
print "Sent"
##receiving data on ws
print "Receiving..."
result = ws.recv()
##getting new content
getUrl = "http://"+serverIp+":"+serverPort+"/"+result+"/entries"
getRest(getUrl)
I don't know if using threads is appropriate or not, I'm not expert in that.
If someone could help, it'll be great.
Thanks in advance.
I finished with this code, doing what I'm expecting. Git it from here
import websocket
import thread
import time
def on_message(ws, message):
print message
def on_error(ws, error):
print error
def on_close(ws):
print "### closed ###"
def on_open(ws):
def run(*args):
for i in range(3):
time.sleep(1)
ws.send("Hello %d" % i)
time.sleep(1)
ws.close()
print "thread terminating..."
thread.start_new_thread(run, ())
if __name__ == "__main__":
websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://localhost:5000/chat",
on_message = on_message,
on_error = on_error,
on_close = on_close)
ws.on_open = on_open
ws.run_forever()