We Keep Coding

Under what circumstances is a class member function not passed self as the first parameter?

Problem
I am using a library to facilitate client side websocket communication with a server.
The websocket library allows you to specify call back functions for when the socket opens, closes, errors or receives a message
If I set my callback functions to be instance functions of another class, then they need to be passed the self parameter when the are called.
I had understood that if you call a class instance method it will always be passed self as the first parameter. However,my callbacks are not getting passed self
For example
from websocket import WebSocketApp
import websocket
class X(object):
def run(self):
self.ws = WebSocketApp('wss://api.bitfinex.com/ws/2'
,on_open=self.on_open
,on_message=self.on_message
,on_error=self.on_error
,on_close=self.on_close)
websocket.enableTrace(True)
self.ws.run_forever()
def on_open(self, ws):
print('open')
def on_close(self, ws):
print('close')
def on_message(self, ws, message):
print('message')
def on_error(self, ws, error):
print('error')
if __name__=='__main__':
x = X().run()
Output
error from callback <bound method X.on_open of <__main__.X object at 0x7fd7635e87f0>>: on_open() missing 1 required positional argument: 'ws'
File "/home/arran/.local/lib/python3.6/site-packages/websocket/_app.py", line 343, in _callback
callback(*args)
I am probably missing something basic here. But any help would be greatly appreciated
Edit
Looks like this might be a version specific issue with the websocket-client library https://github.com/home-assistant/home-assistant/issues/17532
I have downgraded to an earlier version and fixed my problem.
I would still be curious to know how this issue can arise though. My understanding was that class instance methods will always be passed self as the first parameter

It looks to be an issue with the WebSocket class not passing the ws argument that your on_open method expects. I tried to reproduce it with my own dummy class, and it works fine.
class WS:
def __init__(self, on_call):
self.on_call = on_call
def call(self):
print("hi")
self.on_call(self)
class X:
def on_call(self, ws):
print(ws)
def run(self):
self.ws = WS(self.on_call)
self.ws.call()
X().run()
hi
<__main__.WS instance at 0x029AB698>

I am probably missing something basic here.
No, you were spot on. However, the on_open callback does not get called with the ws argument, although it should according to the documentation:
class WebSocketApp(object):
(...)
on_open: callable object which is called at opening websocket.
this function has one argument. The argument is this class object.
(...)
This is a known bug that was closed despite some discussion around the way it was fixed.
would still be curious to know how this issue can arise though.
I guess it's an honest mistake in an attempted bug fix. As there is no test for your particular scenario, it did not get caught.
I have downgraded to an earlier version and fixed my problem
Please kindly submit a bug report or write a pull request to fix the problem.
My understanding was that class instance methods will always be passed self as the first parameter
Yes, your understanding is correct. Here is an example mirroring what you tried.
class Server(object):
def __init__(self, callback):
self.callback = callback
def run(self):
self.callback(5)
class Client(object):
def on_message(self, n):
print("got", n)
client = Client()
server = Server(client.on_message)
server.run()

How to mock a Django internal library using patch decorator

I'm mocking an internal library class (Server) of python that provides connection to HTTP JSON-RPC server. But when running the test the class is not mocking. The class is used calling a project class that is a wrapper for other class that effectively instantiates the Server class.
I extract here the parts of the code that give sense for what I'm talking about.
Unit test:
#patch('jsonrpc_requests.jsonrpc.Server')
def test_get_question_properties(self, mockServer):
lime_survey = Questionnaires()
# ...
Class Questionnaires:
class Questionnaires(ABCSearchEngine):
""" Wrapper class for LimeSurvey API"""
def get_question_properties(self, question_id, language):
return super(Questionnaires, self).get_question_properties(question_id, language)
Class Questionnaires calls the method get_question_properties from class ABCSearchEnginge(ABC). This class initializes the Server class to provide the connection to the external API.
Class ABCSearchEnginge:
class ABCSearchEngine(ABC):
session_key = None
server = None
def __init__(self):
self.get_session_key()
def get_session_key(self):
# HERE the self.server keep getting real Server class instead the mocked one
self.server = Server(
settings.LIMESURVEY['URL_API'] + '/index.php/admin/remotecontrol')
As the test is mocking Server class why it's not mocking? What is the missing parts?

From what i see you didnt add a return value.
Were did you put the mocked value in : #patch('jsonrpc_requests.jsonrpc.Server') ?
If you try to add a MagicMock what happend (Dont forget to add from mock import patch, MagicMock)?
#patch('jsonrpc_requests.Server', MagicMock('RETURN VALUE HERE'))
You also need to Mock the __init__ method (Where Server is this one from jsonrpc_requests import Server):
#patch.object(Server, '__init__', MagicMock(return_value=None))
I extrapolated your problem from my own understanding, maybe you need to fix some path (Mock need the exact path to do the job).

How to Log to a variable or write observer that sends messages to variable in Twisted / Autobahn

I am writing a websocket client that will receive updates every few seconds or so utilizing autobahn with twisted. I am successfully logging the data using multiple observers, however I want to use part of the messages I am receiving to send to a dataframe (and eventually plot in real time). My assumption is that I can log to a variable as well as a file-like object, but I cannot figure out how to do that. What is the correct way to achieve this.
I have very thoroughly read the docs for the current and legacy twisted loggers:
twisted.log https://twistedmatrix.com/documents/current/core/howto/logging.html
twisted.logger https://twistedmatrix.com/documents/current/core/howto/logger.html
In my code I have tried to use a zope.interface and #provider as referenced in the new twisted.logger package to create a custom log observer but have had no luck thus far even getting a custom log observer to print, let alone even send data to a variable.
from twisted.internet import reactor
from autobahn.twisted.websocket import WebSocketClientFactory, WebSocketClientProtocol, connectWS
from twisted.logger import (globalLogBeginner, Logger, globalLogPublisher,
jsonFileLogObserver, ILogObserver)
import sys
import io
import json
from pandas import DataFrame
def loggit(message):
log.info("Echo: {message!r}", message=message)
class ClientProtocol(WebSocketClientProtocol):
def onConnect(self, response):
print("Server connected: {0}".format(response.peer))
def initMessage(self):
message_data = {}
message_json = json.dumps(message_data)
print "sendMessage: " + message_json
self.sendMessage(message_json)
def onOpen(self):
print "onOpen calls initMessage()"
self.initMessage()
def onMessage(self, msg, binary, df):
loggit(msg)
def onClose(self, wasClean, code, reason):
print("WebSocket connection closed: {0}".format(reason))
if __name__ == '__main__':
factory = WebSocketClientFactory("wss://ws-feed.whatever.com")
factory.protocol = ClientProtocol
#provider(ILogObserver)
def customObserver(whatgoeshere?):
print event
observers = [jsonFileLogObserver(io.open("loga.json", "a")),
jsonFileLogObserver(io.open("logb.json", "a")), customObserver(Whatgoeshere?)]
log = Logger()
globalLogBeginner.beginLoggingTo(observers)
connectWS(factory)
reactor.run()

A log observer is simply a callable object that takes a dictionary containing all the values that are part of the log message.
This means you can have an instance of a class with a __call__ method decorated with #zope.interface.implementer(ILogObserver), or a function decorated with #zope.interface.provider(ILogObserver), which can perform that role.
Here's an example of some code which logs some values to a text file, a JSON file, and an in-memory statistics collector which sums things up on the fly.
import io
from zope.interface import implementer
from twisted.logger import (globalLogBeginner, Logger, jsonFileLogObserver,
ILogObserver, textFileLogObserver)
class Something(object):
log = Logger()
def doSomething(self, value):
self.log.info("Doing something to {value}",
value=value)
#implementer(ILogObserver)
class RealTimeStatistics(object):
def __init__(self):
self.stats = []
def __call__(self, event):
if 'value' in event:
self.stats.append(event['value'])
def reportCurrent(self):
print("Current Sum Is: " + repr(sum(self.stats)))
if __name__ == "__main__":
stats = RealTimeStatistics()
globalLogBeginner.beginLoggingTo([
jsonFileLogObserver(io.open("log1.json", "ab")),
textFileLogObserver(io.open("log2.txt", "ab")),
stats, # here we pass our log observer
], redirectStandardIO=False)
something = Something()
something.doSomething(1)
something.doSomething(2)
something.doSomething(3)
stats.reportCurrent()

how to call object instance methods from places they weren't defined

I have a top level script that creates instances of objects and executes instance's methods. I am very confused as how I can execute fetcher's methods from within the alarm_handler file. Specifically, line 'status = fetcher.get_status' below. Hopefully this makes sense as to what I'm asking. Please let me know if I can clarify anything.
***file: master***
import fetcher
import alarm_handler
fetcher = fetcher.Fetcher()
alarms = alarm_handler.AlarmHandler()
site_status = alarms.compare_status()
***file: fetcher***
class Fetcher(object):
def fetch(self)
fetch a bunch of internet data
def get_status(self)
fetch some other internet data
***file: alarm_handler***
from master import fetcher
class AlarmHandler(object):
def compare_status (self)
status = fetcher.get_status()
status_comparison = status comparison stuff
return status_comparison

You are creating a circular import. Don't.
Perhaps you wanted to pass the fetcher instance to the AlarmHandler.compare_status() method instead?
import fetcher
import alarm_handler
fetcher = Fetcher()
alarms = AlarmHandler()
site_status = alarms.compare_status(fetcher)
or perhaps pass it in when you create the AlarmHandler() instance:
import fetcher
import alarm_handler
fetcher = Fetcher()
alarms = AlarmHandler(fetcher)
site_status = alarms.compare_status()

Passing extra metadata to a RequestHandler using python's SocketServer and Children

I'm implementing a python application which is using ThreadingTCPServer and a custom subclass of BaseRequestHandler. The problem with this is that the ThreadingTCPServer seems to automatically spawn threads and create instances of the handler, calling their handle() function. However this leaves me with no way to pass data to the handler other than using global variables or class variables, both of which seem hackish. Is there any better way to do it?
Ideally this should be something like:
class ThreadedTCPServer(ThreadingTCPServer):
def process_request(self, *args, **kwargs):
ThreadingTCPServer.process_request(self, data, *args, **kwargs)
with the handler like
class ThreadedTCPRequestHandler(BaseRequestHandler):
def handle(self,data):
#do something with data

I stumbled upon the very same thing. My solution was the following:
class ThreadedTCPRequestHandler(SocketServer.StreamRequestHandler):
def handle(self):
print(self.server.mycustomdata)
class ThreadedTCPServer(SocketServer.ThreadingTCPServer):
pass
server = ThreadedTCPServer((args.host, args.port), ThreadedTCPRequestHandler)
server.mycustomdata = 'foo.bar.z'
server.serve_forever()
The RequestHandler is called with a server object as a third parameter, and it is saved as self.server attribute, so you can access it. If you would set this attribute to a callable, you could easily call it, too:
def handle(self):
mycustomdata = self.server.mycustomdata()

The first answer worked for me, but I think it is cleaner to alter the __init__ method and pass the attribute in the constructor:
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, host_port_tuple, streamhandler, Controllers):
super().__init__(host_port_tuple, streamhandler)
self.Controllers = Controllers
Note the third parameter 'Controllers' in the constructor, then the call to super without that parameter, then setting the new attribute Controllers to the property self.Controllers. The rest of the class is unchanged. Then, in your Requesthandler, you get access to the parameter using the 'server' attribute, as described above:
def handle(self):
self.Controllers = self.server.Controllers
<rest of your code>
It's much the same as the answer above but I find it a little cleaner because the constructor is overloaded and you simply add the attribute you want in the constructor:
server = ServerInterface.ThreadedTCPServer((HOST, PORT), ServerInterface.ThreadedTCPRequestHandler, Controllers)

Since handle is implemented by your BaseRequest subclass, it can get the data from itself without having it passed by the caller. (handle could also be a callable attribute of the request instance, such as a lambda—explicit user_data arguments are normally unnecessary in idiomatically designed python.)
Looking at the SocketServer code, it should be straightforward to override finish_request to pass the additional data to your BaseRequestHandler subtype constructor which would store it in the instance for handle to use.