We Keep Coding

variables inside BaseHTTPRequestHandler Python

I am creating a chatbot using Python and MS Bot Builder SDK for Python.
The bot is a HTTPServer using a handler. What I want are variables to help me keeping track of the conversation, for example a message counter. But I can't get it to work, each time the bot receives a request (me sending something), it's like another handler is created, cause the number of messages is always 1. I'm not sure what is being called on each request.
Here is the (important) code:
class BotRequestHandler(BaseHTTPRequestHandler):
count = 0
#staticmethod
def __create_reply_activity(request_activity, text):
# not important
def __handle_conversation_update_activity(self, activity):
# not important
def __handle_message_activity(self, activity):
self.count += 1 ############## INCREMENTATION ##############
self.send_response(200)
self.end_headers()
credentials = MicrosoftAppCredentials(APP_ID, APP_PASSWORD)
connector = ConnectorClient(credentials, base_url=activity.service_url)
reply = BotRequestHandler.__create_reply_activity(activity, '(%d) You said: %s' % (self.count, activity.text))
connector.conversations.send_to_conversation(reply.conversation.id, reply)
def __handle_authentication(self, activity):
# not important
def __unhandled_activity(self):
# not important
def do_POST(self):
body = self.rfile.read(int(self.headers['Content-Length']))
data = json.loads(str(body, 'utf-8'))
activity = Activity.deserialize(data)
if not self.__handle_authentication(activity):
return
if activity.type == ActivityTypes.conversation_update.value:
self.__handle_conversation_update_activity(activity)
elif activity.type == ActivityTypes.message.value:
self.__handle_message_activity(activity)
else:
self.__unhandled_activity()
class BotServer(HTTPServer):
def __init__(self):
super().__init__(('localhost', 9000), BotRequestHandler)
def _run(self):
try:
print('Started http server')
self.serve_forever()
except KeyboardInterrupt:
print('^C received, shutting down server')
self.socket.close()
server = BotServer()
server._run()
What I get if enter the message 'a' 4 times is '(1) You said: a' 4 times.
I tried overrifing init method of BaseHTTPRequestHandler but it didn't work.
For those who know: the thing is with Python SDK we don't have Waterfall dialogs like in Node.js, or I didn't find how it works, if someone knows just tell me, cause here I need to keep track of a lot of things from the user and I need variables. And I really want to use Python because I need some ML and other modules in Python.
Thank you for your help.

How to send RabbitMQ messages to Pykka actor?

UPDATE Aug, 2015: For people wanting to use messaging, I currently would recommend zeromq. Could be used in addition to, or as a complete replacement of, pykka.
How can I listen to a RabbitMQ queue for messages and then forward them to an actor within Pykka?
Currently, when I try to do so, I get weird behavior and the system halts to a stop.
Here is how I have my actor implemented:
class EventListener(eventlet.EventletActor):
def __init__(self, target):
"""
:param pykka.ActorRef target: Where to send the queue messages.
"""
super(EventListener, self).__init__()
self.target = target
def on_start(self):
ApplicationService.listen_for_events(self.actor_ref)
And here is my method inside the ApplicationService class that is supposed to check the queue for new messages:
#classmethod
def listen_for_events(cls, actor):
"""
Subscribe to messages and forward them to the given actor.
"""
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='test')
def callback(ch, method, properties, body):
message = pickle.loads(body)
actor.tell(message)
channel.basic_consume(callback, queue='test', no_ack=True)
channel.start_consuming()
It seems like start_consuming is blocking indefinitely. Is there a way I can "poll" the queue periodically myself?

All your code looks correct to me. If you would like to check the queue used by each actor, you can check their actor_inbox property available on the actor reference returned from Actor#start.
I have run into similar issues when inheriting from EventletActor so to test I tried the same code using an EventletActor and using a ThreadingActor. As far as I can tell from the source code they both are using eventlet to do work. The ThreadingActor works great for me but the EventletActor doesn't work with ActorRef#tell, it does work with ActorRef#ask.
I started with two files in the same directory as shown below.
my_actors.py: Initializes two actors which will respond to messages by printing the message content prefaced by their class name.
from pykka.eventlet import EventletActor
import pykka
class MyThreadingActor(pykka.ThreadingActor):
def __init__(self):
super(MyThreadingActor, self).__init__()
def on_receive(self, message):
print(
"MyThreadingActor Received: {message}".format(
message=message)
)
class MyEventletActor(EventletActor):
def __init__(self):
super(MyEventletActor, self).__init__()
def on_receive(self, message):
print(
"MyEventletActor Received: {message}".format(
message=message)
)
my_threading_actor_ref = MyThreadingActor.start()
my_eventlet_actor_ref = MyEventletActor.start()
my_queue.py: Sets up a queue in pika, sends a message to the queue which is forwarded to the two actors setup before. After each actor is told about the message, their current actor inbox is checked for anything in the queue.
from my_actors import my_threading_actor_ref, my_eventlet_actor_ref
import pika
def on_message(channel, method_frame, header_frame, body):
print "Received Message", body
my_threading_actor_ref.tell({"msg": body})
my_eventlet_actor_ref.tell({"msg": body})
print "ThreadingActor Inbox", my_threading_actor_ref.actor_inbox
print "EventletActor Inbox", my_eventlet_actor_ref.actor_inbox
channel.basic_ack(delivery_tag=method_frame.delivery_tag)
queue_name = 'test'
connection = pika.BlockingConnection()
channel = connection.channel()
channel.queue_declare(queue=queue_name)
channel.basic_consume(on_message, queue_name)
channel.basic_publish(exchange='', routing_key=queue_name, body='A Message')
try:
channel.start_consuming()
except KeyboardInterrupt:
channel.stop_consuming()
# It is very important to stop these actors, otherwise you may lockup
my_threading_actor_ref.stop()
my_eventlet_actor_ref.stop()
connection.close()
When I run my_queue.py the output is as follows:
Received Message A Message
ThreadingActor Inbox <Queue.Queue instance at 0x10bf55878>
MyThreadingActor Received: {'msg': 'A Message'}
EventletActor Inbox <Queue maxsize=None queue=deque([{'msg': 'A Message'}]) tasks=1 _cond=<Event at 0x10bf53b50 result=NOT_USED _exc=None _waiters[0]>>
When I hit CTRL+C to stop the queue, I notice that the EventletActor finally receives the message and prints it:
^CMyEventletActor Received: {'msg': 'A Message'}
All this leads me to believe that there may be a bug in EventletActor, I think your code is fine and a bug exists which I was unable to find in the code on first inspection.
I hope this information helps.

Wait for specific server response command code in IRC

I've made an IRC bot in Python and I've been trying to figure out a way to wait for an IRC command and return the message to a calling function for a while now. I refuse to use an external library for various reasons including I'm trying to learn to make these things from scratch. Also, I've been sifting through documentation for existing ones and they're way too comprehensive. I'me trying to make a simple one.
For example:
def who(bot, nick):
bot.send('WHO %s' % nick)
response = ResponseWaiter('352') # 352 - RPL_WHOREPLY
return response.msg
Would return a an object of my Message class that parses IRC messages to the calling function:
def check_host(bot, nick, host):
who_info = who(bot, nick)
if who_info.host == host:
return True
return False
I have looked at the reactor pattern, observer pattern, and have tried implementing a hundred different event system designs for this to no avail. I'm completely lost.
Please either provide a solution or point me in the right direction. There's got to be a simple way to do this.
So what I've done is use grab messages from my generator (a bot method) from the bot's who method. The generator looks like this:
def msg_generator(self):
''' Provides messages until bot dies '''
while self.alive:
for msg in self.irc.recv(self.buffer).split(('\r\n').encode()):
if len(msg) > 3:
try: yield Message(msg.decode())
except Exception as e:
self.log('%s %s\n' % (except_str, str(e)))
And now the bot's who method looks like this:
def who(self, nick):
self.send('WHO %s' % nick)
for msg in self.msg_generator():
if msg.command == '352':
return msg
However, it's now taking control of the messages, so I need some way of relinquishing the messages I'm not using for the who method to their appropriate handlers.
My bot generally handles all messages with this:
def handle(self):
for msg in self.msg_generator():
self.log('◀ %s' % (msg))
SpaghettiHandler(self, msg)
So any message that my SpaghettiHandler would be handling is not handled while the bot's who method uses the generator to receive messages.
It's working.. and works fast enough that it's hard to lose a message. But if my bot were to be taking many commands at the same time, this could become a problem. I'm pretty sure I'll find a solution in this direction, but I didn't create this as the answer because I'm not sure it's a good way, even when I have it set to relinquish messages that don't pertain to the listener.

Python: Accepting Raw Input While Another Thread Checks For Messages

I am trying to write a basic chat client in Python for a project and have completed the task, easy. However when I handed it in they asked if I could get it to accept user input while checking for messages (an extra unmarked task for people who complete work early).
I assume this is something to do with threading, so I tried creating a thread for accepting user input and one for checking for messages, however it would appear that the raw_input stops the other thread.
How would I do this in python? Perhaps I have misunderstood how threading works? - Python Noob
Second try:
#Update last connection
s[user] = str(time.time());
#Start chat server
class chatServer ( threading.Thread ):
def __init__ (self, channel):
self.channel = channel
self.lastMessage = ""
threading.Thread.__init__ ( self ) #Pass to thread constructor
def messageOut ( self ):
while 1:
print "Asking for input"
message = raw_input("Message: ");
s[self.channel] = message;
time.sleep(1)
def messageIn ( self ):
while 1:
print "Checking for new messages"
if s[self.channel]!=self.lastMessage:
print s[self.channel]
self.lastMessage = s[self.channel]
time.sleep(1)
print "Welcome " + user + " type to send a message"
chatServer("channel1").messageIn()
chatServer("channel1").messageOut()
First try:
#Start chat server
class chatServer ( threading.Thread ):
def __init__ (self, user, channel, server):
self.channel = channel
self.lastMessage = ""
self.user = user
self.s = server
threading.Thread.__init__ ( self ) #Pass to thread constructor
def start ( self ):
print "Welcome " + self.user + " type to send a message"
self.messageIn()
self.messageOut()
def messageOut ( self ):
while 1:
message = raw_input("Message: ");
s['message'] = message;
time.sleep(1)
def messageIn ( self ):
while 1:
print "Checking for new messages"
if s[self.channel]!=self.lastMessage:
print s[self.channel]
lastMessage = s[self.channel]
time.sleep(1)
chatServer(user, "channel1", server).start()
Many thanks for your time
P.s. server is a simple class that gets/puts messages it is given
P.p.s This is not homework, more for my personal interest

Not a real answer to your question but as an aside you may want to to look at eventlet.
It allows you to have co-routines which will enable you to handle the kind of things you want to do but in a way thats very easy to read/understand and (imho) very pythonic.
Heres a great video to get you started: PyCon 2010: Eventlet: Asynchronous I/O with a synchronous interface
The main project website: http://eventlet.net/
A chat example using telnet: http://eventlet.net/doc/examples.html#multi-user-chat-server
Hope it helps and you get a real answer to your question too.

Actually you are creating only one thread, reading and posting a single message in a sequential way.
You have to create two threads that read and write messages and are independent from each other. The real problem is synchronization between the two and the sharing of resources in common.

How to implement a two way jsonrpc + twisted server/client

Hello I am working on develop a rpc server based on twisted to serve several microcontrollers which make rpc call to twisted jsonrpc server. But the application also required that server send information to each micro at any time, so the question is how could be a good practice to prevent that the response from a remote jsonrpc call from a micro be confused with a server jsonrpc request which is made for a user.
The consequence that I am having now is that micros are receiving bad information, because they dont know if netstring/json string that is comming from socket is their response from a previous requirement or is a new request from server.
Here is my code:
from twisted.internet import reactor
from txjsonrpc.netstring import jsonrpc
import weakref
creds = {'user1':'pass1','user2':'pass2','user3':'pass3'}
class arduinoRPC(jsonrpc.JSONRPC):
def connectionMade(self):
pass
def jsonrpc_identify(self,username,password,mac):
""" Each client must be authenticated just after to be connected calling this rpc """
if creds.has_key(username):
if creds[username] == password:
authenticated = True
else:
authenticated = False
else:
authenticated = False
if authenticated:
self.factory.clients.append(self)
self.factory.references[mac] = weakref.ref(self)
return {'results':'Authenticated as %s'%username,'error':None}
else:
self.transport.loseConnection()
def jsonrpc_sync_acq(self,data,f):
"""Save into django table data acquired from sensors and send ack to gateway"""
if not (self in self.factory.clients):
self.transport.loseConnection()
print f
return {'results':'synced %s records'%len(data),'error':'null'}
def connectionLost(self, reason):
""" mac address is searched and all reference to self.factory.clientes are erased """
for mac in self.factory.references.keys():
if self.factory.references[mac]() == self:
print 'Connection closed - Mac address: %s'%mac
del self.factory.references[mac]
self.factory.clients.remove(self)
class rpcfactory(jsonrpc.RPCFactory):
protocol = arduinoRPC
def __init__(self, maxLength=1024):
self.maxLength = maxLength
self.subHandlers = {}
self.clients = []
self.references = {}
""" Asynchronous remote calling to micros, simulating random calling from server """
import threading,time,random,netstring,json
class asyncGatewayCalls(threading.Thread):
def __init__(self,rpcfactory):
threading.Thread.__init__(self)
self.rpcfactory = rpcfactory
"""identifiers of each micro/client connected"""
self.remoteMacList = ['12:23:23:23:23:23:23','167:67:67:67:67:67:67','90:90:90:90:90:90:90']
def run(self):
while True:
time.sleep(10)
while True:
""" call to any of three potential micros connected """
mac = self.remoteMacList[random.randrange(0,len(self.remoteMacList))]
if self.rpcfactory.references.has_key(mac):
print 'Calling %s'%mac
proto = self.rpcfactory.references[mac]()
""" requesting echo from selected micro"""
dataToSend = netstring.encode(json.dumps({'method':'echo_from_micro','params':['plop']}))
proto.transport.write(dataToSend)
break
factory = rpcfactory(arduinoRPC)
"""start thread caller"""
r=asyncGatewayCalls(factory)
r.start()
reactor.listenTCP(7080, factory)
print "Micros remote RPC server started"
reactor.run()

You need to add a enough information to each message so that the recipient can determine how to interpret it. Your requirements sounds very similar to those of AMP, so you could either use AMP instead or use the same structure as AMP to identify your messages. Specifically:
In requests, put a particular key - for example, AMP uses "_ask" to identify requests. It also gives these a unique value, which further identifies that request for the lifetime of the connection.
In responses, put a different key - for example, AMP uses "_answer" for this. The value matches up with the value from the "_ask" key in the request the response is for.
Using an approach like this, you just have to look to see whether there is an "_ask" key or an "_answer" key to determine if you've received a new request or a response to a previous request.
On a separate topic, your asyncGatewayCalls class shouldn't be thread-based. There's no apparent reason for it to use threads, and by doing so it is also misusing Twisted APIs in a way which will lead to undefined behavior. Most Twisted APIs can only be used in the thread in which you called reactor.run. The only exception is reactor.callFromThread, which you can use to send a message to the reactor thread from any other thread. asyncGatewayCalls tries to write to a transport, though, which will lead to buffer corruption or arbitrary delays in the data being sent, or perhaps worse things. Instead, you can write asyncGatewayCalls like this:
from twisted.internet.task import LoopingCall
class asyncGatewayCalls(object):
def __init__(self, rpcfactory):
self.rpcfactory = rpcfactory
self.remoteMacList = [...]
def run():
self._call = LoopingCall(self._pokeMicro)
return self._call.start(10)
def _pokeMicro(self):
while True:
mac = self.remoteMacList[...]
if mac in self.rpcfactory.references:
proto = ...
dataToSend = ...
proto.transport.write(dataToSend)
break
factory = ...
r = asyncGatewayCalls(factory)
r.run()
reactor.listenTCP(7080, factory)
reactor.run()
This gives you a single-threaded solution which should have the same behavior as you intended for the original asyncGatewayCalls class. Instead of sleeping in a loop in a thread in order to schedule the calls, though, it uses the reactor's scheduling APIs (via the higher-level LoopingCall class, which schedules things to be called repeatedly) to make sure _pokeMicro gets called every ten seconds.