I need to set up a jabber bot, using python, that will send messages based on the online/offline availability of several contacts.
I've been looking into pyxmpp and xmpppy, but couldn't find any way (at least nothing straightforward) to check the status of a given contact.
Any pointers on how to achieve this?
Ideally I would like something like e.g. bot.status_of("contact1#gmail.com") returning "online"
I don't think it is possible in the way you want it because the presence of contacts (which contains the information about their availability) is received asynchronously by the bot.
You will have to write a presence handler function and registered it with the connection. This function will get called whenever a presence is received from a contact. The parameter of the call will tell you if the contact is online or not. Depending upon it you can send the message to the contact.
Using xmpppy you do it something like this:
def connect(jid, password, res, server, proxy, use_srv):
conn = xmpp.Client(jid.getDomain())
if not conn.connect(server=server, proxy=proxy, use_srv=use_srv):
log( 'unable to connect to server.')
return None
if not conn.auth(jid.getNode(), password, res):
log( 'unable to authorize with server.')
return None
conn.RegisterHandler( 'presence', callback_presence)
return conn
conn = connect(...)
def callback_presence(sess, pres):
if pres.getStatus() == "online":
msg = xmpp.Message(pres.getFrom(), "Hi!")
conn.send(msg)
PS: I have not tested the code but it should be something very similar to this.
What you want is done via a <presence type="probe"/>. This is done on behalf of the client, and SHOULD not be done by them (as per the RFC for XMPP IM). Since this is a bot, you could implement the presence probe, and receive the current presence of a given entity. Remember to send the probe to the bare JID (sans resource), because the server responds on behalf of clients for presence probes. This means your workflow will look like:
<presence/> // I'm online! BOT
<presence from="juliet#capulet.org/balcony"/> RESPONSE
<presence from="romeo#montague.net/hallway"/> // and so on... RESPONSE
<presence type="probe" to="benvolio#montague.net"/> BOT
<presence from="benvoio#montague.net/hallway"> RESPONSE
<status>Huzzah!</status>
<priority>3</priority>
</presence>
Take a look at that portion of the RFC for more in depth information on how your call flow should behave.
What you need to do is:
Connect.
Declare a presence handler. That handler maintains a cache of every contact's presence (see details below)
Send initial presence to the server, which will provoke the reception of presence statuses from all of your online contacts, which will in turn trigger the handler.
The status_of() method reads the cache and deduces the contact's presence status immediately.
Now, it depends on what presence information you need. For the sake of simplicity, let's pretend you just need an "online"/"offline" value. The cache would be a dictionary whose keys are the bare (no resource) JIDs, and the values are a set of connected resources for this JID. For example:
{'foo#bar.com': set(['work', 'notebook']), 'bob#example.net': set(['gtalk'])}
Now, when you receive an "available" presence from a certain JID/resource:
if jid not in cache:
cache[jid] = set()
cache[jid].add(resource)
Reciprocally, when you receive an "unavailable" presence:
if jid in cache: # bad people send "unavailable" just to make your app crash
cache[jid].discard(resource)
if 0 == len(cache[jid]):
del cache[jid]
And now:
def is_online(jid):
return jid in cache
Of course, if you want more detailed information, you could maintain not only the list of available resources for a contact but also the status, status message, priority, etc. of each resource.
Related
I am working on a Python flask app, and the main method start() calls an external API (third_party_api_wrapper()). That external API has an associated webhook (webhook()) that receives the output of that external API call (note that the output that webhook() receives is actually different from the response returned in the third_party_wrapper())
The main method start() needs the result of webhook(). How do I make start() wait for webhook() to be executed? And how do wo pass the returned value of webhook() back to start()?
Here's is a minimal code snippet to capture the scenario.
#app.route('/webhook', methods=['POST'])
def webhook():
return "webhook method has executed"
# this method has a webhook that calls webhook() after this method has executed
def third_party_api_wrapper():
url = 'https://api.thirdparty.com'
response = requests.post(url)
return response
# this is the main entry point
#app.route('/start', methods=['POST'])
def start():
third_party_api_wrapper()
# The rest of this code depends on the output of webhook().
# How do we wait until webhook() is called, and how do we access the returned value?
The answer to this question really depends on how you plan on running your app in production. It's much simpler if we make the assumption that you only plan to have a single instance of your app running at once (as opposed to multiple behind a load balancer, for example), so I'll make that assumption first to give you a place to start, and comment on a more "production-ready" solution afterwards.
A big thing to keep in mind when writing a web application is that you have to understand how you want the outside world to interact with your app. Do you expect to have the /start endpoint called only once at the beginning of your app's lifetime, or is this a generic endpoint that may start any number of background processes that you want the caller of each to wait for? Or, do you want the behavior where any caller after the first one will wait for the same process to complete as the first one? I can't answer these questions for you, it depends on the use-case you're trying to implement. I'll give you a relatively simple solution that you should be able to modify to fulfill any of the ones I mentioned though.
This solution will use the Event class from the threading standard library module; I added some comments to clarify which parts you may have to change depending on the specifics of the API you're calling and stuff like that.
import threading
import uuid
from typing import Any
import requests
from flask import Flask, Response, request
# The base URL for your app, if you're running it locally this should be fine
# however external providers can't communicate with your `localhost` so you'll
# need to change this for your app to work end-to-end.
BASE_URL = "http://localhost:5000"
app = Flask(__name__)
class ThirdPartyProcessManager:
def __init__(self) -> None:
self.events = {}
self.values = {}
def wait_for_request(self, request_id: str) -> None:
event = threading.Event()
actual_event = self.events.setdefault(request_id, event)
if actual_event is not event:
raise ValueError(f"Request {request_id} already exists.")
event.wait()
return self.values.pop(request_id)
def finish_request(self, request_id: str, value: Any) -> None:
event = self.events.pop(request_id, None)
if event is None:
raise ValueError(f"Request {request_id} does not exist.")
self.values[request_id] = value
event.set()
MANAGER = ThirdPartyProcessManager()
# This is assuming that you can specify the callback URL per-request, otherwise
# you may have to get the request ID from the body of the request or something
#app.route('/webhook/<request_id>', methods=['POST'])
def webhook(request_id: str) -> Response:
MANAGER.finish_request(request_id, request.json)
return "webhook method has executed"
# Somehow in here you need to create or generate a unique identifier for this
# request--this may come from the third-party provider, or you can generate one
# yourself. There are three main paths I see here:
# - If you can specify the callback/webhook URL in each call, you can just pass them
# <base>/webhook/<request_id> and use that to identify which request is being
# responded to in the webhook.
# - If the provider gives you a request ID, you can return it from this function
# then retrieve it from the request body in the webhook route
# For now, I'll assume the first situation but you should be able to implement the second
# with minimal changes
def third_party_api_wrapper() -> str:
request_id = uuid.uuid4().hex
url = 'https://api.thirdparty.com'
# Just an example, I don't know how the third party API you're working with works
response = requests.post(
url,
json={"callback_url": f"{BASE_URL}/webhook/{request_id}"}
)
# NOTE: unrelated to the problem at hand, you should always check for errors
# in HTTP responses. This method is an easy way provided by requests to raise
# for non-success status codes.
response.raise_for_status()
return request_id
#app.route('/start', methods=['POST'])
def start() -> Response:
request_id = third_party_api_wrapper()
result = MANAGER.wait_for_request(request_id)
return result
If you want to run the example fully locally to test it, do the following:
Comment out lines 62-71, which actually make the external API call
Add a print statement after line 77, so that you can get the ID of the "in flight" request. E.g. print("Request ID", request_id)
In one terminal, run the app by pasting the above code into an app.py file and running flask run in that directory.
In another terminal, start the process via:
curl -XPOST http://localhost:5000/start
Copy the request ID that will be logged in the first terminal that's running the server.
In a third terminal, complete the process by calling the webhook:
curl -XPOST http://localhost:5000/webhook/<your_request_id> -H Content-Type:application/json -d '{"foo":"bar"}'
You should see {"foo":"bar"} as the response in the second terminal that made the /start request.
I hope that's enough to help you get started w/ whatever problem you're trying to solve.
There are a couple of design-y comments I have based on the information provided as well:
As I mentioned before, this will not work if you have more than one instance of the app running at once. This works by storing the state of in-flight requests in a global state inside your python process, so if you have more than one process, they won't all be working and modifying the same state. If you need to run more than one instance of your process, I would use a similar approach with some database backend to store the shared state (assuming your requests are pretty short-lived, Redis might be a good choice here, but once again it'll depend on exactly what you're trying to do).
Even if you do only have one instance of the app running, flask is capable of being run in a variety of different server contexts--for example, the server might be using threads (the default), greenlets via gevent or a similar library, or multiple processes, or maybe some other approach entirely in order to handle multiple requests concurrently. If you're using an approach that creates multiple processes, you should be able to use the utilities provided by the multiprocessing module to implement the same approach as I've given above.
This approach probably will work just fine for something where the difference in time between the API call and the webhook response is small (on the order of a couple of seconds at most I'd say), but you should be wary of using this approach for something where the difference in time can be quite large. If the connection between the client and your server fails, they'll have to make another request and run the long-running process that your third party is completing for you again. Some proxies and load balancers may also have time out behavior that could terminate the request after a certain amount of time even if nothing goes wrong in the connection between your server and the client making a request to it. An alternative approach would be for your /start endpoint to return quickly and give the client a request_id that they could poll for updates. As an example, AWS Athena's API is structured like this--there is a StartQueryExecution method, and separate GetQueryExecution and GetQueryResults methods that the client makes requests to check the status of a query and retrieve the results respectively (there are also other methods like StopQueryExecution and GetQueryRuntimeStatistics available as well). You can check out the documentation here.
I know that's a lot of info, but I hope it helps. Happy to update the answer w/ more specific info if you'll provide some more details about your use-case.
I am modifying pika headers using
properties.headers = {
'myheader': myheader
}
But I am acking and nacking with the delivery_tag
channel.basic_nack(delivery_tag=delivery_tag, requeue=False)
How can I pass the update properties with the headers to the ack and nack response functions? Or what is the pika way of doing this?
It is correct that basic_nack cannot change the headers.
The way to do this is not to use NACK at all but to generate and return a 'new' message (which is simply the current message you are handling, but adding new headers to it).
It appears that a NACK is basically doing this anyway according to the AMQP spec.
So my logic is using basic_ack on success and message generation with updated headers on failure. And in my case I 'redirect' the new message to a dead letter exchange which a dead letter queue is binded to.
An extract from pika docs say that you misspell basic_nck... is just a question error or is your actual issue?
def basic_nack(self, delivery_tag=None, multiple=False, requeue=True):
"""This method allows a client to reject one or more incoming messages.
It can be used to interrupt and cancel large incoming messages, or
return untreatable messages to their original queue.
:param integer delivery-tag: int/long The server-assigned delivery tag
:param bool multiple: If set to True, the delivery tag is treated as
"up to and including", so that multiple messages
can be acknowledged with a single method. If set
to False, the delivery tag refers to a single
message. If the multiple field is 1, and the
delivery tag is zero, this indicates
acknowledgement of all outstanding messages.
:param bool requeue: If requeue is true, the server will attempt to
requeue the message. If requeue is false or the
requeue attempt fails the messages are discarded or
dead-lettered.
"""
self._raise_if_not_open()
return self._send_method(
spec.Basic.Nack(delivery_tag, multiple, requeue))
Sorry for that but as far as I know is not possible for a basic_nack (or basic_ack) modify the headers. The problem is that the modified message will be put in the dead letter queue and the new one has a newer ID.
L-
Using the code below I'm sending an email on error. I'm trying to include a link to the Cloud Console logs in the email but the request ID seems to be wrong about 30% of the time.
If I find the request with the wrong ID it's always almost a perfect match except the last three characters are 0 (in the Stackdriver console) instead of 101 (returned from the env variable), always the same substitution - is this a bug with cloud console or am I trying to use these IDs wrong?
The code (stripped down version):
class ErrorAlertMiddleware(object):
def process_response(self, request, response):
if response.status_code == 500:
logger.info(os.environ.get('REQUEST_LOG_ID'))
msg = 'Link to logs: https://console.cloud.google.com/logs/viewer?' + '&'.join((
'project=%s' % MY_APP_ID,
'expandAll=true',
'filters=request_id:%s' % os.environ.get('REQUEST_LOG_ID'),
'resource=gae_app',
))
# this is a utility func that simply sends email
sendemail(ERROR_RECIPIENT, msg)
return response
Note I've also logged the REQUEST_LOG_ID to ensure it's not being encoded or something and the log output matches what shows in the link
Instead of os.environ.get('REQUEST_LOG_ID'), use request.environ.get('REQUEST_LOG_ID').
It may be possible that os.environ['REQUEST_LOG_ID'] changes between the start of the current request and the time you access it, but request.environ['REQUEST_LOG_ID'] should not change once the request is initialized. The docs state that if one request ID is greater than another, than it occurred later than the other. This implies that the requestID in the Stackdriver console was generated before the one in your email link. This makes me think that somewhere along the line, os.environ['REQUEST_LOG_ID'] is being updated from '....000' to '....101' before you access it, while the copy in request.environ['REQUEST_LOG_ID'] should remain unchanged.
For more info on the request.environ, take a look at the source code in google.appengine.runtime.request_environment.py. I haven't really found documentation on that, but that code led me to believe that the os.environ is not as safe to access as the request.environ.
I am using the standard asynchronous publisher example. and i noticed that the publisher will keep publishing the same message in a loop forever.
So i commented the schedule_next_message call from publish_message to stop that loop.
But what i really want is for the publissher to start and publish only when a user give it a "message_body" and "Key"
basically publisher to publish the user inputs.
i was not able to fin any examples or hints of how to make the publisher take inputs from user in real time.
I am new to raabitmq, pika, python e.t.c
here is the snippet of code i am talking about :-
def publish_message(self):
"""If the class is not stopping, publish a message to RabbitMQ,
appending a list of deliveries with the message number that was sent.
This list will be used to check for delivery confirmations in the
on_delivery_confirmations method.
Once the message has been sent, schedule another message to be sent.
The main reason I put scheduling in was just so you can get a good idea
of how the process is flowing by slowing down and speeding up the
delivery intervals by changing the PUBLISH_INTERVAL constant in the
class.
"""
if self._stopping:
return
message = {"service":"sendgrid", "sender": "nutshi#gmail.com", "receiver": "nutshi#gmail.com", "subject": "test notification", "text":"sample email"}
routing_key = "email"
properties = pika.BasicProperties(app_id='example-publisher',
content_type='application/json',
headers=message)
self._channel.basic_publish(self.EXCHANGE, routing_key,
json.dumps(message, ensure_ascii=False),
properties)
self._message_number += 1
self._deliveries.append(self._message_number)
LOGGER.info('Published message # %i', self._message_number)
#self.schedule_next_message()
#self.stop()
def schedule_next_message(self):
"""If we are not closing our connection to RabbitMQ, schedule another
message to be delivered in PUBLISH_INTERVAL seconds.
"""
if self._stopping:
return
LOGGER.info('Scheduling next message for %0.1f seconds',
self.PUBLISH_INTERVAL)
self._connection.add_timeout(self.PUBLISH_INTERVAL,
self.publish_message)
def start_publishing(self):
"""This method will enable delivery confirmations and schedule the
first message to be sent to RabbitMQ
"""
LOGGER.info('Issuing consumer related RPC commands')
self.enable_delivery_confirmations()
self.schedule_next_message()
the site does not let me add the solution .. i was able to solve my issue using raw_input()
Thanks
I know I'm a bit late to answer the question but have you looked at this one?
Seems to be a bit more related to what you need than using a full async publisher. Normally you use those with a Python Queue to pass messages between threads.
I am playing around with a small project to get a better understanding of web technologies.
One requirement is that if multiple clients have access to my site, and one make a change all the others should be notified. From what I have gathered Server Sent events seems to do what I want.
However when I open my site in both Firefox and Chrome and try to send an event, only one of the browsers gets it. If I send an event again only one of the browsers gets the new event, usually the browser that did not get event number one.
Here is the relevant code snippets.
Client:
console.log("setting sse handlers")
viewEventSource = new EventSource("{{ url_for('viewEventRequest') }}");
viewEventSource.onmessage = handleViewEvent;
function handleViewEvent(event){
console.log("called handle view event")
console.log(event);
}
Server:
#app.route('/3-3-3/view-event')
def view_event_request():
return Response(handle_view_event(), mimetype='text/event-stream')
def handle_view_event():
while True:
for message in pubsub_view.listen():
if message['type'] == 'message':
data = 'retry: 1\n'
data += 'data: ' + message['data'] + '\n\n'
return data
#app.route('/3-3-3/test')
def test():
red.publish('view-event', "This is a test message")
return make_response("success", 200)
My question is, how do I get the event send to all connected clients and not just one?
Here are some gists that may help (I've been meaning to release something like 'flask-sse' based on 'django-sse):
https://gist.github.com/3680055
https://gist.github.com/3687523
also useful - https://github.com/jkbr/chat/blob/master/app.py
The first thing I notice about your code is that 'handle_view_event' is not a generator.
Though it is in a 'while' loop, the use of 'return' will always exit the function the first time we return data; a function can only return once. I think you want it to be 'yield' instead.
In any case, the above links should give you an example of a working setup.
As Anarov says, websockets and socket.io are also an option, but SSE should work anyway. I think socket.io supports using SSE if ws are not needed.