I am working on MQTT and using python paho-mqtt https://pypi.python.org/pypi/paho-mqtt
I am unable to understand how can I publish msg to a specific client or list of clients?
I'll appreciate your help.
This isn't directly possible with strict MQTT, although some brokers may offer that functionality, or you can construct your application so that the topic design works to do what you need.
Although I do agree that in some cases it would be useful to send a message to a particular client (or list of clients) that's simply not how the publish/subscribe messaging paradigm works. Read more on the publish-subscribe pattern on Wikipedia. If all your system needs to do is send messages to unique clients, then I would perhaps suggest thinking of a different architecture for the system you are designing. That being said, you can leverage off pub/sub to achieve what you want using a clever topic design architecture.
For example, let's assume all clients are part of a group (list), you could think of the following topic design:
Unique per client: P2P/< client-name >
List/Group subscription: LIST/< list-name >
For example, P2P/user12345 and LIST/QA where only user12345 subscribes to P2P/user12345 but all users of the QA group subscribe to LIST/QA.
It would be the client's responsibility to ensure that it is subscribed to its own topic(s) (or if your broker allows it, you could also add the topics administratively to non-clean clients).
With this design, a publisher would be able to send a message to a specific user or all members of a defined group (list).
Related
If I have one client connect to multiple servers, and try to send a message,
socket = context.socket(zmq.REQ)
socket.connect ("tcp://127.0.0.1:5565")
socket.connect ("tcp://127.0.0.1:5566")
socket.connect ("tcp://127.0.0.1:5567")
socket.send("Hello all")
only one server will actually get the message. The documentation says that pyzmq preforms some simple load balancing across all available servers.
Is there a way send a message to all servers, rather than just one?
Background:
I am trying to control a network of raspberry pis with my computer. I need to send a message to all of them at once, but I can't use PUB/SUB model, because then they all need to respond to that message.
I have one requester (master computer) that sends a request to all of the repliers (raspberry pis), and they all reply individually. For example I could send one message asking to get the reading from a tempurature sensor, and I want all of the raspberry pis to get read a tempurature sensor and send it back.
Yes.
Use an appropriate Formal Communication Pattern.
ZMQ.REQ formalism indeed expects, that the component is asking some other process, via sending a REQUEST, to do some job in response to the message. Thus the multiple exgress targets the .connect() has built a transport relation with, are served in a round-robin mode, selecting one after another, in a fair-queue-policy mode. So the component works but for a different purpose, than you are asking it to do.
Solution
Try some more complex Formal Communication Pattern that "spreads" the message to all relevant peers ( PUB/SUB alike ) but more complex, smarter, fail-safe derived schemes, that would serve your Raspberry PI solution needs.
The greatest strength of the ZeroMQ is in that it off-loads the low-level details from you and leaves you an immense power in designing all the needed distributed scaleable Formal Communication Patterns, that you need. Forget about just the few primitives ( building blocks ) directly listed in the ZeroMQ binding. Think about your abstract message/event processing scheme and then assemble ZeroMQ elements to meet that scheme.
ZeroMQ [socket] is not a hose from A to B. It is rather an access port for dialogues with smart Formal Communication Pattern Nodes. You may benefit, that [socket] may work over many transport classes at the same time ... so your Formal Communication Patterns may span over L3-networks [TCP:] + go into [IPC:] and [INPROC:] process-to-process channels inside the [localhost].
All working in parallel ( well, sure - almost in parallel once inspected in lower detail )
All working in smooth co-integrated environment.
Where to source from?
A best next step you may do for this is IMHO to get a bit more global view, which may sound complicated for the first few things one tries to code with ZeroMQ, but if you at least jump to the page 265 of the Code Connected, Volume 1 [asPdf->], if it were not the case of reading step-by-step there.
The fastest-ever learning-curve would be to have first an un-exposed view on the Fig.60 Republishing Updates and Fig.62 HA Clone Server pair for a possible High-availability approach and then go back to the roots, elements and details.
Use PUB/SUB to send the request, and an entirely separate PUSH/PULL socket to get the answers back. The response message should probably include a field saying which Pi it has come from.
An alternate way is using PUSH/PULL instead of PUB/SUB, because with PUB/SUB method your message may be lost if a subscriber has not been executed, but in PUSH/PULL method when a Client/Sender post a message (PUSH), the Server/Getter can get it any time with the PULL attribute.
Here's a simple example:
Client side snippet code:
import zmq
def create_push_socket(ip, port):
print('PUB')
context = zmq.Context()
socket = context.socket(zmq.PUSH)
zmq_address = "tcp://{}:{}".format(ip, port)
socket.connect(zmq_address)
return socket
sock1 = create_push_socket('RPi-1-IP', RPi-1-PORT)
sock2 = create_push_socket('RPi-1-IP', RPi-1-PORT)
sock3 = create_push_socket('RPi-1-IP', RPi-1-PORT)
sock1.send('Hello')
sock2.send('Hello')
sock3.send('Hello')
Server side snippet code:
import zmq
def listen():
context = zmq.Context()
zmq_ = context.socket(zmq.PULL)
zmq_.bind('tcp://*:6667')
print(zmq_.recv())
listen()
I just used an array of req/rep pairs. Each client has multiple req sockets and each server has one rep socket. Is this not a scalable solution? The data being sent does not require high scalability. If it does be a problem, I could work something out with pub/sub.
I have an XMPP client working with Google's GTalk XMPP server. I'd like to make it so that my JID/resource can receive messages from anyone (whether they are subscribed to me or not). Right now, if a client sends a messages to my username without being subscribed, Google's server returns a service-unavailable error (as it should).
<service-unavailable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
But, I'd like to make it so that the message would go through. The use case here is to provide a public support chat entity so that users can chat me but I don't want them to be subscribed to all my activity (like status messages, etc)
Google Talk explicitly blocks messages from entities you don't share presence with as a spam prevention measure. You can't turn that off, I'm afraid.
I am looking to have my client "react" to a received message through some xml based communications medium. I was looking into xmpp with google talk, but I just need something that can quickly relay messages on an event based basis (i.e. without having a thread over a "check messages" function.)
I am using twisted to do the rest of my project, so if at all possible, it would be very helpful to use twisted for the rest.
wokkel is an extension of twisted words that makes it super easy to develop clients and components.
Here is echobot as an example.
I am working on location-based services project where I have several sensors that need to send asynchronous readings to a server, which will correlate the readings and generate a result. There will be some level of sensor to sensor communication as well, and I am interested in using XMPP as a transport due to its efficient messaging, real-time nature and NAT traversal.
I am hoping to find an example of (python, or any other langauge) XMPP machine to machine (M2M) services, hopefully using a PubSub model for asynchronous communication rather than a polling-based RPC. I have not been able to find any examples online or in XMPP books that I have seen, as they seem to be mostly focused on XMPP for human interaction such as chat, video, etc.
The general requirements that I have to work with are:
1. Multiple sensors sharing data with each other over XMPP
2. Asynchronous (PubSub) communication, subscribing to messages of interest
3. Hopefully written in Python, but any language would be a good starting point
4. Server correlates data from all the sensors and generates results, which can be made available to subscribers
5. Easy configuration / setup through discovery
Any ideas about where to look, or a good starting point would be much appreciated.
Thanks!
XMPP for M2M sounds like a nice idea.
About clients and servers, see http://xmpp.org/about-xmpp/technology-overview/pubsub/
In pubsub server does basically all the hard work, and you have to implement very little intelligence to clients. But this depends on what you want to do with published information. I haven't tested any clients which actually do something with the published information.
This fits the pubsub model of XMPP pretty well.
All your machines would be both publishers and subscribers.
Your processing server in this case would also be another subscriber that will do its data processing as it receives published items.
Any example you find dealing with pubsub is easily applicable. In XMPP, whether the JID (Jabber ID) represents a user of a machine is irrelevant, and pubsub is not actually oriented toward human interaction, unlike say, Multi User Chat.
There are many XMPP servers that support pubsub. I have used Smack and OpenFire for a similar purpose myself. The server is of less importance to you, since any off the shelf product that supports PubSub will do the job. More importantly is a client library that has pubsub support. I know Smack has this, but it is a Java library not python.
I do not know anything with all those requisites but you can use SleekXMPP to build your own. It is pure python and well documented XMPP library. XMPP has been used to do computer-to-computer communication which is quite nice because you can just test it from your own chat client. Look for example, http://www.python.org/about/success/projectpipe/
Good luck
Is there a messaging solution out there (preferably supporting Python) that I can use like a mailbox, e.g. retrieve messages from any given queue without having to subscribe? I suppose message queues could work, but I would have to repeatedly subscribe, grab messages from the queue, then unsubscribe, which does not sound optimal.
Most (if not all) Messaging solutions support two modes of messaging
Publish \ Subscribe -that is, you need to subscribe to get the message.
Queuing - one party sends a message to the queue, the other reads the message from the Queue - no subscription needed, and the message is consumed when it's read.
Actually, standard Queuing is more common then publish subscribe - you have better chances of finding a tool that supports queuing, but not pub\sub, then find a tool that supports pub\sub but not queuing.
You are probably looking for the 2nd mode
There are quite a few options. Here are two:
Take a look at Redis. There are two Python client libraries for it (see redis-py and txRedis). The operation you describe (mailbox-like operations on queues), can be simulated by performing a blpop on a list in Redis.
Another option is RabbitMQ. There are quite a few Python client libraries for it py-ampqlib and txAMQP. You can treat this as a mailbox-like queue by doing a basic.get and basic.ack (see this reference for more info).
RabbitMQ - http://www.rabbitmq.com
ZeroMQ - http://www.zeromq.org
Amazon SQS - http://aws.amazon.com/sqs
All three have libraries for python.
The first two are free. SQS costs are pretty low if you don't send and receive millions of messages and it has the advantages of high availability and freeing you from the need to host manage it yourself.
Regarding subscription and unsubscription, if you're pulling from the queue rather than having the queue push messages to you (pub/sub) you're not subscribing and unsubscribing. In all the examples above you do not incur any overhead.