We are trying to connect to AWS Amazon MQ using python but facing an issue.
Please find below code and error.
code:
import stomp
#Establish a connection
con = stomp.Connection([('stomp+ssl://xxxxxxxxxxxxxxxxxx.mq.xxxxxxxxxxxxxxxx.amazonaws.com',61616)])
#listener class to be instantiated.
class Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
con.set_listener('', Listener())
#wait will ensure it waits till connection is established and acknowledged.
# con.start()
con.connect('xxxxxxxx', 'xxxxxxxxx', wait=True)
#subscribe to a particular topic or queue by giving the path and headers if required by the server.
con.subscribe('#', headers={})
Error:
Could not connect to host stomp+ssl://xxxxxxxxxxxxxxxxxxxx.mq.xxxxxxx.amazonaws.com, port 61616
Could not connect to host stomp+ssl://xxxxxxxxxxxxxxxxxxxx.mq.xxxxxxx.amazonaws.com, port 61616
Could not connect to host stomp+ssl://xxxxxxxxxxxxxxxxxxxx.mq.xxxxxxx.amazonaws.com, port 61616
Traceback (most recent call last):
File "stomp_mqtt_subscribe.py", line 34, in <module>
con.connect('xxxxxxxx', 'xxxxxxxx', wait=True)
File "/home/ubuntu/.local/lib/python3.8/site-packages/stomp/connect.py", line 150, in connect
self.transport.start()
File "/home/ubuntu/.local/lib/python3.8/site-packages/stomp/transport.py", line 130, in start
self.attempt_connection()
File "/home/ubuntu/.local/lib/python3.8/site-packages/stomp/transport.py", line 834, in attempt_connection
raise exception.ConnectFailedException()
stomp.exception.ConnectFailedException
Any help appreciated.
This is what I use with stand-alone ActiveMQ Artemis, not AWS. I also filter message subscription on a selector field called "key". Remove that selector from the headers if you want. I'm using 30 second heartbeats here.
import stomp
def connect_and_subscribe(conn):
conn.connect('username', 'p#$$w0rd', wait=True)
conn.subscribe(destination='my-topic', id=12345, ack='auto', headers={'subscription-type': 'MULTICAST', 'selector': "key like '"+key+"'"})
class MyListener(stomp.ConnectionListener):
def __init__(self):
self.counter=0
def on_message(self, frame):
self.counter+=1
print('counter', self.counter)
print('got:', frame)
def on_disconnected(self):
print('disconnected')
connect_and_subscribe(self.conn)
key='test'
hosts = [('mqserver', 61616)]
conn = stomp.Connection(host_and_ports=hosts, heartbeats=(30000, 30000))
conn.set_listener('', MyListener())
connect_and_subscribe(conn)
while True:
pass
conn.disconnect()
There's a handful of problems (and potential problems) with your code.
First, you're using this as the hostname:
stomp+ssl://xxxxxxxxxxxxxxxxxx.mq.xxxxxxxxxxxxxxxx.amazonaws.com
The stomp+ssl:// should not be included in the hostname.
Second, you're using this as the port:
61616
By default the broker will not accept STOMP connection on port 61616. You should likely use 61613 instead.
Third, you're subscribing to # which looks like something you might do with an MQTT client (although even that is discouraged with MQTT). The destination # has no special meaning in STOMP as the STOMP specification defines no wildcards. Your STOMP client will subscribe to a destination literally named # which is likely not what you want.
Related
Hy, I'm created a AmazonMQ using broker as RabbitMQ. Now I want to publish a message and read that message from the queue using python. So I followed the steps given in the AWS docs.
Link: Using Python Pika with Amazon MQ for RabbitMQ
I follow the same steps in docs but when I tried to send a message to the queue it gives me this error!
Traceback (most recent call last):
File "E:/Axcer/Using-Python-Pika-with-Amazon-MQ-for-RabbitMQ/publisher.py", line 26, in
basic_message_sender = BasicMessageSender(
File "E:\Axcer\Using-Python-Pika-with-Amazon-MQ-for-RabbitMQ\basicClient.py", line 15, in init
parameters = pika.URLParameters(url)
File "E:\Axcer\Using-Python-Pika-with-Amazon-MQ-for-RabbitMQ\Intern\lib\site-packages\pika\connection.py", line 757, in init
if parts.port is not None:
File "C:\Users\Yomal\Python38-32\lib\urllib\parse.py", line 174, in port
raise ValueError(message) from None
ValueError: Port could not be cast to integer value as 'xxxxxyyyy'
I have really no Idea about this issue! I hope that someone can help me with this. Thank you!
basicClient.py
import ssl
import pika
class BasicPikaClient:
def __init__(self, rabbitmq_broker_id, rabbitmq_user, rabbitmq_password, region):
# SSL Context for TLS configuration of Amazon MQ for RabbitMQ
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
ssl_context.set_ciphers('ECDHE+AESGCM:!ECDSA')
url = f"amqps://{rabbitmq_user}:{rabbitmq_password}#{rabbitmq_broker_id}.mq.{region}.amazonaws.com:5671"
parameters = pika.URLParameters(url)
parameters.ssl_options = pika.SSLOptions(context=ssl_context)
self.connection = pika.BlockingConnection(parameters)
self.channel = self.connection.channel()
publisher.py
from basicClient import BasicPikaClient
class BasicMessageSender(BasicPikaClient):
def declare_queue(self, queue_name):
print(f"Trying to declare queue({queue_name})...")
self.channel.queue_declare(queue=queue_name)
def send_message(self, exchange, routing_key, body):
channel = self.connection.channel()
channel.basic_publish(exchange=exchange,
routing_key=routing_key,
body=body)
print(f"Sent message. Exchange: {exchange}, Routing Key: {routing_key}, Body: {body}")
def close(self):
self.channel.close()
self.connection.close()
if __name__ == "__main__":
# Initialize Basic Message Sender which creates a connection
# and channel for sending messages.
basic_message_sender = BasicMessageSender(
"*******************",
"xxxxx",
"xxxxxyyyy#zzzzzzz",
"********"
)
# Declare a queue
basic_message_sender.declare_queue("hello world queue")
# Send a message to the queue.
basic_message_sender.send_message(exchange="", routing_key="hello world queue", body=b'Hello World!')
# Close connections.
basic_message_sender.close()
I have configured MQTT Broker, receiving published messages from another piece of code (not written or accessible by me). I added another topic to the broker configuration and am now trying to publish data to this new topic from a piece of python code. I get the feedback that the message is published by the callback function, but no actual data is received.
Is there anything I am missing?
I am using the following code:
import paho.mqtt.client as mqtt
import time
#=========================================================================
def on_connect(client, userdata, flags, rc) :
print "on_connect()"
#=========================================================================
def on_publish(client, userdata, mid) :
print "on_publish()"
#=========================================================================
def send() :
mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
#host = "localhost"
host = "127.0.0.1"
port = 1883
keepalive = 60
print "\nConnect to '%s', port '%s', keepalive '%s'" % (host, port, keepalive)
mqttc.connect(host=host, port=port, keepalive=keepalive)
time.sleep(3)
mqttc.loop_start()
time.sleep(3)
topic = "data/MY/TOPIC"
msg = "MY_MESSAGE"
print "Publish to '%s' msg '%s'" % (topic, msg)
mqttc.publish(topic, msg, qos=2)
time.sleep(3)
mqttc.loop_stop()
# end send()
#=========================================================================
if __name__ == "__main__":
send()
# end if
Getting the stdout
Connect to '127.0.0.1', port '1883', keepalive '60'
on_connect()
Publish to 'data/MY/TOPIC' msg 'MY MESSAGE'
on_publish()
I am not sure if the loop() functions are necessary, but if I do not embed the publishing in the loop_start() and loop_stop(), I do not get a on_connect callback.
The loop functions are necessary as these are where all the network traffic is processed.
Manually setting up a connection to the broker to just send a single message like this is not a good idea, it would be better to start the client, leave it running (by calling loop_start() and not calling loop_stop()) in the background and then just call the publish method on the mqttc client object.
If you don't want to keep a instance of the client running then you should use the single message publish helper method provided by the paho python library (https://pypi.python.org/pypi/paho-mqtt/1.1#single):
import paho.mqtt.publish as publish
publish.single("paho/test/single", "payload", hostname="iot.eclipse.org")
I have a simple twisted TCP server that I am using to send and receive data from another application. I am able to communicate and send messages to this application once a connection has been established.
What I want to be able to do have an external module command when and what to send to the 3rd party application (instead of when connected). For example, here is my server code:
from twisted.internet import reactor, protocol
# Twisted
class Tcp(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
print 'data received: ', data
# do something with data
def connectionMade(self, msg):
# Want to only transport message when I command not immediately when connected
self.transport.write(msg)
class TcpFactory(protocol.Factory):
def buildProtocol(self, addr):
return Tcp()
reactor.listenTCP(8001, TcpFactory())
reactor.run()
I have another module (operated by a GUI interface) which defines what message to send and this message needs to be passed as 'msg' into connectionMade
What is the best way to do this?
Your 3rd party call will happen from within connectionMade and should not call it directly. The connectionMade is a Twisted callback and it's there to signal that a client is on the other side. It's up to you to dictate what happens after that. So you can do something like make a series of Deferreds/callbacks which take self.transport (this holds relevant client data). In those series of callbacks, you'll have your logic for
communicating with the 3rd party
validating client
sending a message to the client
This is the gist of it:
def connectionMade(self):
d = Deferred()
d.addCallback(self.thirdPartyComm).addErrBack(self.handleError)
d.addCallback(sendMsg)
d.callback(self.transport)
def thirdPartyComm(self, transport):
# logic for talking to 3rd party goes here
# 3rd party should validate the client then provide a msg
# or raise exception if client is invalid
return msg
def handleError(self, error):
# something went wrong, kill client connection
self.transport.loseConnection()
def sendMsg(self, msg):
self.transport.write(msg)
I am using Kombu in Python to consume a durable RabbitMQ queue.
There is only one consumer consuming the queue in Windows. This consumer produces the below error:
Traceback (most recent call last):
File ".\consumer_windows.py", line 66, in <module>
message.ack()
File "C:\Users\Administrator\Anaconda2\lib\site-packages\kombu\message.py", line 88, in ack
self.channel.basic_ack(self.delivery_tag)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\amqp\channel.py", line 1584, in basic_ack
self._send_method((60, 80), args)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\amqp\abstract_channel.py", line 56, in _send_method
self.channel_id, method_sig, args, content,
File "C:\Users\Administrator\Anaconda2\lib\site-packages\amqp\method_framing.py", line 221, in write_method
write_frame(1, channel, payload)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\amqp\transport.py", line 182, in write_frame
frame_type, channel, size, payload, 0xce,
File "C:\Users\Administrator\Anaconda2\lib\socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
error: [Errno 10054] An existing connection was forcibly closed by the remote host
There are at most 500 messages in the queue at any one time. Each message is small in size however it is a task and takes up to 10 minutes to complete (although it usually takes less then 5 mins per message).
I have tried restarting the consumer, RabbitMQ server and deleting the queue however the error still persists.
I've seen this question however the answer is from 2010 and my rabbitmq.log has different entries:
=ERROR REPORT==== 24-Apr-2016::08:26:20 ===
closing AMQP connection <0.6716.384> (192.168.X.X:59602 -> 192.168.Y.X:5672):
{writer,send_failed,{error,timeout}}
There were no recent events in the rabbitmq-sasl.log.
Why is this error happening and how can I prevent it from occurring?
I'm still looking for an answer. In the meantime I restart the connection to my rabbit server:
while True:
try:
connection = pika.BlockingConnection(params)
channel = connection.channel() # start a channel
channel.queue_declare(queue=amqp_q, durable=True) # Declare a queue
...
except pika.exceptions.ConnectionClosed:
print('connection closed... and restarted')
I had the same issue with MySQL server which was hosted...
I came to understand that it happened if we open the connection for a long time or unmodified for a long time..
If your program opens the DB or anything until the whole program runs make it in a such a way that it opens the DB writes everything and closes and repeats
I don't know what exactly rabbitmq is but I think the error you wrote as title may be for this reason
I had the same error (using pure PIKA library) and trying to connect to a Rabbitmq broker through Amazon MQ.
The problem resolved when setting up correctly the ssl configuration.
Please check full blog post here: https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/amazon-mq-rabbitmq-pika.html
Core snippets that I used:
Define Pika Client:
import ssl
import pika
class BasicPikaClient:
def __init__(self, rabbitmq_broker_id, rabbitmq_user, rabbitmq_password, region):
# SSL Context for TLS configuration of Amazon MQ for RabbitMQ
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
ssl_context.set_ciphers('ECDHE+AESGCM:!ECDSA')
url = f"amqps://{rabbitmq_user}:{rabbitmq_password}#{rabbitmq_broker_id}.mq.{region}.amazonaws.com:5671"
parameters = pika.URLParameters(url)
parameters.ssl_options = pika.SSLOptions(context=ssl_context)
self.connection = pika.BlockingConnection(parameters)
self.channel = self.connection.channel()
Producer:
from basicClient import BasicPikaClient
class BasicMessageSender(BasicPikaClient):
def declare_queue(self, queue_name, durable):
print(f"Trying to declare queue({queue_name})...")
self.channel.queue_declare(queue=queue_name, durable=durable)
def send_message(self, exchange, routing_key, body):
channel = self.connection.channel()
channel.basic_publish(exchange=exchange,
routing_key=routing_key,
body=body)
print(f"Sent message. Exchange: {exchange}, Routing Key: {routing_key}, Body: {body}")
def close(self):
self.channel.close()
self.connection.close()
Calling Producer:
# Initialize Basic Message Sender which creates a connection
# and channel for sending messages.
basic_message_sender = BasicMessageSender(
credentials["broker_id"],
credentials["username"],
credentials['password'],
credentials['region']
)
# Declare a queue
basic_message_sender.declare_queue("q_name", durable=True)
# Send a message to the queue.
basic_message_sender.send_message(exchange="", routing_key="q_name", body=b'Hello World 2!')
# Close connections.
basic_message_sender.close()
Define Consumer:
class BasicMessageReceiver(BasicPikaClient):
def get_message(self, queue):
method_frame, header_frame, body = self.channel.basic_get(queue)
if method_frame:
print(method_frame, header_frame, body)
self.channel.basic_ack(method_frame.delivery_tag)
return method_frame, header_frame, body
else:
print('No message returned')
def close(self):
self.channel.close()
self.connection.close()
Calling Consumer:
# Create Basic Message Receiver which creates a connection
# and channel for consuming messages.
basic_message_receiver = BasicMessageReceiver(
credentials["broker_id"],
credentials["username"],
credentials['password'],
credentials['region']
)
# Consume the message that was sent.
basic_message_receiver.get_message("q_name")
# Close connections.
basic_message_receiver.close()
I hope the above helps.
Thanks
I have uploaded my python socket, (cloud service project), on azure ,and when ever I connected to Hercules client side socket ....after a message or two, connection closed by remote host... forcefully...?
Server Code
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
import SocketServer
class IphoneChat(Protocol):
def connectionMade(self):
self.factory.clients.append(self)
print('clients are'), self.factory.clients
def connectionLost(self, reason):
self.factory.clients.remove(self)
def dataReceived(self, data):
msg = ""
msg = data.strip()
for c in self.factory.clients:
c.message(msg)
def message(self, message):
self.transport.write(message + '\n')
print ('Iphone Chat server startedd')
factory = Factory()
factory.protocol = IphoneChat
factory.clients = []
reactor.listenTCP(9073, factory)
reactor.run()
I tried to reproduce your issue on my environment, but failed. Base on my experience, we often pay attention to 3 points if we use socket in azure worker role.
Open input TCP port. If we open a port as listener port, we'd better to add this port into the endpoints setting.
Considered the worker role status, I suggest we can code logic into the while True loop function, as following,
while True:
reactor.listenTCP(65534, f)
print "server run"
reactor.run()
sleep(1.0)
According to the error message, I guess the reason is that azure load balancer will kill the idle connection in 240s. I recommend you can refer to this blog to configure your idleTimeoutInMinutes value.
Please try to check your project and configuration. Any further findings, please let me know.