When I built about 5000 MQTT clients by using threading, I got the error message which is [WinError 10061] as following code.
import paho.mqtt.client as mqtt
import threading
def client(id):
subscriber=mqtt.Client(id)
subscriber.on_connect=on_connect
subscriber.on_message=on_message
subscriber.connect('192.168.233.142',keepalive=20)
subscriber.loop_forever()
def on_connect(subscriber,userdata,flags,rc):
if int(subscriber._client_id.decode())%1000==0:
logging.info('連接伺服器')
def on_message(subscriber,userdata,msg):
message=str(loads(msg.payload.decode()))
if int(subscriber._client_id.decode())%1000==0:
logging.info(message)
for i in range(5001):
threading.Thread(target=client,args=(str(i),)).start()
sleep(0.01)
I have seen this error before when I using ZMQ.SUB in another program, but the numbers of clients were over 10000.
Does too much connections make this problem?
If true, what makes these two package have the huge difference?
Google says 'Win10061' is a "Connection Refused" error....so go look at the log file of your MQTT Broker and see what it says....that is where the error is occuring.
Most MQTT Brokers should be able to handle 5000 connections, but maybe you are starting them too close together and the MQTT Broker gets too busy trying to update its table of clients and push out CONACKs??
Where in your code does the error come back from??
Related
I have been struggeling with a strange case of random client MQTT publish failing for certain payloads. It happends randomly when trying to publish some large amount of BASE64 data.
I've finally managed to narrow it down to payloads containing a lot of consequtive forwards slashes (/). I've searched the net to find a good reason why this happends, but havent found anythong. Is it a MQTT feature, a Paho client feature or a broker feature, or just some bug...
Setup:
Python 3.8.8 (Windows 10)
paho-mqtt 1.5.0
mosquitto 1.6.9-1 amd64
On my setup, it fails when I send a payload of 255 '/' to a 1 character topic 'a'. Larger topic length, reduces the possible number of forward slashes.
Code to reproduce error:
import paho.mqtt.client as mqtt_client
import time
address = 'some.server.com'
port = 1883
connected = False
def on_connect(client, userdata, flags, rc):
global connected
connected = True
print("Connected!")
client = mqtt_client.Client()
client.on_connect = on_connect
client.connect(host=address, port=port, keepalive=60)
client.loop_start()
while not connected:
time.sleep(1)
payload = '/'*205
print('Payload: {}'.format(payload))
client.publish(topic='a', payload=payload)
time.sleep(2)
client.loop_stop()
client.disconnect()
print('Done!')
This generates this output:
Connected!
Payload: /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Connected!
Done!
This produces the following error in /var/log/mosquitto/mosquitto.log for the mosquitto broker:
1616605010: New connection from a.b.c.d on port 1883.
1616605010: New client connected from a.b.c.d as auto-CEF15129-E74C-F00A-A6FA-5B5FDA0CEF1D (p2, c1, k60).
1616605011: Socket error on client auto-CEF15129-E74C-F00A-A6FA-5B5FDA0CEF1D, disconnecting.
1616605012: New connection from a.b.c.d on port 1883.
1616605012: New client connected from a.b.c.d as auto-0149B6DB-5997-9E08-366A-304F21FDF2E1 (p2, c1, k60).
1616605013: Client auto-0149B6DB-5997-9E08-366A-304F21FDF2E1 disconnected.
I observe that the client() connects twice, but do not know why, but this is probably caused by a disconnect...
Any Ideas?
Update 1: I've tested this on Linux Ubunit running Python 3.7.3, and same paho-mqtt version, and this does not produce the same error... Seems like some problem in Windows then.
Update 2:
I also tried running mosquitto_pub and experienced the same error, so this has to be Windows-related (or system related) in some way. Possibly firewall? I will close question if I find manage to solve this.
"C:\Program Files\mosquitto\mosquitto_pub.exe" -h some.server.com -t a -m '/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////'
Update 3 The issue is related to OpenVPN. I closed my open VPN connection and MQTT messages were passing! I'm running OpenVPN Client (Windows version 3.2.2 (1455). I have no idea what causes this conlflict...
I'm a beginner in python socket programming and I have to send a message from the server to the client side . I have 2 python IDLES one for the server and one for the client. I have made the server file with no errors but when I create a connection socket in my client file and try to connect to server I get the error:
clientSocket.connect((servername,port))
ConnectionRefusedError: [WinError 10061] no connection could be made because the target machine actively refused it
I don't know how to deal with this error and I would appreciate your help with guiding me.
Thank you in advance.
My code:
Server:
from socket import *
port = 1234
serverSocket = socket(AF_INET,SOCK_STREAM)
serverSocket.bind(('',port))
serverSocket.listen()
print("Server has started")
data = "Network labs"
while True:
connectionSocket , addr = serverSocket.accept()
connectionSocket.send(data)
connectionSocket.close()
Client:
from socket import *
port = 1234
servername = 'localhost'
clientSocket = socket(AF_INET,SOCK_STREAM)
clientSocket.connect((servername,port)) #this is where the error happens
I would gradually try to understand where the problem is coming from.
Try to identify where the problem is:
Write an example which is available online and will surely works, and then you will know that the problem is in your code.
Try to use the same code on different computer, or a VM. If it will work there you will know that the problem is with the environment.
Try to find people with similar problems, you will usually won't be the first. - Errno 10061 : No connection could be made because the target machine actively refused it ( client - server ) - this seems nice.
Find out if your server is running correctly, checks if someone is listening on port 1234 before running the client. (Use netstat)
Few things which unrelated to the subject but will improve your coding:
1. Don't import *, it's just an easy way to get name collision.
2. Use conventions, it's just make everything nicer to read. https://www.python.org/dev/peps/pep-0008/
Of course you can ignore all of this, it's just an advice.
So I have a publisher which is using schedule python package to read data from a file and every 5-10 mins and publish each line to a queue.
On the other side I have consumer using something like:
self.connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
self.channel = self.connection.channel()
while True:
method, properties, body = self.channel.basic_get(queue=conf.UNIVERSAL_MESSAGE_QUEUE, no_ack=False)
if body is not None:
self.assign_task(body=body)
self.channel.basic_ack(delivery_tag=method.delivery_tag)
else:
self.logger.info('channel empty')
self.move_to_done()
time.sleep(5)
Assign task function looks like:
def assign_task(body=body):
<do something with the message body>
For some reason after a while it throws the following error:
2017-08-03 15:27:43,756: ERROR: base_connection.py: _handle_error: 335: Socket Error: 10054
2017-08-03 15:27:43,756: WARNING: base_connection.py: _check_state_on_disconnect: 180: Socket closed when connection was open
2017-08-03 15:27:43,756: WARNING: connection.py: _on_disconnect: 1360: Disconnected from RabbitMQ at localhost:5672 (0): Not specified
Essentially both publisher and consumer are 2 different python programs intended to run on a single machine with Windows Server 2012. Can community help understand what might be going wrong here.
The same code runs absolutely fine locally on my windows machine
Following is the output from my log file.
=ERROR REPORT==== 3-Aug-2017::15:06:48 ===
closing AMQP connection <0.617.0> ([::1]:53485 -> [::1]:5672):
missed heartbeats from client, timeout: 60s
Simple answer to this was to create a durable queue and set heartbeat_interval to 0.
I used the following code and I get this error :
Time out error : [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host failed to respond.
I also turned off the firewall in the systems i am using , still i get that error.
#!/usr/bin/env python3
import paho.mqtt.client as mqtt
# This is the Publisher
client = mqtt.Client()
client.connect("10.12.114.103",1883,60)
client.publish("topic/test", "Hello world!");
client.disconnect();
The problem probably isn't with your python code as I tried a slightly modified version with hivemq's public test server and it worked fine. Here is the version I tried:
#!/usr/bin/env python3
import paho.mqtt.client as mqtt
# This is the Publisher
def on_log(client, userdata, level, buf):
print(level, buf)
client = mqtt.Client()
client.on_log = on_log
client.connect("broker.hivemq.com",1883,60)
client.publish("topic/test", "Hello world!");
client.disconnect();
Note that I modified it to return the log output which would be useful in debugging this. The script returned:
16 Sending PUBLISH (dFalse, q0, r0, m1, 'topic/test', ... (12 bytes)
Which is what I would expect from a successful connection. I also checked your code with the mosquitto test server and it worked fine.
It looks like your broker is not accepting your connection attempt. If you tried a public test server and it still didn't work then it suggests that something is interfering with your traffic on port 1883.
I think this is a broker issue, however, I did notice that you are not using any of the network loop functions (like client.loop_start) which, according to the paho documentation, may cause unpredictable behavior. You could try adding a loop like this to see if that helps
client = mqtt.Client()
client.on_log = on_log
client.connect("broker.hivemq.com",1883,60)
client.loop_start
client.publish("topic/test", "Hello world!");
client.disconnect();
client.loop_stop
I'm trying to get my program, which uses Pika, to continually retry connecting to RabbitMQ on failure. From what I've seen of the Pika docs, there's a SimpleReconnectionStrategy class that can be used to accompish this but it doesn't seem to be working very well.
strategy = pika.SimpleReconnectionStrategy()
parameters = pika.ConnectionParameters(server)
self.connection = pika.AsyncoreConnection(parameters, True, strategy)
self.channel = self.connection.channel()
The connection should wait_for_open and setup the reconnection strategy.
However, when I run this, I get the following errors thrown:
error: uncaptured python exception, closing channel <pika.asyncore_adapter.RabbitDispatcher at 0xb6ba040c> (<class 'socket.error'>:[Errno 111] Connection refused [/usr/lib/python2.7/asyncore.py|read|79] [/usr/lib/python2.7/asyncore.py|handle_read_event|435] [/usr/lib/python2.7/asyncore.py|handle_connect_event|443])
error: uncaptured python exception, closing channel <pika.asyncore_adapter.RabbitDispatcher at 0xb6ba060c> (<class 'socket.error'>:[Errno 111] Connection refused [/usr/lib/python2.7/asyncore.py|read|79] [/usr/lib/python2.7/asyncore.py|handle_read_event|435] [/usr/lib/python2.7/asyncore.py|handle_connect_event|443])
These errors are continually thrown whilst Pika tries to connect. If I start the RabbitMQ server while my client is running, it will connect. I just don't like the sight of these errors... Are they normal? Am I doing this wrong?
import socket
...
while True:
connectSucceeded = False
try:
self.channel = self.connection.channel()
connectSucceeded = True
except socket.error:
pass
if connectSucceeded:
break
Something like the above is usually used. You could also add time.sleep() every time through the loop to try less frequently because sometimes servers do go down. In real production code I would also count the number of retries (or track the amount of time spent retrying) and give up after some interval. Sometimes it is better to log an error and crash.