I'm learning thenextquant quant framework, but always get connection error while connecting to RabbitMQ. after some troubleshooting, i found asynico/aioamqp module always prompts error:An open stream was garbage collected prior to establishing network connection; call "stream.close()" explicitly.
while trying to connect to RabbitMQ server, RabbitMQ is up and running and other client can connect to it without any error.
Then I followed the hello world example from aioamqp(https://github.com/Polyconseil/aioamqp/blob/master/examples/send.py) to write a test program, then reproduced the error.
import asyncio
import aioamqp
async def send():
transport, protocol = await aioamqp.connect(host="127.0.0.1", port=5672, login="guest", password="guest", login_method="PLAIN")
channel = await protocol.channel()
await channel.queue_declare(queue_name='hello')
await channel.basic_publish(
payload='Hello World!',
exchange_name='',
routing_key='hello'
)
print(" [x] Sent 'Hello World!'")
await protocol.close()
transport.close()
asyncio.get_event_loop().run_until_complete(send())
(venv_380) appledeMBP:Market apple$ python hello.py
An open stream was garbage collected prior to establishing network connection; call "stream.close()" explicitly.
Traceback (most recent call last):
File "hello.py", line 31, in <module>
asyncio.get_event_loop().run_until_complete(send())
File "/Users/apple/.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
return future.result()
File "hello.py", line 14, in send
transport, protocol = await aioamqp.connect(host="127.0.0.1", port=5672, login="guest", password="guest", login_method="PLAIN")
File "/Users/apple/.virtualenvs/venv_380/lib/python3.8/site-packages/aioamqp/__init__.py", line 59, in connect
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
File "/Users/apple/.pyenv/versions/3.8.0/lib/python3.8/asyncio/trsock.py", line 82, in setsockopt
self._sock.setsockopt(*args, **kwargs)
OSError: [Errno 9] Bad file descriptor
finally I found it's caused by version of Python. the issue is gone once using 3.6 instead of 3.8 which result in connection issue with aioamqp.connect()
Related
So I'm new to PostgreSQL and I made my server and database and the table and stuff then I do back to repl.it discord.py and I tried running this:
async def create_db_pool():
client.pg_con = await asyncpg.create_pool(database= "mainbank", user = "postgres", password = password)
client.loop.run_until_complete(create_db_pool())
Error:
Traceback (most recent call last):
File "main.py", line 204, in <module>
client.loop.run_until_complete(create_db_pool())
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "main.py", line 39, in create_db_pool
client.pg_con = await asyncpg.create_pool(database= "mainbank", user = "postgres", password = password)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/pool.py", line 407, in _async__init__
await self._initialize()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/pool.py", line 435, in _initialize
await first_ch.connect()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/pool.py", line 127, in connect
self._con = await self._pool._get_new_connection()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/pool.py", line 477, in _get_new_connection
con = await connection.connect(
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/connection.py", line 1980, in connect
return await connect_utils._connect(
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 677, in _connect
raise last_error
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 661, in _connect
con = await _connect_addr(
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 634, in _connect_addr
tr, pr = await compat.wait_for(connector, timeout=timeout)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/compat.py", line 103, in wait_for
return await asyncio.wait_for(fut, timeout)
File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
return fut.result()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 544, in _create_ssl_connection
tr, pr = await loop.create_connection(
File "/usr/lib/python3.8/asyncio/base_events.py", line 1033, in create_connection
raise OSError('Multiple exceptions: {}'.format(
OSError: Multiple exceptions: [Errno 111] Connect call failed ('127.0.0.1', 5432), [Errno 99] Cannot assign requested address
What is wrong? It is my side or is it PostgreSQL? I have the Host as localhost and the port as 5432 (all default)
It looks like you have too many TCP connections on that machine. Whenever you connect(2) to establish a TCP connections (and you didn't bind(2) the socket to a port), the network socket is assigned an ephemeral port by the operating system. If there is no free ephemeral port on the machine, you get the error EADDRNOTAVAIL:
EADDRNOTAVAIL
(Internet domain sockets) The socket referred to by sockfd had not previously been bound to an address and, upon attempting to
bind it to an ephemeral port, it was determined that all port numbers in the ephemeral port range are currently in use. See the
discussion of /proc/sys/net/ipv4/ip_local_port_range in ip(7).
Either open fewer TCP connections or increase the range for ephemeral ports.
I had the same exact error running a Python FastAPI app in a Docker container trying to connect to another container with http3.AsyncClient (looks in both cases asyncio library is used) to GET a response from another app. The answer above by #Laurenz Albe directed me to
docker system prune -a
The app ran after that. I also can reproduce this error if trying to connect to 127.0.0.1 but container not on the host network.
I am trying out pyro4 connection between my PC and Raspberry Pi 4.
Code on my PC is:
# saved as server.py
import Pyro4, Pyro4.naming
import socket, threading
# Define an object that will be accessible over the network.
# This is where all your code should go...
#Pyro4.expose
class MessageServer(object):
def show_message(self, msg):
print("Message received: {}".format(msg))
# Start a Pyro nameserver and daemon (server process) that are accessible
# over the network. This has security risks; see
# https://pyro4.readthedocs.io/en/stable/security.html
hostname = socket.gethostname()
ns_thread = threading.Thread(
target=Pyro4.naming.startNSloop, kwargs={'host': hostname}
)
ns_thread.daemon = True # automatically exit when main program finishes
ns_thread.start()
main_daemon = Pyro4.Daemon(host=hostname)
# find the name server
ns = Pyro4.locateNS()
# register the message server as a Pyro object
main_daemon_uri = main_daemon.register(MessageServer)
# register a name for the object in the name server
ns.register("example.message", main_daemon_uri)
# start the event loop of the main_daemon to wait for calls
print("Message server ready.")
main_daemon.requestLoop()
And code on my Raspberry is:
import Pyro4
import sys
print("Message:")
msg=sys.stdin.readline().strip()
message_server = Pyro4.Proxy("PYRONAME:192.168.1.5")
message_server.show_message(msg)
Code on my PC doesn t show any errors, but when I try to send a message from raspberry i get this:
What s your message?
test
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 102, in getIpAddress
return getaddr(config.PREFER_IP_VERSION) if ipVersion is None else getaddr(ipVersion)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 94, in getaddr
ip = socket.getaddrinfo(hostname or socket.gethostname(), 80, family, socket.SOCK_STREAM, socket.SOL_TCP)[0][4][0]
File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 515, in connect_and_handshake
sslContext=sslContext)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 266, in createSocket
if getIpVersion(connect[0]) == 4:
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 68, in getIpVersion
address = getIpAddress(hostnameOrAddress)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 106, in getIpAddress
return getaddr(0)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 94, in getaddr
ip = socket.getaddrinfo(hostname or socket.gethostname(), 80, family, socket.SOCK_STREAM, socket.SOL_TCP)[0][4][0]
File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/pi/Desktop/client.py", line 10, in <module>
message_server.show_message(msg)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 275, in __getattr__
self._pyroGetMetadata()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 1915, in _resolve
return nameserver.lookup(uri.object)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 275, in __getattr__
self._pyroGetMetadata()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 596, in __pyroCreateConnection
connect_and_handshake(conn)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 549, in connect_and_handshake
raise ce
Pyro4.errors.CommunicationError: cannot connect to ('JAKOB-PC', 9090): [Errno -2] Name or service not known
My PC has its firewall disabled, so there shouldn t be any problem with that. My local ip is 192.168.1.5.
I am using a headless Raspberry and write code on it with puTTY and VNC.
I have googled this error but couldn t find any answers. Any help would be appreciated.
I did this
#
# Server.py
#
from __future__ import print_function
import Pyro4
#Pyro4.expose
#Pyro4.behavior(instance_mode="single")
class Messenger(object):
# This is a constructor
def __init__(self):
pass
# This method will be called on the server
def send_message(self, name, message):
print("[{0}] {1}".format(name, message))
def main():
Pyro4.Daemon.serveSimple(
{
Messenger: "example.messenger"
},
ns=True)
if __name__ == "__main__":
main()
#
# Client.py
#
# This is the code that visits the warehouse.
import sys
import Pyro4
import Pyro4.util
sys.excepthook = Pyro4.util.excepthook
messenger = Pyro4.Proxy("PYRONAME:example.messenger#192.168.1.5")
messenger.send_message("Tim", "Hello!")
Then ran
python -m Pyro4.naming -n 192.168.1.5
python Server.py
python Client.py
In short I couldn't solve the problem with Pyro and (almost) no one helped so I decided to use 'websockets' instead.
You can read the documentation here but I'll explain it here anyway.
First of all you need two devices with network connection. You also have to run python 3.6.1 on both of them. After that you also need to install websockets if you don't have them already with pip install websockets or as I had to do it with pip3 install websockets.
Code below runs on the server and handles messages you send to it from client. Function 'hello' is a simple example of processing request and sending back a response. 'request' is the data server receives, that data must be bytes, string on iterable. Response is made by converting request to integer, squaring it and converting it back to string. This response is then sent back to client.
'start_server' defines the server, function that will define its behavior(hello), ip address on witch the server is running on(192.168.1.117) and port on witch it will receive requests(8765).
!/usr/bin/env python
import asyncio
import websockets
print("Running...")
async def hello(websocket, path):
request = await websocket.recv()
print("Request: " + request)
response = str(int(request)*int(request))
await websocket.send(response)
print("Response:" + response)
start_server = websockets.serve(hello, "192.168.1.117", 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Next bit is code on the client. 'uri' is ip and port of the server. Function 'tellServ' asks you to input some data('tell' variable) and sends it to the server. After that it waits for reply and once it gets it it prints it out. In this case if I would input number "6" server would reply with "36". Function loop is in a while loop so I can send multiple numbers without having to restart the script.
#!/usr/bin/env python
import asyncio
import websockets
uri = "ws://192.168.1.117:8765"
async def tellServ():
async with websockets.connect(uri) as websocket:
tell = input("Podatek ki ga posles: ")
await websocket.send(tell)
reply = await websocket.recv()
print("Odgovor:")
print(reply)
while 1:
asyncio.get_event_loop().run_until_complete(tellServ())
I'm trying to create a simple WebSocket server using websockets module in Python.
My task is to create a WebSocket server that sends a message to all the clients that are listening to it. The problem is that everytime a client disconnects, websockets launches an exception (ConnectionClosedOK or ConnectionClosedError, depending of what happened) that I need to catch because I have to unregister the clients that have disconnected so that the WebSocket stops to send messages in the ether (and stopping the WebSocket server throwing these exceptions everytime that he tries to send a message to a closed client).
Here's the server's code:
async def foo_call():
await asyncio.sleep(3)
print('Hi!')
async def register(websocket):
users.add(websocket)
await notify_users()
async def unregister(websocket):
users.remove(websocket)
async def start(websocket, path):
await register(websocket)
try:
await foo_call()
except websockets.ConnectionClosedError:
await unregister(websocket)
# WebSocket clients that are listening this WebSocket server
users = set()
# Open WebSocket server
start_server = websockets.serve(start, "localhost", 8765)
# Run asyncio event loop
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
And here's the traceback:
Task exception was never retrieved
future: <Task finished coro=<WebSocketCommonProtocol.send() done, defined at /home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py:521> exception=ConnectionClosedError('code = 1006 (connection closed abnormally [internal]), no reason',)>
Traceback (most recent call last):
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py", line 827, in transfer_data
message = await self.read_message()
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py", line 895, in read_message
frame = await self.read_data_frame(max_size=self.max_size)
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py", line 971, in read_data_frame
frame = await self.read_frame(max_size)
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py", line 1051, in read_frame
extensions=self.extensions,
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/framing.py", line 105, in read
data = await reader(2)
File "/usr/lib/python3.6/asyncio/streams.py", line 672, in readexactly
raise IncompleteReadError(incomplete, n)
asyncio.streams.IncompleteReadError: 0 bytes read on a total of 2 expected bytes
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py", line 555, in send
await self.ensure_open()
File "/home/vlado/Projects/dahua/venv/lib/python3.6/site-packages/websockets/protocol.py", line 803, in ensure_open
raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: code = 1006 (connection closed abnormally [internal]), no reason
I don't really understand how can I catch those exceptions thrown by websockets module.
I am trying out MQTT for the first time using Python and the mosquitto library. My client program is below. I'm trying to use the public demo MQTT server at http://www.mqtt-dashboard.com/subscribe. However the client code is failing, see error below. Any ideas on what's going on?
#!/usr/bin/env python
import mosquitto
client = mosquitto.Mosquitto("fredtest", clean_session=True)
client.connect("broker.mqttdashboard.com", 1883)
client.publish("fred.test", "hello world", 1)
client.loop_forever()
Error message:
C:\tmp>python mqttclient.py
Traceback (most recent call last):
File "mqttclient.py", line 6, in
client.connect("broker.mqttdashboard.com", 1883)
File "build\bdist.win-amd64\egg\mosquitto.py", line 582, in connect
File "build\bdist.win-amd64\egg\mosquitto.py", line 657, in reconnect
File "c:\python27\lib\socket.py", line 571, in create_connection
raise err
socket.error: [Errno 10060] A connection attempt failed because the connected pa
rty did not properly respond after a period of time, or established connection f
ailed because connected host has failed to respond
I'm not currently able to connect any client to broker.mqttdashboard.com:1883 - so this probably isn't an issue with your code.
To sanity check, have you tried connected to another broker, such as iot.eclipse.org:1883 ?
i have a rabbit mq running on machine
both client and rabbitMQ are running on the same network
rabbitMQ has many clients
i can ping client from rabbitMQ and back
longest latency measured between the machine is 12.1 ms
network details : Standard Switch network (network of virtual machines running on single physical machine - using vmware VC)
im getting random timeouts when initializing RPC connection
/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/blocking_connection.py
problem is that the timeout isn't consistent and happens from time to time.
when manually testing this issue and running the blocking_connection.py 1000 times from the same machine that it fails no timeout accrue.
this is the error i get when failing :
2013-04-23 08:24:23,396 runtest-trigger.24397 24397 DEBUG producer_rabbit initiate_rpc_connection Connecting to RabbitMQ RPC queue rpcqueue_java on host: auto-db1
2013-04-23 08:24:25,350 runtest-trigger.24397 24397 ERROR testrunner go Run 1354: cought exception: timed out
Traceback (most recent call last):
File "/testrunner.py", line 193, in go
self.set_runparams(jobid)
File "/testrunner.py", line 483, in set_runparams
self.runparams.producers_testrun = self.initialize_producers_testrun(self.runparams)
File "/basehandler.py", line 114, in initialize_producers_testrun
producer.set_testcase_checkout()
File "/baseproducer.py", line 73, in set_testcase_checkout
self.checkout_handler = pm_checkout.get_producer(self.testcasecheckout)
File "/producer_manager.py", line 101, in get_producer
producer = self.load_producer(plugin_dir, producer_name)
File "/producer_manager.py", line 20, in load_producer
producer = getattr(producer_module, 'Producer')(producer_name, self.runparams)
File "/producer_rabbit.py", line 13, in __init__
self.initiate_rpc_connection()
File "/producer_rabbit.py", line 67, in initiate_rpc_connection
self.connection = pika.BlockingConnection(pika.ConnectionParameters( host=self.conf.rpc_proxy))
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/blocking_connection.py", line 32, in __init__
BaseConnection.__init__(self, parameters, None, reconnection_strategy)
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 50, in __init__
reconnection_strategy)
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/connection.py", line 170, in __init__
self._connect()
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/connection.py", line 228, in _connect
self.parameters.port or spec.PORT)
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/blocking_connection.py", line 44, in _adapter_connect
self._handle_read()
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 151, in _handle_read
data = self.socket.recv(self._suggested_buffer_size)
timeout: timed out
please assist
I had a similar issue. If everything looks fine, then you most likely have some sort of miss configuration, e.g. bad binding. If miss configured, then you'll get a timeout because the script can't reach where it thinks it needs to go, so the error can be miss leading in this case.
For my problem, I specifically had issues with both my rabbitmq.config file and my bindings and had to use my python solution shown in: RabbitMQ creating queues and bindings from a command line over the command line example I showed. Once updated and configured properly, everything worked fine. Hopefully this gets you in the right direction.
Pika provides some time out issue when connecting different hosts.Solution is to pass a socket_timeout argument in connection parameter.Pika should upgrade to >=0.9.14
credentials = pika.PlainCredentials(RABBITMQ_USER, RABBITMQ_PASS)
connection = pika.BlockingConnection(pika.ConnectionParameters(
credentials=credentials,
host=RABBITMQ_HOST,
socket_timeout=300))
channel = connection.channel()