I've been trying to write code that collects crypto data from Binance. Binance auto disconnects after 24 hours. Is there any way for me to reconnect after disconnection? I believe running forever should take care of that for me, but it dies when an error is thrown. I will be running this program on a server 24/7. I will also need a way to be notified maybe telegram/discord bot that I can build where do I type the code to send when it is disconnected
This is the error I get.
Traceback (most recent call last):
File "exchanges/binance/binance_ticker.py", line 97, in <module>
start()
File "exchanges/binance/binance_ticker.py", line 94, in start
rel.dispatch()
File "/home/pyjobs/.local/lib/python3.8/site-packages/rel/rel.py", line 205, in dispatch
registrar.dispatch()
File "/home/pyjobs/.local/lib/python3.8/site-packages/rel/registrar.py", line 72, in dispatch
if not self.loop():
File "/home/pyjobs/.local/lib/python3.8/site-packages/rel/registrar.py", line 81, in loop
e = self.check_events()
File "/home/pyjobs/.local/lib/python3.8/site-packages/rel/registrar.py", line 232, in check_events
self.callback('read', fd)
File "/home/pyjobs/.local/lib/python3.8/site-packages/rel/registrar.py", line 125, in callback
self.events[etype][fd].callback()
File "/home/pyjobs/.local/lib/python3.8/site-packages/rel/listener.py", line 108, in callback
if not self.cb(*self.args) and not self.persist and self.active:
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_app.py", line 349, in read
op_code, frame = self.sock.recv_data_frame(True)
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_core.py", line 401, in recv_data_frame
frame = self.recv_frame()
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_core.py", line 440, in recv_frame
return self.frame_buffer.recv_frame()
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_abnf.py", line 352, in recv_frame
payload = self.recv_strict(length)
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_abnf.py", line 373, in recv_strict
bytes_ = self.recv(min(16384, shortage))
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_core.py", line 524, in _recv
return recv(self.sock, bufsize)
File "/home/pyjobs/.local/lib/python3.8/site-packages/websocket/_socket.py", line 122, in recv
raise WebSocketConnectionClosedException(
websocket._exceptions.WebSocketConnectionClosedException: Connection to remote host was lost.
My code:
import websocket
import rel
uri = "wss://stream.binance.com:9443/ws/!ticker#arr"
def on_message(ws, message):
print(message)
def on_error(ws, error):
print(error)
write_logs(error)
def on_close(ws, close_status_code, close_msg):
print("### closed ###")
write_logs(str(close_status_code) + str(close_msg))
start(
def on_open(ws):
print("Opened connection")
start()
websocket.enableTrace(True)
ws = websocket.WebSocketApp(uri,
on_open = on_open,
on_message=on_message,
on_error = on_error,
on_close (on_close)
ws.run_forever(dispatcher=rel) #Set the dispatcher to automatic reconnection.
rel.signal(2, rel.abort) # Keyboard Interrupt
rel.dispatch()
start()
The comment in this line of code ws.run_forever(dispatcher=rel) #Set the dispatcher to automatic reconnection. could auto reconnection depending on rel module? And how the module rel and func dispatcher work together?
Related
I have created a telegram bot. where I am asking the users to send the location.
import aioschedule as schedule
import asyncio
from asyncio import futures
#bot.on(events.CallbackQuery(pattern=r'sendloc'))
async def sendlocation(event):
sender = await event.get_input_sender()
my_user = await bot.get_entity(sender.user_id)
userId = my_user.id
async with bot.conversation(userId) as conv:
await conv.send_message('receiving', buttons=[Button.request_location(text='send Location', resize=10)])
try:
response = await conv.get_response()
except futures.TimeoutError as e:
await conv.cancel_all()
return
conv.send_message('location accepted')
async def methodToSendLocation():
users = pd.read_csv('telegramId.csv')
for user in users['Telegram ID']:
try:
await bot.send_message(user, 'send location', buttons=Button.inline('sendloc'))
except ValueError as e:
print('Error occurred for user: {telId}'.format(telId=user))
def main():
'''start the bot'''
bot.start()
schedule.every().day.at("11:00").do(methodToSendLocation)
loop = asyncio.get_event_loop()
while True:
loop.run_until_complete(schedule.run_pending())
bot.run_until_disconnected()
if __name__ == '__main__':
main()
what does my program do?
It will run every day at 11:00 AM.
It will fetch the telegram Id from the CSV file and send a message to the users to send the location and we process the location.
My Query:
Sometimes there occurs a timeout error in sendlocation method of my program and recently I observed that for some users timeout error occurred within a minute after the user clicks the send Location button.
I am not able to understand why the error occurred in a very short interval.
And sometimes there has been a delay to send the location to the bot after the user clicks the send Location button.
stack trace:
Traceback (most recent call last):
File "app.py", line 118, in sendlocation
response = await conv.get_response()
File "C:\Users\Swarnim\AppData\Local\Programs\Python\Python37-32\lib\asyncio\tasks.py", line 449, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
Stack (most recent call last):
File "app.py", line 450, in <module>
main()
File "app.py", line 444, in main
loop.run_until_complete(schedule.run_pending())
File "/usr/lib/python3.6/asyncio/base_events.py", line 471, in run_until_complete
self.run_forever()
File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
self._run_once()
File "/usr/lib/python3.6/asyncio/base_events.py", line 1451, in _run_once
handle._run()
File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
self._callback(*self._args)
File "/home/epluser/.venv/automation/lib/python3.6/site-packages/telethon/client/updates.py", line 443, in _dispatch_update
await callback(event)
File "app.py", line 121, in sendlocation
Minimal demonstration example:
import asyncio
async def main():
c1_reader, c1_writer = await asyncio.open_connection(host='google.com', port=80)
c1_socket = c1_writer.get_extra_info('socket')
c1_socket.close()
c2_reader, c2_writer = await asyncio.open_connection(host='google.com', port=80)
asyncio.run(main())
Running this program gives this error:
$ python3 asyncio_fd_used.py
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 469, in _sock_connect
sock.connect(address)
BlockingIOError: [Errno 36] Operation now in progress
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "asyncio_fd_used.py", line 11, in <module>
asyncio.run(main())
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
return future.result()
File "asyncio_fd_used.py", line 9, in main
c2_reader, c2_writer = await asyncio.open_connection(host='google.com', port=80)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/streams.py", line 77, in open_connection
lambda: protocol, host, port, **kwds)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 941, in create_connection
await self.sock_connect(sock, address)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 463, in sock_connect
self._sock_connect(fut, sock, address)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 477, in _sock_connect
self.add_writer(fd, self._sock_connect_cb, fut, sock, address)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 333, in add_writer
self._ensure_fd_no_transport(fd)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 244, in _ensure_fd_no_transport
f'File descriptor {fd!r} is used by transport '
RuntimeError: File descriptor 8 is used by transport <_SelectorSocketTransport fd=8 read=polling write=<idle, bufsize=0>>
Just for explanation why I am doing the low-level socket.close() and not asyncio-level writer.close(): I was trying some code to send RST packet. But I can imagine other reasons why people would call socket.close(), maybe even unintentionally.
The problem is that the low-level socket is closed, but asyncio doesn't know about that and thinks it is still open. For some reason (performance?) asyncio remembers the socket file descriptor (fileno).
When a new connection is opened, operating system gives to it the same file descriptor number, and asyncio starts panicking, because it has the same exact fd number associated with that previous connection.
Solution: tell asyncio the socket is closed :)
import asyncio
async def main():
c1_reader, c1_writer = await asyncio.open_connection(host='google.com', port=80)
c1_socket = c1_writer.get_extra_info('socket')
c1_socket.close()
c1_writer.close() # <<< here
c2_reader, c2_writer = await asyncio.open_connection(host='google.com', port=80)
asyncio.run(main())
This code runs without raising an error.
What I want to do is set up an XMPP receiver using the Slixmpp library which listens for messages throughout the script execution. I think the threading library might provide a solution so tried that.
This script is for an online PvP game and I need to receive information from the opponent continuously.
This is the code I have tried:
import time
import threading as td
import slixmpp as slix
import logging
import asyncio
my_jid = "test#xmpp.jp"
my_pass = "test"
class receiver(slix.ClientXMPP):
def __init__(self, jid, password):
slix.ClientXMPP.__init__(self, jid, password)
self.add_event_handler("session_start", self.start)
self.add_event_handler("message", self.message)
def start(self, event):
self.send_presence()
self.get_roster()
def message(self, msg):
if msg['type'] in ('chat', 'normal'):
msg.reply("Thanks for sending\n%(body)s" % msg).send()
print(msg['body'])
def function():
recv = receiver(my_jid, my_pass)
recv.connect()
recv.process()
newthread = threading.Thread(target=function)
logging.basicConfig(level="DEBUG", format='%(levelname)-8s %(message)s')
newthread.start()
input("Press Enter to continue")
This returns the following error:
Exception in thread Thread-1:
Press Enter to continueTraceback (most recent call last):
File "C:\Program Files (x86)\Python37-32\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\Program Files (x86)\Python37-32\lib\threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "C:/Users/dps/Documents/GitHub/Python-Problems/UltimateTicTacToe-scripts/xmppp.py", line 46, in good
recv = receiver(my_jid, my_pass)
File "C:/Users/dps/Documents/GitHub/Python-Problems/UltimateTicTacToe-scripts/xmppp.py", line 32, in __init__
slix.ClientXMPP.__init__(self, jid, password)
File "C:\Users\dps\venv\lib\site-packages\slixmpp\clientxmpp.py", line 70, in __init__
BaseXMPP.__init__(self, jid, 'jabber:client', **kwargs)
File "C:\Users\dps\venv\lib\site-packages\slixmpp\basexmpp.py", line 48, in __init__
XMLStream.__init__(self, **kwargs)
File "C:\Users\dps\venv\lib\site-packages\slixmpp\xmlstream\xmlstream.py", line 219, in __init__
self.disconnected = asyncio.Future()
File "C:\Program Files (x86)\Python37-32\lib\asyncio\events.py", line 644, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'Thread-1'.
In my case executing:
newthread.daemon = True
Before calling:
newthread.start()
Works as expected creating a non-blocking thread.
I was learning some async/await in python, and i wanted to try it, but
I'm getting this error while trying to connect to chatango via websocket and i don't know what means.
I'm using python 3.6.1 and aiohttp 2.2.3
This is my code:
import asyncio
import aiohttp
msgs = []
async def main():
async with aiohttp.ClientSession() as session:
async with session.ws_connect("ws://s12.chatango.com:8081/") as ws:
for msg in ws:
msgs.append(msg)
print(msg)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
Full traceback:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client_reqrep.py", line 559, in start
(message, payload) = yield from self._protocol.read()
File "C:\Program Files\Python36\lib\site-packages\aiohttp\streams.py", line 509, in read
yield from self._waiter
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client_proto.py", line 165, in data_received
messages, upgraded, tail = self._parser.feed_data(data)
File "aiohttp\_http_parser.pyx", line 274, in aiohttp._http_parser.HttpParser.feed_data (aiohttp/_http_parser.c:4364)
aiohttp.http_exceptions.BadHttpMessage: 400, message='invalid constant string'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/Users/joseh/Desktop/a.ws.py", line 42, in <module>
loop.run_until_complete(main())
File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete
return future.result()
File "C:/Users/joseh/Desktop/a.ws.py", line 34, in main
async with session.ws_connect("ws://s12.chatango.com:8081/") as ws:
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 603, in __aenter__
self._resp = yield from self._coro
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 390, in _ws_connect
proxy_auth=proxy_auth)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\helpers.py", line 91, in __iter__
ret = yield from self._coro
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client.py", line 241, in _request
yield from resp.start(conn, read_until_eof)
File "C:\Program Files\Python36\lib\site-packages\aiohttp\client_reqrep.py", line 564, in start
message=exc.message, headers=exc.headers) from exc
aiohttp.client_exceptions.ClientResponseError: 400, message='invalid constant string'
invalid constant string is a custom response from chatango, they probably want a protocol or some kind of auth header.
If you don't know much about how chatango uses websockets, reverse engineering their system is probably not a good task for learning asyncio and aiohttp.
Better to use something like httparrot which just echos back the message you send it.
Here's your code modified to use httparrot and send 5 messages, get 5 responses, then exit.
import asyncio
import aiohttp
msgs = []
async def main():
async with aiohttp.ClientSession() as session:
async with session.ws_connect('ws://httparrot.herokuapp.com/websocket') as ws:
ws.send_str('hello')
async for msg in ws:
msgs.append(msg)
print(msg)
ws.send_str('hello')
if len(msgs) >= 5:
break
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
print(msgs)
I am using "hello world" tutorial in :http://www.rabbitmq.com/tutorials/tutorial-two-python.html .
worker.py looks like this
import pika
import time
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='task_queue', durable=True)
print ' [*] Waiting for messages. To exit press CTRL+C'
def callback(ch, method, properties, body):
print " [x] Received %r" % (body,)
time.sleep( body.count('.') )
print " [x] Done"
ch.basic_ack(delivery_tag = method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(callback,
queue='task_queue')
channel.start_consuming()
I have used this code to implement in my work. Everything works smoothly untill there comes a point in a queue for which it raises an exception after printing [x] Done
Traceback (most recent call last):
File "hullworker2.py", line 242, in <module>
channel.basic_consume(callback,queue='test_queue2')
File "/usr/local/lib/python2.7/dist-packages/pika/channel.py", line 211, in basic_consume
{'consumer_tag': consumer_tag})])
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 904, in _rpc
self.connection.process_data_events()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 88, in process_data_events
if self._handle_read():
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 184, in _handle_read
super(BlockingConnection, self)._handle_read()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 300, in _handle_read
return self._handle_error(error)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 264, in _handle_error
self._handle_disconnect()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 181, in _handle_disconnect
self._on_connection_closed(None, True)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 232, in _on_connection_closed
self._channels[channel]._on_close(method_frame)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 817, in _on_close
self._send_method(spec.Channel.CloseOk(), None, False)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 920, in _send_method
self.connection.send_method(self.channel_number, method_frame, content)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 120, in send_method
self._send_method(channel_number, method_frame, content)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1331, in _send_method
self._send_frame(frame.Method(channel_number, method_frame))
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 245, in _send_frame
super(BlockingConnection, self)._send_frame(frame_value)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1312, in _send_frame
raise exceptions.ConnectionClosed
pika.exceptions.ConnectionClosed
I don't understand how the connection is closing automatically in between the process. Process runs fine for 100's of messages in the queue then suddenly this error comes up.
Any help appreciated.
There is a concept of heartbeats. It's basically a way how the server can make sure that the client is still connected.
when you do
time.sleep( body.count('.') )
You blocking the code by N number of seconds. It means that if server would like to send a heartbeat frame to check if your client is still alive, then it will not get a response back, because your code is blocked and doesn't know if heartbeat arrived.
Instead of using time.sleep() you should use connection.sleep() this will also make the code "sleep" for N number of seconds, but it will also communicate with the server and will respond back.
sleep(duration)[source]
A safer way to sleep than calling time.sleep() directly which will keep the adapter from ignoring frames sent from RabbitMQ. The connection will “sleep” or block the number of seconds specified in duration in small intervals.