We Keep Coding

NATS JetStream is Timing out when calling stream_info(STREAM_NAME)

raise errors.TimeoutError
nats.errors.TimeoutError: nats: timeout
if self.nc.is_connected:
log.info(f"NATS client successfully connected to {SERVERS}")
# Create jetstream context and check if stream exists
self.nc = self.nc.jetstream()
log.info("Created contextual JetStream object!")
try:
# Check if the JetStream exists [Client has already created JetStream and subject associations]
acc_info = await self.nc.account_info()
info = await self.nc.stream_info(STREAM_NAME)
The code is erroring out when using the JetStream context prior to adding a stream. Upon deletion of a stream, and calling stream_info() after, the NotFoundError is returned and handled. Why does the returned exception change so suddenly? Any help is greatly appreciated.
File "/usr/local/lib/python3.9/site-packages/nats/js/manager.py", line 65, in stream_info
resp = await self._api_request(
File "/usr/local/lib/python3.9/site-packages/nats/js/manager.py", line 158, in _api_request
msg = await self._nc.request(req_subject, req, timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/nats/aio/client.py", line 899, in request
msg = await self._request_new_style(
File "/usr/local/lib/python3.9/site-packages/nats/aio/client.py", line 939, in _request_new_style
raise errors.TimeoutError
nats.errors.TimeoutError: nats: timeout

Is your nats-server configured to enable JetStream functionality (i.e. either in it's config file or passing the -js command line argument)?
Is your account allowed to use JetStream? nats account info will tell you.

How to communicate between discord.py and Tweepy?

My code is is similar to this example:
import discord
import tweepy
import asyncio
class Client(discord.Client):
async def on_ready(self):
print("ready")
global GUILD
GUILD = discord.utils.get(self.guilds, name = "Any Guild")
class TweepyStream(tweepy.Stream):
def on_connect(self):
print("connceted")
def on_status(self, status):
print(status.text)
channel = discord.utils.get(GUILD.channels, name = "twitter-posts")
asyncio.run(channel.send(status.text))
#From here the Discord message should be send
auth = tweepy.OAuthHandler(keys)
auth.set_access_token(tokens)
global api
api = tweepy.API(auth)
follow_list = []
follow = int(api.get_user(screen_name = "Any User").id)
print(follow)
follow_list.append(follow)
print(str(follow_list))
stream = TweepyStream(tokens and keys)
stream.filter(follow = follow_list, threaded=True) #track = track,
client = Client()
client.run(token)
I try to receive Twitter posts and send them into a Discord channel, but it doesn't work. How can I do this (maybe without asyncio)?
Or is there a way to work with a Python Twitter API, which works with async functions?
My error:
Stream encountered an exception
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\tweepy\streaming.py", line 133, in _connect
self.on_data(line)
File "C:\Python39\lib\site-packages\tweepy\streaming.py", line 387, in on_data
return self.on_status(status)
File "c:\Users\morit\Desktop\FLL 2021 Bot\DiscordBot\DiscordBotV0.7\example.py", line 18, in on_status
asyncio.run(channel.send(status.text))
File "C:\Python39\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
return future.result()
File "C:\Python39\lib\site-packages\discord\abc.py", line 1065, in send
data = await state.http.send_message(channel.id, content, tts=tts, embed=embed,
File "C:\Python39\lib\site-packages\discord\http.py", line 192, in request
async with self.__session.request(method, url, **kwargs) as r:
File "C:\Python39\lib\site-packages\aiohttp\client.py", line 1117, in __aenter__
self._resp = await self._coro
File "C:\Python39\lib\site-packages\aiohttp\client.py", line 448, in _request
with timer:
File "C:\Python39\lib\site-packages\aiohttp\helpers.py", line 635, in __enter__
raise RuntimeError(
RuntimeError: Timeout context manager should be used inside a task

asyncio.run creates a new event loop. You need to use the existing event loop that the discord.Client uses. You can retrieve this with Client.loop, asyncio.get_running_loop, or asyncio.get_event_loop.
You should probably also use asyncio.run_coroutine_threadsafe.
If you're using the current latest development version of Tweepy on the master branch, set to be released as v4.0, which it seems like you are, then you can also look into using AsyncStream

SSL: APPLICATION_DATA_AFTER_CLOSE_NOTIFY Error

I am sending multiple requests with aiohttp using tor's http proxy (with aiohttp_socks)
After some requests are done I am getting the following error:
Traceback (most recent call last):
File "main.py", line 171, in <module>
loop.run_until_complete(future)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "main.py", line 95, in get_market_pages
async with session.get(active_link, headers=headers) as response:
File "/home/mrlalatg/.local/lib/python3.8/site-packages/aiohttp/client.py", line 1012, in __aenter__
self._resp = await self._coro
File "/home/mrlalatg/.local/lib/python3.8/site-packages/aiohttp/client.py", line 504, in _request
await resp.start(conn)
File "/home/mrlalatg/.local/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 847, in start
message, payload = await self._protocol.read() # type: ignore # noqa
File "/home/mrlalatg/.local/lib/python3.8/site-packages/aiohttp/streams.py", line 591, in read
await self._waiter
aiohttp.client_exceptions.ClientOSError: [Errno 1] [SSL: APPLICATION_DATA_AFTER_CLOSE_NOTIFY] application data after close notify (_ssl.c:2745)
I can't find any information about this error, except the git discussion about a similar one - github
There I found a workaround (link) that I can modify to ignore this error, but it didn't work, the error is still there.
The modified version of a workaround:
SSL_PROTOCOLS = (asyncio.sslproto.SSLProtocol,)
try:
import uvloop.loop
except ImportError:
pass
else:
SSL_PROTOCOLS = (*SSL_PROTOCOLS, uvloop.loop.SSLProtocol)
def ignore_aiohttp_ssl_eror(loop):
"""Ignore aiohttp #3535 / cpython #13548 issue with SSL data after close
There is an issue in Python 3.7 up to 3.7.3 that over-reports a
ssl.SSLError fatal error (ssl.SSLError: [SSL: KRB5_S_INIT] application data
after close notify (_ssl.c:2609)) after we are already done with the
connection. See GitHub issues aio-libs/aiohttp#3535 and
python/cpython#13548.
Given a loop, this sets up an exception handler that ignores this specific
exception, but passes everything else on to the previous exception handler
this one replaces.
Checks for fixed Python versions, disabling itself when running on 3.7.4+
or 3.8.
"""
orig_handler = loop.get_exception_handler()
def ignore_ssl_error(loop, context):
if context.get("message") in {
"SSL error in data received",
"Fatal error on transport",
}:
# validate we have the right exception, transport and protocol
exception = context.get('exception')
protocol = context.get('protocol')
if (
isinstance(exception, ssl.SSLError)
and exception.reason == 'APPLICATION_DATA_AFTER_CLOSE_NOTIFY'
and isinstance(protocol, SSL_PROTOCOLS)
):
if loop.get_debug():
asyncio.log.logger.debug('Ignoring asyncio SSL KRB5_S_INIT error')
return
if orig_handler is not None:
orig_handler(loop, context)
else:
loop.default_exception_handler(context)
loop.set_exception_handler(ignore_ssl_error)

Discord bot python - RuntimeError can't start new thread

I'm developing a Discord bot in Python 3 and can't get it running properly. I'm hosting the bot on a 1&1 server, booting it with nohup python3 main.py &. The bot is running and working perfectly for a while then crash with every times a huge Traceback of errors ending with RuntimeError: can't start new thread.. And furthermore the source line of the error from my code isn't the same each time. It always comes from an await [...] line.
I've been searching for a while on internet but other developers using import threading and related functions while I don't, i'm only using the async / await keywords. Is this a mistake ? I'm certain that i don't reach the thread limit, i've been testing my bot with only a few friends, so a maximum of 3 to 4 requests at a time. The bot is using asynchronous methods only for sending messages or reactions on Discord, there is no recursion anywhere.
Here is one of the multiple Traceback i've had. Hope somebody can help me, thanks !
`
Ignoring exception in on_message
Traceback (most recent call last):
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/discord/client.py", line 333, in _run_event
await coro(*args, **kwargs)
File "main.py", line 261, in on_message
await message.channel.send(lines + "```")
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/discord/abc.py", line 905, in send
nonce=nonce, allowed_mentions=allowed_mentions)
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/discord/http.py", line 185, in request async with self.__session.request(method, url, **kwargs) as r:
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/client.py", line 1012, in __aenter__
self._resp = await self._coro
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/client.py", line 483, in _request
timeout=real_timeout
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/connector.py", line 523, in connect
proto = await self._create_connection(req, traces, timeout)
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/connector.py", line 859, in _create_connection
req, traces, timeout)
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/connector.py", line 967, in _create_direct_connection
traces=traces), loop=self._loop)
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/connector.py", line 830, in _resolve_host
self._resolver.resolve(host, port, family=self._family)
File "/kunden/homepages/44/d758963141/htdocs/.local/lib/python3.7/site-packages/aiohttp/resolver.py", line 30, in resolve
host, port, type=socket.SOCK_STREAM, family=family)
File "/usr/lib/python3.7/asyncio/base_events.py", line 789, in getaddrinfo
None, getaddr_func, host, port, family, type, proto, flags)
File "/usr/lib/python3.7/asyncio/base_events.py", line 752, in run_in_executor
executor.submit(func, *args), loop=self)
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 160, in submit
self._adjust_thread_count()
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 181, in _adjust_thread_count
t.start()
File "/usr/lib/python3.7/threading.py", line 847, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
`
===== EDIT =====
Here is a sample of my code :
`
#client.event
async def on_message(message) :
global messagesId
# ===== Block self messages =====
if message.author == client.user :
return
# ===== $register =====
elif message.content == "$register" :
if not(await isRegistered(message.author.id, message.channel)) :
try :
refreshDb()
sql = "INSERT INTO player VALUES (%s, %s, 0)"
dbCurs.execute(sql, (message.author.id, message.author.name))
sql = "INSERT INTO schedule(player_id) VALUES (%s)"
dbCurs.execute(sql, (message.author.id))
db.commit()
except Exception as e :
await message.channel.send("Erreur interne #9")
raise e
await message.channel.send("Tu as été enregistré.")
else :
await message.channel.send("Tu es déjà enregistré.")
# ===== $schedule =====
elif message.content == "$schedule" :
# Get player's current schedule
data = None
try :
if await isRegistered(message.author.id, message.channel) :
try :
refreshDb()
sql = "SELECT * FROM schedule WHERE player_id = %s"
dbCurs.execute(sql, (message.author.id))
data = dbCurs.fetchall()[0]
except Exception as e :
await message.channel.send("Erreur interne #4")
raise e
else :
await message.channel.send("Tu dois t'enregistrer pour utiliser cette commande.")
return
except Exception as e :
await message.channel.send("Erreur interne #5")
raise e
# Create lines
text = "<#{}>\n{}".format(message.author.id, getScheduleText(data, 1))
sentMessage = await message.channel.send(text)
# register message's id
messagesId[sentMessage.id] = message.author.id
# Add reactions
i = 0
keys = list(cols.keys())
for key in keys :
await sentMessage.add_reaction(bytes.decode(numbers[i]))
i += 1
# Register every messages id and delete everyone after 60 sec
await deleteSched(sentMessage)
`

Your code apparently is using threads, via a thread-pool-executor; this is intended to run long-lasting tasks but leave the main asyncio thread available to handle more events. Creating new threads in the executor will always happen when your code hits some await condition, as this is the only time the event loop is free to work.
So..., it seems like the tasks you are running in the executor are not ending, and you are ending up with very many stale threads. Without seeing the code, I don't think we can say more.

Python asyncio: is the error me or the API?

I am using a third party API Wrapper that supports async. I am having a difficult time figuring out how to use it as I am new to asyncio, and I'm not sure if the errors I'm getting are my usage (most likely) or the wrapper itself.
The API can be found here: https://clashroyale.readthedocs.io/en/latest/api.html#royaleapi
The goal of my code is to make three API requests at once (the .get_clan(*lClanGroup)), asynchronously instead of synchronously.
My Code:
import clashroyale
import asyncio
import aiohttp
# Define Tokens
unofficialAPIToken = "secretToken"
# Get Client Objects
session1 = aiohttp.ClientSession()
unofficialClient = clashroyale.royaleapi.Client(unofficialAPIToken, session=session1, is_async=True)
lClanGroups = ['PPCLCJG9', '2LRU2J', 'PGLQ0VQ', 'YU2RQG9', '2LVRQ29'],['PYP8UPJV', 'P9L0CYY0', 'Y2RGQPJ', '8P2GYJ8', '9VQJPL2L'],['RYPRGCJ', '809R8PG8', 'PJY9PP98', '2GCQLC', '2GL2QPPL']
async def x(lClanGroup):
print(*lClanGroup)
a = await unofficialClient.get_clan(*lClanGroup) # Iterates five tags for the API to request at once
return a
async def y(lClanGroups):
result = await asyncio.gather(x(lClanGroups[0]),x(lClanGroups[1]),x(lClanGroups[2]))
return result
async def close_sessions():
await session1.close()
asyncio.run(y(lClanGroups))
asyncio.run(close_sessions())
The error I'm getting is long and difficult to make sense of:
C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\Scripts\python.exe C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py
PPCLCJG9 2LRU2J PGLQ0VQ YU2RQG9 2LVRQ29
PYP8UPJV P9L0CYY0 Y2RGQPJ 8P2GYJ8 9VQJPL2L
RYPRGCJ 809R8PG8 PJY9PP98 2GCQLC 2GL2QPPL
C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py:16: DeprecationWarning: The object should be created from async function
session1 = aiohttp.ClientSession()
Traceback (most recent call last):
File "C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py", line 33, in <module>
asyncio.run(y(lClanGroups))
File "C:\Program Files\Python38\lib\asyncio\runners.py", line 43, in run
return loop.run_until_complete(main)
File "C:\Program Files\Python38\lib\asyncio\base_events.py", line 612, in run_until_complete
return future.result()
File "C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py", line 27, in y
result = await asyncio.gather(x(lClanGroups[0]),x(lClanGroups[1]),x(lClanGroups[2]))
File "C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py", line 23, in x
a = await unofficialClient.get_clan(*lClanGroup)
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\clashroyale\royaleapi\client.py", line 203, in _aget_model
raise e
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\clashroyale\royaleapi\client.py", line 196, in _aget_model
data, cached, ts, resp = await self._request(url, **params)
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\clashroyale\royaleapi\client.py", line 150, in _arequest
async with self.session.get(url, timeout=timeout, headers=self.headers, params=params) as resp:
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\aiohttp\client.py", line 1012, in __aenter__
self._resp = await self._coro
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\aiohttp\client.py", line 426, in _request
with timer:
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\aiohttp\helpers.py", line 579, in __enter__
raise RuntimeError('Timeout context manager should be used '
RuntimeError: Timeout context manager should be used inside a task
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x0000016C5AD50EB0>
Process finished with exit code 1
Thanks for your help!

Each asyncio.run() creates new event loop. Moreover, if you create objects like aiohttp.ClientSession() globally, they may be bind to default event loop.
Using multiple event loops in your program is dangerous and may lead to non-obvious problems. To avoid the situation:
Run asyncio.run() only once
Create all asyncio-related stuff inside main coroutine
I didn't work with royaleapi, but it seems API developers do things differently than you in their tests. Here's how they innit/close stuff and here's how they get clan.
Let's give it a try?
import clashroyale
import asyncio
TOKEN = "secretToken"
clan_groups = [
['PPCLCJG9', '2LRU2J', 'PGLQ0VQ', 'YU2RQG9', '2LVRQ29'],
['PYP8UPJV', 'P9L0CYY0', 'Y2RGQPJ', '8P2GYJ8', '9VQJPL2L'],
['RYPRGCJ', '809R8PG8', 'PJY9PP98', '2GCQLC', '2GL2QPPL']
]
async def get_clans(cr, clan_groups):
return await asyncio.gather(*[
cr.get_clan(tag)
for group in clan_groups
for tag in group
])
async def main():
cr = clashroyale.RoyaleAPI(
TOKEN,
is_async=True,
timeout=30
)
try:
results = await get_clans(cr, clan_groups)
print(results)
finally:
await cr.close()
await asyncio.sleep(2)
asyncio.run(main())
Didn't test it.