We Keep Coding

Retry logic being ignored for server errors with Google Cloud Text-to-speech API

I'm trying to use Google Cloud's text-to-speech API. The problem I'm running into is that periodically the API returns a status of "500 Internal Server Error." The correct logic for these errors is usually to just retry the call. Unfortunately, I can't get any of Google Cloud's retry logic to work. As soon as I hit the exception my script exits.
My API function:
async def get_audio_from_google(input_text: str, output_file: str):
"""
Convert the provided text to audio using the Google text-to-speech API.
Args:
input_text: Text to conver to speech.
output_file: File path to write. File extension will be added automatically.
Returns: Writes the audio file to disk. Does not return a result.
"""
client = texttospeech.TextToSpeechAsyncClient()
# Create and configure the Synthesis object.
synthesis_input = texttospeech.SynthesisInput()
synthesis_input.text = input_text
voice_parameters = texttospeech.VoiceSelectionParams()
voice_parameters.language_code = VOICE_ENCODING
voice_parameters.name = VOICE
audio_parameters = texttospeech.AudioConfig()
if AUDIO_FORMAT == AudioFormat.MP3:
audio_parameters.audio_encoding = texttospeech.AudioEncoding.MP3
elif AUDIO_FORMAT == AudioFormat.OPUS:
audio_parameters.audio_encoding = texttospeech.AudioEncoding.OGG_OPUS
else:
print("Invalid audio format specified")
sys.exit(1)
logging.info(f"Synthesizing speech for {output_file}")
# Build our request.
request = texttospeech.SynthesizeSpeechRequest()
request.input = synthesis_input
request.voice = voice_parameters
request.audio_config = audio_parameters
# Get audio.
# Configure when to retry on error.
retry_object = retry.Retry(initial=5, timeout=90)
response = await client.synthesize_speech(request=request, retry=retry_object)
with open(f"{output_file}.{AUDIO_FORMAT}", "wb") as out:
# Write the response to the output file.
out.write(response.audio_content)
logging.info(f'Audio content written to file "{output_file}.{AUDIO_FORMAT}"')
TextToSpeechAsyncClient's synthesize_speech method accepts an instance of Retry, which is part of the Google Core API and can be used as a decorator or passed to some methods. Unfortunately, I can't seem to get the retry logic to work. By default it should retry on any error classed as transient, which includes Internal Server Error (error 500):
if_transient_error = if_exception_type(
exceptions.InternalServerError,
exceptions.TooManyRequests,
exceptions.ServiceUnavailable,
requests.exceptions.ConnectionError,
requests.exceptions.ChunkedEncodingError,
auth_exceptions.TransportError,
I've tried both passing retry to synthesize_speech and using it as a decorator for get_audio_from_google. In either case, as soon as my script gets an error response from the server, it exits.
How I'm calling get_audio_from_google:
def process_audio(text: List[str]):
"""
Process text asynchronously in segments and output a bunch of audio files ready for stitching.
Args:
text (List[str]): List of text snippets to process.
"""
async def gather_with_concurrency(max_tasks: int, *coroutines):
"""
Run tasks in parallel with a limit.
https://stackoverflow.com/questions/48483348/how-to-limit-concurrency-with-python-asyncio
Args:
max_tasks: Maximum number of tasks to run at once.
*coroutines: All async tasks that should be run.
"""
semaphore = asyncio.Semaphore(max_tasks)
async def sem_coro(coro):
async with semaphore:
return await coro
return await asyncio.gather(*(sem_coro(c) for c in coroutines))
async def main():
snippet_counter = 1
subtasks = []
for text_snippet in text:
snippet_filename = str(snippet_counter)
snippet_counter += 1
subtasks.append(
get_audio_from_google(input_text=text_snippet, output_file=snippet_filename)
)
await gather_with_concurrency(2, *subtasks)
logging.info("Starting audio processing tasks…")
# Begin execution.
asyncio.run(main())

Telegram Bot not responding after 3rd function call

The issue is that the third function never seems to respond.
I haven't been able to find a reason why this happens in the telegram documentation.
Please let me know if you have this issue or seen it and know the solution.
Even a post that references an issue like this would work.
Thank you so much for the assistance.
from email import message
import os
import re
import html
import json
import telebot
import requests
import http.client
from pytube import *
from dotenv import load_dotenv
load_dotenv()
# Creating hiding, and using API Keys
API_KEY = os.getenv("API_KEY")
RAPID_KEY = os.getenv("RAPID_API")
bot = telebot.TeleBot(API_KEY)
#bot.message_handler(commands="start")
# Creating a help message for guidance on how to use bot.
def help(message):
# Trying to send help message, if unable to send, throw an error message for the user.
try:
bot.send_message(message.chat.id, "Use \"Youtube\" and the video name to search for a video.\n")
except:
bot.send_message(message.chat.id, "There was an error fetching help, the bot may be offline.\n")
# Checking data and seeing if the word "YouTube" was used in order to start the search
def data_validation(message):
query = message.text.split()
if("youtube" not in query[0].lower()): # Set flag false if regular text
return False
else:
return True
#bot.message_handler(func=data_validation)
# Searching for youtube videos
# using RAPID API
def search(message):
query = message.text.split()
# Check if data is valid, and change variable to be lowercase for easy use.
if(data_validation(message) == True and query[0].lower() == "youtube"):
try:
if(data_validation(message) == True and query[1].lower() != "-d"):
# Removing the word "YouTube" and sending the results to the YouTube search engine.
for item in query[:]:
if(item.lower() == "youtube"):
query.remove(item)
search_query = ' '.join(query)
else:
pass #If it's not term we're looking to convert, ignore it.
# RAPID API for Youtube
try:
url = "https://youtube-search-results.p.rapidapi.com/youtube-search/"
querystring = {"q":search_query}
headers = {
"X-RapidAPI-Key": RAPID_KEY,
"X-RapidAPI-Host": "youtube-search-results.p.rapidapi.com"
}
response = requests.request("GET", url, headers=headers, params=querystring) # Grabbing response information from URL
request = json.loads(response.text) # Parsing json string for python use
# Testing to see if the RAPID API service responds and is online.
if(response.status_code == 503):
# If the service is not online, let the user know.
bot.send_message(message.chat.id, f"The RAPID API service appears to be offline try back later.\n")
if(response.status_code == 429):
# If the service has reached max quota for the day, let the user know.
bot.send_message(message.chat.id, f"Max quota reached, try back in 24 hours.\n")
# Grabbing first link from json text and sending direct url and title.
first_link = str((request["items"][0]["url"]))
bot.send_message(message.chat.id, f"{first_link}\n") # Sending first link that was queried.
# If there are no results found for the requested video, sending an error message to alert the user.
except:
bot.send_message(message.chat.id, "Unable to load video.\n")
except:
pass #ignoring if not the phrase we're looking for.
def test(message):
string = message.text.split()
print(string)
if(string[0] == "test" and data_validation(message) == True):
print("This is a test and i should be printed")
bot.send_message(message.chat.id, "Test message")
# Stay alive function for bot pinging / communication
bot.infinity_polling(1440)

The first problem in your code is your first line
from email import message
You import the message from email and also pass a parameter to the data_validation function with the same name, then return False in the data_validation function. If you return false, the function never will be executed.
first give an alias to first line you imported
Try This
from email import message as msg
import os
import re
import html
import json
import telebot
import requests
import http.client
from pytube import *
from dotenv import load_dotenv
load_dotenv()
# Creating hiding, and using API Keys
API_KEY = os.getenv("API_KEY")
RAPID_KEY = os.getenv("RAPID_API")
bot = telebot.TeleBot(API_KEY)
# Creating a help message for guidance on how to use bot.
#bot.message_handler(commands=["start"])
def help(message):
# Trying to send help message, if unable to send, throw an error message for the user.
try:
bot.send_message(message.chat.id, "Use \"Youtube\" and the video name to search for a video.\n")
except:
bot.send_message(message.chat.id, "There was an error fetching help, the bot may be offline.\n")
# Checking data and seeing if the word "YouTube" was used in order to start the search
def data_validation(message):
query = message.text.split()
print(query)
if("youtube" not in query[0].lower()): # Set flag false if regular text
return False # if you return false, the function never will be executed
else:
return True
# Searching for youtube videos
# using RAPID API
#bot.message_handler(func=data_validation)
def search(message):
query = message.text.split()
print(query) # if function executed you see the query result
# Check if data is valid, and change variable to be lowercase for easy use.
if(data_validation(message) == True and query[0].lower() == "youtube"):
try:
if(data_validation(message) == True and query[1].lower() != "-d"):
# Removing the word "YouTube" and sending the results to the YouTube search engine.
for item in query[:]:
if(item.lower() == "youtube"):
query.remove(item)
search_query = ' '.join(query)
else:
pass #If it's not term we're looking to convert, ignore it.
# RAPID API for Youtube
try:
url = "https://youtube-search-results.p.rapidapi.com/youtube-search/"
querystring = {"q":search_query}
headers = {
"X-RapidAPI-Key": RAPID_KEY,
"X-RapidAPI-Host": "youtube-search-results.p.rapidapi.com"
}
response = requests.request("GET", url, headers=headers, params=querystring) # Grabbing response information from URL
request = json.loads(response.text) # Parsing json string for python use
# Testing to see if the RAPID API service responds and is online.
if(response.status_code == 503):
# If the service is not online, let the user know.
bot.send_message(message.chat.id, f"The RAPID API service appears to be offline try back later.\n")
if(response.status_code == 429):
# If the service has reached max quota for the day, let the user know.
bot.send_message(message.chat.id, f"Max quota reached, try back in 24 hours.\n")
# Grabbing first link from json text and sending direct url and title.
first_link = str((request["items"][0]["url"]))
bot.send_message(message.chat.id, f"{first_link}\n") # Sending first link that was queried.
# If there are no results found for the requested video, sending an error message to alert the user.
except:
bot.send_message(message.chat.id, "Unable to load video.\n")
except:
pass #ignoring if not the phrase we're looking for.
def test(message):
string = message.text.split()
print(string)
if(string[0] == "test" and data_validation(message) == True):
print("This is a test and i should be printed")
bot.send_message(message.chat.id, "Test message")
# Stay alive function for bot pinging / communication
bot.infinity_polling(1440)

I found that using "if name == 'main':" and keeping all the functions in "main():" as a function handler everything ran smoothly.
I'm still trying to figure out why this works.

Webhook status code 400: {"embeds": ["Must be 10 or fewer in length."]}

I´m developing a bot that sends messages to the discord channel every second, however when I run the program and only sent 10 messages and after only print [1/1] Webhook status code 400: {"embeds": ["Must be 10 or fewer in length."]}. I don't find why is this happening, I'm using discord_webhook library and python to do this. Here is my code
async def __scrape_api(session, pid):
async with session.get(API_URL + pid) as response:
data = await response.json()
print(f"scrape {data}")
if not __search_product(pid):
name = data["name"]
image_url = data['skus'][0]['image']
for size in data['skus']:
if size['available']:
print("sent")
message = DiscordEmbed(title=f'{name}', url=f'{PRODUCT_URL_0}{size["sku"]}{PRODUCT_URL_1}',
description=f'talla: {size["dimensions"]["Tallas Calzado"]}\nPrecio: {size["bestPriceFormated"]}')
message.set_thumbnail(url=image_url)
message.set_timestamp()
webhook.add_embed(message)
response = webhook.execute()
time.sleep(1)
# save to database
__insert_new_product(pid, name, image_url, data['available'])

I found the solution for this after few minutes of taking a deeper look inside it's code. Basically each embed you send over is being held in the webhook object, once you made 10 requests with embeds then it gives you that error. All you have to do is:
webhook.execute(remove_embeds=True)

on_direct_message is not listening the direct message streaming using tweepy

The on_direct_message never called when a message arrives. i have used python3.7 and latest tweepy library. i n twitter account it works fine but not working with the code snippet am using. But the code snippet is working well to listen tweets.
twitter_stream=Stream(auth,StdOutListener())
print('Stream created...')
twitter_stream.filter(follow=[user.id_str], is_async=True)
NB:Permission taken for read, write and direct message. And all access parameters are correct
The StdOutListener is:
class StdOutListener( StreamListener ):
def __init__( self ):
self.tweetCount = 0
def on_connect( self ):
print("Connection established!!")
def on_disconnect( self, notice ):
print("Connection lost!! : ", notice)
def on_data( self, status ):
print("Entered on_data()")
print(status, flush = True)
return True
def on_direct_message( self, status ):
print("Entered on_direct_message()")
try:
print(status, flush = True)
return True
except BaseException as e:
print("Failed on_direct_message()", str(e))
def on_error( self, status ):
print(status)

Direct Messages are not supported in the Twitter streaming API (they were a part of the User Streams API, which was removed in 2018 and replaced with the Account Activity API).
To receive Direct Messages in realtime, you will need to implement a webhook handler for the Account Activity API. You could try the twitivity library, or look at this Python sample app. Tweepy does not have built-in support for this API.

Threads can only be started once in Django Channels

I created a simple Django Channels consumer that should connects to an external source, retrieve data and send it to the client. So, the user opens the page > the consumer connects to the external service and gets the data > the data is sent to the websocket.
Here is my code:
import json
from channels.generic.websocket import WebsocketConsumer, AsyncConsumer, AsyncJsonWebsocketConsumer
from binance.client import Client
import json
from binance.websockets import BinanceSocketManager
import time
import asyncio
client = Client('', '')
trades = client.get_recent_trades(symbol='BNBBTC')
bm = BinanceSocketManager(client)
class EchoConsumer(AsyncJsonWebsocketConsumer):
async def connect(self):
await self.accept()
await self.send_json('test')
bm.start_trade_socket('BNBBTC', self.process_message)
bm.start()
def process_message(self, message):
JSON1 = json.dumps(message)
JSON2 = json.loads(JSON1)
#define variables
Rate = JSON2['p']
Quantity = JSON2['q']
Symbol = JSON2['s']
Order = JSON2['m']
asyncio.create_task(self.send_json(Rate))
print(Rate)
This code works when i open one page; if i try to open a new window with a new account, though, it will throw the following error:
File "C:\Users\User\Desktop\Heroku\github\master\myapp\consumers.py", line 54, in connect
bm.start()
File "C:\Users\User\lib\threading.py", line 843, in start
raise RuntimeError("threads can only be started once")
threads can only be started once
I'm new to Channels, so this is a noob question, but how can i fix this problem? What i wanted to do was: user opens the page and gets the data, another user opens the page and gets the data; is there no way to do that? Or am i simply misunderstanding how Django Channels and websockets works?

Do you really need a secondary thread ?
class EchoConsumer(AsyncJsonWebsocketConsumer):
symbol = ''
async def connect(self):
self.symbol = 'BNBBTC'
# or, more probably, retrieve the value for "symbol" from query_string
# so the client can specify which symbol he's interested into:
# socket = new WebSocket("ws://.../?symbol=BNBBTC");
await self.accept()
def process_message(self, message):
# PSEUDO-CODE BELOW !
if self.symbol == message['symbol']:
await self.send({
'type': 'websocket.send',
'text': json.dumps(message),
})
For extra flexibility, you might also accept al list of symbols from the client, instead:
//HTML
socket = new WebSocket("ws://.../?symbols=XXX,YYY,ZZZ");
then (in the consumer):
class EchoConsumer(AsyncJsonWebsocketConsumer):
symbols = []
async def connect(self):
# here we need to parse "?symbols=XXX,YYY,ZZZ" ...
# the code below has been stolen from another project of mine and should be suitably adapted
params = urllib.parse.parse_qs(self.scope.get('query_string', b'').decode('utf-8'))
try:
self.symbols = json.loads(params.get('symbols', ['[]'])[0])
except:
self.symbols = []
def process_message(self, message):
if message['symbol'] in self.symbols:
...

I'm no Django developer, but if I understand correctly, the function connect is being called more than once-- and bm.start references the same thread most likely made in bm.start_trade_socket (or somewhere else in connect). In conclusion, when bm.start is called, a thread is started, and when it is done again, you get that error.

Here start() Start the thread’s activity.
It should be called at most once per thread object. You have made a global object of BinanceSocketManager as "bm".
It will always raise a RuntimeError if called more than once on the same thread object.
Please refer the below mentioned code, it may help you
from channels.generic.websocket import WebsocketConsumer, AsyncConsumer, AsyncJsonWebsocketConsumer
from binance.client import Client
import json
from binance.websockets import BinanceSocketManager
import time
import asyncio
class EchoConsumer(AsyncJsonWebsocketConsumer):
client = Client('', '')
trades = client.get_recent_trades(symbol='BNBBTC')
bm = BinanceSocketManager(client)
async def connect(self):
await self.accept()
await self.send_json('test')
self.bm.start_trade_socket('BNBBTC', self.process_message)
self.bm.start()
def process_message(self, message):
JSON1 = json.dumps(message)
JSON2 = json.loads(JSON1)
#define variables
Rate = JSON2['p']
Quantity = JSON2['q']
Symbol = JSON2['s']
Order = JSON2['m']
asyncio.create_task(self.send_json(Rate))
print(Rate)