We Keep Coding

404 error when polling Reddit's developer API

I am trying to use Reddit's developer API to build a simple scraper that grabs posts and their replies in a target subreddit and produces JSON with the information.
I am getting a 404 error that I don't understand.
This is my code:
import praw
import json
def scrape(subreddit, limit):
r = praw.Reddit(user_agent='Reddit data organizer 1.0 by /u/reallymemorable', client_id='none of your business', client_secret='none of your business')
submissions = r.subreddit(subreddit).get_hot(limit=limit)
for submission in submissions:
data = {}
data['title'] = submission.title
data['score'] = submission.score
data['url'] = submission.url
data['author'] = str(submission.author)
data['subreddit'] = str(submission.subreddit)
data['num_comments'] = submission.num_comments
data['over_18'] = submission.over_18
data['selftext'] = submission.selftext
data['is_self'] = submission.is_self
data['name'] = submission.name
data['created_utc'] = submission.created_utc
data['permalink'] = submission.permalink
data['domain'] = submission.domain
data['id'] = submission.id
data['kind'] = submission.kind
json.dumps(data)
scrape('https://www.reddit.com/r/funny/', 25)
When I run it, I get this:
reallymemorable#Christians-MBP Desktop % python3 fetch-data-subreddit.py
Traceback (most recent call last):
File "/Users/reallymemorable/Desktop/fetch-data-subreddit.py", line 26, in <module>
scrape('https://www.reddit.com/r/augmentedreality/comments/yv7sn8/ar_maximum_distance/', 25)
File "/Users/reallymemorable/Desktop/fetch-data-subreddit.py", line 6, in scrape
submissions = r.subreddit(subreddit).get_hot(limit=limit)
File "/opt/homebrew/lib/python3.9/site-packages/praw/models/reddit/base.py", line 34, in __getattr__
self._fetch()
File "/opt/homebrew/lib/python3.9/site-packages/praw/models/reddit/subreddit.py", line 583, in _fetch
data = self._fetch_data()
File "/opt/homebrew/lib/python3.9/site-packages/praw/models/reddit/subreddit.py", line 580, in _fetch_data
return self._reddit.request(method="GET", params=params, path=path)
File "/opt/homebrew/lib/python3.9/site-packages/praw/util/deprecate_args.py", line 43, in wrapped
return func(**dict(zip(_old_args, args)), **kwargs)
File "/opt/homebrew/lib/python3.9/site-packages/praw/reddit.py", line 941, in request
return self._core.request(
File "/opt/homebrew/lib/python3.9/site-packages/prawcore/sessions.py", line 330, in request
return self._request_with_retries(
File "/opt/homebrew/lib/python3.9/site-packages/prawcore/sessions.py", line 266, in _request_with_retries
raise self.STATUS_EXCEPTIONS[response.status_code](response)
prawcore.exceptions.NotFound: received 404 HTTP response

r.subreddit(subreddit) - subreddit should just be the name of the subreddit e.g. "funny" and not the full URL.
See the docs here: https://praw.readthedocs.io/en/stable/getting_started/quick_start.html#obtain-a-subreddit

can't loop through chats and get premissions in telethon

I'm trying to get my group/channel permissions with the help of telethon but it gives an error whenever I run the get_permissions function inside a for-loop. This is the code:
client = TelegramClient(phone, apiId, apiHash)
client.connect()
if not client.is_user_authorized():
client.send_code_request(phone)
client.sign_in(phone, input("Enter the verification code:"))
chats = []
last_date = None
chunk_size = 200
groups = []
channels = []
result = client(GetDialogsRequest(
offset_date=last_date,
offset_id=0,
offset_peer=InputPeerEmpty(),
limit=chunk_size,
hash=0
))
chats.extend(result.chats)
me = client.get_me()
for chat in chats:
permissions = client.get_permissions(chat, client.get_me())
if permissions.is_admin():
client.get_participants(chat)
else:
print("you don't have access to this channel/group")
and this is the error that it's giving me:
Request caused struct.error: required argument is not an integer: GetFullChatRequest(chat_id=InputPeerChat(chat_id=701294847))
Traceback (most recent call last):
File "/Users/artin/Documents/coding/work/Cryptal/cryptal-aggregator/src/sample.py", line 45, in <module>
permissions = client.get_permissions(chat, client.get_me())
File "/opt/homebrew/lib/python3.10/site-packages/telethon/sync.py", line 39, in syncified
return loop.run_until_complete(coro)
File "/opt/homebrew/Cellar/python#3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/opt/homebrew/lib/python3.10/site-packages/telethon/client/chats.py", line 1278, in get_permissions
chat = await self(functions.messages.GetFullChatRequest(
File "/opt/homebrew/lib/python3.10/site-packages/telethon/client/users.py", line 30, in __call__
return await self._call(self._sender, request, ordered=ordered)
File "/opt/homebrew/lib/python3.10/site-packages/telethon/client/users.py", line 63, in _call
future = sender.send(request, ordered=ordered)
File "/opt/homebrew/lib/python3.10/site-packages/telethon/network/mtprotosender.py", line 176, in send
state = RequestState(request)
File "/opt/homebrew/lib/python3.10/site-packages/telethon/network/requeststate.py", line 17, in __init__
self.data = bytes(request)
File "/opt/homebrew/lib/python3.10/site-packages/telethon/tl/tlobject.py", line 194, in __bytes__
return self._bytes()
File "/opt/homebrew/lib/python3.10/site-packages/telethon/tl/functions/messages.py", line 2065, in _bytes
struct.pack('<q', self.chat_id),
struct.error: required argument is not an integer

Error when saving data to MongoDB database

I want to make a role assignment using the support approval on the discord server, in general, a complex system.. To do this, I need to save the message id and the id of the message author who needs to be assigned a role, all the code is working, an error in the line of code associated with the database.
event code
cluster = MongoClient('mongodb+srv://corry:pa4-pD6-3Dj-AdT#cluster0.gi8x6.mongodb.net/?retryWrites=true&w=majority')
db = cluster.RolesData
roles = db.RolesCollection
#bot.event
async def on_message(message):
msg = message.content.lower()
request = ['роль', 'хочу роль', 'дайте роль', 'нужна роль']
if msg in request:
for i in orgs:
if not message.author.display_name.split(' ')[0].replace('[', '') in orgs:
await message.channel.send(f'`[ROLES]`: {message.author.mention}, тег организации не найден!')
break
if i == message.author.display_name.split(' ')[0].replace('[', ''):
guild = bot.get_guild(831511376425123882)
role = discord.utils.get(guild.roles, id=orgs[i])
await message.channel.send(f'`[ROLES]`: {message.author.mention}, ваш запрос на получение роли был отправлен саппортам!')
embed = discord.Embed(
title='Запрос на получение роли!',
description='✅ - выдать роль\n❌ - отклонить запрос\n🖊️ - изменить никнейм\n📋 - запросить статистику',
color=0x000000
)
embed.add_field(name=f'Пользователь:', value=f'{message.author.mention}', inline=False)
embed.add_field(name=f'Никнейм:', value=f'{message.author.display_name}', inline=False)
embed.add_field(name=f'Роль для выдачи:', value=f'{role.mention}', inline=False)
embed.set_thumbnail(url='https://psv4.userapi.com/c537232/u356224017/docs/d1/416eb92aec38/rodina.png?extra=2E9CwKZ0PIjnG8aaNkwehlHwjxlycBfGx-4p20ABm3mPI4jNpdV1OXaUUA9zGA4Q04VM21UezrXsjtqC411Xbh-Ro7rW1L4OGgNxpcQv3lvfOYCb-Irn-_-51AaQa2fpNDZhHm80dvZm1HlR1ZWoDigw')
embed.set_image(url=message.author.avatar_url)
msgemb = await bot.get_channel(851413212485779466).send(embed=embed)
await msgemb.add_reaction('✅')
await msgemb.add_reaction('❌')
await msgemb.add_reaction('🖊️')
await msgemb.add_reaction('📋')
roles_post = {
'm_id':msg_db.id,
'user':msg_db.author
}
if roles.count_documents({'m_id':message.id}) == 0:
roles.insert_one(roles_post)
else:
await message.channel.send(f'`[ROLES]`: {message.author.mention}, вы уже отправили запрос на получение роли!')
error code
Traceback (most recent call last):
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\srv_resolver.py", line 89, in _resolve_uri
results = _resolve(
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\srv_resolver.py", line 43, in _resolve
return resolver.resolve(*args, **kwargs)
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\dns\resolver.py", line 1305, in resolve
return get_default_resolver().resolve(qname, rdtype, rdclass, tcp, source,
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\dns\resolver.py", line 1176, in resolve
timeout = self._compute_timeout(start, lifetime)
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\dns\resolver.py", line 997, in _compute_timeout
raise Timeout(timeout=duration)
dns.exception.Timeout: The DNS operation timed out after 21.201510190963745 seconds
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Хозяин\Desktop\rodina\main.py", line 5, in <module>
cluster = MongoClient('mongodb+srv://corry:pa4-pD6-3Dj-AdT#cluster0.gi8x6.mongodb.net/?retryWrites=true&w=majority')
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\mongo_client.py", line 704, in __init__
res = uri_parser.parse_uri(
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\uri_parser.py", line 542, in parse_uri
nodes = dns_resolver.get_hosts()
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\srv_resolver.py", line 121, in get_hosts
_, nodes = self._get_srv_response_and_hosts(True)
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\srv_resolver.py", line 101, in _get_srv_response_and_hosts
results = self._resolve_uri(encapsulate_errors)
File "C:\Users\Хозяин\AppData\Local\Programs\Python\Python39\lib\site-packages\pymongo\srv_resolver.py", line 97, in _resolve_uri
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: The DNS operation timed out after 21.201510190963745 seconds
Для продолжения нажмите любую клавишу . . .

Judging by this bit self._raise_connection_failure(error) in your error message, I believe the issue lies within the fact that you're not successfully connecting to your db in the first place.

DialogFlow CX V3 - Upsert Entities (Batch Insert)

I am looking for some advice on how I can upsert or replace existing user entity.
I tried couple of API's documented here and also here.
The entities are read from database and the plan is to keep them in sync with database values as a scheduled job.
Update: Code Snippet
client_options = {"quota_project_id": gcp_default_project_id,
"api_endpoint": "us-central1-dialogflow.googleapis.com:443"}
client = EntityTypesClient(credentials=credentials_det, client_options=client_options)
entity_type = v3beta.EntityType()
entity_type.display_name = entity_display_name
entity_type.kind = "KIND_REGEXP"
print(client_options)
entity_type.entities = entity_json
# Initialize request argument(s)
request = v3beta.UpdateEntityTypeRequest(
entity_type=entity_type,
)
print(request)
response = client.update_entity_type(request=request)
print(response)
entity_json is fetched from DB and created as JSON object as below.
df = get_data.get_df_details(config_dir, entity_data_source, sql)
username = df['username'].tolist()
entity_json = []
for each in username:
each_entity_value = {}
each_entity_value['value'] = each
each_entity_value['synonyms'] = [each]
entity_json.append(each_entity_value)
Here's the Trace
Traceback (most recent call last):
File "/Users/<some_dir>/df_ins_entities/df_ins_entities/ins_entity_val.py", line 116, in
ins_now(config_dir, input_entity_name, entity_data_source)
File "/Users/<some_dir>/df_ins_entities/df_ins_entities/ins_entity_val.py", line 96, in ins_now
response = client.update_entity_type(request=request)
File "/Users/<some_dir>/df_ins_entities/lib/python3.9/site-packages/google/cloud/dialogflowcx_v3beta1/services/entity_types/client.py", line 902, in update_entity_type
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "/Users/<some_dir>/df_ins_entities/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 154, in call
return wrapped_func(args, **kwargs)
File "/Users/<some_dir>/df_ins_entities/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Resource name '' does not match 'projects//locations//agents//entityTypes/*'.
Process finished with exit code 1

Discord.py Random errors from running bot?

This question is mostly code because I have no clue what is happening and how to fix it...
I will also provide the main file. Thanks!
Code (looks something like this; Microsoft Lens is good, but not great) :
import discord
from discord.ext.commands import CommandNotFound
from discord_slash import SlashCommand
def mixedCase(*args) :
total = []
for string in args:
a = map( ''. join,
itertools.product(*((c.upper(), c.lower( ))
for c in string) ) )
for x in list(a):
total.append(x)
return list(total)
client = commands.Bot(case_insensitive=True, command_prefix=mixedCase( "mc" ) )
slash = SlashCommand ( client, sync_commands=True, sync_on_cog_reload=True)
startTime = int(time.time( ))
client.main_startTime = startTime
class MyHelpCommand (commands.MinimalHelpCommand):
async def send_pages(self):
destination = self.get_destination( )
e = discord.Embed(color=discord.Color.green( ), description='')
for page in self.paginator. pages:
e.description += page
e.set_footer (text="Need help? Do 'mcsupport to join the support server!")
await destination.send(embed=e)
client.help_command = MyHelpCommand( )
client.run(TOKEN)
Error:
Traceback (most recent call last):
File "Bot.py", line 127, in <module> client. run( "TOKEN" )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 723, in run return future. result( )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 702, in runner await self. start( *args,** kwargs )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 666, in start await self. connect (reconnect=reconnect)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 566, in connect await self.ws. poll_event( )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/gateway.py", line 559, in poll_event await self. received_message(msg. data)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/gateway.py", line 509, in received_message func (data)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/state.py", line 490, in parse_message_crea te message = Message(channel=channel, data=data, state=self )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/message. py", line 584, in _init ref. resolved = self.__class__(channel=chan, data=resol ved, state=state)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/message.py", line 562, in _init self. stickers = [Sticker(data=data, state=state) for d ata in data. get( 'stickers', [])]
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/message.py", line 562, in <listcomp> self. stickers = [Sticker(data=data, state=state) for d ata in data. get( 'stickers', [])]
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/sticker.py", line 77, in _init_ self. pack_id = int(data['pack_id'])
KeyError: 'pack_id'
Extracted using Microsoft Lens, I'll try to correct missed words.

Looking at the output and traceback I can see that the error comes from within the discord module. When sticker.py is used, and the class within it is initialized, the pack_id attribute is instantiated with int(data['pack_id']) which should return the value from the dictionary data with key 'pack_id'. It seems that it cannot find such an item. The keyerror suggests that the key 'pack_id' cannot be found in the dictionary data. Usually, data.get('pack_id', 'default value') is better used in order to avoid problems (if value doesn't exist, just replaced with a default value. I am not sure where the issue comes from here, but I don't believe its due to your code.