We Keep Coding

Data Scraping Discord Messages Python

I am trying to web scrape Discord messages using Python from specific server channels.
import requests
import json
def retrieve_messages(channelid):
headers = {
'authorization': 'enter the authorization code'
}
# need to make a request to the url
r = requests.get(
f'https://discord.com/api/v9/channels/{channelid}/messages', headers=headers)
# create JSON object
jsonn = json.loads(r.text)
# we can now use a for loop on this JSON object
for value in jsonn:
print(value, '\n') # new line as well to separate each message
retrieve_messages('channel server id')
I am expecting the messages to be outputted in the terminal but instead I keep receiving the following output.
The output that I am getting instead of the messages

you're probably using a bot
token = "token"
headers = {
'authorization': f'Bot {token}'
}
try this

Spotify Web API not detecting body of POST request

I'm currently working on an app, one functionality of it being that it can add songs to the user's queue. I'm using the Spotify API for this, and this is my code to do so:
async def request():
...
uri = "spotify:track:5QO79kh1waicV47BqGRL3g" # temporary, can change later on
header = {'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': "{} {}".format(TOKEN_TYPE, ACCESS_TOKEN)}
data = {'uri': uri}
resp = requests.post(url="https://api.spotify.com/v1/me/player/queue", data=data, headers=header)
...
I've tried a lot of things but can't seem to understand why I'm getting Error 400 (Error 400: Required parameter uri missing).

so the Spotify API for the endpoint you're using suggests that the uri parameter required should be passed as part of the url, instead of as a data object.
Instead of data = {'uri': uri} can you please try adding your uri to the end of the url as such:
resp = requests.post(url="https://api.spotify.com/v1/me/player/queue?uri=?uri=spotify%3Atrack%3A5QO79kh1waicV47BqGRL3g", headers=header)
I also suggest using software like postman or insomnia to play around with the requests you send.

Requests package and API documentation

I'm having trouble understanding where to add parameters defined by API documentation. Take BeeBole's documentation for example, which specifies that to get an absence by ID, the following request is required:
{
"service": "absence.get",
"id": "absence_id"
}
They provide only one URL in the documentation:
BeeBole is accepting HTTP POST resquests in a json-doc format to the following URL:
https://beebole-apps.com/api/v2
How would this be implemented in the context of Python requests? The following code I've tried returns 404:
import requests
payload = {
"service": "absence.get",
"id": "absence_id"
}
auth = {
"username": "API_token",
"password": "x"
}
url = "https://beebole-apps.com/api/v2"
req = requests.get(url, params=payload, auth=auth).json()

BeeBole is accepting HTTP POST resquests in a json-doc format to the following URL: https://beebole-apps.com/api/v2
The JSON document format here is the part you missed; you need to pass the information as a JSON encoded body of the request. The params argument you used only sets the URL query string (the ?... part in a URL).
Use
import requests
payload = {
"service": "absence.get",
"id": "absence_id"
}
auth = ("API_token", "x")
url = "https://beebole-apps.com/api/v2"
req = requests.get(url, json=payload, auth=auth).json()
The json= part ensures that the payload dictionary is encoded to JSON and sent as a POST body. This also sets the Content-Type header of the request.
I've also updated the API authentication, all that the auth keyword needs here is a tuple of the username and password. See the Basic Authentication section.
You may want to wait with calling .json() on the response; check if the response was successful first:
req = requests.get(url, json=payload, auth=auth)
if not req.ok:
print('Request not OK, status:', req.status_code, req.reason)
if req.content:
print(req.text)
else:
data = req.json()
if data['status'] == 'error':
print('Request error:', data['message'])
This uses the documented error responses.

From the site documentation it would appear that this particular vendor has chosen an unusual API. Most people use different endpoints to implement different operations, but BeeBole appears to implement everything off the one endpoint, and then selects the operation by examining the "service" key in the request data.
Try
response - request.post('https://beebole-apps.com/api/v2',
json={"service": "company.list"},
headers={"authorization": TOKEN)
From the documentation I can't guarantee that will put the request in the right format, but at least if it doesn't work it should give you some clue as to how to proceed. Establishing the correct value of TOKEN is described under "Authorization" in the BeeBole documentation.
It's an unusual way to offer an API, but it seems workable.

Uber API reports JSON is invalid for /requests/estimates endpoint

I have made a fork of the Uber Application for Heroku
My Fork: https://github.com/CuriosityGym/Python-Sample-Application
I have modified the original code and the /price URL to get me estimates of price for a specific product ID, using the /requests/estimates endpoint documented here https://developer.uber.com/docs/riders/references/api/v1.2/requests-estimate-post.
#app.route('/price', methods=['GET'])
def price():
"""Example call to the price estimates endpoint.
Returns the time estimates from the given lat/lng given below.
"""
url = config.get('base_uber_url') + 'requests/estimate'
params = {
"product_id": "83941b0d-4be1-4979-a9c0-f0af5ee2b89b",
"start_latitude": config.get('start_latitude'),
"start_longitude": config.get('start_longitude'),
"end_latitude": config.get('end_latitude'),
"end_longitude": config.get('end_longitude')
}
print params
print generate_ride_headers(session.get('access_token'))
response = app.requests_session.post(
url,
headers=generate_ride_headers(session.get('access_token')),
data=params
)
return render_template(
'results.html',
endpoint='price',
data=response.text,
)
Here is the snippet of my code which uses the 1.2 version of the Uber Api. Other end points are working fine, its this one that does not work.
The print statements print to the Heroku logs and this is the output
{'product_id': '83941b0d-4be1-4979-a9c0-f0af5ee2b89b', 'end_longitude': '72.8811862', 'start_latitude': '18.936404', 'end_latitude': '19.0822507', 'start_longitude': '72.832546'}
{'Content-Type': 'application/json', 'Authorization': 'Bearer KA.eyJ2ZXJzaW9uIjkgsdshdJpZCI6IkNmcjAvRzhrUUNPaDNhSnRsUVZ6QlE9PSIsImV4cGlyZXNfYXQiOjE1MTAzMjA3NzgsInBpcGVsaW5lX2tleV9pZCI6Ik1RPT0iLCJwaXBlbGluZV9pZCI6MX0.JDoDTgaYJitK8Rtr35C6gTh5IQc7-P4T7mGg_wOYXu0'}
The error Reported by the api is
{"message":"Unable to parse JSON in request body.","code":"invalid_json"}

You need to encode your json as a string. Luckily requests can do this for you or you can use json.dumps() to dump the object as a string.
Here are two examples:
Either do this:
import json
response = app.requests_session.post(
url,
headers=generate_ride_headers(session.get('access_token')),
data=json.dumps(params)
)
Or pass it as a kwarg json:
response = app.requests_session.post(
url,
headers=generate_ride_headers(session.get('access_token')),
json=params
)

FB Messenger can't receive messages

I am trying to setup a messenger chatbot using the newly released messenger platform api. I setup a Python Flask server hosted on Heroku and have been adapting these instruction to try to get my page receiving the messages my server sends it: https://developers.facebook.com/docs/messenger-platform/quickstart
Thus far I have validated a callback url and have been able to receive messages when I post to my page on FB (i.e. when I message my page which is linked to my app on FB, my heroku logs show that the POST request is being received). However, when I try to send messages from my server to my app, I get the following JSON error response:
400: {"error":{"message":"(#100) param recipient must be non-empty.","type":"OAuthException","code":100,"fbtrace_id":"B3cni+LAmYU"}}
I am using the requests library to send requests to the page. Below is the code I am using to service POST requests:
import json
import os
import requests
from flask import Flask, request
app = Flask(__name__)
FB_MESSAGES_ENDPOINT = "https://graph.facebook.com/v2.6/me/messages"
FB_TOKEN = "EAADKWAcVj...AZDZD"
#app.route('/', methods=["POST"])
def chatbot_response():
req_data = request.data
data = json.loads(req_data)
print "Data: ", data
sender_id = data["entry"][0]["messaging"][0]["sender"]["id"]
print "Sender id: ", sender_id
send_back_to_fb = {
"entry": [{"messaging": [{"recipient": {"id": str(sender_id)}}]}],
"recipient": {
"id": str(sender_id)},
"message": "this is a test response message",
"recipient": str(sender_id), "access_token": FB_TOKEN
}
params_input = {"access_token": FB_TOKEN, "recipient": sender_id}
fb_response = requests.post(FB_MESSAGES_ENDPOINT,
params={"access_token": FB_TOKEN, "recipient": {"id": str(sender_id)}, "json": "recipient": {"id": str(sender_id)}},
data=json.dumps(send_back_to_fb))
print "Json of response: ", fb_response.json()
# handle the response to the subrequest you made
if not fb_response.ok:
# log some useful info for yourself, for debugging
print 'jeepers. %s: %s' % (fb_response.status_code, fb_response.text)
return "OK", 200
if __name__ == '__main__':
app.run(host="0.0.0.0")
I've tried countless different types of key/value encodings of the 'recipients' element into json but none of them seem to be understood by the FB graph service. How can I encode my request so that FB knows what the 'recipient' param is?
Thanks!
Edit:
It turns out I had to manually set the encoding type in the header of the POST request. Adding the following line made it so I could send interpretable text responses to FB:
headers = {'content-type': 'application/json'}

You can try these, both should work
headers = {'Content-type': 'application/json'}
response = requests.post(url, data=json.dumps(send_back_to_fb), headers=headers)
OR
response = requests.post(url, json=send_back_to_fb)

how about python library for using facebook messenger platform?
https://github.com/conbus/fbmq
from flask import Flask, request
from fbmq import Page
page = fbmq.Page(PAGE_ACCESS_TOKEN)
#app.route('/webhook', methods=['POST'])
def webhook():
page.handle_webhook(request.get_data(as_text=True))
return "ok"
#page.handle_message
def message_handler(event):
page.send(event.sender_id, "HI!!!")