We Keep Coding

Log in to a website with csrfmiddlewaretoken verification using python

I am using the following code to log in to a website with csrfmiddlewaretoken verification, but it throws me the following error:
"csrfmiddlewaretoken = HTML.find_all('input')[1]['value']
IndexError: list index out of range"
What do you think is the problem, I'm new using python :)
import requests
from bs4 import BeautifulSoup
request_url = 'https://text.gob.pe/accounts/login/'
with requests.session() as session:
get_url = session.get('https://text.gob.pe/accounts/login/')
HTML = BeautifulSoup(get_url.text, 'html.parser')
csrfmiddlewaretoken = HTML.find_all('input')[1]['value']
#logging in
payload = {
'next' : '/ profile /',
'username' : 'secret',
'password' : 'secret',
'next': '/ profile /',
'csrfmiddlewaretoken': csrfmiddlewaretoken
}
headers = {
'Referer': 'https://text.gob.pe/accounts/login/'
}
login_request = session.post(request_url,payload, headers=headers)
home_page = session.get("https://text.gob.pe/ficha/buscar/")
print(home_page.content)

Without a username and password, it is difficult to show how to proceed. At the moment your mistake is that you didn't specify headers
headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'content-type': 'application/x-www-form-urlencoded',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36'
}
url = "https://logincovid19.minsa.gob.pe/accounts/login/"
session = requests.Session()
response = session.get(url, headers=headers)
csrfmiddlewaretoken = BeautifulSoup(response.text, 'lxml').find('input', {'name': 'csrfmiddlewaretoken'}).get('value')
Now csrfmiddlewaretoken is something like this - eyHLYFv7HOYxglzFS9a3JDxOT38u8mrakdwhatOnkvcJJzwN9dNi6olBxJxD1HZi
I think further the code will look like this, but you need to check it:
user = 'YourUserNmaeHere'
password = 'UserPaswordHere'
payload = f'csrfmiddlewaretoken={csrfmiddlewaretoken}&username={user}&password={password}'
response = session.post(url, data=payload, headers=headers)

Python: Use Solscan API to get transactions

I am using the following code:
import requests
from requests.structures import CaseInsensitiveDict
url='https://public-api.solscan.io/account/transactions?account=24jvtWN7qCf5GQ5MaE7V2R4SUgtRxND1w7hyvYa2PXG6'
headers = CaseInsensitiveDict()
headers["accept"] = "application/json"
resp = requests.get(url, headers=headers)
print(resp.json())
I get an error: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I expect to get an output with in the form of key:value and then covnvert that to a DataFrame. Why does my error occur?

Use headers that actually work.
Try this:
import requests
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'
}
url = 'https://public-api.solscan.io/account/transactions?account=24jvtWN7qCf5GQ5MaE7V2R4SUgtRxND1w7hyvYa2PXG6'
resp = requests.get(url, headers=headers)
print(resp.json()[0]['signer'])
Sample output:
['7BjLjdJEGLaLscYkpw57YKzYqRY1i3ypnfLK8R2bgDrC', '7qfzWZmyYU1PBYJG5Y2ksSbaf6xHc77Tx47urzySFins']

How get id of the message I sent with my Telegram bot via Python?

To send the message on my Telegram channel, I use the following template that works perfectly:
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36"
}
chat_telegram = ['XXXXXXXXXXXXX']
textalert = f'Test'
botalert = 'YYYYYYYYYYYYYYYYYYYYY'
urlalert = f'https://api.telegram.org/bot{botalert}/sendMessage'
params = {'text':textalert, 'chat_id':chat_telegram, 'parse_mode':'HTML'}
return_request = requests.get(urlalert, headers=headers, params=params)
I've tried using all four of the following ways to try to retrieve the message ID that was sent:
print(return_request['update']['message']['message_id'])
print(return_request['message']['message_id'])
print(return_request.update.message.message_id)
print(return_request.message.message_id)
But none returned positively, how should I proceed to recover the value I need?
Here is the API map

The message id is stored in the response.text part of the response, you can retrieve it via the json method:
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36"
}
chat_id = ['chat id']
textalert = f'Test'
botalert = 'some token'
urlalert = f'https://api.telegram.org/bot{botalert}/sendMessage'
params = {'text':textalert, 'chat_id':chat_id, 'parse_mode':'HTML'}
response = requests.get(urlalert, headers=headers, params=params)
json_data = response.json()
# get the message id
print('message id', json_data['result']['message_id'])
alternatively, you can retrieve it with the json module:
import json
json_data = json.loads(response.text)
# json_data['result']['message_id']

Trouble scraping all the follower names from a profile page using requests

I'm trying to scrape all the follower names from a profile page using requests module. The problem is when I run the script below, I get the first 20 names over and over again.
The parameters used in post requests only have two keys and values like size:20 and continuation:timestamp. I tried to use the parameters in the right way but still I get the same results repeatedly.
import time
import requests
link = 'https://api-mainnet.rarible.com/marketplace/api/v4/followers'
params = {'user': '0xe744d23107c9c98df5311ff8c1c8637ec3ecf9f3'}
payload = {"size": 20}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36',
'origin': 'https://rarible.com',
'referer': 'https://rarible.com/'
}
with requests.Session() as s:
s.headers.update(headers)
while True:
res = s.post(link,params=params,json=payload)
print(s.headers)
for item in res.json():
print(item['owner'].get('name',''))
payload['continuation'] = f"{int(time.time() * 1000)}"
time.sleep(2)
How can I parse all the follower names from that page using requests?

Your next continuation value is in X-CONTINUATION response header, so this will work when increasing size in payload doesn't:
import requests
link = 'https://api-mainnet.rarible.com/marketplace/api/v4/followers'
params = {'user': '0xe744d23107c9c98df5311ff8c1c8637ec3ecf9f3'}
payload = {"size": 20}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36',
'origin': 'https://rarible.com',
'referer': 'https://rarible.com/'
}
res = requests.post(link, headers=headers, params=params, json=payload)
print(res.headers["X-CONTINUATION"])
while True:
for item in res.json():
print(item['owner'].get('name',))
if not res.headers["X-CONTINUATION"]:
break
payload['continuation'] = res.headers["X-CONTINUATION"]
res = requests.post(link, headers=headers, params=params, json=payload)

some api may block you from extracting values more than certain limit and also may show in pages with limits.
For me just increasing the size payload worked with your code.
import time
import requests
link = 'https://api-mainnet.rarible.com/marketplace/api/v4/followers'
params = {'user': '0xe744d23107c9c98df5311ff8c1c8637ec3ecf9f3'}
payload = {"size": 10000}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36',
'origin': 'https://rarible.com',
'referer': 'https://rarible.com/'
}
with requests.Session() as s:
s.headers.update(headers)
res = s.post(link,params=params,json=payload)
print(len(res.json()))
for item in res.json():
print(item['owner'].get('name',''))

How to upload a photo to Instagram using request post python?

I'm trying to compose a post request for uploading a photo to Instagram, but the third request returns a 403 error to "instagram.com/create/configure/"
My code:
import re
import requests
import urllib.request
from datetime import datetime
link = 'https://www.instagram.com/accounts/login/'
login_url = 'https://www.instagram.com/accounts/login/ajax/'
login = 'login_name'
password = '0000'
time = int(datetime.now().timestamp())
print(time)
payload = {
'username': login,
'enc_password': f'#PWD_INSTAGRAM_BROWSER:0:{time}:{password}',
'queryParams': {},
'optIntoOneTap': 'false'
}
with requests.Session() as s:
r = s.get(link)
a = r.cookies['csrftoken']
print(r.cookies)
print(a)
csrf = re.findall(r"csrf_token\":\"(.*?)\"", r.text)[0]
print(csrf)
r = s.post(login_url, data=payload, headers={
"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36",
"X-Requested-With": "XMLHttpRequest",
"Referer": "https://www.instagram.com/accounts/login/",
"x-csrftoken": csrf
})
print(r.status_code)
print(r.url)
print(r.text)
print(s.cookies)
r = s.get('https://www.instagram.com/accounts/edit/')
print(login in r.text)
microtime = int(datetime.now().timestamp())
headers = {
"content-type": "image / jpg",
"X-Entity-Name" : f"fb_uploader_{microtime}",
"Offset": "0",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36",
"x-entity-length": "299255",
"X-Instagram-Rupload-Params": f'{{"media_type": 1, "upload_id": {microtime}, "upload_media_height": 1080, "upload_media_width": 1080}}',
"x-csrftoken": csrf,
"x-ig-app-id": "1217981644879628"
}
img = urllib.request.urlopen('https://sun9-64.userapi.com/RVgUHSq9fXrDr8YBJ4a4h9xwN4EQA_8BXuQ5Vg/Mdx3LwawEmY.jpg')
photo = img.read()
r = s.post(f'https://www.instagram.com/rupload_igphoto/fb_uploader_{microtime}', data=open("4.jpg", "rb"), headers=headers)
print(r.text)
headers = {
'Content-Length': '104',
'content-type': 'application/x-www-form-urlencoded',
"origin": "https://www.instagram.com",
"referer": "https://www.instagram.com/create/details/",
'user-agent': "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36",
"x-csrftoken": csrf,
"x-ig-app-id": "1217981644879628",
"X-Requested-With": "XMLHttpRequest"
}
body = {
'upload_id': f'{microtime}',
"caption":'00000000000000000000',
'usertags':'',
'custom_accessibility_caption': '',
'retry_timeout':''
}
r = s.post('https://www.instagram.com/create/configure/', data=body, headers=headers)
print(r.status_code)
print(r.text)
Process:
First authorization, everything is okay with that
The second request is to upload a photo, everything is ok too
The third request, with the parameters of the photo, there is an error ...
error 403 is returned

You need to give the csrf_token and session_id as cookies in the third request.
you can find the complete guide here:
Share a post into your Instagram account using the requests library