Instagram pagination tag - python

I am doing a development based on instagram, I do not use the API because they do not allow applications in development mode but in production mode.
So, I'm trying to get the following pages regarding a hashtag, for example:
https://www.instagram.com/explore/tags/plebiscito/
The next page is done by sending a POST method, to a /query / and a /ajax/bz, however, trying to try it with cURL does not work for me.
I leave as I have been doing with cURL.
curl "https://www.instagram.com/explore/tags/plebiscito/" --http1.1 -k "https://www.instagram.com/query/" -H "Content-Type: application/x-www-form-urlencoded" -X POST -d "q=ig_hashtag(plebiscito)+{+media.after(j0hwe66aaaaaf0hwexjmwaaafkwa,+12)+{++count,++nodes+{++++caption,++++code,++++comments+{++++++count++++},++++comments_disabled,++++date,++++dimensions+{++++++height,++++++width++++},++++display_src,++++id,++++is_video,++++likes+{++++++count++++},++++owner+{++++++id++++},++++thumbnail_src,++++video_views++},++page_info}+}&ref=tags::show&query_id=/" --next --http1.1 -k "https://www.instagram.com/ajax/bz" -H "Content-Type: application/json" -X POST -d '{"q":[{"page_id":"7mj51x","posts":[["timespent_bit_array",{"tos_id":"7mj51x","start_time":1481556875,"tos_array":[3,0],"tos_len":2,"tos_seq":2,"tos_cum":19,"log_time":1481556876912},1481556876912,0]],"trigger":"timespent_bit_array"}],"ts":1481556877336}' --next -k "https://www.instagram.com/query/" -H "Content-Type: application/x-www-form-urlencoded" -X POST -d "q=ig_hashtag(plebiscito)+{+media.after(j0hwe66aaaaaf0hwexjmwaaafkwa,+8)+{++count,++nodes+{++++caption,++++code,++++comments+{++++++count++++},++++comments_disabled,++++date,++++dimensions+{++++++height,++++++width++++},++++display_src,++++id,++++is_video,++++likes+{++++++count++++},++++owner+{++++++id++++},++++thumbnail_src,++++video_views++},++page_info}+}&ref=tags::show&query_id=/" --next -k "https://www.instagram.com/ajax/bz" -H "Content-Type: application/json" -X POST -d '{"q":[{"page_id":"7mj51x","posts":[["timespent_bit_array",{"tos_id":"7mj51x","start_time":1481556875,"tos_array":[3,0],"tos_len":2,"tos_seq":2,"tos_cum":19,"log_time":1481556876912},1481556876912,0]],"trigger":"timespent_bit_array"}],"ts":1481556877336}'
Here I was trying to get page two, however, it responds with a page not found.
In short, I need to automate the pages with Python, but I was testing it with cURL.
Could you help me? thank you very much.

First of all, you can get a json https://www.instagram.com/explore/tags/plebiscito/?__a=1 It contains page_info with end_cursor. Paginate using max_id=VALUE-FROM-end_cursor in GET-request. More info here.
If you want to use POST /query you need right cookie.

Related

collect others parameters in my flask api [duplicate]

This question already has answers here:
Using cURL to upload POST data with files
(11 answers)
Closed 2 months ago.
I send a json file by curl with this command and its work :
curl -X POST -H "Content-Type: application/json" -d #/Users/test/testjson.json 'http://127.0.0.1:5000
I collect this data like and its work :
#app.route('/json_test', methods=['POST'])
def process_data():
# Collect the JSON file from the request
data = json.loads(request.data)
#
I want to add both parameters source and id.
How can I send these parameters in Curl plus my json file . I tried
curl -i -X POST -H "Content-Type: multipart/form-data"
-F "data /Users/test/testjson.json” -F "source=Value1" -F "ident=Value2" http://127.0.0.1:5000
But it's not working
if you can help me to have the command and how i can read this data with python.
Please provide more information of the content on both values you want to add.
I changed a quotation mark inside your curl, try with:
curl -i -X POST -H 'Content-Type: multipart/form-data'
-F 'data /Users/test/testjson.json' -F "source=Value1" -F 'ident=Value2' http://127.0.0.1:5000
To use variables from the curl, you need to update your method. The below curl is another example you can try for now:
#app.route('/json_test', methods=['POST'])
def process_data(source: str, ident: str):
# Collect the JSON file from the request
data = json.loads(request.data)
#
Curl Command:
curl -i -X POST \
'http://127.0.0.1:5000/json_test/?source=value1/?ident=value2' \
-H 'Content-Type: multipart/form-data' \
-F 'data /Users/test/testjson.json'
Try this:
curl -i -X POST -H "Content-Type: multipart/form-data"
-F "user_input=</Users/test/testjson.json” -F "source=Value1" -F "ident=Value2" http://127.0.0.1:5000/json_test
Note that this will create a form with three fields: user_input, source, and ident. You'll need to modify your controller endpoint as well:
#app.route('/json_test', methods=['POST'])
def process_data():
form_content_as_json = json.loads(request.form)

Posting a file with a REST API: from a curl example to Python code

I'm trying to code the upload of a file to a web service through a REST API in Python. The service's documentation shows a example using curl as client:
curl -X POST -H \
-H "Content-Type: multipart/form-data" \
-F "file=filename.ext" \
-F "property1=value1" \
-F "property2=value2" \
-F "property3=value3" \
https://domain/api/endpoint
The difficulty for me is that this syntax doesn't match multipart form-data examples I found, including the requests documentation. I tried this, which doesn't work (rejected by the API):
import requests
file_data = [
("file", "filename.ext"),
("property1", "value1"),
("property2", "value2"),
("property3", "value3"),
]
response = requests.post("https://domain/api/endpoint",
headers={"Content-Type": "multipart/form-data"}, files=file_data)
With the error: "org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found"
Can anybody help in transposing that curl example to proper Python code?
Thanks!
R.
OK, looks like the web services documentation is wrong, and metadata simply needs to be sent as parameters. Moreover, I found in another request that you shouldn't set the header. So I was starting from a wrong example.

Django DRF JWT authentication get token - JSON parse error - Expecting value

I have simple Django DRF application setup which I have implemented JWT authentication.
I used the Django REST framework JWT documentation
I am using curl to test the implementation.
I can successfully get a token using the following notation used in the documentation:
$ curl -X POST -d "username=admin&password=password123" http://localhost:8000/api-token-auth/
The token is returned in following format:
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InNpdHJ1Y3AiLCJleHAiOjE1MTE2NTEyMTQsInVzZXJfaWQiOjEsImVtYWlsIjoiY3VydGlzLnBva3JhbnRAZ21haWwuY29tIn0.F1TSkxe5tQVpddetUdOJDdAPP1XB9Bimb5U3c75oWd0"}
However, when I try using this other variation, I get an error:
$ curl -X POST -H "Content-Type: application/json" -d '{"username":"admin","password":"password123"}' http://localhost:8000/api-token-auth/
The error I get is:
{"detail":"JSON parse error - Expecting value: line 1 column 1 (char 0)"}
I also had the same error when trying to refresh or verify the token:
Refresh:
$ curl -X POST -H "Content-Type: application/json" -d '{"token":"<EXISTING_TOKEN>"}' http://localhost:8000/api-token-refresh/
Verify:
$ curl -X POST -H "Content-Type: application/json" -d '{"token":"<EXISTING_TOKEN>"}' http://localhost:8000/api-token-verify/
I was adding the token as follows:
curl -X POST -H "Content-Type: application/json" -d '{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InNpdHJ1Y3AiLCJleHAiOjE1MTE2NDg5MjIsInVzZXJfaWQiOjEsImVtYWlsIjoiY3VydGlzLnBva3JhbnRAZ21haWwuY29tIn0.T5h_PSvzvKOZCPTS60x5IUm3DgAsRCRmbMJeGWZk3Tw"}' http://localhost:8800/api-token-refresh/
Am I perhaps adding the token incorrectly? Does it need some other formatting with quotes?
Those requests are sending data in two different ways. The first request sends it as form data (x-www-form-urlencoded) which is what your endpoint is expecting and the second request sends it as application/json.
I'm not sure that the library you're using will handle a json request out of the box so one option would be to create a custom endpoint and use something like the following:
import json
def ParseFormData(self, request):
payload = json.loads(request.body.decode('utf-8'))
// use django auth to authorize request and return token
You can read more about it in this answer here: https://stackoverflow.com/a/29514222/5443056
There's instructions for manually creating auth tokens in your library's documentation. Here's the code:
from rest_framework_jwt.settings import api_settings
jwt_payload_handler = api_settings.JWT_PAYLOAD_HANDLER
jwt_encode_handler = api_settings.JWT_ENCODE_HANDLER
payload = jwt_payload_handler(user)
token = jwt_encode_handler(payload)
i suggest use json.dumps({key:value})

Why Pocket API returns 403 Forbidden always?

I'm trying to call this line:
curl https://getpocket.com/v3/oauth/authorize --insecure -X POST -H "Content-Type: application/json" -H "X-Accept: application/json" -d "{\"consumer_key\":\"61999-492f79db0bd3292f0b4...1\",\"code\":\"c9166709-0c45-2b1f-a22f-e...r\"}"
and each time I get 403 Forbidden.
I do not know and understand the reason of that.
Does anyone knows? I tried it through Python too:
import requests
auth_params = {'consumer_key': 'key_here', 'redirect_uri': 'https://www.twitter.com/'}
tkn = requests.post('https://getpocket.com/v3/oauth/request', data=auth_params)
tkn.content
Above code gives me a code:
usr_params = {'consumer_key': 'key_here', 'code': 'code_here'}
usr = requests.post('https://getpocket.com/v3/oauth/authorize', data=usr_params)
usr.content
here I'm getting 403 too.
How can I fix that?
From Pocket Authentication API Documentation, you need to register an application to get a consumer key, then request OAuth token via :
curl -X POST \
-H 'Content-Type: application/json' \
-d '{"consumer_key":"XXXXX-XXXXXXXXXXXXXXXXXXXXXX","redirect_uri":"AppTest:authorizationFinished"}' \
https://getpocket.com/v3/oauth/request
Then the step 2 is to authorize this request token (this is the step you are missing). On a browser open the following URL with the request token you got from the previous step :
https://getpocket.com/auth/authorize?request_token=XXXXXXXX-XXXX-XXXX-XXXX-XXXX&redirect_uri=AppTest:authorizationFinished
Click on "authorize" :
Once the request token is authorized, you can call your request on https://getpocket.com/v3/oauth/authorize to convert a request token into a Pocket access token:
curl -X POST \
-H 'Content-Type: application/json' \
-d '{"consumer_key":"XXXXX-XXXXXXXXXXXXXXXXXXX","code":"XXXXXXXXX-XXXX-XXXX-XXXX-XXXXX"}' \
https://getpocket.com/v3/oauth/authorize
The consumer key is the one you got when you created the app on Pocket and the request token the one generated from v3/oauth/request endpoint
Then you get as expected :
{ "access_token":"5678defg-5678-defg-5678-defg56", "username":"pocketuser" }

Django JWT Auth, why one request works and the other not

I was trying Django JWT Auth and noticed that the URL responds well to one type of post but doesn't respond well to another, but i can figure out why.
Basically, if i use the cURL POST referred in the readme.md, everything goes accordingly to planned:
$ curl -X POST -H "Content-Type: application/json" -d '{"username":"admin","password":"abc123"}' http://localhost:8000/api-token-auth/
but if you i use another type of cURL POST with the same info, it doesn't work:
$ curl -d 'username=admin&password=abc123' http://localhost:8000/api-token-auth/
I know that the "Content-Type" is diferent, but shouldn't the request be accepted in the same manner, they are both well formed posts?
Curl's -d option actually sends the request like it's a web browser. My guess is that the URL you're testing against doesn't have a standard web form, so it can't actually process the request.
TL;DR Pretty sure Django JWT Auth doesn't support the application/x-www-form-urlencoded content type.
From curl manual:
-d --data
(HTTP) Sends the specified data in a POST request to the HTTP
server, in the same way that a browser does when a user has
filled in an HTML form and presses the submit button. This will
cause curl to pass the data to the server using the content-type
application/x-www-form-urlencoded. Compare to -F, --form.
Hope this helps!

Categories