I m trying to learn django and I'm to implement csrf token for some senstive actions.
But when I intercept the request/response I get csrf_token in every request in cookie field and the webpages where I have actually implemented the csrf_token, in those request I get another csrf token as csrfMiddleware parameter in data.
So I want to know why do I get two csrf_tokens in my request and response.
POST /demo/login/ HTTP/1.1
Host: xx.xx.xx.xx:8000
User-Agent: xxxxxx
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://xx.xx.xx.xx:8000/demo/login/
Content-Type: application/x-www-form-urlencoded
Content-Length: 125
Connection: close
Cookie: csrftoken=n4bSbWP8p9Uce3b5iAxI0UvDG0qQq7B3OwBXisww754LYztEm0wFf9ARLpVM2v7W
Upgrade-Insecure-Requests: 1
csrfmiddlewaretoken=VIqUszlij0OLyTgYEp7V2TRsNUtBqkISmaQZz52G1WYkipyxIP6Sh8WGSjYx2IeL&username=qwerty6&password=password%40123
According to the Django documentation:
For the value stored in the cookie:
In order to protect against BREACH attacks, the token is not simply the secret; a random salt is prepended to the secret and used to scramble it.
For the csrfmiddlewaretoken:
A hidden form field with the name csrfmiddlewaretoken present in all outgoing POST forms. The value of this field is, again, the value of the secret, with a salt which is both added to it and used to scramble it. The salt is regenerated on every call to get_token() so that the form field value is changed in every such response.
Therefore the values differ due to being scrambled in a different way.
Related
Normally, when sending a HTTP request, the actually traffic is like below:
GET /abc?hello HTTP/1.1
Host: localhost:8080
User-Agent: python-requests/2.7
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
However, I would like to send URLs without the leading slash, for example:
GET abc?hello HTTP/1.1
GET ftp://abc?hello HTTP/1.1
I understand that's not compliant with RFCs, but just need to send such request for testing purpose in Python.
Have checked requests, urllib, urllib2, urllib3, haven't figured out how to do it.
Anyone can help me out?
Where is the csrftoken stored?
When I access an API endpoint (logout API, it do not need the params):
POST /rest-auth/logout/ HTTP/1.1
Host: 10.10.10.105:8001
Connection: keep-alive
Content-Length: 0
Accept: application/json, text/plain, */*
Origin: http://localhost:8080
Authorization: Token 0fe2977498e51ed12ddc93026b08ab0b1a06a434
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36
Referer: http://localhost:8080/register
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cookie: sessionid=b95zopro0qvkrexj8kq6mzo1d3z2hvbl; csrftoken=z53lKL0f7VHkilYS5Ax8FMaQCU2ceouje9OeTJOgTy4gH0UgHVltAlOe2KFNNNB6
the header is upper. In the Response I get an error:
{"detail":"CSRF Failed: CSRF token missing or incorrect."}
So, the backend must have verified the csrftoken.
In the backend database, I can not find the csrftoken field:
So I want to know where it is saved in the encrypted session_data?
Given this QA in the django docs, you can see that the framework by default uses the Double Submit Cookie approach (rather than the synchronizer pattern).
This approach does not require the server to store the CSRF token, as the only check it does is comparing the token within the cookie with the one in the header (or parameter) and verify that they are equal.
The synhronizer pattern, on the other hand, does store the CSRF token somewhere in the server, and for each request it verifies its validity by comparing it with the one sent over the header ( or as before, in a POST parameter ).
You can read more about the two approaches here.
I guess you are testing your API with a web service testing application, in which case you are missing the second token somewhere in your request.
This section explains how to place the token for AJAX calls:
AJAX
While the above method can be used for AJAX POST requests, it has some inconveniences: you have to remember to pass the CSRF token in as POST data with every POST request. For this reason, there is an alternative method: on each XMLHttpRequest, set a custom X-CSRFToken header to the value of the CSRF token. This is often easier, because many JavaScript frameworks provide hooks that allow headers to be set on every request.
Seeing your request above, therefore you should place this header (with the value of the current token, of course):
X-CSRFToken: z53lKL0f7VHkilYS5Ax8FMaQCU2ceouje9OeTJOgTy4gH0UgHVltAlOe2KFNNNB6
This is a follow up on Security Dialogflow fulfillment thread.
the answer there
explore the req.headers.authorization you will find an authentication variable
(concat these three things:
Your dialogflow username
The character ':'
Your dialogflow password
and encode it in base64)
makes sense but in my python implementation the
request headers I get is:
Accept: */*
Content-Type: application/json; charset=UTF-8
Content-Length: 571
Host: xxxxxxxx
User-Agent: Apache-HttpClient/4.5.4 (Java/1.8.0_151)
Accept-Encoding: gzip,deflate
X-Forwarded-Proto: https
X-Forwarded-For: xx.xxxx.xx..xx
PS: I tried both V1 and V2
not sure how to take care of authorization
You have to set the basic auth fields in the Fullfilment settings (the ones below the Fullfilment-URL). Only then you will receive the base64 encoded part in the Authorization Header.
This has nothing to do with your personal credentials you use to login to dialogflow! Do not use them for basic auth!
I need to log into a website using python but the login page requires a sessionID cookie in the request header. Using Google developer tools along with a webclient(hurl.it), I was able to determine the required format of the request header that is acceptable by the webserver:
Accept: */*
Accept-Encoding: gzip, deflate
Content-Length: 85
Content-Type: application/x-www-form-urlencoded
Cookie: www_amsterdam-dance-event_nl_session=l9Abno8a1UyHPof%2BOyVqk8BxHjesGMi78z6Ot0ZXCCbI%2BxVKqjm30ALTfW%2FR7yKcDaqfEtFOyysTrjIeU8lU5ylv1TOlW6GLHY8jDfKKWSULKsUUJiTh92DbvkuYBuE6zt%2FeLs44lDna6Nz3uMCOaSARN7gCpoSz0TOcFaes8Hk9q6FikP1F9e%2B%2FsMwfUP0RTA0Rc5gJFyJPxHXNCdn%2BT49mhHYnzoIWVlxGHhlaEkZX1PPsYx1xq0BCgpb0WnPViuiZiBnQY2nz%2BBO4Uur0WPNfpSSWZg5Qxz79nYeChlRe16JhYjVOdaiUhnfEvp1jM7h%2BCdR6cUeatd7HGbftRCjINDrVuPeyB5ltVihStmzKEjOmWetI0xNuaNswsPIKKuo%2BV6JFNfdLcA6h3iy1K8o%2FA49tKGMP2rmGe4e5Jec%3Df395212364d1ffc80cf95ebf5abf3b40f9dc6441;
User-Agent: runscope/0.1
email=******%40beatswitch.com&login_token=545a46230b291&password=*****&submission=
I have produced the following request using Python requests module:
POST /my-ade/login/ HTTP/1.1
Host: www.amsterdam-dance-event.nl
Content-Length: 85
Accept-Encoding: gzip,deflate
Accept: */*
User-Agent: runscope/0.1
Connection: keep-alive
Cookie: www_amsterdam-dance-event_nl_session=l9Abno8a1UyHPof%2BOyVqk8BxHjesGMi78z6Ot0ZXCCbI%2BxVKqjm30ALTfW%2FR7yKcDaqfEtFOyysTrjIeU8lU5ylv1TOlW6GLHY8jDfKKWSULKsUUJiTh92DbvkuYBuE6zt%2FeLs44lDna6Nz3uMCOaSARN7gCpoSz0TOcFaes8Hk9q6FikP1F9e%2B%2FsMwfUP0RTA0Rc5gJFyJPxHXNCdn%2BT49mhHYnzoIWVlxGHhlaEkZX1PPsYx1xq0BCgpb0WnPViuiZiBnQY2nz%2BBO4Uur0WPNfpSSWZg5Qxz79nYeChlRe16JhYjVOdaiUhnfEvp1jM7h%2BCdR6cUeatd7HGbftRCjINDrVuPeyB5ltVihStmzKEjOmWetI0xNuaNswsPIKKuo%2BV6JFNfdLcA6h3iy1K8o%2FA49tKGMP2rmGe4e5Jec%3Df395212364d1ffc80cf95ebf5abf3b40f9dc6441;
Content-Type: application/x-www-form-urlencoded
login_token=545a46230b291&password=*****&email=******%40beatswitch.com&submission='
When I load the former request header with hurl.it, everything works perfectly and the webserver lets me log in but trying the almost-same request with the same parameters fails in python. While using python's request, the webserver presents an error page. Any help would be highly appreciated. I need a solution desperately.
EDIT:
Here is the code:
#Open the login page to get sessionID and login_token
loginURL = "https://www.amsterdam-dance-event.nl/my-ade/login/"
loginReq = session.get(loginURL)
loginSoup = BeautifulSoup(loginReq.text)
loginToken = loginSoup.find('input',attrs={'name':'login_token'})['value']
sessionID= loginReq.cookies['www_amsterdam-dance-event_nl_session']
cookie = 'www_amsterdam-dance-event_nl_session='+sessionID
#Construct the header and post it to the webserver
headers = {'Content-Length':'85','Accept':'*/*','User-Agent':' runscope/0.1','Content-Type':'application/x-www-form-urlencoded','Accept-Encoding':'gzip,deflate','Cookie':cookie}
payload = {'email':'*******#beatswitch.com','password':'********','login_token':loginToken,'submission':''}
loggedinReq = session.post(loginURL,headers=headers,data=payload)
I found the solution, thanks to Md. Mohsin. I was trying to handle the request headers and cookies manually while the requests module can handle them by itself. So I REMOVED the following line from the code and let requests take total control, and everything worked as intended:
headers = {'Content-Length':'85','Accept':'*/*','User-Agent':' runscope/0.1','Content-Type':'application/x-www-form-urlencoded','Accept-Encoding':'gzip,deflate','Cookie':cookie}
I'm trying to write Python code for Twitter OAuth authentication. I'm getting a "401 Unauthorized" error code when I attempt to request a token.
In the process of trying to diagnose my problem, I'm going thru each step of the authentication process and trying to undercover any errors I'm making. With regard to generating the "Signature Base String", I found an online tool that tries to help validate signature base strings: http://quonos.nl/oauthTester/
When I use this tool, it complains:
Bad URL encoding!
Both key and value in the POST body need to be URL encoded.
Here is an example Signature Base String that my Python code generates:
POST&https%3A%2F%2Fapi.twitter.com%2F1.1%2Foauth%2Frequest_token&oauth_callback%3Doob%26oauth_consumer_key%3DeXL46FKblmfiXHvmC3wcew%26oauth_nonce%3DTAHTO%2FmlyeJ1x9FrgFixosZPYVhvWLXmq%2BdKKTL1rTY%3D%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1391813822%26oauth_version%3D1.0
When I paste this string into the validator, it says:
Bad URL encoding!
Both key and value in the POST body need to be URL encoded.
In this case: "TAHTO/mlyeJ1x9FrgFixosZPYVhvWLXmq+dKKTL1rTY" is bad
I'm very confused because all key/value pairs in the URL are, in fact, URL encoded (I'm assuming "URL encoded" means "percent encoded" here.)
Is there anything wrong with my base string here?
Edit:
The actual HTTP request headers I'm sending to Twitter to request a token are:
POST /1.1/oauth/request_token HTTP/1.1
Accept-Encoding: identity
Content-Length: 0
Connection: close
Accept: */*
User-Agent: Python-urllib/3.2
Host: api.twitter.com
Content-Type: application/x-www-form-urlencoded format
Authorization: OAuth oauth_callback="oob", oauth_consumer_key="eXL46FKblmfiXHvmC3wcew", oauth_nonce="nBcVYSqv8FEi0d7MEs8%2BqtqvdYA9JcbnW%2BVqoP%2FGlrI%3D", oauth_signature="WT9c3U5Puam7dEnMt3DWDsyVAHw%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1391815422", oauth_version="1.0"