For those who read this later: This was part of a hobby project I was doing and back then I wasn't aware of such a thing called Let's Encrypt! Also, I didn't want to purchase an SSL cert; so I was trying to have SOME security without https. Signing does not replace encryption. If you wanna do what I wanted to do, don't. Get a cert from Let's Encrypt. It's Easier and more secure!
I am implementing a web service using Django for a mobile app. I decided to use Hash-MAC to authenticate requests (no Oath, no https) but the challenge is what should I use as the shared secret key?
First I considered using The user's password but it required to store passwords in plain text in server-side database. The other solution that came to my mind was salting and hashing password in the mobile app just the way that Django auth app does in order to compute hashed password on client-side.
What does Django use as salt? Are they "secret"? Is there any problem with sending them as plain-text to users? We don't have to keep salts as "secret"s in general, but Django may use something that should be kept secret as salts, I don't know, it is about how Django implemented this.
Users have to know their salts to compute a hashed password, so the server should provide them with it. An adversary can ask for all users salts and eventually (s)he can have all the salts (Even if there is a limit to the number of times someone can ask for this in a certain time period). Even though salts are not secret but I guess knowing "all" of them can be dangerous. (or maybe I am overly concerned about security!)
Requests format:
HTTP request header:
x-mac-digest: 1d186b9c0fd5cd393f23623f0d167f7b17ac7d1cd74d8442647991d61e756c19
HTTP request body:
{
"username": "mjafar",
... rest of request in json
}
Authenticating request (simplified):
hash_digest = request.META['HTTP_X_MAC_DIGEST']
request_body = request.body.decode('utf-8')
request_json = json.loads(request_body)
user = UserModel.objects.get(username=request_json['username'])
sharedKey = getSharedKey(user) # What should it return?
hash = hmac.new(sharedKey, request_body, hashlib.sha256).hexdigest()
if hash != hash_digest:
return HttpResponseBadRequest('MAC authentication failed')
HMAC is used to authenticate a block of messages. It's used to verify block of cipher/Text was not changed during the transmission. You will need to use asymmetric encryption (RSA, DH,..etc) to transfer your shared key.
Using a plain text without using digital signature is useless. You will be vulnerable to MITM attack. Unless you manage to put the shared secret on the mobile app without internet (GSM SIM cards have shared key inside the sim itself and it's used to encrypt GSM calls).
Django uses random function to generate its secret_key. Random function on Linux will call /dev/urandom, on windows will call what's equivalent to that.
In your case, create a rest api to create a username and password, then return a hash value to be used to access your views.
If you want to compute a custom hash value, you can call make_password function, and save it directly in the password attribute in User
Read this: https://docs.djangoproject.com/en/1.8/topics/auth/passwords/
Example: Django make_password too slow for creating large list of users programatically
Related
I create a client (plain python script) and server (flask app) in Python. I wonder how to perform simple authentication using a access key in Python. Authentication is for scripts - they are not real users (no registration required). The key will be assigned to the script during implementation.
Scenario 1:
Client sends message to server.
Server (flask app) reads token from Authorization header.
Server looks for the token in the database.
If server finds token in database it will authenticate the client.
The access token is explicitly passed to the client before sending messages and the client stores it in an environment variable.
Disadventages:
Tokens are not stored as hashes
Scenario 2:
Client sends message to server.
Server (flask app) reads token (jwt) from Authorization header.
Server decodes jwt token to: {'accessKeyId': 'fake_id', 'accessKey': 'fake_key'}
Server looks for the key in database by id.
If server finds row in db and key matches to decoded server will authenticate the client.
Adventages:
Keys are stored as hashes
The access token is explicitly passed to the client too. It requires:
create {'accessKeyId': 'fake_id', 'accessKey': 'fake_key'} object,
hash 'fake_key' (and save to server database),
generate jwt token (finally it will passed for client and it will be in environment variable).
Is the 2nd approach correct? How is this implemented in applications where the user gets an access key after registration in web app and can query the api (via REST) if the key is correct - sth like OpenWeatherMaps?
Edit:
I found out that I need to implement the KEY API mechanism. Are there any guides on how to do this in Flask/Python?
This decision is very situation dependent. I've read over your question and personally, I'd go with scenario 1. Unless of course, you don't want the script to know its own key? But that defeats the purpose in some ways because you'd need to refresh that key since JWTs have an expiry value and common practice is to have them short-lived. It seems like scenario 2 just adds unnecessary complexity.
You're still validating the token against the database in both scenarios. Unless you're encrypting the JWT then you'll still be enabling users of that JWT to view that same token. If you want to encrypt it, then go for it...but you've just created an encrypted token-like header which just adds more overhead because instead of just going token to database you're going to JWT -> decrypt -> Token -> database on every request and you'll still need some sort of access token for them to regenerate that JWT.
If you have special claims to give them and want to communicate that via JWT then that's valid, but you'll still need them to be short-lived and they have to refresh them.
tl;dr I'd go with scenario 1, it just seems based on the example to be the better solution in my opinion.
Not sure how much you've looked into JWT's but this may be a good place to start.
I am planning design an encrypted time-limited API on both Client and Server sides, the server side is written in Django, the client side is a GUI program which call the API by
import requests
c = requests.post("http://127.0.0.1:8000/VideoParser/", data={'videoUrl': videoUrl })
The way it call the API is desperately exposed to those who can use network traffic capturing tools like wireshark and fiddler, while I don't want anyone else could call the API with their customized videoUrl, and if people made the post call with the same parameters 2 minutes later after the client initially made the call, the call should be valid or expired, so how to design the encrypted time-limited API on both Client and Server side in this case ?
P.S. I think add an identifier to the post data could prevent them using the API
import requests
c = requests.post("http://127.0.0.1:8000/VideoParser/", data={'videoUrl': videoUrl, 'identifier':value_of_identifier })
provided there is something encrypted in the value_of_identifier and it changes with each call, but I don't know how to get started, any idea ?
It would be better to show some code , I really don't know how to start to write code.
I think this will address your objectives:
Create an access key and a secret key for each user who is allowed to make requests.
Give one copy of each key to the user and keep one copy of each key on the server.
Add the following two parameters to each request: timestamp, and access key
Require that when the user sends a request, she creates a string of the post request, along with a timestamp parameter, and hash it using SHA256, using the secret key. (any hashing algo will do)
Create a parameter named Signature in your Post request, and use the hashed string as the value of the signature.
When the request reaches the server, use the access key to retrieve that user's secret key, and hash the request again. This will re-create the signature. The signature created at the server and one sent by the user must match.
If the two signatures don't match, discard. This way, anyone listening on the wire will have the url as well as the access keys, but will not be able to create a valid signature without the secret key associated with the access key.
Since there is a timestamp involved, the attacker will not be able to re-use a stolen signature.
Amazon uses this technique. http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
Writing a REST API with Pyramid/Cornice using JWT for authentication, I'll have to implement some CSRF protection.
Having thoroughly read up on the topic I understand the problem, but I'm pretty confused about the best way to implement it, it's a bit tricky considering all the possible attack vectors.
Since this API gives access to sensitive data and will be published as open source software, it requires a self-contained protection. It will be used in environments with untrusted subdomains and I can not rely on users to follow security guidelines.
For my stateless service I can either use "Double Submit Cookies" or the "Encrypted Token Pattern"-method.
Double Submit Cookies
To prevent "cookie tossing", the token in the Double Submit method needs to be verifiable. MiTM attacks are an additional threat, which I hope to mitigate sufficiently by forcing HTTPS-cookies only.
To get a verifiable token that can't be easily guessed and replicated by an attacker, I imagine a hashed token like this should work:
pbkdf2_sha256.encrypt($userid + $expire + $mycsrfsecret, salt=$salt)
"exp" is the expire-value from the JWT. The JWT will be issued together with the CSRF-cookie and "exp" can be read by the server, which adds some additional protection as it's variable and the attacker doesn't know it (Might be superfluous?).
On a request I can easily compare the two tokens I receive with each other and use pbkdf2_sha256.verify($tokenfromrequest, $userid + $exp + $mycsrfsecret) to compare it with the values from the JWT-token ('Verifiablity').
Would that approach follow recommended practices?
I've selected pbkdf2 over bcrypt since its verify-method is noticeably quicker.
Expiry would be set to 7 days, after that both the JWT and the CSRF-token would be renewed by a fresh login (They would also be renewed on an intermediate relogin).
Encrypted Token Pattern
The alternative is to send a string to the client, consisting of userid, expiry and nonce, encrypted with a server-side secret. On a request this string is sent along and the server can decrypt it and verify userid and expiry.
This seems the simpler approach, but I'm unsure how to implement it, I don't intend to roll my own crypto and I have not found good examples:
What cipher/library should I use in Python? How do I do Encrypt-then-MAC?
How would I persist the token until its natural expiration? I don't want the users to have to login freshly every time they restart their browsers. Local Storage is not a safe place - but there is no alternative.
Writing a REST API with Pyramid/Cornice using JWT for authentication
While I am not familiar with those frameworks, I suggest you ensure the JWT token is passed within a HTTP header (e.g. My-JWT-Token: ... ) which is NOT the cookie. Then you do not have to worry about the CSRF vector.
Cross Site Request Forgery is an issue due to the nature of the browser's tendency to always submit cookies, which often contain authentication information, to a particular domain. A browser will not automatically submit a custom header, ergo you do not have to worry.
Double Submit Cookies
Your method is overly complicated, you could simply use a GUID. Put that GUID in a cookie, and put it in any other part of the request. If they equal, CSRF check passed. You could also put the GUID into the JWT, then validate the GUID is also in a header/body/query parameter.
Encrypted Token Pattern
This is almost exactly what JWT is, just pass the token in the header as suggested 😄
To answer the questions:
I would suggest hmac as in import hmac. I would not bother encrypting but merely ensure there is no sensitive information in the token. Else PyCrypto may do you well.
This is why cookies exist, which does raise the CSRF issue again. If this is a hard requirement then I suggest the double submit cookie method.
As far as my knowledge says in digest authentication, a client does an irreversible computation, using the password and a random value supplied by the server as input values. The result is transmitted to the server who does the same computation and authenticates the client if he arrives at the same value. Since the computation is irreversible, an eavesdropper can't obtain the password.
Keeping eye on the above definition, I used CryptoJS.HmacSHA256("password", "key") in Javascript to send the information to django server, now the problem is:
I need to check that in server using same logic but django already has hashed the password in its own format, for example using pbkdf2_sha256.
Should I use some reversible algorithm like AES? I don't think it is possible to crack django's hashing algorithm and write the same for client side?
Why are you trying to do authentication in this manner? You can not use djangos default authentication for this.
What you could do though is (for example):
Create a new model, linked to the user, that has a shared secret key, and a token
Create your own authentication backend which accepts the signed value, validates the signed value, and then logs the user in
I used the details in this article to implement the algorithm. Even if you aren't using this method, you're still going to need to create your own custom authentication backend.
AES is not a hashing algorithm. It's an encryption algorithm.
You can use hashing algorithms like SHA1 or MD5.
I'm looking at using a crypto lib such as pycrypto for encrypting/decrypting fields in my python webapp db. But encryption algorithms require a key. If I have an unencrypted key in my source it seems silly to attempt encryption of db fields as on my server if someone has access to the db files they will also have access to my python sourcecode.
Is there a best-practice method of securing the key used? Or an alternative method of encrypting the db fields (at application not db level)?
UPDATE: the fields I am trying to secure are oauth tokens.
UPDATE: I guess there is no common way to avoid this. I think I'll need to encrypt the fields anyway as it's likely the db files will get backed up and moved around so at least I'll reduce the issue to a single vulnerable location - viewing my source code.
UPDATE: The oauth tokens need to be used for api calls while the user is offline, therefore using their password as a key is not suitable in this case.
If you are encrypting fields that you only need to verify (not recall), then simple hash with SHA or one-way encrypt with DES, or IDEA using a salt to prevent a rainbow table to actually reveal them. This is useful for passwords or other access secrets.
Python and webapps makes me think of GAE, so you may want something that is not doing an encrypt/decrypt on every DB transaction since these are already un-cheap on GAE.
Best practice for an encrypted databased is to encrypt the fields with the users own secret, but to include an asymmetric backdoor that encrypts the users secret key so you (and not anyone who has access to the DB source files, or the tables) can unencrypt the users key with your secret key, should recovery or something else necessitate.
In that case, the user (or you or trusted delegate) can retireve and unencrypt their own information only. You may want to be more stringent in validating user secrets if you are thinking you need to secure their fields by encryption.
In this regards, a passphrase (as opposed to a password) of some secret words such "in the jungle the mighty Jungle" is a good practice to encourage.
EDIT: Just saw your update. The best way to store OAuth is to give them a short lifespan, only request resources your need and re-request them over getting long tokens. It's better to design around getting authenticated, getting your access and getting out, than leaving the key under the backdoor for 10 years.
Since, if you need to recall OAuth when the user comes online, you can do as above and encrypt with a user specfic secret. You could also keygen from an encrypted counter (encrypted with the user secret) so the actual encryption key changes at each transaction, and the counter is stored in plaintext. But check specific crypto algo discussion of this mode before using. Some algorithms may not play nice with this.
Symmetric encryption is indeed useless, as you have noticed; however for certain fields, using asymmetric encryption or a trapdoor function may be usable:
if the web application does not need to read back the data, then use asymmetric encryption. This is useful e.g. for credit card data: your application would encrypt the data with the public key of the order processing system, which is on a separate machine that is not publically accessible.
if all you need is equality comparison, use a trapdoor function, such as a message digest, ideally with a salt value. This is good for passwords that should be unrecoverable on the server.
Before you can determine what crypto approach is the best, you have to think about what you are trying to protect and how much effort an attacker will be ready to put into getting the key/information from your system.
What is the attack scenario that you are trying to remedy by using crypto? A stolen database file?