I have a server implementing a python API. I am calling functions from a frontend that uses Angular.js. Is there any way to add an authentication key to my calls so that random people cannot see the key through the Angular exposed code? Maybe file structure? I am not really sure.
As long as you send the sensitive data outside, you are at risk. You can obfuscate your code so that first grade malicious users have a hard time finding the key, but basically breaking your security is just a matter of time as an attacker will have all the elements to analyse your protocol and exchanged data and design a malicious software that will mimic your original client.
One possible (although not unbreakable) solution would be to authenticate the users themselves so that you keep a little control over who is accessing the data and revoke infected accounts.
Related
TLDR
I am making a REST Session management solution for industrial automation purposes and need to automatically log into devices to perform configurations.
NOTE:
These devices are 99% of the time going to be isolated to private networks/VPNs (i.e., Will not have a public IP)
Dilemma
I am being tasked with creating a service that can store hardware device credentials so automated configurations (& metrics scraping) can be done. The hardware in question only allows REST Session logins via a POST method where the user and (unencrypted) password are sent in the message body. This returns a Session cookie that my service then stores (in memory).
The service in question consists of:
Linux (Ubuntu 20.04) server
FastAPI python backend
SQLITE3 embedded file DB
Storing Credentials?
My background is not in Security so this is all very new to me but it seems that I should prefer storing a hash (e.g., bcrypt) of my password in my DB for future verification however there will not be any future verification as this is all automated.
This brings me to what seems like is the only solution - hashing the password and using that as the salt to encrypt the password, then storing the hashed password in the DB for decryption purposes later. I know this provides almost 0 security given the DB is compromised but I am at a loss for alternate solutions. Given the DB is embedded, maybe there is some added assurance that the server itself would have to be compromised before the DB itself is compromised? I don't know if there is a technical "right" approach to this, maybe not, however if anyone has any advice I am all ears.
You should consider using a hardware security module (HSM). There are cloud alternatives (like AWS Secrets manager, an encrypted secrets repository based on keys stored in an actual HSM, AWS KMS). Or if your app is not hosted in a public cloud, you can consider buying an actual HSM too, but that's expensive. So it all comes down to the risk you want to accept vs the cost.
You can also consider building architecture to properly protect your secrets. If you build a secure secrets store service and apply appropriate protection (which would be too broad to describe for an answer here), you can at least provide auditing of secret usage, you can implement access control, you can easily revoke secrets, you can monitor usage patterns in that component and so on. Basically your secrets service would act like a very well protected "HSM", albeit it might not involve specialized hardware at all. This would not guarantee that secrets (secret encryption keys, typically) cannot ever be retrieved from the service like a real HSM would, but it would have many of the benefits as described above.
However, do note that applying appropriate protection is the key there - and that's not straightforward at all. One approach that you can take is model your potential attackers, list ways (attack paths) for compromising different aspects of different components, and then design protections against those, as long as it makes sense financially.
I'm working on a personal project which makes use of Python, FastAPI and a microservices architecture.
I want to learn more about security so I'm trying to add some into this. I have read through the fastapi security intro and it mostly makes sense to me.
One thing I'm not sure about though is going about handling this cleanly in a microservices architecture.
Let's assume I have 2 services, user service and bankAccount service. The user service is supposed to handle everything with regards to a new user registering on my site, to logging them in, etc. At this point, it shouldn't be too difficult to authenticate the user as the user service can access it's db.
The part where I'm not sure about the best way to go forward would be with the bankAccount service. If a user makes a request to an endpoint within that service, how should I go about authenticating/authorising them?
The two options I can think of are as follows:
Create an /authenticate endpoint which has the sole purpose of other services being able to call it. Then, create a wrapper function in bankAccount service which wraps every endpoint and calls the /authenticate endpoint before running it's function
Create an /authenticate endpoint which has the sole purpose of other services being able to call it. Then, using something like NGINX or some sort of gateway, have this called before sending the request to the bankAccount service.
I lack experience/knowledge in this area so I'm not sure which of these would be the better option. I am leaning towards 2 so that I don't have to copy the wrapper code from the bankAccount service to any new service I create, but I don't know anything about NGINX or other gateways so any advice on how best to proceed here would be appreciated.
I'm not an expert in the subject, since I started recently diving into the microservices topic. So, take what I'm saying with a pinch of salt.
JWT AUTH WITH PUBLIC AND PRIVATE KEY
One thing you could do, is to use JWT authentication in all of your microservices. Basically, every service is capable of decrypting/reading the JWT token, handle the necessary verifications and respond accordingly.
The authentication service would be the one in charge of generating the tokens, so the idea is to use asymmetric encryption, where one key, owned by the authentication service, is used to generate the tokens, while other (public) keys are used to assess the authenticity of the token provided by users. The public/private keys could also be a pair of public/private certificates.
This is no way a scalable approach, as all the public keys have to be updated in case of an update of the key. Also, if the content of the token or the checks that are to be performed on the key change, then all the microservices have to be updated accordingly, which can be a tedious and long process.
Unfortunately, I haven't got any occasion to dive deeper as the topic is not simple and experimenting approaches in production isn't a good idea.
If someone more experienced than me can fill in missing details or other approaches, feel free to edit this answer or to comment below and I'll try to learn and update my answer.
I am aware that these questions has been asked before several times separately, and most of the answers I've found are "Python is not easy to obfuscate, because that's the nature of the language. If you really need obfuscation, use another tool" and "At some point you need a tradeoff" (see How do I protect Python code and How to keep the OAuth consumer secret safe, and how to react when it's compromised?).
However, I have just made a small Python app which makes use of Twitter's API (and therefore needs OAuth). OAuth requires a Consumer Secret, which is to be kept away from users. The app needs that information, but the user should not be able to access it easily. If that information cannot be protected (and I am using obfuscation and protection as synonyms, because I do not know of any other way), what is the point of having a OAuth API for Python in the first place?
The question(s) then are:
Would it be possible to hardcode the secret in the app and then
obfuscate it in an effective manner?
If not, what would be the best way to use OAuth in Python? I have thought of "shipping" the encrypted consumer secret along with the app and using a hardcoded key to recover it, but the problem remains the same (how to protect the key); having the consumer secret in a server, and have the application retrieve it at start up (if information is sent unencrypted, it would be even easier for a malicious attacker to just use Wireshark and get the consumer secret from the network traffic than decompiling the bytecode, plus how could I make sure that I am sending that secret to my app and not to a malicious attacker? Any form of authentication I know would require having secret information in the app side, the problem remains the same); a mixture of both (have the server send the encryption key, same problems as before). The basic problem is the same: how can you have something secret if critical information cannot be hidden?
I have also seen comments saying that one should use a C/C++ extension for those critical parts, but I do not know anything about that, so if that were the answer, I'd appreciate some extra information.
If you want to deploy on servers (or laptop) you own, you can store secrets in env var or files. If you want to deploy to user, suggestion is that you, or your user should register an API key, generate ssl key, or similar.
You can code your own simple symetric crypt fucntion with a lot of data manipulation to make it harder to reverse.
It is unclear why you'd need to ship your OAuth key with the script. That would mean giving anyone access to your Twitter account, whether or not the key itself is obfuscated inside the app.
The more typical scenario is that you develop some Twitter client, and anyone who wants to run it locally will have to input their own OAuth token before being able to run it. You simply do not hardcode the token and require any user to supply the token.
I am a newbie and I want to do the following
I have service endpoints like
#login_required
#app.route('/')
def home():
pass
#login_required
#app.route('/add')
def add():
pass
#login_required
#app.route('/save')
def save():
pass
#login_required
#app.route('/delete')
def delete():
pass
I would like user to user to be authenticated while making these calls.
Question
How can I authenticate REST calls using python?
How do I make sure that if call lands to execute any of the endpoints, they are authenticated?
How do I basically do all authentication at the HTTP header level without saving any state so that it can scale better in future (Like Amazon S3), meaning any call might go to a different server and still be able to authenticate it self.
I am entirely new to REST world and don't really know how to achieve this.
Thank you
First, a question, are you authenticating a user, a client, or both?
For authenticating a client I like HTTP MAC Authentication for REST service authentication. Take a look at the Mozilla Services macauthlib and how it's used in their pyramid_macauth project. You should be able to learn from pyramid_macauth as an example in applying macauthlib to secure your services. A search to see if anyone else has tried this with Flask is a good idea, too.
For authenticating users and clients, perhaps take a look at OAuth 2.0 proper (HTTP MAC Auth is a related specification).
I had hoped to post more links, however, this is my first post and it seems I have to earn my way to more links in a response. :)
Security is not for noobs. Use a framework and rely on its implementation. Study the source code, read blogs and papers, and at some point you'll be able to architect your own system.
There are many things that may go wrong, and once you deploy a protocol you may not be able to come back without breaking existing clients.
That said, the usual way fot authenticating a request is by using a couple of tokens, usually called a public key and a private (secret) key. A variant is using the private key to generate a short lived session token. Another variant is using an API key specific per client. Anyway, this token is usually sent in a HTTP header (either a standard cookie or a custom one), but it's also possible to use the request body. Usually they are not appended to the URL because the secret may end in a log file. Also, you should pay attention to how and where store the secret key.
Depending on the channel (plain HTTP) you may want to use a HMAC to sign requests instead of sending secrets in the wild. You have to watch against replay attacks. Timing attacks are possible. Cryptographic collisions may be used to defeat your scheme. You may need tokens to avoid CSRF (this is not really needed if web browsers don't come into play, but you don't specify this)
Again, choose a framework and don't code anything by yourself. Broken software is usually ok to fix, but security holes can do real damages.
Looking at your API, it does not look like restful endpoints. The URI should represent a certain entity and not actions. For an instance if you are dealing with an entity such as user you could have yourdomain.com/user and perform various operations such as create, delete, update and fetch using HTTP verbs like POST, DELETE, PATCH and GET (Given that you use flask this can be achieved very easily).
In terms of security, I assume there are multiple schemes but the one which I have used is generating a session token given a key and secret via an initial authenticate call. I suggest you look for specialized online resources on generating key and secret pair as well as the session token.
In terms of scaling I guess your concern is that the sessions should not be specific to a given machine. The authentication data can be stored in a store separately from the HTTP front-ends. This way you can add additional webservers and scale your front-end or add additional data stores and scale either on a need basis.
The attack
One possible threat model, in the context of credential storage, is an attacker which has the ability to :
inspect any (user) process memory
read local (user) files
AFAIK, the consensus on this type of attack is that it's impossible to prevent (since the credentials must be stored in memory for the program to actually use them), but there's a couple of techniques to mitigate it:
minimize the amount of time the sensitive data is stored in memory
overwrite the memory as soon as the data is not needed anymore
mangle the data in memory, keep moving it, and other security through obscurity measures
Python in particular
The first technique is easy enough to implement, possibly through a keyring (hopefully kernel space storage)
The second one is not achievable at all without writing a C module, to the best of my knowledge (but I'd love to be proved wrong here, or to have a list of existing modules)
The third one is tricky.
In particular, python being a language with very powerful introspection and reflection capabilities, it's difficult to prevent access to the credentials to anyone which can execute python code in the interpreter process.
There seems to be a consensus that there's no way to enforce private attributes and that attempts at it will at best annoy other programmers who are using your code.
The question
Taking all this into consideration, how does one securely store authentication credentials using python? What are the best practices? Can something be done about the language "everything is public" philosophy? I know "we're all consenting adults here", but should we be forced to choose between sharing our passwords with an attacker and using another language?
There are two very different reasons why you might store authentication credentials:
To authenticate your user: For example, you only allow the user access to the services after the user authenticates to your program
To authenticate the program with another program or service: For example, the user starts your program which then accesses the user's email over the Internet using IMAP.
In the first case, you should never store the password (or an encrypted version of the password). Instead, you should hash the password with a high-quality salt and ensure that the hashing algorithm you use is computationally expensive (to prevent dictionary attacks) such as PBKDF2 or bcrypt. See Salted Password Hashing - Doing it Right for many more details. If you follow this approach, even if the hacker retrieves the salted, slow-hashed token, they can't do very much with it.
In the second case, there are a number of things done to make secret discovery harder (as you outline in your question), such as:
Keeping secrets encrypted until needed, decrypting on demand, then re-encrypting immediately after
Using address space randomization so each time the application runs, the keys are stored at a different address
Using the OS keystores
Using a "hard" language such as C/C++ rather than a VM-based, introspective language such as Java or Python
Such approaches are certainly better than nothing, but a skilled hacker will break it sooner or later.
Tokens
From a theoretical perspective, authentication is the act of proving that the person challenged is who they say they are. Traditionally, this is achieved with a shared secret (the password), but there are other ways to prove yourself, including:
Out-of-band authentication. For example, where I live, when I try to log into my internet bank, I receive a one-time password (OTP) as a SMS on my phone. In this method, I prove I am by virtue of owning a specific telephone number
Security token: To log in to a service, I have to press a button on my token to get a OTP which I then use as my password.
Other devices:
SmartCard, in particular as used by the US DoD where it is called the CAC. Python has a module called pyscard to interface to this
NFC device
And a more complete list here
The commonality between all these approaches is that the end-user controls these devices and the secrets never actually leave the token/card/phone, and certainly are never stored in your program. This makes them much more secure.
Session stealing
However (there is always a however):
Let us suppose you manage to secure the login so the hacker cannot access the security tokens. Now your application is happily interacting with the secured service. Unfortunately, if the hacker can run arbitrary executables on your computer, the hacker can hijack your session for example by injecting additional commands into your valid use of the service. In other words, while you have protected the password, it's entirely irrelevant because the hacker still gains access to the 'secured' resource.
This is a very real threat, as the multiple cross-site scripting attacks have shows (one example is U.S. Bank and Bank of America Websites Vulnerable, but there are countless more).
Secure proxy
As discussed above, there is a fundamental issue in keeping the credentials of an account on a third-party service or system so that the application can log onto it, especially if the only log-on approach is a username and password.
One way to partially mitigate this by delegating the communication to the service to a secure proxy, and develop a secure sign-on approach between the application and proxy. In this approach
The application uses a PKI scheme or two-factor authentication to sign onto the secure proxy
The user adds security credentials to the third-party system to the secure proxy. The credentials are never stored in the application
Later, when the application needs to access the third-party system, it sends a request to the proxy. The proxy logs on using the security credentials and makes the request, returning results to the application.
The disadvantages to this approach are:
The user may not want to trust the secure proxy with the storage of the credentials
The user may not trust the secure proxy with the data flowing through it to the third-party application
The application owner has additional infrastructure and hosting costs for running the proxy
Some answers
So, on to specific answers:
How does one securely store authentication credentials using python?
If storing a password for the application to authenticate the user, use a PBKDF2 algorithm, such as https://www.dlitz.net/software/python-pbkdf2/
If storing a password/security token to access another service, then there is no absolutely secure way.
However, consider switching authentication strategies to, for example the smartcard, using, eg, pyscard. You can use smartcards to both authenticate a user to the application, and also securely authenticate the application to another service with X.509 certs.
Can something be done about the language "everything is public" philosophy? I know "we're all consenting adults here", but should we be forced to choose between sharing our passwords with an attacker and using another language?
IMHO there is nothing wrong with writing a specific module in Python that does it's damnedest to hide the secret information, making it a right bugger for others to reuse (annoying other programmers is its purpose). You could even code large portions in C and link to it. However, don't do this for other modules for obvious reasons.
Ultimately, though, if the hacker has control over the computer, there is no privacy on the computer at all. Theoretical worst-case is that your program is running in a VM, and the hacker has complete access to all memory on the computer, including the BIOS and graphics card, and can step your application though authentication to discover its secrets.
Given no absolute privacy, the rest is just obfuscation, and the level of protection is simply how hard it is obfuscated vs. how much a skilled hacker wants the information. And we all know how that ends, even for custom hardware and billion-dollar products.
Using Python keyring
While this will quite securely manage the key with respect to other applications, all Python applications share access to the tokens. This is not in the slightest bit secure to the type of attack you are worried about.
I'm no expert in this field and am really just looking to solve the same problem that you are, but it looks like something like Hashicorp's Vault might be able to help out quite nicely.
In particular WRT to the problem of storing credentials for 3rd part services. e.g.:
In the modern world of API-driven everything, many systems also support programmatic creation of access credentials. Vault takes advantage of this support through a feature called dynamic secrets: secrets that are generated on-demand, and also support automatic revocation.
For Vault 0.1, Vault supports dynamically generating AWS, SQL, and Consul credentials.
More links:
Github
Vault Website
Use Cases