We have built a Python based REST API. We are planning to give to other developers as well. Is there a Python library which could manage authentication keep track of API calls made be each client etc?
Well, ideally you'll be using public-key cryptography, supplying the developers with an API key and secret both. If the service is accessible via HTTPS to a limited number of consumers, you might be tempted to defer to issuing a simple API key alone, but your committing yourself to remain a small, closed and insecure service forever.
As for managing API calls themselves, since you have the RESTful interfaces developed already, I would suggest that you begin decorating the functions or methods to extract the service consumer and keep track of API calls in MonogoDB--it's simple and perfectly tuned for such a requirement. It would also allow you to start throttling consumer connections at the application level where, in time, you can develop the system to encompass some low-level solutions for managing service connections, such as iptables.
Related
I came across an issue with performance testing of payments-related endpoints.
Basically I want to test some endpoints that make request themselves to a 3rd-party providers' API.
Is it possible from Locust's tests level to mock those 3rd-party API for the endpoints I intend to actually test (so without interference with the tested endpoints)?
If I understand correctly, you have a service you'd like to load/performance test but that service calls out to a third-party. But when you do your testing, you don't want to actually make any calls to the third-party service?
Locust is used for simulating client behavior. You can define that client behavior to be whatever you want; typically it's primary use case is for making http calls but almost any task can be done.
If it's your client that makes a request to your service and then makes a separate request to the other third-party service for payment processing, yes, you could define some sort of mocking behavior in Locust to make a real call to your service and then mock out a payment call. But if it's your service that takes a client call and then makes its own call to the third-party payment service, no, Locust can't do anything about that.
For that scenario, you'd be best off making your own simple mock/proxy service of the third-party service. It would take a request from your service, do basic validation to ensure things are coming in as expected, and then just return some canned response that looks like what your service would expect from the third-party. But this would be something you'd have to host yourself and have a method of telling your service to point to this mock service instead (DNS setting, environment variable, etc.). Then you could use Locust to simulate your client behavior as normal and you can test your service in an isolated manner without making any actual calls to the third-party service.
I actually skipped the most important part of the issue, namely I am testing the endpoints from outside of the repo containing them (basically my load test repo calls my app repo). I ended up mocking the provider inside of the app repo, which I initially intended to avoid but turned out to be only reasonable solution at the moment.
I've been looking into the hyperledger indy framework and I wanted to start to build an app to get started but I noticed that there's the sdk that uses Libindy but there's also the Libvcx that is on top of Libindy but I don't know which one to use since they both seem to do the same.
As you've said, LibVCX is built on top of LibIndy.
LibIndy
Provides low level API to work with credentials and proofs. It provides operations to create create credential requests, credentials, proofs. It also exposes operations for communication with Hyperldger Indy ledger.
What Libindy doesn't handle is the credential exchange. If you write backend which issues credential and a mobile app which can request and receive credentials using Libindy, you'll have to come up with some communication protocol to do so. Is it gonna be HTTP? ZMQ? How are you going to format messages? This is what LibVCX does for you. You will also have to come up with solution how will you securely deliver messages and credentials from server to client when the client is offline.
LibVCX
LibVCX is one of several implementations of Hyperledger Aries specification. LibVCX is built on top of LibIndy and provides consumer with OOP-style API to manage connections, credentials, proofs, etc. It's written in Rust and has API Wrappers available for Python, Javascript, Java, iOS.
LibVCX was designed with asynchronicity in mind. LibVCX assumes existence of so called "Agency" between the 2 parties communicating - a proxy which implements certain Indy communication protocol, receives and forwards messages. Therefore your backend server can now issue and send a credential to someone whom it has talked to days ago. The credential will be securely stored in the agency and the receiver can check whether there's any new messages/credentials addressed for him at the agency.
You can think of agency as a sort of mail server. The message is stored there and the client can pull its messages/credentials and decrypt them locally.
What to use?
If you want to leverage tech in IndySDK perhaps for a specific use case and don't care about Aries, you can use vanilla libindy.
If you want to be interoperably exchange credentials with other apps and agents, you should comply with Aries protocol. LibVCX is one of the ways to achieve that.
The indy-sdk repository is the Indy software that enables building components (called agents) that can interact with an Indy ledger and with each other.
In 2019, at a "Connect-a-thon" in Utah, USA, developers from a variety of organizations gathered to demonstrate interoperability across a set of independently developed agent implementations. At that time, a further idea developed that led to the creation of Hyperledger Aries. What if we had agents that could use DIDs and verifiable credentials from multiple ecosystems? Aries is a toolkit designed for initiatives and solutions focused on creating, transmitting, storing and using verifiable digital credentials. At its core are protocols enabling connectivity between agents using secure messaging to exchange information.
Libvcx is a c-callable library built on top of libindy that provides a high-level credential exchange protocol. It simplifies creation of agent applications and provides better agent-2-agent interoperability for Hyperledger Indy infrastructure.
You need LibVCX if you want to be interoperably exchange credentials with other apps and agents, in others words if you want to be comply with Aries protocol.
In this case LibVCX Agency can be used with mediator agency which enables asynchronous communication between 2 parties.
I am currently working with MS Azure. There I have a worker role and a web role. In worker role I start an infinite loop to process some data continously. The web role is performing the interaction with the client. There I use a MVC Framework, which on server side is written in C# and on client side in Javascript.
Now I'm interested in GAE engine. I read a lot about the app engine. I want to build an application in Python. But I don't really understand the architecture. Is there a counterpart in the project structure like the worker and web role in Azure?
The closest thing to what you want is what Google App Engine calls modules. Modules are (roughly) pools of instances that can be set up with different runtimes and performance characteristics:
https://cloud.google.com/appengine/docs/python/modules/
I'm not an expert with Azure, but the big difference I see between GAE's approach and Azure's is that, unlike in Azure, "back-end modules" (not an official term) in GAE are still basically web services at heart. Everything in the module is still basically written as HTTP handlers. So, the main ways you control that module are via HTTP: using push queues to hit HTTP endpoints, using cron to trigger HTTP endpoints that read from pull queues/the datastore/Google Cloud Storage, and/or making HTTP requests from your "front-end module" directly to your "back-end module".
Note that Google App Engine historically provided the concept of "backends" and "backend instances" that you could use for much the same purpose as modules for longer-running background processes. However, the module system is more flexible, and it is now recommended.
yes there is. look at backend and frontend instances. your question is too broad to go into more detail. in general the backend type of instance is used for long running tasks but you could also do everyrhing in the frontend instance.
I would like build a REST API using the Django REST framework. Initially its client would be a web application, but conceivably future clients could include mobile applications.
Unfortunately I'm finding the list of authentication classes listed in the documentation a little confusing. It looks like TokenAuthentication would meet my needs. I would rather avoid the cognitive overhead of OAuth unless there is a compelling security reason to go that way.
This is a decision I want to get right at this very early stage. Can anyone provide any advice?
Edit: Although hopefully not relevant, I thought I'd mention I'll be using Neo4j as a back-end for the application, not a conventional SQL database.
Django REST Framework gives you the flexibility of having multiple authentication methods. Since I've got some time, and it will be useful to future visitors who have similar questions, I'll outline the benefits of the most common authentication methods.
Initially its client would be a web application, but conceivably future clients could include mobile applications.
Typically when working with web applications that are on the same domain and Django instance as the API, most people use SessionAuthentication as it interacts with the server using the existing authentication methods. Authentication works seamlessly, so you don't need to go through the second authentication step.
Most APIs also support some form of BasicAuthentication, most likely because it is the easiest to test with but also because it is the easiest to implement. For your web application, this isn't the recommended authentication method, but for your mobile application it's not uncommon to see it being used. I personally would recommend a token-based authentication, so you don't have to worry about clients intercepting user's credentials.
It looks like TokenAuthentication would meet my needs.
Many people use TokenAuthentication because it is relatively simple to understand and use, and it seems to meet everyone's needs at first. Tokens are directly attached to users, and they do not automatically rotate (though you can make them automatically rotate), so every client working on behalf of the user gets the same token. This can be an issue if you ever need to revoke the token, as all other clients will have their token invalidated as well.
I would rather avoid the cognitive overhead of OAuth unless there is a compelling security reason to go that way.
OAuth 2 (OAuth2Authentication) gives you token rotation and token expiration on top of the benefits of TokenAuthentication. There's also the benefit of being able to revoke individual tokens without affecting other clients who are authenticating for the user. You can also limit clients to individual areas of your API through the use of scopes, which is useful if you have certain areas of the API that are more often used than others.
I'm also going to mention JSON Web Tokens, because while I haven't used it, it's been showing up quite a bit in the support channels. It works very similar to TokenAuthentication as far as retrieving tokens, but it has the added benefit of unique tokens for clients and token expiration.
I have been looking into using ZODB as a persistence layer for a multiplayer video game. I quite like how seamlessly it integrates with arbitrary object-oriented data structures. However, I am stumbling over one issue, where I can't figure out, whether ZODB can resolve this for me.
Apparently, one can use the ClientStorage from ZEO to access a remote data storage used for persistence. While this is great in a trusted local network, one can't do this without proper authorization and authentication in an open network.
So I was wondering, if there is any chance to realize the following concept with ZODB:
On the server-side I would like to have a ZEO server running plus a simulation of the game world that might operate as a fully authorized client on the ZEO server (or use the same file storage as the ZEO server).
On the client side I'd need very restricted read/write access to the ZEO server, so that a client can only view the information its user is supposed to know about (e.g. the surrounding area of their character) and can only modify information related to the actions that their character can perform.
These restrictions would have to be imposed by the server using some sort of fine-grained authorisation scheme. So I would need to be able to tell the server whether user A has permissions to read/write object B.
Now is there way to do this in ZODB or third-party solutions for this kind of problem? Or is there a way to extend ZEO in this way?
No, ZEO was never designed for such use.
It is designed for scaling ZODB access across multiple processes instead, with authentication and authorisation left to the application on top of the data.
I would not use ZEO for anything beyond a local network anyway. Use a different protocol to handle communication between game clients and game server instead, keeping the ZODB server side only.