We Keep Coding

django-ratelimit stack keys. Not the intended behaviour

I think my understanding of django-ratelimit is incorrect. I am using v3.0.0 but v2.0 produces the same results.
Lets say I have this code:
#ratelimit(key='post:username', rate='5/h', block=True)
#ratelimit(key='post:tenant', rate='5/h', block=True)
#csrf_exempt
def index(request):
print(request.POST["username"])
print(request.POST["tenant"])
print("")
return HttpResponse('hallo', content_type='text/plain', status=200)
Let's say tenant A submits username "Antwon" 6 times, then tenant A will be blocked for 1 hour, which is good. But, lets say tenant B also has a user "Antwon", then that user for tenant B will not be able to log in.
I would assume that Antwon for tenant B should still be able to log in, otherwise tenant A can DOS other tenants?
Is this intended behavior or is my implementation incorrect?

My understanding is incorrect. Here is a response from the creator of django-ratelimit:
Hi there! I think there might be some confusion about how multiple ratelimits interact, and how the cache keys get set/rotated.
Multiple limits are ORed together—that is, if you fail any of them,
you fail—but order matters. In this case, the outer limit,
post:username, is tested first, then the tenant is tested. If the user
succeeds, it counts as an attempt, regardless of what happens later.
That's why the user keeps accruing attempts. If you want tenant to
count first, you could re-order the decorators. However...
If you want a single limit to the (username, tenant) pair, you could
combine the two fields with custom logic, by creating a callable key
that returned some combination of tenant and username (maybe
{tenant}\u04{username} or hash them together).
In terms of locking out tenants, there are a couple of things:
First, ratelimit uses a staggered fixed-window strategy, instead of a
sliding windows. So based on the rate, the period, the key value, and
the group, there'll be some calculated window. For example, for 1
hour, 11:03:29 will be the reset time, and the next reset time for
that same combination will be 12:03:29, then 13:03:29, etc... The
downside is that you if you had a limit of 100/h, you could do 200
requests in a short span around the reset point. The upsides are that
the calculation is share-nothing and can be done independently, the
window reset even if you accidentally try again a little too early.
Second, yes if you're doing hard blocks on user-supplied data like
username (instead of e.g. using the authenticated user) it creates a
denial-of-service vector. Where possible, another option is to use
block=False, and do something like require a captcha rather than fully
blocking.

How do you design Page Object Model without duplicates?

I'm working on UI automating testing, using POM with Python and Selenium.
I want to know how to handle duplicate test cases.
For example, you have two webpages: Login page and Homepage.
I want to test three test cases.
Homepage functions without login: test_homepage_before_login.py
Login with valid/invalid username and password: test_login.py
Homepage functions with login: test_homepage_after_login.py
(1 and 3 have a lot in common. 3 has additional functions. 1 is subset of 3)
There are three files for each test case, and I already implemented 1 and 2. But for the third one, I just imported relevant functions from 1 and 2 modules.
The thing is validating login is duplicate. In this case, do you do login validation every time? Also do you give order or dependency when automating these cases by using pytest-ordering or pytest-dependency?
Another case I can think of is "logout". When you automate logout function, you need to log in first. In this case, do you add login validation beforehand again and implement logout? Do you give dependency in this case as well or just make scripts independent?
Thank you in advance.

You can use cookies to handle authentication. It will greatly speed up your tests. An example:
public void setAuthenticationCookies() {
Cookie at = new Cookie("Cookie_AccessToken", prop.getAccessToken(), "/", DatatypeConverter.parseDateTime("2030-01-01T12:00:00Z").getTime());
Cookie rt = new Cookie("Cookie_RefreshToken", prop.getRefreshToken(), "/", DatatypeConverter.parseDateTime("2030-01-01T12:00:00Z").getTime());
driver.manage().addCookie(at);
driver.manage().addCookie(rt);
}
For more: https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/Cookie.html
For logout issue, I advice you to login first then logout in order to make your tests independent of each other.

As this is a design question, there's no one right answer, but I'll give you some tips and guidelines that I follow.
I don't design each test to correspond to a page. Instead, I design short scenarios that verify behaviors and outcomes that are most relevant to the customer
Login is a very specific and special case. Usually most tests need to start with login in order to get to what they actually need to test, though they don't really care to verify the login process per se. In addition, you usually want to have some other tests that specifically verify the login process. For the first category of tests, you can perform the login either before each test and even before all the tests using a fixture. The fixture may verify that the login succeeded, just to prevent the test from continuing in case of a failure, but I don't consider it to be part of what the test validates. This fixture should perform the login in the simplest and most reliable way, because it's purpose is not find more bugs, but rather to help us get to where we really care about in the test. This means that you may use API or take any other shortcut in order to be in a logged in state.
For the tests that are specific to login I'll often combine them with the registration process, as the main purpose of the registration is that it will allow the user to login, and the success of logging in is the result of the registration process, so there's no much purpose to test one without the other.
Regarding reuse of code and page objects: following my first bullet, the process I follow is to write each test in a way that describes the scenario in the most readable fashion, using classes and methods that don't exists yet. Then I implement theses classes and methods to make the test pass. On the second test and on, if I realize that I already implemented an action, I reuse this code, and if I need something similar to something I already implemented, but not exactly the same, I refactor the original function to be usable in my new test, removing any duplication along the way. This help me design my code in a very reusable and duplication-free manner. Sometimes it leads me to create page objects, and other times it leads me to other design patterns. You can find a much more detailed explanation about this method including a step by step tutorial on my book Completed Guide to Test Automation.
HTH.

Lock the system

I am writing a mini-CRM system that two users can login at the same time and they can answer received messages. However, the problem is that they might response the same message because messages can only disappear when they click "Response" button. Is there any suggestion to me to lock the system?

This sounds like a great case for an 'optimistic locking' approach. Here are two methods I've used with much success. Often, I combine the two methods to ensure no data is lost by mis-matched object instances on POSTs.
The easy way: Add a version field to your model. On POST, check the POSTed version number vs. the object's version number. If they don't match, raise a validation error. If they do match, increment the version by 1.
More elegant approach: Django's Generic Relations (part of the content types framework). A table which stores the content_type and object_id of the object that's locked, along with the user who 'owns' that lock. Check this lock on GET requests, and disable POSTing if it's 'locked' by another user. 'Release' the lock on a page unload, session end, or browser exit. You can get very creative with this approach.

Add some boolean field (answered, is_answered.. etc) and check on every "Response" click if it answered.
Hope it will help.

Define the order of scenarios (or required scenario) with behave (python)

I'm using behave to test my little Django app.
I've already created the file user_management.feature that contains also this Scenario:
Scenario: register
Given I can access registration form
When I put "doctor" in "username" field
And I put "tardisBlue" in "password" field
And I put "doctor#tardis.com" in "email" field
And I press the "Register" button
Then the registration is successful
And I am logged in
Everythig works fine.
The next feature I want to develop is in file project_management.feature:
Scenario: create a project
Given I am logged in
When I go to the home page
And I click on "Create new Project" link
And I fill the fields
| field | text |
| name | Save Gallifrey |
And I click on "Save" button
And I go to the home page
Then I see the project name in the project list
Now when I execute my test, behave executes the feature files in alphabetical order, so project_management.feature is executed first.
It raise an error in the first given, because the user has not been created yet.
I've tested renamin the first file in 01_user_management.feature to make it work.
Do you know a better solution?
Is there some configuration file where I can specify the order of the feature file?
Or can I tell that a Scenario needs another Scenario tu run first?

You should not make scenarios dependent on one another. It is absolutely possible to do this. I have multiple large and complex test suites with hundred of scenarios. No scenario of mine depends on another scenario having run before it.
When you have a large suite and there's a single scenario failing, it is extremely useful to be able to do:
behave -n 'failing scenario name'
This gets Behave to run only the failing scenario. Alternatively, there's the #wip tag that can do the same thing. However, if the scenario you want to test depends on another one, Behave won't automatically know that it should run the other scenario so you have a) to know the dependency and b) manually take care of selecting all scenarios that the one you really want to run depends on.
What I'd do in your situation (which is pretty much what I've done in the past) is implement a step Given I am logged in as .... I implement it with a regex so that I can use
Given I am logged in as an administrator
Given I am logged in as a regular user
Given I am logged in as a user with permissions to delete articles
The application I'm testing has its database preloaded with some test users that correspond to the cases above. (There's still a test for registering new users but that's independent from the preloaded users.) The Given I am logged in as ... step just logs the user in. It does not need to create the user.
One of the side benefits of doing this is that if you run your suite on a test service like Sauce Labs or BrowserStack and use Selenium, you can implement the Given I am logged in as ... step to save a lot of testing time. Each Selenium command in such a case requires a round-trip between your Behave test and the browser running on the test service, which can take significant time transiting through the Internet. Reducing the number of such interactions can make a huge difference in the time it takes to run a whole suite.

There seems to be two ways to do this. One is that you can use Background to setup state for multiple Scenarios. The other is to call previous steps from other steps. The first solution would look something like this:
Feature: logins
Test login functionality
Background: login
Given I can access registration form
And I put "doctor" in "username" field
And I put "tardisBlue" in "password" field
And I put "doctor#tardis.com" in "email" field
And I press the "Register" button
Scenario: successful login
Then the registration is successful
And I am logged in
Scenario: create a project
When I go to the home page
And I click on "Create new Project" link
And I fill the fields
| field | text |
| name | Save Gallifrey |
And I click on "Save" button
And I go to the home page
Then I see the project name in the project list

Use feature-listfiles, like:
behave #my_foo.featureset …
with:
# -- FILE: my_foo.featureset
features/alice.feature
features/bob.feature:10

Answers from #Andrew Johnson and jenisys provides three ways to specify the order of steps/scenarios/features. And here I add a couple of more:
Basically same idea with listfile as jenisys mentioned, when specifying feature files in command line, features will be executed in the order as written. It seems not officially documented but this is what I observed.
One can also use tags to control what scenarios to run in command line. In the example of --tags=t1,t2,t3, those scenarios will be executed exactly the order written there. Handy.

How to setup a 3-tier web application project

EDIT:
I have added [MVC] and [design-patterns] tags to expand the audience for this question as it is more of a generic programming question than something that has direclty to do with Python or SQLalchemy. It applies to all applications with business logic and an ORM.
The basic question is if it is better to keep business logic in separate modules, or to add it to the classes that our ORM provides:
We have a flask/sqlalchemy project for which we have to setup a structure to work in. There are two valid opinions on how to set things up, and before the project really starts taking off we would like to make our minds up on one of them.
If any of you could give us some insights on which of the two would make more sense and why, and what the advantages/disadvantages would be, it would be greatly appreciated.
My example is an HTML letter that needs to be sent in bulk and/or displayed to a single user. The letter can have sections that display an invoice and/or a list of articles for the user it is addressed to.
Method 1:
Split the code into 3 tiers - 1st tier: web interface, 2nd tier: processing of the letter, 3rd tier: the models from the ORM (sqlalchemy).
The website will call a server side method in a class in the 2nd tier, the 2nd tier will loop through the users that need to get this letter and it will have internal methods that generate the HTML and replace some generic fields in the letter, with information for the current user. It also has internal methods to generate an invoice or a list of articles to be placed in the letter.
In this method, the 3rd tier is only used for fetching data from the database and perhaps some database related logic like generating a full name from a users' first name and last name. The 2nd tier performs most of the work.
Method 2:
Split the code into the same three tiers, but only perform the loop through the collection of users in the 2nd tier.
The methods for generating HTML, invoices and lists of articles are all added as methods to the model definitions in tier 3 that the ORM provides. The 2nd tier performs the loop, but the actual functionality is enclosed in the model classes in the 3rd tier.
We concluded that both methods could work, and both have pros and cons:
Method 1:
separates business logic completely from database access
prevents that importing an ORM model also imports a lot of methods/functionality that we might not need, also keeps the code for the model classes more compact.
might be easier to use when mocking out ORM models for testing
Method 2:
seems to be in line with the way Django does things in Python
allows simple access to methods: when a model instance is present, any function it
performs can be immediately called. (in my example: when I have a letter-instance available, I can directly call a method on it that generates the HTML for that letter)
you can pass instances around, having all appropriate methods at hand.

Normally, you use the MVC pattern for this kind of stuff, but most web frameworks in python have dropped the "Controller" part for since they believe that it is an unnecessary component. In my development I have realized, that this is somewhat true: I can live without it. That would leave you with two layers: The view and the model.
The question is where to put business logic now. In a practical sense, there are two ways of doing this, at least two ways in which I am confrontet with where to put logic:
Create special internal view methods that handle logic, that might be needed in more than one view, e.g. _process_list_data
Create functions that are related to a model, but not directly tied to a single instance inside a corresponding model module, e.g. check_login.
To elaborate: I use the first one for strictly display-related methods, i.e. they are somehow concerned with processing data for displaying purposes. My above example, _process_list_data lives inside a view class (which groups methods by purpose), but could also be a normal function in a module. It recieves some parameters, e.g. the data list and somehow formats it (for example it may add additional view parameters so the template can have less logic). It then returns the data set to the original view function which can either pass it along or process it further.
The second one is used for most other logic which I like to keep out of my direct view code for easier testing. My example of check_login does this: It is a function that is not directly tied to display output as its purpose is to check the users login credentials and decide to either return a user or report a login failure (by throwing an exception, return False or returning None). However, this functionality is not directly tied to a model either, so it cannot live inside an ORM class (well it could be a staticmethod for the User object). Instead it is just a function inside a module (remember, this is Python, you should use the simplest approach available, and functions are there for something)
To sum this up: Display logic in the view, all the other stuff in the model, since most logic is somehow tied to specific models. And if it is not, create a new module or package just for logic of this kind. This could be a separate module or even a package. For example, I often create a util module/package for helper functions, that are not directly tied for any view, model or else, for example a function to format dates that is called from the template but contains so much python could it would be ugly being defined inside a template.
Now we bring this logic to your task: Processing/Creation of letters. Since I don't know exactly what processing needs to be done, I can only give general recommendations based on my assumptions.
Let's say you have some data and want to bring it into a letter. So for example you have a list of articles and a costumer who bought these articles. In that case, you already have the data. The only thing that may need to be done before passing it to the template is reformatting it in such a way that the template can easily use it. For example it may be desired to order the purchased articles, for example by the amount, the price or the article number. This is something that is independent of the model, the order is now only display related (you could have specified the order already in your database query, but let's assume you didn't). In this case, this is an operation your view would do, so your template has the data ready formatted to be displayed.
Now let's say you want to get the data to create a specifc letter, for example a list of articles the user bough over time, together with the date when they were bought and other details. This would be the model's job, e.g. create a query, fetch the data and make sure it is has all the properties required for this specifc task.
Let's say in both cases you with to retrieve a price for the product and that price is determined by a base value and some percentages based on other properties: This would make sense as a model method, as it operates on a single product or order instance. You would then pass the model to the template and call the price method inside it. But you might as well reformat it in such a way, that the call is made already in the view and the template only gets tuples or dictionaries. This would make it easier to pass the same data out as an API (see below) but it might not necessarily be the easiest/best way.
A good rule for this decision is to ask yourself If I were to provide a JSON API additionally to my standard view, how would I need to modify my code to be as DRY as possible?. If theoretical is not enough at the start, build some APIs for the templates and see where you need to change things to the API makes sense next to the views themselves. You may never use this API and so it does not need to be perfect, but it can help you figure out how to structure your code. However, as you saw above, this doesn't necessarily mean that you should do preprocessing of the data in such a way that you only return things that can be turned into JSON, instead you might want to make some JSON specifc formatting for the API view.
So I went on a little longer than I intended, but I wanted to provide some examples to you because that is what I missed when I started and found out those things via trial and error.