I'm working with the fixture module for the first time, trying to get a better set of fixture data so I can make our functional tests more complete.
I'm finding the fixture module a bit clunky, and I'm hoping there's a better way to do what I'm doing. This is a Flask/SQLAlchemy app in Python 2.7, and we're using nose as a test runner.
So I have a set of employees. Employees have roles. There are a few pages with rather complex permissions, and I'd like to make sure those are tested.
I created a DataSet that has each type of role (there are about 15 roles in our app):
class EmployeeData(DataSet):
class Meta:
storable = Employee
class engineer:
username = "engineer"
role = ROLE_ENGINEER
class manager:
username = "manager"
role = ROLE_MANAGER
class admin:
username = "admin"
role = ROLE_ADMIN
and what I'd like to do is write a functional test that checks only the right people can access a page. (The actual permissions are way more complicated, I just wanted a toy example to show you.)
Something like this:
def test_only_admin_can_see_this_page():
for employee in Employee.query.all():
login(employee)
with self.app.test_request_context('/'):
response = self.test_client.get(ADMIN_PAGE)
if employee.role == ROLE_ADMIN
eq_(200, response.status_code)
else:
eq_(401, response.status_code)
logout(employee)
Is there a way to generate the fixture data so my devs don't have to remember to add a line to the fixtures every time we add a role? We have the canonical list of all roles as configuration elsewhere in the app, so I have that.
I'm not wedded to any of this or the fixture module, so I'm happy to hear suggestions!
An option would be to use factory_boy to create your test data.
Assuming that you keep and update accordingly a list of roles (that will be used later on) like this one:
roles = [ROLE_ENGINEER, ROLE_ADMIN, ROLE_MANAGER, ...]
Let's create a factory for the Employee table:
import factory
from somewhere.in.the.app import roles
class EmployeeFactory(factory.alchemy.SQLAlchemyModelFactory):
class Meta:
model = Employee
sqlalchemy_session = session
username = factory.Sequence(lambda n: u'User %d' % n)
# Other attributes
...
# Now the role choice
role = factory.fuzzy.FuzzyChoice(roles)
The FuzzyChoice method takes a list of choices and makes a random choice from this list.
Now this will be able to create any amount of Employee objects on demand.
Using the factory:
from factory.location import EmployeeFactory
def test_only_admin_can_see_this_page():
EmployeeFactory.create_batch(size=100)
for employee in session.query(Employee).all():
login(employee)
with self.app.test_request_context('/'):
response = self.test_client.get(ADMIN_PAGE)
if employee.role == ROLE_ADMIN
eq_(200, response.status_code)
else:
eq_(401, response.status_code)
logout(employee)
Breakdown:
EmployeeFactory.create_batch(size=100) Creates 100 Employee objects in the test session.
We can access those objects from the factory session.
More information about using factory_boy with SQLAlchemy: https://factoryboy.readthedocs.io/en/latest/orms.html?highlight=sqlalchemy#sqlalchemy.
Be careful with session management especially: https://factoryboy.readthedocs.io/en/latest/orms.html?highlight=sqlalchemy#managing-sessions
Related
I'm currently working on a practice social media app. In this app, current users can invite their friends by email to join the app (specifically, joining a 'channel' of the app, like Discord). For this project, I'm working on functionality where a user will get an error if they try to invite someone who is already in the app (meaning people who are already in the app's database). I'm working on unit tests that ensures the error messages pop up when users are detected as already existing.
I managed to get my first scenario working, but I'm a bit stumped for the second one.
Here is a file that is central to both tests.
factories.py
class ChannelFactory(factory.django.DjangoModelFactory)
class Meta:
model = Channel
id = int
name = str
class CurrentUserFactory(factory.django.DjangoModelFactory)
class Meta:
model = CurrentUser
user_email = user_email
channel = models.ForeignKey(Channel)
Scenario #1 (currently working) - one new user is invited to join the app but already exists in the app's database
test_forms.py
from tests.factories import ChannelFactory, CurrentUserFactory
#pytest.mark.django_db
def test_that_current_user_cannot_be_reinvited(email, error_message):
"""user that is already in the specific channel cannot be reinvited"""
email = "user#test.com"
error_message = "user#test.com already exists in this channel"
# I am not specifying the channel name because the factory object is supposed to generate it automatically
current_user = CurrentUserFactory(user_email='user#test.com')
invite_form = forms.UserRequestForm({"email":email, channel=current_user.channel)
assert not invite_form is valid()
assert invite_form.errors["email"][0] = error_message
Result: Test passes!
However, the test passes mainly because there's only one user being tested.
So now, my task is to create a test to see what happens if several people are invited at once. The app allows for a comma separated string to be entered, so theoretically up to ten emails can be invited at a time.
Scenario #2 - two new users are invited to the same channel and both exist in the app's database. Here's where I'm running into issues because I need to somehow make sure the CurrentUsers are generated into the same channel.
from tests.factories import CurrentUserFactory
#pytest.mark.django_db
def tests_that_multiple_current_users_cannot_be_reinvited(emails, error_message):
"""users that are already in the specific channel cannot be reinvited"""
emails = ["user#test.com", "user2#test.com"]
error_message = "The following users already exist in this channel: user#test.com, user2#test.com"
#here, I attempt to "force" a Channel instance
channel = Channel(id="5", name="Hometown Friends")
current_users = [
(CurrentUserFactory(user_email='user#test.com', channel=channel)),
(CurrentUserFactory(user_email='user2#test.com', channel=channel)),
]
invite_form = forms.UserRequestForm({"email":emails, "channel":current_users.channel})
assert not invite_form is valid()
assert invite_form.errors["email"][0] = error_message
When I try running the test:
E AttributeError: 'tuple' object has no attribute 'channel'
I'm wondering if I can easily solve this by trying to work with the tuple, or if there's a much easier method that I'm somehow not seeing. Help would be greatly appreciated!
Can Django's user_passes_test() access view parameters?
For example I have view that receives an id to retrieve specific record:
def property(request, id):
property = Property.objects.get(id=int(id))
The record has a field named user_id that contains the id for user that originally created record. I want users to be able to view only their own records otherwise be redirected.
I'd like to use a custom decorator which seems simple and clean.
For custom decorator is there some variation of something like this that will work?
#user_passes_test(request.user.id = Property.objects.get(id=int(id)).id, login_url='/index/')
def property(request, id):
property = Property.objects.get(id=int(id))
I have tried creating separate test_func named user_is_property_owner to contain logic to compare current user to property record user_id
#user_passes_test(user_is_property_owner(id), login_url='/index/')
def property(request, id):
property = Property.objects.get(id=int(id))
def user_is_property_owner(property_id):
is_owner = False
try:
Property.objects.filter(id=property_id, user_id=user_id).exists()
is_owner = True
except Property.DoesNotExist:
pass
But having trouble getting current user id and the property id from the request into the user_is_property_owner decorator function.
EDIT to add solution I was using. I did test inside each view test was required. It is simple. i thought using a decorator might be prettier and slightly more simple.
def property(request, id):
# get object
property = Property.objects.get(id=int(id))
# test if request user is not user id on property record
if request.user.id != property.user_id:
# user is not same as property user id so redirect to index
return redirect('index')
# rest of the code if request user is user_id on property record
# eg it is ok to let user into view
Typically, (using class based views), I'll handle this in the get_queryset method so it would be something like
class PropertyDetail(DetailView):
def get_queryset(self):
return self.request.user.property_set.all()
and that will give a 404 if the property isn't for the current user. You might prefer to use a project like django-guardian if you end up with more permission relationships than just Property.
If you take a look at UserPassesTestMixin you'll see that it processes the test_func before calling dispatch so you'll have to call self.get_object(request) yourself if you decide to go that route.
Context: I am writing an API (using Flask and MongoEngine) with multiple account types, including perhaps buildings. I need the database to hold some temporary accounts until a particular building registers.
This is how I've been referencing just one type of user:
current_holder_of_stuff = ReferenceField(ActiveUser)
I know GenericReferenceField is also an option, but what if I only want to allow two types of references? Is there anything like:
current_holder_of_stuff = ReferenceField(ActiveUser, TempUser)
Muchos thankos!
It may work to create a parent class of type User and then have inherited classes of ActiveUser and TempUser to deal with the various user types. As for the requirement for current_holder_of_stuff to be two possible document types, you cannot use a single reference field. As you've dismissed using GenericReferenceField then one way might be to add a property method and a StringField with options such as this:
import mongoegine as mdb
class User(mdb.Document):
name = mdb.StringField()
meta = {'allow_inheritance': True}
class ActiveUser(User):
activation_date = mdb.DateTimeField()
class TempUser(User):
date_limit = mdb.DateTimeField()
class Building(mdb.Document):
address = mdb.StringField()
class Stuff(mdb.Document):
user = mdb.ReferenceField(User)
building = mdb.ReferenceField(Building)
currently_with = mdb.StringField(options=['user','building'],required=True)
#property
def current_holder_of_stuff(self):
if self.currently_with == "user":
return self.user
else:
return self.building
You can also use mongoengine's signals to perform checks pre-save to ensure there is only a user or building defined.
tl;dr: is it possible, with endpoints-proto-datastore, to receive a list with objects from a POST and insert it in the db?
Following the samples, when building my API i didn't got how could i let the users POST a list of objects so that i could be more efficient about putting a bunch of data in the db using ndb.put_multi, for example.
From this comment here at endpoints_proto_datastore.ndb.model i imagine that it is not possible with how it is designed. Am i right or i am missing something?
Extending the sample provided by endpoints achieved the desired with:
class Greeting(messages.Message):
message = messages.StringField(1)
class GreetingCollection(messages.Message):
items = messages.MessageField(Greeting, 1, repeated=True)
# then inside the endpoints.api class
#endpoints.method(GreetingCollection, GreetingCollection,
path='hellogretting', http_method='POST',
name='greetings.postGreeting')
def greetings_post(self, request):
result = [item for item in request.items]
return GreetingCollection(items=result)
-- edit --
See the docs about POSTing into the datastore, your only issue is that your models aren't EndpointsModels. Instead define a datastore model for both your Greeting and GreetingCollection:
from endpoints_proto_datastore.ndb import EndpointsModel
class Greeting(EndpointsModel):
message = ndb.StringProperty()
class GreetingCollection(EndpointsModel):
items = ndb.StructuredProperty(Greeting, repeated=True)
Once you've done this, you can use
class MyApi(remote.Service):
# ...
#GreetingCollection.method(path='hellogretting', http_method='POST',
name='greetings.postGreeting')
def greetings_post(self, my_collection):
ndb.put_multi(my_collection.items)
return my_collection
In my schema, as described in the below test data generation example, I want to know a good way to:
Dereference all instances of Favourites that have reference keys to instances of Pictures that have been deleted. Just delete any Favourite that links to a deleted picture.
The Person class is a user
The Picture class is something that can be a Favourite
The Favourite class is an example of the Link-Model way of having many-to-many relationships.
Why this question?
First I hope it doesn't fall out of the scope here, second because this can happen and third because it's interesting.
How?
Let's say that a person can have up to thousands favourites, something like Likes are on social networks or to make it worse, orders, accounts or invalid data in a scientific application.
In our example for some reason (and these reasons happen) a person is experiencing lot of dead favourite link, or I do know, that there are dead favourites.
What would be a good way to do this, reducing ndb.get() operations and not iterating through every Favourite.
Lets not complicate things. Lets make the assumption that we have only one user suffering from dead favourites. He has a class of Person and stubbed user_id property of '123'.
In the following example you can use the following handlers and their corresponding functions.
import time
import sys
import logging
import random
import cgi
import webapp2
from google.appengine.ext import ndb
class Person(ndb.Expando):
pass
class Picture(ndb.Expando):
pass
class Favourite(ndb.Expando):
user_id = ndb.StringProperty(required=True)
#picture = ndb.KeyProperty(kind=Picture, required=True)
pass
class GenerateDataHandler(webapp2.RequestHandler):
def get(self):
try:
number_of_models = abs(int(cgi.escape(self.request.get('n'))))
except:
number_of_models = 10
logging.info("GET ?n=parameter not defined. Using default.")
pass
user_id = '123' #stub
person = Person.query().filter(ndb.GenericProperty('user_id') == user_id).get()
if not person:
person = Person()
person.user_id = user_id #Stub
person.put()
logging.info("Created Person instance")
if not self._gen_data(person, number_of_models):
return
self.response.write("Data generated successfully")
def _gen_data(self, person, number_of_models):
first, last = Picture.allocate_ids(number_of_models)
picture_keys = [ndb.Key(Picture, id) for id in range(first, last+1)]
pictures = []
favourites = []
for picture_key in picture_keys:
picture = Picture(key=picture_key)
pictures.append(picture)
favourite = Favourite(parent=person.key,
user_id=person.user_id,
picture=picture_key
)
favourites.append(favourite)
entities = favourites
entities[1:1] = pictures
ndb.put_multi(entities)
return True
class CorruptDataHandler(webapp2.RequestHandler):
def get(self):
if not self._corrupt_data(0.5):#50% corruption
return
self.response.write("Data corruption completed successfully")
def _corrupt_data(self, n):
picture_keys = Picture.query().fetch(99999, keys_only=True)
random_picture_keys = random.sample(picture_keys, int(float(len(picture_keys))*n))
ndb.delete_multi(random_picture_keys)
return True
class FixDataHandler(webapp2.RequestHandler):
def get(self):
user_id = '123' #stub
person = Person.query().filter(ndb.GenericProperty('user_id') == user_id).get()
self._dereference(person)
def _dereference(self, person):
#Here if where you implement your answer
Separate handlers due to eventual consistency in
the NDB Datastore. More info:
GAE put_multi() entities using backend NDB
Of course I am posting an answer as well to show that I tried something before posting this.
A ReferenceProperty is just a key, so if you have the key of the deleted Person, you can use that to query the Favourite.
Otherwise, there's no easy way. You'll have to filter through all Favourites and find ones that have an invalid Picture. It's very simple in a mapreduce job, but could be an expensive query if you have a lot of Favourites.
You could use a pre delete hook (look here for a way to implement it)
Of course this could be done easier if you use the NDB API instead of the Datastore API (hooks on NDB), but then you'll have to change the way you make the referenes