We Keep Coding

How can I intercept database requests(Peewee ORM) in pytest?

I have a main database and a replica (they are the same in the test environment):
core_db = PooledPostgresqlExtDatabase(**DB_COFIG)
replica_db = PooledPostgresqlExtDatabase(**DB_REPLICA_COFIG)
A controllers that executes a query in different databases depending on the model
class BaseController:
def _get_logs(self):
query = self.model.select()
if is_instance(self.model, ModelToReplica)
query = query.bind(replica_db)
return list(query)
class ReplicaExampleController(BaseLogsController):
model = ModelToReplica
def process(self):
return self._get_logs()
class BaseExampleController(BaseLogsController):
model = BaseModel
def process(self):
return self._get_logs()
Controllers are linked to two urls:
/get_core_result/ # Returns the result from BaseExampleController (core_db)
/get_replica_result/ # Returns the result from ReplicaExampleController (replica_db)
I want to check that each of the corners accesses the right base. And I know that the reference to the database object is stored in the request object. How do I get it from the test? I'm using a PyTest. I understand that I probably need to use mock, but I don't understand how.
Unfortunately, this is all I have so far:
class TestSwitchDB:
def test_switch_db_to_replica():
url_core = url_for('core_db_controller')
core_result = self.client.get(url_core)
url_replica = url_for('replica_db_controller')
replica_result = self.client.get(url_replica)

In test_switch_db_to_replica you can mock the db query and patch it, something
with patch(BaseController._get_logs) as mock_get_logs:
mock_get_logs.return_value="your expected return"
As a reference: https://docs.python.org/3/library/unittest.mock-examples.html

Django post save signal not updating the database

Im not getting why this signal is not working. This same code worked once but after that i deleted the objects from admin and ran it again and it stopped working.
#receiver(post_save, sender=FinancePending)
def calcualate_FinancePending(sender, instance, created, **kwargs):
amount_paid = FinancePending.objects.values_list('amountPaid', flat=True)
amount_paid = list(amount_paid)
total_amount = FinancePending.objects.values_list('TotalAmount', flat=True)
total_amount = list(total_amount)
# total - paid
TotalFee = [int(s.replace(',', '')) for s in total_amount]
AmountPaid = [int(s.replace(',', '')) for s in amount_paid]
finance_pending = FinancePending.objects.all()
i = 1
while i <= len(TotalFee):
amount_pending = TotalFee[i-1] - AmountPaid[i-1]
amountpending = FinancePending.objects.filter(invoiceNumber=i)
amountpending.AmountPending = amount_pending
i = 1 + i

You did not call save() method, that is why it is not saving. But I do not think it is a optimized implementation from django's prespective. You can simply try like this using update():
from django.db.models import F
#receiver(post_save, sender=FinancePending)
def calcualate_FinancePending(sender, instance, created, **kwargs):
FinancePending.objects.update(AmountPending=F('TotalAmount')-F('amountPaid'))
Also, it does not make sense to update each and every object one instance of FinancePending is created. Probably you should only update the object which was created. Like this:
#receiver(post_save, sender=FinancePending)
def calcualate_FinancePending(sender, instance, created, **kwargs):
instance.AmountPending=instance.TotalAmount-instance.amountPaid
instance.save()
Finally, please follow pep8 style guide when naming your attributes and function names.

Because signal post_save only triggered after call save() method. You should use post_delete instead.

Wagtail/Django: Query filter to return only the pages the user has acess/permissions?

I'm going through the documentation at: http://docs.wagtail.io/en/v2.7.1/reference/pages/queryset_reference.html.
Is there a filter to return only the pages the user has access to? I can only see public() and not_public().
I have some pages which privacy is set to Private (accessible to users in specific groups). And would like to exclude them from the query results.

There is no such filter in PageQuerySet. You can however create your own QuerySet that adds an authorized filter and use that. The following code comes from the Joyous events EventQuerySet and is based upon PageQuerySet.public_q and BaseViewRestriction.accept_request. It gets all the restrictions that could apply, excludes the ones that the user passes, and then filters out the pages with the remaining restrictions.
from wagtail.core.query import PageQuerySet
from wagtail.core.models import Page, PageManager, PageViewRestriction
class MyQuerySet(PageQuerySet):
def authorized_q(self, request):
PASSWORD = PageViewRestriction.PASSWORD
LOGIN = PageViewRestriction.LOGIN
GROUPS = PageViewRestriction.GROUPS
KEY = PageViewRestriction.passed_view_restrictions_session_key
restrictions = PageViewRestriction.objects.all()
passed = request.session.get(KEY, [])
if passed:
restrictions = restrictions.exclude(id__in=passed,
restriction_type=PASSWORD)
if request.user.is_authenticated:
restrictions = restrictions.exclude(restriction_type=LOGIN)
if request.user.is_superuser:
restrictions = restrictions.exclude(restriction_type=GROUPS)
else:
membership = request.user.groups.all()
if membership:
restrictions = restrictions.exclude(groups__in=membership,
restriction_type=GROUPS)
q = models.Q()
for restriction in restrictions:
q &= ~self.descendant_of_q(restriction.page, inclusive=True)
return q
def authorized(self, request):
self.request = request
if request is None:
return self
else:
return self.filter(self.authorized_q(request))
You could then set this to be your model's default QuerySet.
class MyPage(Page):
objects = PageManager.from_queryset(MyQuerySet)()
Then when filtering your MyPage objects you can say MyPage.objects.live().authorized(request).all()
Hope that is helpful. May contain bugs.

What is the proper way to delineate modules and classes in Python?

I am new to Python, and I'm starting to learn the basics of the code structure. I've got a basic app that I'm working on up on my Github.
For my simple app, I'm create a basic "Evernote-like" service which allows the user to create and edit a list of notes. In the early design, I have a Note object and a Notepad object, which is effectively a list of notes. Presently, I have the following file structure:
Notes.py
|
|------ Notepad (class)
|------ Note (class)
From my current understanding and implementation, this translates into the "Notes" module having a Notepad class and Note class, so when I do an import, I'm saying "from Notes import Notepad / from Notes import Note".
Is this the right approach? I feel, out of Java habit, that I should have a folder for Notes and the two classes as individual files.
My goal here is to understand what the best practice is.

As long as the classes are rather small put them into one file.
You can still move them later, if necessary.
Actually, it is rather common for larger projects to have a rather deep hierarchy but expose a more flat one to the user. So if you move things later but would like still have notes.Note even though the class Note moved deeper, it would be simple to just import note.path.to.module.Note into notes and the user can get it from there. You don't have to do that but you can. So even if you change your mind later but would like to keep the API, no problem.

I've been working in a similar application myself. I can't say this is the best possible approach, but it served me well. The classes are meant to interact with the database (context) when the user makes a request (http request, this is a webapp).
# -*- coding: utf-8 -*-
import json
import datetime
class Note ():
"""A note. This class is part of the data model and is instantiated every
time there access to the database"""
def __init__(self, noteid = 0, note = "", date = datetime.datetime.now(), context = None):
self.id = noteid
self.note = note
self.date = date
self.ctx = context #context holds the db connection and some globals
def get(self):
"""Get the current object from the database. This function needs the
instance to have an id"""
if id == 0:
raise self.ctx.ApplicationError(404, ("No note with id 0 exists"))
cursor = self.ctx.db.conn.cursor()
cursor.execute("select note, date from %s.notes where id=%s" %
(self.ctx.db.DB_NAME, str(self.id)))
data = cursor.fetchone()
if not data:
raise self.ctx.ApplicationError(404, ("No note with id "
+ self.id + " was found"))
self.note = data[0]
self.date = data[1]
return self
def insert(self, user):
"""This function inserts the object to the database. It can be an empty
note. User must be authenticated to add notes (authentication handled
elsewhere)"""
cursor = self.ctx.db.conn.cursor()
query = ("insert into %s.notes (note, owner) values ('%s', '%s')" %
(self.ctx.db.DB_NAME, str(self.note), str(user['id'])))
cursor.execute(query)
return self
def put(self):
"""Modify the current note in the database"""
cursor = self.ctx.db.conn.cursor()
query = ("update %s.notes set note = '%s' where id = %s" %
(self.ctx.db.DB_NAME, str(self.note), str(self.id)))
cursor.execute(query)
return self
def delete(self):
"""Delete the current note, by id"""
if self.id == 0:
raise self.ctx.ApplicationError(404, "No note with id 0 exists")
cursor = self.ctx.db.conn.cursor()
query = ("delete from %s.notes where id = %s" %
(self.ctx.db.DB_NAME, str(self.id)))
cursor.execute(query)
def toJson(self):
"""Returns a json string of the note object's data attributes"""
return json.dumps(self.toDict())
def toDict(self):
"""Returns a dict of the note object's data attributes"""
return {
"id" : self.id,
"note" : self.note,
"date" : self.date.strftime("%Y-%m-%d %H:%M:%S")
}
class NotesCollection():
"""This class handles the notes as a collection"""
collection = []
def get(self, user, context):
"""Populate the collection object and return it"""
cursor = context.db.conn.cursor()
cursor.execute("select id, note, date from %s.notes where owner=%s" %
(context.db.DB_NAME, str(user["id"])))
note = cursor.fetchone()
while note:
self.collection.append(Note(note[0], note[1],note[2]))
note = cursor.fetchone()
return self
def toJson(self):
"""Return a json string of the current collection"""
return json.dumps([note.toDict() for note in self.collection])
I personally use python as a "get it done" language, and don't bother myself with details. This shows in the code above. However one piece of advice: There are no private variables nor methods in python, so don't bother trying to create them. Make your life easier, code fast, get it done
Usage example:
class NotesCollection(BaseHandler):
#tornado.web.authenticated
def get(self):
"""Retrieve all notes from the current user and return a json object"""
allNotes = Note.NotesCollection().get(self.get_current_user(), settings["context"])
json = allNotes.toJson()
self.write(json)
#protected
#tornado.web.authenticated
def post(self):
"""Handles all post requests to /notes"""
requestType = self.get_argument("type", "POST")
ctx = settings["context"]
if requestType == "POST":
Note.Note(note = self.get_argument("note", ""),
context = ctx).insert(self.get_current_user())
elif requestType == "DELETE":
Note.Note(id = self.get_argument("id"), context = ctx).delete()
elif requestType == "PUT":
Note.Note(id = self.get_argument("id"),
note = self.get_argument("note"),
context = ctx).put()
else:
raise ApplicationError(405, "Method not allowed")
By using decorators I'm getting user authentication and error handling out of the main code. This makes it clearer and easier to mantain.

Passing in Session data from Flask to WTForms

So I'm having issues with passing in a list from my Flask session object into the WTForms that I'm using.
I'm trying to access the 'strategy' object (it's a list) within the session object.
The goal is to list all of the strategies that are associated with the current user.
Within the Flask app:
class Create_Indicator_Form(Form):
security = TextField('Ticker Name', [
validators.Required(),
validators.Length(min=1, max=6)])
mva_10 = BooleanField('10 day moving average')
mva_25 = BooleanField('25 day moving average')
strategy = SelectField('strategy', session['strategy'])
def validate_mva_10(form, field):
if form.mva_25.data is True and field.data is True:
raise ValidationError('You can only choose one reference')
if form.mva_25.data is False and field.data is False:
raise ValidationError('You must choose at least one reference')
#app.route('/create_indicator', methods=['GET', 'POST'])
def create_indicator():
check_if_logged_in()
f = request.form
create_indicator_form = Create_Indicator_Form(f)
if request.method == 'POST' and create_indicator_form.validate():
indicator_id = get_next_index('indicator', 'indicator_id')
ticker = create_indicator_form.security.data
if create_indicator_form.mva_10.data is True:
mva_10_day = 'Y'
mva_25_day = 'N'
else:
mva_10_day = 'N'
mva_25_day = 'Y'
row = [indicator_id, ticker, mva_10_day, mva_25_day]
add_data('indicator', row)
# adding relation
criteria_row = [session['strategy'][0], indicator_id]
add_data('criteria', criteria_row)
return redirect(url_for('home'))
create_indicator_form.strategies = session['strategy']
return render_template('create_indicator.html',
form=create_indicator_form)
When I try to run the flask app I'm thrown this error:
RuntimeError: working outside of request context
with a trace back to where I access the session object in the Create_Indicator_Form class.
I've realized since starting to try to fix this that it'd be more efficient to chose between the mva_10 and mva_25 with a select field but I would like to resolve this issue before refactoring.

Why
This is because the strategy = SelectField('strategy', session['strategy']) line is executed when the file is loaded, as it is part of the Create_Indicator_Form class definition.
At this time, you're indeed working outside of request context
How to fix it
Using a class factory will work here:
def CreateIndicatorForm():
class IndicatorForm(Form):
security = TextField('Ticker Name', [
validators.Required(),
validators.Length(min=1, max=6)])
mva_10 = BooleanField('10 day moving average')
mva_25 = BooleanField('25 day moving average')
strategy = SelectField('strategy', session['strategy'])
def validate_mva_10(form, field):
if form.mva_25.data is True and field.data is True:
raise ValidationError('You can only choose one reference')
if form.mva_25.data is False and field.data is False:
raise ValidationError('You must choose at least one reference')
return IndicatorForm
Be mindful that calling CreateIndicatorForm() returns a Form class, so to instantiate an actual form, you need to use: CreateIndicatorForm()() (i.e. "call" the class, like you would any other class to create a new instance).
When instantiating the form, you can (and should) pass arguments to the form constructor, such as the request data: CreateIndicatorForm()(request.form).
Also, making session['strategy'] an argument of the factory function would be better practice here - this would make the factory reusable outside of request context and would ensure you don't have to create a new form class for every single request.