We Keep Coding

Why I get an error when adding __init__(self) method to Django rest framework viewset class?

I Am keep getting a error when trying to build a Django API.
I have this class:
from uuid import UUID
from django.shortcuts import render
from django.http.response import JsonResponse
from django.http.request import HttpRequest
from rest_framework import viewsets, status
from rest_framework.parsers import JSONParser
from rest_framework.response import Response
from Instruments import serializers
from Instruments.services import InstrumentsService
from rest_framework.decorators import api_view
from Instruments.services import InstrumentsService
from Instruments.models import Instrument
from Instruments.serializers import InstrumentsSerializer
# Application views live here
class InstrumentViewSet(viewsets.ViewSet):
# instruments = Instrument.objects.all()
def __init__(self):
# self.instrument_service = InstrumentsService()
# self.instruments = Instrument.objects.all()
super().__init__()
def list(self, request: HttpRequest):
try:
self.instruments = Instrument.objects.all()
serializer = InstrumentsSerializer(self.instruments, many=True)
# data = self.instrument_service.get_instruments()
data = serializer.data
return JsonResponse(data, status=status.HTTP_200_OK, safe=False)
except Exception as exc:
return JsonResponse(
{"Status": f"Error: {exc}"},
status=status.HTTP_400_BAD_REQUEST,
safe=False,
)
when the init() method is defining even if it is just doing pass the django server gives me this error when I send a request:
TypeError at /api/
__init__() got an unexpected keyword argument 'suffix'
If I remove or comment out the init() method it works.why??

The “got an unexpected keyword argument” exception is rather descriptive. What it means is your class instance (in this case, your ViewSet instance) was initialized with a keyword argument that you’re not handling. That can be fixed by doing the following in your init method:
def __init__(self, **kwargs):
# self.instrument_service = InstrumentsService()
# self.instruments = Instrument.objects.all()
super().__init__(**kwargs)
This is required because the super class (View) utilizes **kwargs when the instance is initialized.
For the record, this is not using Django as it was intended. Django was never meant for service layers and using the init method like this is counterproductive since a ViewSet takes a queryset variable. I would encourage you to read the documentation much more thoroughly before continuing with this project of yours.

Django Test Object created is empty

I am trying to follow instructions from the Django documentation:
https://docs.djangoproject.com/en/3.2/topics/testing/overview/
However when I try to create the object for the test. The object is empty??
This is my code:
from django.test import TestCase, Client
from django.urls import reverse
from .models import ShortURL
from .views import generate_shortURL, redirect, home
from datetime import datetime
url = "https://stackoverflow.com/"
time = datetime.now()
class TestURLShortener(TestCase):
def setup(self):
self.client = Client()
obj= ShortURL.objects.create(original_url=url, short_url='zqSkSQ', time_date_created=time, count=2)
obj.save()
def test_creating_short_URL_POST(self):
"""
Test to create short Urls
"""
short_url_from_db = ShortURL.objects.all()
print(f'short_url_from_db : {short_url_from_db}')
response = self.client.post(reverse('generate_shortURL'), data={'original_url': url})
generated_short_url = response.context["chars"]
self.assertEquals(generated_short_url, 'afasdf')
This is the results when I run the test:
short_url_from_db prints out this <QuerySet []> instead of the object I wanted it to print out from the setup function.
How can I get the object I created to use in this test?

You need to use setUp and not setup as the function for setting up your test case.
You also don't need to call save() if you use create().
An alternative you could make use of is setUpTestData()
This technique allows for faster tests as compared to using setUp().
class TestURLShortener(TestCase):
#classmethod
def setUpTestData(cls):
# Set up data for the whole TestCase
cls.obj = ShortURL.objects.create(original_url=url, short_url='zqSkSQ', time_date_created=time, count=2)
...
def setUp(self):
self.client = Client()
def test_obj_type(self):
self.assertTrue(isinstance(self.obj, ShortURL))

Authenticate Flask unit test client from another service (microservices architecture)?

Problem:
So my problem is I have a Flask microservice want to implement the unit tests to it so when I start writing my test cases I found that I need to authenticate the unit test client because of some endpoints need authorization and here comes the problem the whole authentication system in another service this service all can do about the authentication is to validate the JWT token and get user ID from it so here is one of the views.py
from flask_restful import Resource
from common.decorators import authorize
class PointsView(Resource):
decorators = [authorize]
def get(self, user):
result = {"points": user.active_points}
return result
and authorize decorator from decorators.py
import flask
import jwt
from jwt.exceptions import DecodeError, InvalidSignatureError
from functools import wraps
from flask import request
from flask import current_app as app
from app import db
from common.models import User
from common.utils import generate_error_response
def authorize(f):
"""This decorator for validate the logged in user """
#wraps(f)
def decorated_function(*args, **kwargs):
if 'Authorization' not in request.headers:
return "Unable to log in with provided credentials.", 403
raw_token = request.headers.get('Authorization')
if raw_token[0:3] != 'JWT':
return generate_error_response("Unable to log in with provided credentials.", 403)
token = str.replace(str(raw_token), 'JWT ', '')
try:
data = jwt_decode_handler(token)
except (DecodeError, InvalidSignatureError):
return generate_error_response("Unable to log in with provided credentials.", 403)
user = User.query.filter_by(id=int(data['user_id'])).first()
return f(user, *args, **kwargs)
return decorated_function
and the test case from tests.py
import unittest
from app import create_app, db
from common.models import User
class TestMixin(object):
"""
Methods to help all or most Test Cases
"""
def __init__(self):
self.user = None
""" User Fixture for testing """
def user_test_setup(self):
self.user = User(
username="user1",
active_points=0
)
db.session.add(self.user)
db.session.commit()
def user_test_teardown(self):
db.session.query(User).delete()
db.session.commit()
class PointsTestCase(unittest.TestCase, TestMixin):
"""This class represents the points test case"""
def setUp(self):
"""Define test variables and initialize app."""
self.app = create_app("testing")
self.client = self.app.test_client
with self.app.app_context():
self.user_test_setup()
def test_get_points(self):
"""Test API can create a points (GET request)"""
res = self.client().get('/user/points/')
self.assertEqual(res.status_code, 200)
self.assertEquals(res.data, {"active_points": 0})
def tearDown(self):
with self.app.app_context():
self.user_test_teardown()
# Make the tests conveniently executable
if __name__ == "__main__":
unittest.main()
My authentication system work as the following:
Any service (include this one) request User service to get user JWT
token
Any service take the JWT token decoded and get the user ID
from it
Get the user object from the database by his ID
so I didn't know how to make the authentication flow in the test cases.

Here is just an example. I skipped some little things such as create_app, jwt.decode(token) etc. I'm sure you can understand the main approach. Structure:
src
├── __init__.py # empty
├── app.py
└── auth_example.py
app.py:
from flask import Flask
from src.auth_example import current_identity, authorize
app = Flask(__name__)
#app.route('/')
#authorize()
def main():
"""
You can use flask_restful - doesn't matter
Do here all what you need:
user = User.query.filter_by(id=int(current_identity['user_id'])).first()
etc..
just demo - return current user_id
"""
return current_identity['user_id']
auth_example.py:
from flask import request, _request_ctx_stack
from functools import wraps
from werkzeug.local import LocalProxy
current_identity = LocalProxy(lambda: getattr(_request_ctx_stack.top, 'current_identity', None))
def jwt_decode_handler(token):
"""
just do here all what you need. Should return current user data
:param str token:
:return: dict
"""
# return jwt.decode(token), but now - just demo
raise Exception('just demo')
def authorize():
def _authorize(f):
#wraps(f)
def __authorize(*args, **kwargs):
if 'Authorization' not in request.headers:
return "Unable to log in with provided credentials.", 403
raw_token = request.headers.get('Authorization')
if raw_token[0:3] != 'JWT':
return "Unable to log in with provided credentials.", 403
token = str.replace(str(raw_token), 'JWT ', '')
try:
# I don't know do you use Flask-JWT or not
# this is doesn't matter - all what you need is just to mock jwt_decode_handler result
_request_ctx_stack.top.current_identity = jwt_decode_handler(token)
except Exception:
return "Unable to log in with provided credentials.", 403
return f(*args, **kwargs)
return __authorize
return _authorize
Our test:
import unittest
from mock import patch
from src.app import app
app.app_context().push()
class TestExample(unittest.TestCase):
def test_main_403(self):
# just a demo that #authorize works fine
result = app.test_client().get('/')
self.assertEqual(result.status_code, 403)
def test_main_ok(self):
expected = '1'
# we say that jwt_decode_handler will return {'user_id': '1'}
patcher = patch('src.auth_example.jwt_decode_handler', return_value={'user_id': expected})
patcher.start()
result = app.test_client().get(
'/',
# send a header to skip errors in the __authorize
headers={
'Authorization': 'JWT=blabla',
},
)
# as you can see current_identity['user_id'] is '1' (so, it was mocked in view)
self.assertEqual(result.data, expected)
patcher.stop()
So, in your case you need just mock jwt_decode_handler. Also I recommend do not add any additional arguments inside a decorators. It will be hard to debugging when you have more than two decorators with a different arguments, recursion, hard processing etc.
Hope this helps.

Could you create some mock tokens in your unit testing framework (that your decorator can actually decode like in a real request) and send them in with your test client? An example of how that might look can be seen here: https://github.com/vimalloc/flask-jwt-extended/blob/master/tests/test_view_decorators.py#L321

Can't get Pyramid to recognize Mongo

I am trying to get the Pyramid Web framework to handle a request using Mongo but I am a relative newbie to both. I cannot get my view to recognize a database attached to a request.
In development.ini:
###
# configure mongodb
###
mongo_uri = mongodb://localhost:27017/nomad
The __init__.py imports and main function:
# imports for Mongodb
from urllib.parse import urlparse
from gridfs import GridFS
from pymongo import MongoClient
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
config = Configurator(settings=settings)
init_includes(config)
init_routing(config)
db_url = urlparse(settings['mongo_uri'])
config.registry.db = MongoClient(
host=db_url.hostname,
port=db_url.port,
)
def add_db(request):
db = config.registry.db[db_url.path[1:]]
if db_url.username and db_url.password:
db.authenticate(db_url.username, db_url.password)
return db
def add_fs(request):
return GridFS(request.db)
config.add_request_method(add_db, 'db', reify=True)
config.add_request_method(add_fs, 'fs', reify=True)
config.scan()
return config.make_wsgi_app()
In jobscontroller.py, which is the handler view making the request:
import pyramid_handlers
from nomad.controllers.base_controller import BaseController
class JobsController(BaseController):
#pyramid_handlers.action(renderer='templates/jobs/index.pt')
def index(request):
all_jobs = request.db['jobs'].find()
return {'all_jobs': all_jobs}
I get an error:
all_jobs = request.db['jobs'].find()
AttributeError: 'JobsController' object has no attribute 'db'
I am using Pyramid handlers to manage routing and views, and I know that all of this works because all my routes resolve and deliver web pages. It's only the jobs controller that's funky, and only after I tried adding that request.db call.
Can someone help me understand what's going on?

You're not referring to the request - you're referring to the object itself (usually named self, but you have named it request - which would work if it was just a function and not a method on an object). Since you're inside an object of a class, the first parameter is always the object itself:
class JobsController(BaseController):
#pyramid_handlers.action(renderer='templates/jobs/index.pt')
def index(self, request):
all_jobs = request.db['jobs'].find()
return {'all_jobs': all_jobs}

Django Testing - loading data into the database before loading the apps

I'm currently writting some tests for a Django app (+ REST framework), and having some issues loading the test data into the database.
Let me explain with some (very simplified) code :
I have a django view which is something like :
view.py
from myapp.models import Item
from myapp.utils import MyClass
# need to initialize with the set of items
item_set = {item.name for item in Item.objects.all()}
my_class_object = MyClass(item_set)
class MyView(APIView):
def post(selfself, request):
result = my_class_object.process(request.data)
return Response(result)
So basically I need to initialize a class with some data from the database, and I then use this class in my view to process the data received by the endpoint.
Now the test :
my_test.py
from rest_framework.test import APILiveServerTestCase
from myapp.models import Item
class MyTest(APILiveServerTestCase):
def setUp(self):
self.URL = '/some_url_linking_to_myview/'
# load some data
Item.objects.create(name="first item")
Item.objects.create(name="second item")
def test_myview_return_correct_result(self):
post_data = {"foo"}
response = self.client.post(self.URL,
data=post_data,
format='json')
self.assertEqual(response.status_code, 200)
self.assertEqual(response.data, {"my_expected_result"})
When running the test, what currently happens is that view.py is loaded before the setUp() method get excecuted, so when I instantiate the class with these two lines :
item_set = {item.name for item in Item.objects.all()}
my_class_object = MyClass(item_set)
the database is still empty.
I'm wondering if there is a way to either get the data into the database before view.py get executed, or maybe somehow force reloading the app after setUp(), or instantiate my class somewhere else so it gets called after loading the data ?
thanks !

I think you are looking for setUpTestData().
Here is roughly how I set this up if I am going to use a significant amount of data:
tests.py
from django.test import TestCase
from .tests.test_data import base_data
class MyClassTest(TestCase):
#classmethod
def setUpTestData(cls):
base_data.base_data(cls)
base_data.py
from .models import MyClass
def base_data(cls):
cls.MyClass1 = MyClass.objects.create(
name="first_name"
)
cls.MyClass2 = MyClass.objects.create(
name="second_name"
)
Of course, you can do everything directly in the setUpTestData() function, if you would rather have your test data sitting up top.

Why not put the initiate code in the function and call from inside the post.
class MyView(APIView):
def initialize(self):
item_set = {item.name for item in Item.objects.all()}
my_class_object = MyClass(item_set)
def post(self, request):
self.initialize()
result = my_class_object.process(request.data)
return Response(result)
Edit 1
Optionally, you can use fixture to load MyClass objects in the database beforehand
class MyTest(APILiveServerTestCase):
fixtures = [
// my class objects fixtures file
]
def setUp():
// rest of the code