We Keep Coding

How to type dynamically created class methods

I have a class which is used to send requests to server.
There's a ROUTES dictionary which is a map of class method names to the server endpoint used in request function.
These methods are added to class in __init__ with setattr. I could've just added these methods to the class, but there are a lot of them and I hate code duplication.
Class looks like this:
ROUTES = {
"send_test": ["GET", "/test/"],
"send_start": ["GET", "/start/"]
}
class Response(TypedDict):
...
class Client(object):
def __init__(self) -> None:
for mname, endpoint in ROUTES.items():
setattr(self, mname, self.make_request_func(endpoint[0], endpoint[1]))
def make_request_func(self, method, path):
def _func(*args, **kwargs):
return self.request(method, path, *args, **kwargs)
return _func
def request(self, method, path, data: dict = {}, files: Optional[dict] = None) -> Response:
...
I basically need these send_test and send_start methods to just point to an alias of request function, but without method and path arguments, only data and files.
I have a very basic knowledge of python's typing, I will appreciate any additional explanation of typing this!

If you are willing to be a little less DRY and define instance methods explicitly, the following should work. (Let's face it, the repeated calls to makerequest aren't really any more repetitive than the original definition of ROUTES.)
from typing import Optional, Callable
def makerequest(method: str, path: str) -> Callable[[Client, Optional[dict], Optional[dict]], Response]:
def wrapper(self: Client, data: Optional[dict] = None, files: Optional[dict] = None) -> Response:
return self.request(method, path, data, files)
return wrapper
class Client(object):
def __init__(self) -> None:
...
send_test = makerequest("GET", "/test/")
send_start = makerequest("GET", "/start/")
def request(self, method, path, data: Optional[dict] = None, files: Optional[dict] = None) -> Response:
...
(I took the liberty of replacing your default dict value with None, to avoid problems with mutable default values.)
With this, mypy reports the revealed type of Client.send_test as
tmp.py:32: note: Revealed type is "def (tmp.Client, Union[builtins.dict[Any, Any], None], Union[builtins.dict[Any, Any], None]) -> TypedDict('tmp.Response', {})"
Adapting this to allow defining the aliases by iterating over ROUTES will be at best, I think, a mess, and at worst not possible, due to the class attributes send_test etc not being statically defined.
If typing weren't an issue, I would recommend a class decorator to define the methods.
ROUTES = {...}
def add_aliases(d: dict):
def _(cls):
for name, args in d.items():
setattr(cls, name, makereqeust(*args))
return cls
return _
def makerequest(method, path):
...
#add_aliases(ROUTES)
class Client:
def request(self, ...):
...
The problem is annotating add_aliases and _ in a way that captures the idea of the class that _ returns having methods of the desired type, since nothing can be inferred from the body of add_aliases alone.

How to update/change attribute in Django Class Based View Test

I created a mixin that requires an attribute to be added to a Class Based View (cbv) and it works as expected - but I'm having a tough time 'testing' that mixin specifically.
Here is my test:
class TestView(GroupRequiredMixin, View):
group_required = ("test_group",)
raise_exception = True
def get(self, request):
return HttpResponse("OK")
class GroupRequiredMixinTest(TestCase):
def setUp(self):
group = GroupFactory(name="test_group")
self.user = UserFactory(groups=(group,))
def test_view_without_group_required_improperly_configured(self):
rfactory = RequestFactory()
request = rfactory.get("/fake-path")
request.user = self.user
view = TestView()
view.group_required = None
with self.assertRaises(ImproperlyConfigured):
view.as_view()(request)
This test errors with this message: AttributeError: This method is available only on the class, not on instances.
What is the 'appropriate' way to test this?
Should I be setting up a unique class for each situation I want to test? Or is there a way I can dynamically change that attribute on an instance like in my test that fails?
*Edit to add the mixin (almost identical to the core Django version for Permissions):
class GroupRequiredMixin(AccessMixin):
"""Verify that the current user has all specified groups."""
group_required = None
def handle_no_group(self):
if self.raise_exception or self.request.user.is_authenticated:
raise PermissionDenied(self.get_permission_denied_message())
path = self.request.build_absolute_uri()
resolved_login_url = resolve_url(self.get_login_url())
# If the login url is the same scheme and net location then use the
# path as the "next" url.
login_scheme, login_netloc = urlparse(resolved_login_url)[:2]
current_scheme, current_netloc = urlparse(path)[:2]
if (not login_scheme or login_scheme == current_scheme) and (
not login_netloc or login_netloc == current_netloc
):
path = self.request.get_full_path()
return redirect_to_login(
path,
resolved_login_url,
self.get_redirect_field_name(),
)
def get_group_required(self):
"""
Override this method to override the group_required attribute.
Must return an iterable.
"""
if self.group_required is None:
raise ImproperlyConfigured(
f"{self.__class__.__name__} is missing the "
f"group_required attribute. Define "
f"{self.__class__.__name__}.group_required, or override "
f"{self.__class__.__name__}.get_group_required()."
)
if isinstance(self.group_required, str):
groups = (self.group_required,)
else:
groups = self.group_required
return groups
def has_group(self):
"""
Override this method to customize the way groups are checked.
"""
groups = self.get_group_required()
if hasattr(self.request.user, "has_groups"):
return self.request.user.has_groups(groups)
else:
return self.handle_no_group()
def dispatch(self, request, *args, **kwargs):
if not self.has_group():
return self.handle_no_group()
return super().dispatch(request, *args, **kwargs)
with self.assertRaises(ImproperlyConfigured):
> view.as_view()(request)
my_app/tests/test_mixins.py:52:
_ _ _ _ _
def __get__(self, instance, cls=None):
if instance is not None:
> raise AttributeError("This method is available only on the class, not on instances.")
E AttributeError: This method is available only on the class, not on instances.

I was able to get it working with the following:
rfactory = RequestFactory()
request = rfactory.get("/fake-path")
request.user = self.user
view = TestView.as_view(group_required=None)
with self.assertRaises(ImproperlyConfigured):
view(request)
I could not call TestView() to instantiate a new view - it needed to be TestView.as_view() to instantiate it. Then the arguments get passed inside the as_view() portion.
Working as intended now.

Is this the right way to do dependency injection in Django?

I'm trying to inject dependencies into my Django view (controller?). Here's some background.
Normally, the urls.py file is what handles the routing. It is usually something like this:
urlpatterns = [
path("", views.get_all_posts, name="get_all_posts"),
path("<int:post_id>", views.get_post, name="get_post"),
path("create", views.create_post, name="create_post"),
]
The problem with this, is that once you get to create_post for instance, you might have a dependency on a service that creates posts:
# views.py
...
def create_post(self):
svc = PostCreationService()
svc.create_post()
This kind of pattern is difficult to test. While I know python testing libraries have tools to mock this sort of thing, I'd rather inject the dependency into the view. Here's what I came up with.
A Controller class that has a static method, export(deps) that takes in a list of dependencies and returns a list of url pattern objects:
class ApiController(object):
#staticmethod
def export(**deps):
ctrl = ApiController(**deps)
return [
path("", ctrl.get_all_posts, name="get_all_posts"),
path("<int:post_id>", ctrl.get_post, name="get_post"),
path("create", ctrl.create_post, name="create_post"),
]
def __init__(self, **deps):
self.deps = deps
def get_all_posts():
pass
...
This looks janky, but I'm not aware of any other way to do what I'm trying to do. The controller needs to return a list of url patterns, and it also needs to take in a list of dependencies. Using the above technique, I can do this in urls.py:
urlpatterns = ApiController.export(foo_service=(lambda x: x))
I am now free to use foo_service in any of the methods of ApiController.
Note:
One alternative would be for the constructor to return the list of urls, but I don't see that as a huge improvement over this. In fact, it strikes me as being more confusing because the class constructor would return a list instead of an instance of the class.
Note 2:
I'm aware that python has mocking tools for mocking class members. Please don't suggest using them. I'd like to use DI as the way to control and manage dependencies.
Any ideas on what the best way to do this is?

Consider injecting using decorators:
from functools import wraps
class ServiceInjector:
def __init__(self):
self.deps = {}
def register(self, name=None):
name = name
def decorator(thing):
"""
thing here can be class or function or anything really
"""
if not name:
if not hasattr(thing, "__name__"):
raise Exception("no name")
thing_name = thing.__name__
else:
thing_name = name
self.deps[thing_name] = thing
return thing
return decorator
def inject(self, func):
#wraps(func)
def decorated(*args, **kwargs):
new_args = args + (self.deps, )
return func(*new_args, **kwargs)
return decorated
# usage:
si = ServiceInjector()
# use func.__name__, registering func
#si.register()
def foo(*args):
return sum(args)
# we can rename what it's been registered as, here, the class is registered
# with name `UpperCase` instead of the class name `UpperCaseRepresentation`
#si.register(name="UpperCase")
class UpperCaseRepresentation:
def __init__(self, value):
self.value = value
def __str__(self):
return self.value.upper()
#register float
si.register(name="PI")(3.141592653)
# inject into functions
#si.inject
def bar(a, b, c, _deps): # the last one in *args would be receiving the dependencies
UpperCase, PI, foo = _deps['UpperCase'], _deps['PI'], _deps['foo']
print(UpperCase('abc')) # ABC
print(PI) # 3.141592653
print(foo(a, b, c, 4, 5)) # = 15
bar(1, 2, 3)
# inject into class methods
class Foo:
#si.inject
def my_method(self, a, b, _deps, kwarg1=30):
return _deps['foo'](a, b, kwarg1)
print(Foo().my_method(1, 2, kwarg1=50)) # = 53

You could take a look at https://github.com/ets-labs/python-dependency-injector, but that is a pretty big setup.
You could also create something small like a Service factory
# services.py
class ServiceFactory:
def __init__(self):
self.__services = {}
def register(self, name, service_class):
# Maybe add some validation
self.__services[name] = service_class
def create(self, name, *args, **kwargs):
# Maybe add some error handling or fallbacks
return self.__services[name](*args, **kwargs)
factory = ServiceFactory()
# In your settings.py for example
from services import factory
factory.register('post_creation', PostCreationService)
# Or maybe in apps.ready do auto_load that will loop all apps and get config from services.py
# In your views.py
from services import factory
def create_post(self):
svc = factory.create('post_creation')
svc.create_post()
# In your tests.py
from services import factory
def setUp(self):
factory.register('post_creation', FakePostCreationService)

While reading Dependency Injection Principles, Practices, and Patterns and trying to apply the examples to a django app I came up with the following:
# views.py
class IndexView(View):
# Must include this to bypass django's validation
product_service: IProductService = None
# Init method not necessary but more explicit
def __init__(self, product_service: IProductService):
self.product_service = product_service
def get(self, request):
self.product_service.do_stuff()
...
# urls.py
# Construct dependencies. I guess this is the closest to the entry-point we can get
# with Django.
repo = DjangoProductRepository()
product_service = ProductService(repo)
urlpatterns = [
path('admin/', admin.site.urls),
path("",
IndexView.as_view(product_service=product_service),
name="index"),
]

This is only an updated version of rabbit.aaron reply above. My idea is to be able to specify which dependencies to inject instead of getting a dictionary with all registered dependencies.
from functools import wraps
class ServiceInjector:
deps = {}
def register(self, name=None):
name = name
def decorator(thing):
"""
thing here can be class or function or anything really
"""
if not name:
if not hasattr(thing, '__name__'):
raise Exception('no name')
thing_name = thing.__name__
else:
thing_name = name
self.__class__.deps[thing_name] = thing
return thing
return decorator
class inject:
def __init__(self, *args):
self.selected_deps = args
def __call__(self, func):
#wraps(func)
def decorated(*args, **kwargs):
selected_deps = {k: v for k, v in ServiceInjector.deps.items() if k in self.selected_deps}
new_kwargs = {**kwargs, **selected_deps}
return func(*args, **new_kwargs)
return decorated
Usage:
si = ServiceInjector()
# use func.__name__, registering func
#si.register()
def foo(*args):
return sum(args)
Custom naming still works
#si.register(name='uppercase')
class UpperCaseRepresentation:
def __init__(self, value):
self.value = value
def __str__(self):
return self.value.upper()
Register float
si.register(name="PI")(3.141592653)
Inject into functions
#si.inject('foo', 'PI', 'uppercase')
def bar(a, b, c, uppercase: UpperCaseRepresentation, **kwargs):
"""
You can specify dependencies as keyword arguments and add typehint annotation.
"""
UpperCase, foo = kwargs['UpperCase'], kwargs['foo']
print(uppercase('abc')) # ABC
print(PI) # 3.141592653
print(foo(a, b, c, 4, 5)) # = 15
bar(1, 2, 3)
Inject into class methods
class Bar:
#si.inject('foo')
def my_method(self, a, b, foo, kwarg1=30):
return foo(a, b, kwarg1)
print(Bar().my_method(1, 2, kwarg1=50)) # = 53

You could go the flask route and export a class instance with a property that initializes and caches the service on first access. E.g:
def default_factory():
pass
# service.py
class ServiceProvider:
def __init__(self, create_instance=default_factory):
self.create_instance = create_instance
_instance = None
#property
def service(self):
if self._instance:
return self._instance
self._instance = self.create_instance()
return self._instance
service_provider = ServiceProvider()
from .service import service_provider
# views.py
def view(request):
service_provider.service.do_stuff()
# etc.
This has the advantages of being easy to mock and not having any magic.

The most boring solution I could come up with involves using class variables:
# Module services.post_service
def default_create_post():
return "foo"
class Provider:
create_post = default_create_post
Then you could import and use normally in a view or elsewhere:
from services import post_service
post_service.Provider.create_post()
# Should return "foo"
And when testing it could be swapped out before being called:
from django.test import TestCase
from services import post_service
from unittest.mock import patch
class MyTestCase(TestCase):
#patch('services.post_service.default_create_post')
def test_some_view(self, mock_create_post):
mock_create_post.return_value = "bar"
post_service.Provider.create_post = mock_create_post
# Now when calling post_service.Provider.create_post it should just return "bar"

Find name of dynamic method in Python

I want to proxy an API over a network. I have the API in a dictionary. I'd like to create a class with the API methods from the dictionary so I can use the API as if I was local. The trouble is finding the name of my dynamically created method. (My approach is based on Adding a Method to an Existing Object and Python dynamic class methods.)
class MainClass(object):
def build_API(self):
methods = dict(meth1='arg1', meth2='arg2')
for key in methods.iterkeys():
setattr(self, key, MethodType(self.default_API, self))
def default_API(self, *args, **kwargs)
called_as_name = ????
self.send_message(called_as_name, args, kwargs)
def send_message(self, called_as_name, *args, **kwargs)
...
# Send API command over network
....
To use this:
api = MainClass()
api.build_API()
api.meth1()
However, everything I try for "called_as_name" always returns "default_API" and never "meth1". How can I get "called_as_name = meth1" when I type "api.meth1()" and "called_as_name = meth2" when I type "api.meth2()"?
I have tried:
curframe = inspect.currentframe()
calframe = inspect.getouterframes(curframe, 2)
called_as_name = calframe[1][3]
from Python: How to get the caller's method name in the called method?
called_as_name = inspect.stack()[1][5]
from Getting the caller function name inside another function in Python?
called_as_name = sys._getframe(1).f_code.co_name
from Getting the caller function name inside another function in Python?

Trying to do this with actual methods and grabbing the names from the stack frame with that sort of introspection trickery is a recipe for disaster. Instead, make the "methods" be custom callable objects that know their names. Here's a sketch:
class FakeMethod(object):
def __init__(self, name, parent):
self.name = name
self.parent = parent
def __call__(self, *args, **kwargs):
self.parent.send_message(self.name, args, kwargs)
class MainClass(object):
def build_API(self):
methods = dict(meth1='arg1', meth2='arg2')
for key in methods.iterkeys():
setattr(self, key, FakeMethod(key, self))
def send_message(self, called_as_name, *args, **kwargs):
print("Sending message:", called_as_name, args, kwargs)
Then:
>>> api = MainClass()
>>> api.build_API()
>>> api.meth1()
Sending message: meth1 ((), {}) {}
>>> api.meth2()
Sending message: meth2 ((), {}) {}
In theory you could even use __getattr__ on the MainClass to dynamically generate a FakeMethod every time an attribute name is accessed that is not defined but is listed in some list of API method names.

Why cannot I not create an object like this in Python?

Question about objects in python. I have created the following object....
class http(object):
def __init__(self):
self._resource = None
self._response = None
#property
def resource(self):
return self._resource
#resource.setter
def resource(self, value):
self._resource = "http://127.0.0.1:8000/%s" % value
def get(self, resource=None):
self.resource = resource
self._response = requests.get(self.resource)
return self._response
Init does not need anything at this stage so I was hoping I could create the object like this....
content = http.get("users/")
but it won't let me do this, instead I have to pass use the syntax http() but pass nothing...
content = http().get("users/")
which seems silly if I don't pass anything to __init__. I'm wondering how the a Python package like requests achieves the following syntax....
requests.get('https://api.github.com/user')
without doing this...
requests().get('https://api.github.com/user')
why, what does requests package do different?

Requests defines some extra methods that create an instance of Requests.request behind the scenes. You can do the same thing for your http class.
class http(object):
def get(self, resource=None):
self.resource = resource
self._response = requests.get(self.resource)
return self._response
def get(resource=None):
temp_instance = http()
return temp_instance.get(resource)