how to override a method defined at __init__.py? - python

I'm trying to use a module (pandas_access) witch has an issue on a method (_extract_dtype) defined at __init__.py
What is the proper way (or any) to overwrite a method defined on __init__.py?

You can just "monkey patch" the method e.g.
def _extract_dtype_patched(self):
print("Patched version")
TargetClass._extract_dtype = _extract_dtype_patched
Although care must be taken, per #bracco23's link above, and if you've discovered a bug it would be cool to report it to the upstream project!

Related

Python: How to deprecate a function alias

As the title states, what I want to achieve is the following:
I have a python code base for which I want to rename all the functions from camelCasing to underscore_naming. In order to maintain backwards compatibility, I have renamed all the functions, but have created function aliases for all the old names. So far, so good.
Now what I want to do is add deprecation warnings to the function aliases, preferably in a fashion like this:
def do_something():
...
#deprecated(deprecated_in='2.0',
details='All functions have been adapted to fit the PEP8 standard. Please use "do_something" instead')
doSomething = do_something
So that if somebody uses the old API call, they will get a deprecation warning. I've taken a look at deprecation and Deprecated, but neither of them seem to work on aliases.
I realise that I can create a function definition for every deprecated name and decorate that, but that loses the elegance of the function alias, and makes for more mucky code. Does anybody have a good suggestion to achieve what I want?
I actually ended up going for a different solution, similar to what's suggested in Method and property aliases with custom docstring in Python.
I modified my code to
#alias('doSomething', deprecated=True)
def do_something():
...
and added a deprecation warning to the alias decorator.

what is Pytorch's add_module()?

I stumbled upon the method add_module() in a Pytorch model.
The doc only states
Adds a child module to the current module.
The module can be accessed as an attribute using the given name.
I don't understand what "adding a child module" means.
How is it different from just setting a pointer to the other module using self._other module = other_module?
What are the nuances?
As mentioned here: https://discuss.pytorch.org/t/when-to-use-add-module-function/10534
In general, you won’t need to call add_module. One potential use case is the following:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
modules = [...] # some list of modules
for module in modules:
self.add_module(...)
nn.Modules have a hierarchy of child modules that you can access via methods like module.named_children() or module.children().
As mentioned in the forum post above, doing self._other module = other_module will type check other_module, see it's an nn.Module, and also add it to the child list, so add_module isn't really necessary.
Adding a module as an attribute works fine, as you say. But it can be a bit difficult to do at runtime if you don't know how many modules you have in advance, and you have to construct names programmatically. In such a case, add_module() is a very convenient way to do this. I've just written a short blog post showing this in action:
https://blog.d-and-j.net/deep-learning/2021/04/23/pytorch-add_module.html

How to create a Mock which has functions' special (dunder) attributes

I have some code that uses functions as parameters and I've added some logging that includes __qualname__, this caused my unit tests to fail since the Mock object I passed in raises an AttributeError for __qualname__.
mock_func = Mock()
A simple solution to this problem is to manually add the expected attribute to the mock:
mock_func.__qualname__ = "mock_function"
Or add it to the spec of the mock when I create it:
mock_func = Moc(["__qualname__"])
But these solutions are unsatisfying since I would need to change them whenever I use a different built-in attribute (e.g. __name__).
Is there a simple way to create a Mock which behaves like a function?
The closest I found was this bug report that was opened on the wrong repository, and this request which has no replies.
You can simply use any function as spec for the mock.
mock_func = Mock(spec=max)
mock_func.__qualname__
>>> <Mock name='mock.__qualname__' id='140172665218496'>

How to log the name of the test class, if the test method resides in a class common for all tests?

I have the following project structure:
/root
/tests
common_test_case.py
test_case_1.py
test_case_2.py
...
project_file.py
...
Every test test_case_... is inherited from both unittest.TestCase and common_test_case.CommonTestCase. Class CommonTestCase contains test methods that should be executed by all the tests (though using data unique to each test, stored and accessed in self.something of the test). If some specific tests are needed for an exact test case, they are added directly to that particular class.
Currently I am working on adding logging to my tests. Among other things I would like to log the class the method was run from (since the approach above implies the same test method name for different classes). I would like to stick with the built-in logging module to achieve this.
I have tried the following LogRecord attributes:%(filename)s, %(module)s, %(pathname)s. Though, for methods defined in common_test_case.py they all return path/name to the common_test_case.py and not the test module they were actually run from.
My questions are:
Is there a way to achieve what I am trying to, using only built-in logging module?
Using some third-party/other module (I was thinking maybe some "hacky" solution with inspect)?
Is it possible to achieve (in Python) at all?
Your question appears similar to this one, and solved by:
self.id()
See the function definition here, which calls self.__class__ for the instance of the TestCase class that is instantiated. Given that you are using multiple inheritance the multiple inheritance rules from Python apply:
For most purposes, in the simplest cases, you can think of the search for attributes inherited from a parent class as depth-first, left-to-right, not searching twice in the same class where there is an overlap in the hierarchy.
Which means that common_test_case.CommonTestCase will be searched then unittest.TestCase. If there is no id function in common_test_case.CommonTestCase things should work as if it is only derived from unittest.TestCase. If you feel the need to add an id function to the CommonTestCase, something like this (if really necessary):
def id(self):
if issubclass(self,unittest.TestCase):
return super(unittest.TestCase,self).id()
The solution I've found (that does the trick, so far):
import inspect
class_called_from = inspect.stack()[1][0].f_locals['self'].__class__.__name__
I'm still wondering, though, if there is a "clearer" method, or if this is possible to achieve using logging module.
Recipes, based on West's answer (tested on Python 3.6.1):
test_name = self.id().split('.')[-1]
class_called_from = self.id().split('.')[-2]

sphinx autodoc-skip-member handler: can't show __init__() when using napoleon

I want to include the docstrings for __init__() in my sphinx-generated documentation.
I was following the accepted answer to this stackoverflow question to add a handler for autodoc-skip-member and was still unable to see my __init__() documentation. Trace code inside the if name == "__init__": block shows I am hitting that code.
On a hunch I removed 'sphinx.ext.napoleon' from my extensions definition, leaving
extensions = [
'sphinx.ext.autodoc',
# 'sphinx.ext.napoleon',
]
and then I can see the __init__() documentation.
The only thing I see in the napoleon documentation that seems relevant is napoleon_include_special_with_doc, which it says defaults to True. Explicitly setting it to True in conf.py doesn't seem to change anything.
ETA: If I add the following method:
def __blah__(self):
'''blah blah blah'''
print self.__class__
I see __blah__() in my generated documentation.
If I change the name of __blah__ to __repr__ or __str__,
I see them in the generated documentation.
If I comment out the existing __init__ and change
__blah__ to __init__ I don't see it.
So it seems specific to __init__().
Is this a known issue, and is there another way to control this when using napoleon?
Napoleon defers to your autodoc configuration for how you want to handle the __init__ method.
Check your autodoc settings in conf.py. In particular, make sure autoclass_content is set to either init or both.
Per Rob's followon at https://github.com/sphinx-doc/sphinx/issues/2374, if you're using any extension that also sets a handler for the "autodoc-skip-member" event only one of the handlers will be used. This would seem to be the issue at hand. Thanks Rob!

Categories