I've built a module in Python in one single file without using classes. I do this so that using some api module becomes easier. Basically like this:
the_module.py
from some_api_module import some_api_call, another_api_call
def method_one(a, b):
return some_api_call(a + b)
def method_two(c, d, e):
return another_api_call(c * d * e)
I now need to built many similar modules, for different api modules, but I want all of them to have the same basic set of methods so that I can import any of these modules and call a function knowing that this function will behave the same in all the modules I built. To ensure they are all the same, I want to use some kind of abstract base module to build upon. I would normally grab the Abstract Base Classes module, but since I don't use classes at all, this doesn't work.
Does anybody know how I can implement an abstract base module on which I can build several other modules without using classes? All tips are welcome!
You are not using classes, but you could easily rewrite your code to do so.
A class is basically a namespace which contains functions and variables, as is a module.
Should not make a huge difference whether you call mymodule.method_one() or mymodule.myclass.method_one().
In python there is no such thing as interfaces which you might know from java.
The paradigm in python is Duck typing, that means more or less that for a given module you can tell whether it implements your API if it provides the right methods.
Python does this i.e. to determine what to do if you call myobject[i] on an instance of your class myclass. It looks whether the class has the method __getitem__ and if it does so, it replaces myobject[i] by myobject.__getitem__(i).
Yout don't have to tell python that your class supports this kind of access, python just figures it out from the way you defined your class.
The same way you should determine whether your module implements your API.
Maybe you want to look inside the hidden dictionary mymodule.__dict__ after import mymodulewhich contains all function names and pointers to them of your module. You could then check whether the right functions are present and raise an error otherwise
import my_module_4
#check if my_module_4 implements api
if all(func in my_module_4.__dict__ for func in ("method_one","method_two"):
print "API implemented"
else:
print "Warning: Not all API functions found in my_module_4"
Related
I'm working with Tensorflow in Python. In a custom written function I found #tf_export() before the function definition like below, the function of which I don't understand. Could somebody explain?
#tf_export("signal.ifftshift")
def ifftshift(x, axes=None, name=None):
As I understand, it allows Tensorflow to expose a function or class under a different name. For example, the Server class within the distribute module actually lives in the training/server_lib.py file within the repo, but, since it is exported as distribute.Server, you can use it like tf.distribute.Server().
# training/server_lib.py
#tf_export("distribute.Server", v1=["distribute.Server", "train.Server"])
#deprecation.deprecated_endpoints("train.Server")
class Server(object):
...
It makes it confusing to find the code, but I imagine it's a more flexible way to create these "logical" modules.
It is a convenient way to output dot delimited symbols directly to the tf API. Namely, a user can access ifftshift() with tf.signal.ifftshift(), without caring about the true path (here tf.python.ops.signal.fft_ops.ifftshif()).
I have the following project structure:
/root
/tests
common_test_case.py
test_case_1.py
test_case_2.py
...
project_file.py
...
Every test test_case_... is inherited from both unittest.TestCase and common_test_case.CommonTestCase. Class CommonTestCase contains test methods that should be executed by all the tests (though using data unique to each test, stored and accessed in self.something of the test). If some specific tests are needed for an exact test case, they are added directly to that particular class.
Currently I am working on adding logging to my tests. Among other things I would like to log the class the method was run from (since the approach above implies the same test method name for different classes). I would like to stick with the built-in logging module to achieve this.
I have tried the following LogRecord attributes:%(filename)s, %(module)s, %(pathname)s. Though, for methods defined in common_test_case.py they all return path/name to the common_test_case.py and not the test module they were actually run from.
My questions are:
Is there a way to achieve what I am trying to, using only built-in logging module?
Using some third-party/other module (I was thinking maybe some "hacky" solution with inspect)?
Is it possible to achieve (in Python) at all?
Your question appears similar to this one, and solved by:
self.id()
See the function definition here, which calls self.__class__ for the instance of the TestCase class that is instantiated. Given that you are using multiple inheritance the multiple inheritance rules from Python apply:
For most purposes, in the simplest cases, you can think of the search for attributes inherited from a parent class as depth-first, left-to-right, not searching twice in the same class where there is an overlap in the hierarchy.
Which means that common_test_case.CommonTestCase will be searched then unittest.TestCase. If there is no id function in common_test_case.CommonTestCase things should work as if it is only derived from unittest.TestCase. If you feel the need to add an id function to the CommonTestCase, something like this (if really necessary):
def id(self):
if issubclass(self,unittest.TestCase):
return super(unittest.TestCase,self).id()
The solution I've found (that does the trick, so far):
import inspect
class_called_from = inspect.stack()[1][0].f_locals['self'].__class__.__name__
I'm still wondering, though, if there is a "clearer" method, or if this is possible to achieve using logging module.
Recipes, based on West's answer (tested on Python 3.6.1):
test_name = self.id().split('.')[-1]
class_called_from = self.id().split('.')[-2]
I am currently learning how to write Python (v3.5) extension modules using the Python C API. Some operations, like fast numerical work, are best done in C, while other operations, like string manipulation, are far easier to implement in Python. Is there an agreed-upon way to use both Python and C code to define a new type?
For example, I've written a Matrix type in C that supports basic storage and arithmetic operations. I want to define the Matrix.__str__ using Python, where string manipulations are much easier and I don't need to worry about cstrings.
I attempted to define the __str__ method when the module loads in __init__.py as follows:
from mymodule._mymodule import Matrix;
def as_str(self):
print("This is a matrix!");
Matrix.__str__ = as_str;
When I run this code, I get a TypeError: can't set attributes of built-in/extension type 'matey.Matrix'. Is there an acceptable way to do this? If the solution is to subclass Matrix instead, what is the best way to keep my C base classes / Python subclasses organized within a module?
Personally, I wouldn't try and do object-oriented stuff in C. I'd stick to writing a module which exposes some (stateless) functions.
If I wanted the Python interface to be object oriented, I'd write a class in Python which imports that (C extension) module and consumes functions from it. Maintaining of any state would all be done in Python.
You could instead define a _Matrix type which you then extend with a traditional OOP approach
from mymodule._mymodule import _Matrix;
class Matrix(_Matrix):
def __str__(self):
return "This is a matrix!"
I have a file that I read from which has definitions of ctypes that are used in a separate project. I can read the file and obtain all the necessary information to create a ctype that I want in Python like the name, fields, bitfields, ctype base class (Structure, Union, Enum, etc), and pack.
I want to be able to create a ctype class from the information above. I also want these ctypes to be pickleable.
I currently have two solutions, both of which I feel like are hacks.
Solution 1
Generate a Python code object in an appropriate ctype format by hand or with the use of something like Jinja2 and then evaluate the python code object.
This solution has the downside of using eval. I always try to stay away from eval and I don't feel like this is a good place to use it.
Solution 2
Create a ctype dynamically in a function like so:
from ctypes import Structure
def create_ctype_class(name, base, fields, pack):
class CtypesStruct(base):
_fields_ = fields
_pack_ = pack
CtypesStruct.__name__ = name
return CtypesStruct
ctype = create_ctype_class('ctype_struct_name', ctypes.Structure,
[('field1', ctypes.c_uint8)], 4)
This solution isn't so bad, but setting the name of the class is ugly and the type cannot be pickled.
Is there a better way of creating a dynamic ctype class?
Note: I am using Python 2.7
Solution 2 is probably your better option, though if you're also writing such classes statically, you may want to use a metaclass to deduplicate some of that code. If you need your objects to be pickleable, then you'll need a way to reconstruct them from pickleable objects. Once you've implemented such a mechanism, you can make the pickle module aware of it with a __reduce__() method.
I would go with a variant of Solution 1. Instead of evaling code, create a directory with an __init__.py (i.e. a package), add it to your sys.path and write out an entire python module containing all of the classes. Then you can import them from a stable namespace which will make pickle happier.
You can either take the output and add it to your app's source code or dynamically recreate it and cache it on a target machine at runtime.
pywin32 uses an approach like this for caching classes generated from ActiveX interfaces.
Is it possible to overload the from/import statement in Python?
For example, assuming jvm_object is an instance of class JVM, is it possible to write this code:
class JVM(object):
def import_func(self, cls):
return something...
jvm = JVM()
# would invoke JVM.import_func
from jvm import Foo
This post demonstrates how to use functionality introduced in PEP-302 to import modules over the web. I post it as an example of how to customize the import statement rather than as suggested usage ;)
It's hard to find something which isn't possible in a dynamic language like Python, but do we really need to abuse everything? Anyway, here it is:
from types import ModuleType
import sys
class JVM(ModuleType):
Foo = 3
sys.modules['JVM'] = JVM
from JVM import Foo
print Foo
But one pattern I've seen in several libraries/projects is some kind of a _make_module() function, which creates a ModuleType dynamically and initializes everything in it. After that, the current Module is replaced by the new module (using the assignment to sys.modules) and the _make_module() function gets deleted. The advantage of that, is that you can loop over the module and even add objects to the module inside that loop, which is quite useful sometimes (but use it with caution!).