I am reading this piece of code by Alex Martelli mentioned in this question. I understand that sys.modules[__name__] tells you what module you are currently at, but this line of code at the end of his constant.py really confuses me. What is the meaning and the point of having such a statement that declares the current module by the end of the file?
# Put in const.py...:
class _const:
class ConstError(TypeError): pass
def __setattr__(self,name,value):
if self.__dict__.has_key(name):
raise self.ConstError, "Can't rebind const(%s)"%name
self.__dict__[name]=value
import sys
sys.modules[__name__]=_const() #this I don't understand
# that's all -- now any client-code can
import const
Basically, my question is that in my opinion this line of code does not do anything; am I understanding it wrong?
Since in Python you don't have to put class definitions in separate files, I argue that I don't really need two modules unless I want to reuse the class "const." Then in this case sys.moldules[__name__]=_const() is not necessary either... Am I understanding it correctly?
I believe it is binding an instance to the module. So when you do import const, you actually get an instance of the class _const.
This allows you to call methods on it. Like, for example, the __setattr__, where in this case it checks that you only bind a variable once.
This is described by Guido van Rossum as a hack that is sometimes recommended:
https://mail.python.org/pipermail/python-ideas/2012-May/014969.html
# foo.py
import sys
class Foo:
def funct1(self, <args>): <code>
def funct2(self, <args>): <code>
sys.modules[__name__] = Foo()
This works because the import machinery is actively enabling this
hack, and as its final step pulls the actual module out of
sys.modules, after loading it. (This is no accident. The hack was
proposed long ago and we decided we liked enough to support it in the
import machinery.)
You can easily override __getattr__ / __getattribute__ / __setattr__
this way. It also makes "subclassing" the module a little easier
(although accessing the class to be used as a base class is a little
tricky: you'd have to use foo.__class__). But of course the kind of
API that Greg was griping about would never be implemented this way,
so that's fairly useless. And if you were designing a module as an
inheritable class right from the start you're much better off just
using a class instead of the above hack.
Related
I have a base class, Sample and a derived class SignalSample, and the code for both of these classes normally resides within the same file Sample.py.
I import these classes into my main script in the usual way: from Sample import Sample, SignalSample
While chasing down a bug within one of these classes, I noticed some unusual behavior: deleting the code for the SignalSample derived class changed the behavior of the Signal base class.
So my question is, can the existence of a derived class alter a base class, even if the derived class is never instantiated?
To be concrete, I tried the following combinations.
The code for Sample and SignalSample are both in Sample.py. from Sample import Sample, SignalSample is used in my main script to load these classes. Sample objects are instantiated, there are no SignalSample objects instantiated, the code is just there and unused. In this scenario, I get an error which I will call "type 1".
Delete the code for SignalSample inside Sample.py, and remove the ... import SignalSample statement. In this case, I get a different error which I'll call "type 2".
Note that I don't think the errors are coming from the classes themselves (although they may be), it was more that I found it interesting that the behavior of the code seemed to change because there was an inherited class, even though that class was not used.
This is a stripped-down example of my setup, note that this is not an MWE of the source of my bug, as at the moment I don't know where it is coming from, and so I can't even narrow it down. It's not the solution to the bug I'm looking for, just more information on this seemingly strange behavior of class inheritance.
# file Sample.py
class Sample:
def __init__(self):
self._tfile = None
self._filepath = None
def calculate_filepath(self):
return "my/file/path"
__calculate_filepath = calculate_filepath # private copy
def get_histogram(self, histogram_name):
if not self._filepath:
self._filepath = self.calculate_filepath()
if not self._tfile:
from ROOT import TFile # this is a special filetype
self._tfile = TFile.Open(self._filepath, "READ")
histo = self._tfile.Get(histogram_name)
histo.SetDirectory(0)
self._tfile.Close()
return histo
class SignalSample(Sample):
def __init__(self):
# Inherit
Sample.__init__(self)
self._filepath = self.calculate_filepath()
def calculate_filepath(self):
# Overloaded version of the function in Sample
return "my/very/special/filepath"
Note that I chose to call the calculate_filepath method inside get_histogram because I wanted to avoid possible namespace clashes with the derived class. This is also why I try to make the method "private" with the namespace mangling. This is also why I open the special TFile file inside the get_histogram method, although it is nice that I can then also Close this file inside the same function. Perhaps this isn't the correct usage and maybe this related to the source of my problem?
get_histogram looks potentially broken if it is called more than once. You assign the opened file to the instance (on self._tfile), but then close it before returning... that means that next time the method is called not self._tfile will probably (*) evaluate to False, meaning that you then try to call Get on a closed file. If you are using a sane library this would probably throw a nice error telling you this, but I see you're using ROOT so who knows what might happen :)
Probably simplest would be not to store the file on Sample, and just open the file whenever get_histogram is called?
(*) Implicit booliness is sometimes worth avoiding. In particular when what you actually want to check is whether something is None, prefer writing if x is None: ( https://legacy.python.org/dev/peps/pep-0008/#programming-recommendations )
Incidentally, in this example, __calculate_filepath = calculate_filepath # private copy isn't doing anything, since you never actually use it.
I would like to convert a singleton-object programmatically into a Python module so that I can use the methods of this singleton-object directly by importing them via the module instead of accessing them as object attributes. By "programmatically" I mean that I do not want to have to copy-paste the class methods explicitly into a module file. I need some sort of a workaround that allows me to import the object methods into to global scope of another module.
I would really appreciate if someone could help me on this one.
Here is a basic example that should illustrate my problem:
mymodule.py
class MyClass:
"""This is my custom class"""
def my_method(self):
return "myValue"
singleton = MyClass()
main_as_is.py
from mymodule import MyClass
myobject = MyClass()
print(myobject.my_method())
main_to_be.py
from mymodule import my_method # or from mymodule.singleton import my_method
print(my_method())
You can use the same strategy that the standard random module uses. All the functions in that module are actually methods of a "private" instance of the Random class. That's convenient for most common uses of the module, although sometimes it's useful to create your own instances of Random so that you can have multiple independent random streams.
I've adapted your code to illustrate that technique. I named the class and its instance with a single leading underscore, since that's the usual convention in Python to signify a private name, but bear in mind it's simply a convention, Python doesn't do anything to enforce this privacy.
mymodule.py
class _MyClass:
""" This is my custom class """
def my_method(self):
return "myValue"
_myclass = _MyClass()
my_method = _myclass.my_method
main_to_be.py
from mymodule import my_method
print(my_method())
output
myValue
BTW, the from mymodule import method1, method2 syntax is ok if you only import a small number of names, or it's clear from the name which module it's from (like math module functions and constants), and you don't import from many modules. Otherwise it's better to use this sort of syntax
import mymodule as mm
# Call a method from the module
mm.method1()
That way it's obvious which names are local, and which ones are imported and where they're imported from. Sure, it's a little more typing, but it makes the code a whole lot more readable. And it eliminates the possibility of name collisions.
FWIW, here's a way to automate adding all of the _myclass methods without explicitly listing them (but remember "explicit is better than implicit"). At the end of "mymodule.py", in place of my_method = _myclass.my_method, add this:
globals().update({k: getattr(_myclass, k) for k in _MyClass.__dict__
if not k.startswith('__')})
I'm not comfortable with recommending this, since it directly injects items into the globals() dict. Note that that code will add all class attributes, not just methods.
In your question you talk about singleton objects. We don't normally use singletons in Python, and many programmers in various OOP languages consider them to be an anti-pattern. See https://stackoverflow.com/questions/12755539/why-is-singleton-considered-an-anti-pattern for details. For this application there is absolutely no need at all to use a singleton. If you only want a single instance of _MyClass then simply don't create another instance of it, just use the instance that mymodule creates for you. But if your boss insists that you must use a singleton, please see the example code here.
As you know, when the project's code is very large and there are so many attributes and functions defined in a Class, but some of them never be called by the instance of the Class, and maybe some of them has been discarded. Here is a example:
class Foo(object):
""""""
def __init__(self):
self.a = 1
self.b = 2
self.c = 3
...
self.y = 25
self.z = 26
def func1(self):
pass
def func2(self):
pass
def func3(self):
pass
...
...
def func100(self):
pass
if __name__ == '__main__':
f = Foo()
f.func1()
f.func2()
print f.a, f.b, f.z
In the above code, the instance f of class Foo just called func1() and func2(). And how to find all the attributes and functions of class that never called by the instance of class.
I have tried compiler module but that could not solve my question. And dir(my_instance) is just print all the functions and attributes defined the the class.
Thanks in advance.
You can try coverage.py. It's not static analysis, but actually runs your code and records which statements are executed, outputting annotated html or txt as you wish (quite nicely formatted as well). You can then look for functions and methods whose bodies are not executed at all.
This still doesn't take care of unused attributes. And I don't know the answer to that. Maybe comment them out one at a time and see if tests still pass...
It's pretty hard to prove something is or is not used in the general case. Python is a dynamic language; if even one bit of code calls into code the static analyzer doesn't fully analyze, it could be accessing the variables mentioned.
The pylint and flake8 tools will tell you about local and global names that aren't defined prior to use (unless you break them by using from x import * style imports), and about imports that are never used (an import that is never used is usually wrong, but even then, it could be an intentional part of the interface, where linters would have to be silenced), but I don't believe they can tell you that a given attribute is never accessed; after all, someone else could import your module and access said attributes.
Use the profile module in the standard library.
python -m cProfile -o output_file myscript.py
Then load the stats file and use print_callees() to get all the functions that were called--during that run of the program.
I don't know of any easy way to find out which attributes are used.
Sorry for somewhat unclear question. I'm actually wondering whether it's possible in Python not to mention class name, when you call class's methods iteratively? I mean to write instead of:
SomeClass.Fun1()
SomeClass.Fun2()
...
SomeClass.Fun100()
Something like:
DoWith SomeClass:
Fun1()
Fun2()
...
Fun100()
?
There are several methods to achieve that (from SomeClass import *, locals().update(SomeClass.__dict__())), but what you're trying is not really logical:
In 90% of cases you're not calling static class methods, but member functions, which need a single instance to operate on. You do realize that the first, the self argument that you typically see on methods is important, because it gives you access to the instance's namespace. So even in methods, you use self.my_member instead of my_member. That's an important python concept, and you should not try to avoid it -- there's a difference between the local name space and the attributes of an instance.
What you can do, however, is having a short handle, without any overhead:
my_instance = SomeClass() #notice, this is an instance of SomeClass, not the class or type itself
__ = my_instance
that can save you a lot of typing. But I prefer clarity over saved typing (hell, vim has good autocompletion plugins for Python).
yes, just try from SomeClass import * (after moving SomeClass to an other file of course)
I have a large python code with many modules and classes. I have a special class, whose single instance is needed everywhere throughout the code (it's a threaded application, and that instance of a class also holds Thread Local Storage, locks, etc). It's a bit uncomfortable to always "populate" that instance in every imported module. I know, using globals is not the best practice, but anyway: is there any "import hook" in python, so I can do with hooking on it to have my instance available in every modules without extra work? It should work for normal imports, "from mod import ..." too, and for import constructs too. If this is not possible, can you suggest a better solution? Certenly it's not fun to pass that instance to the constructors of every classes, etc ... Inheritance also does not help, since I have modules without classes, and also I need a single instance, not the class itself ...
class master():
def import_module(self, name):
mod = __import__(name)
mod.m = self
return mod
[...]
m = master()
Currently I am thinking something like that: but then I have to use m.import_module() to import modules, then other modules will have instance of master class with name of "m" available, so I can use m.import_module() too, etc. But then I have to give up to use "normal" import statements, and I should write this:
example_mod = m.module_import("example_mod")
instead of just this:
import example_mod
(but for sure I can do with this too, to assign "m" to example_mod.m then)
Certainly it's not fun to pass that instance to the constructors of
every classes
You don't have to do this. Set up your global class in a module like config and import it
# /myapp/enviroment/__init__.py
class ThatSingleInstanceClass: pass
# create the singleton object directly or have a function init the module
singleton = ThatSingleInstanceClass()
# /myapp/somewhere.py
# all you need to use the object is importing it
from myapp.enviroment import singleton
class SomeClass:
def __init__(self): # no need to pass that object
print "Always the same object:", singleton
What's wrong with having each module import the needed object? Explicit is better than implicit.