I'm writing a module to load a dataset. I want to keep the interface/API as clean as possible - so I've made internal functions and variables hidden by prefacing their names with __. Awesome. My module, however, imports other packages (e.g. numpy) which still appear in my module's namespace, how can I avoid this?
i.e. my file looks something like:
Loader.py:
import numpy as np
__INTERNAL_VAR1 = True
EXTERNAL_VAR = True
def loadData():
data = __INTERNAL_FUNC1()
...
return data
def __INTERNAL_FUNC1():
...
return data
and when I import my module np is exposed:
> import Loader
> Loader.[TAB]
Loader.EXTERNAL_VAR Loader.loadData Loader.np
If the autocompletion you are using is correctly implemented, it should honour the __all__ attribute of modules, if set.
Add a list of all names your module exports in that name:
__all__ = ['loadData', 'EXTERNAL_VAR']
The __all__ variable is used to determine what names are imported if you use a from modulename import * wildcard import, as well as by the help() function when documenting your module.
There is no point in using double-underscore names as globals; it is a single underscore at the start that marks such names as 'internal' (by convention).
Another solution could be to create an __init__.py that contains a line that imports the functions and variables you need:
from Loader import EXTERNAL_VAR,loadData
This file should be placed inside a folder containing your module:
└── Loader
├── __init__.py
└── Loader.py
Now when you import Loader you can access directly only EXTERNAL_VAR and loadData while all the other classes and modules are in Loader.Loader.
This solution should be independent of your autocompletion system.
Related
I have a python application that has grown in size and complexity. I now have 2 folders - one that contain some utility classes and another that contains some other classes. It's quite a long list in each and the classes are referenced all over the place.
MyApp Folder
main_app.py
-- states
- Class1.py
- Class2.py
-- util
- Util1.py
- Util2.py
In main_ap.py is there a way I can just do import states and then reference any classes within that folder as states.Class1? I'd do the same for the util folder but the difference there is some of these classes reference each other.
I've tried __init__.py and some other things but i'm a legacy C++/C developer and relatively new to Python and i think this is handled much differently.
So __init__.py is a file that is executed the first time when you access something inside that package.
Saying that, import states is a way to execute content of states/__init__.py
# states/__init__.py
from states.Class1 import MyClass
# Or with relative path
from .Class2 import MyClass as MyClass2
# main_app.py
import states
states.MyClass() # It works
from states import MyClass2
MyClass2() # It works too.
In case you decide to use asterisk import (as Eldamir stated) you can be interested in __all__ keyword.
If states.__init__.py does from .Class1 import * and from .Class2 import *, then main.py can do import states and then states.SomeClassFromClass1Module
define a __init__.py file and then import the classes you want in the __init__.py file eg from class1 import class11 then at last define __all__ method and write all the class name which you want to be used as states.class11 as __all__ = ['class11', 'class12',.. ]
I have a file, myfile.py, which imports Class1 from file.py and file.py contains imports to different classes in file2.py, file3.py, file4.py.
In my myfile.py, can I access these classes or do I need to again import file2.py, file3.py, etc.?
Does Python automatically add all the imports included in the file I imported, and can I use them automatically?
Best practice is to import every module that defines identifiers you need, and use those identifiers as qualified by the module's name; I recommend using from only when what you're importing is a module from within a package. The question has often been discussed on SO.
Importing a module, say moda, from many modules (say modb, modc, modd, ...) that need one or more of the identifiers moda defines, does not slow you down: moda's bytecode is loaded (and possibly build from its sources, if needed) only once, the first time moda is imported anywhere, then all other imports of the module use a fast path involving a cache (a dict mapping module names to module objects that is accessible as sys.modules in case of need... if you first import sys, of course!-).
Python doesn't automatically introduce anything into the namespace of myfile.py, but you can access everything that is in the namespaces of all the other modules.
That is to say, if in file1.py you did from file2 import SomeClass and in myfile.py you did import file1, then you can access it within myfile as file1.SomeClass. If in file1.py you did import file2 and in myfile.py you did import file1, then you can access the class from within myfile as file1.file2.SomeClass. (These aren't generally the best ways to do it, especially not the second example.)
This is easily tested.
In the myfile module, you can either do from file import ClassFromFile2 or from file2 import ClassFromFile2 to access ClassFromFile2, assuming that the class is also imported in file.
This technique is often used to simplify the API a bit. For example, a db.py module might import various things from the modules mysqldb, sqlalchemy and some other helpers. Than, everything can be accessed via the db module.
If you are using wildcard import, yes, wildcard import actually is the way of creating new aliases in your current namespace for contents of the imported module. If not, you need to use the namespace of the module you have imported as usual.
I have a module structured as follows:
/module
__init__.py
/submod_1
__init__.py
submod_1_class.py
/submod_2
__init__.py
submod_2_class.py
but I find it incredibly annoying to have to import a class within submod_1_class.py with:
from module.submod_1.submod_1_class import my_class
What I would prefer to be able to type is:
from module import my_class
I have browsed through the site-packages folder and looked through popular modules like numpy, but I haven't been able to figure out how, for example:
import numpy
a = numpy.array([1,2,3,4,5])
can be used when the definition of numpy array objects is buried deep within several subfolders of the numpy package.
You can use __init__.py files to define what can be imported from a given module. A very simple addition to your structure, for example, would draw up Class from submod_1_class such that external users can simply from module import Class.
/module
__init__.py
from submod_1 import Class
/submod_1
__init__.py
from submod_1_class import Class
submod_1_class.py
/submod_2
submod_2_class.py
In numpy, for example, the top-level __init__.py contains a line that reads:
from .core import *
This means everything defined within /core/__init__.py is available externally directly in the numpy namespace, even though it may actually be buried deep in some complex structure.
Is it possible to create a package dynamically, something like:
subpackage = create_subpackage(package_name, package_path)
The package should be associated with a physical path so that modules from that path can be imported through it.
The purpose is to be able to have subpackages that are not subdirectories of their parent package.
e.g.
main_package/
__init__.py
sub_package/
__init__.py
some_module.py
Contents of main_package/__init__.py:
sub_package = create_subpackage("sub_package", "/a/path/to/sub_package")
globals()["sub_package"] = sub_package
Contents of some_random_script.py
from main_package.sub_package import some_module
While this won't give you exactly the layout you're asking for, this might help: http://docs.python.org/tutorial/modules.html#packages-in-multiple-directories
Basically, each package has a __path__ attribute that contains a list of places to search for submodules. And you can modify it to your liking.
e.g.
main_package/__init__.py:
__path__ += ['/tmp/some/other/path/']
/tmp/some/other/path/sub_package/__init__.py:
value = 42
test.py:
from main_package.sub_package import value
print value
If that doesn't cut it, you can go read up on import hooks, the all-powerful (and correspondingly complicated) way to modify Python's import behavior.
I want to define a constant that should be available in all of the submodules of a package. I've thought that the best place would be in in the __init__.py file of the root package. But I don't know how to do this. Suppose I have a few subpackages and each with several modules. How can I access that variable from these modules?
Of course, if this is totally wrong, and there is a better alternative, I'd like to know it.
You should be able to put them in __init__.py. This is done all the time.
mypackage/__init__.py:
MY_CONSTANT = 42
mypackage/mymodule.py:
from mypackage import MY_CONSTANT
print "my constant is", MY_CONSTANT
Then, import mymodule:
>>> from mypackage import mymodule
my constant is 42
Still, if you do have constants, it would be reasonable (best practices, probably) to put them in a separate module (constants.py, config.py, ...) and then if you want them in the package namespace, import them.
mypackage/__init__.py:
from mypackage.constants import *
Still, this doesn't automatically include the constants in the namespaces of the package modules. Each of the modules in the package will still have to import constants explicitly either from mypackage or from mypackage.constants.
You cannot do that. You will have to explicitely import your constants into each individual module's namespace. The best way to achieve this is to define your constants in a "config" module and import it everywhere you require it:
# mypackage/config.py
MY_CONST = 17
# mypackage/main.py
from mypackage.config import *
You can define global variables from anywhere, but it is a really bad idea. import the __builtin__ module and modify or add attributes to this modules, and suddenly you have new builtin constants or functions. In fact, when my application installs gettext, I get the _() function in all my modules, without importing anything. So this is possible, but of course only for Application-type projects, not for reusable packages or modules.
And I guess no one would recommend this practice anyway. What's wrong with a namespace? Said application has the version module, so that I have "global" variables available like version.VERSION, version.PACKAGE_NAME etc.
Just wanted to add that constants can be employed using a config.ini file and parsed in the script using the configparser library. This way you could have constants for multiple circumstances. For instance if you had parameter constants for two separate url requests just label them like so:
mymodule/config.ini
[request0]
conn = 'admin#localhost'
pass = 'admin'
...
[request1]
conn = 'barney#localhost'
pass = 'dinosaur'
...
I found the documentation on the Python website very helpful. I am not sure if there are any differences between Python 2 and 3 so here are the links to both:
For Python 3: https://docs.python.org/3/library/configparser.html#module-configparser
For Python 2: https://docs.python.org/2/library/configparser.html#module-configparser