I'm writing a .py file which will be regularly imported at the start of some of my IPython sessions in the first cells but will also be imported from other non-interactive sessions, since it contains functions that can be run in batch in non-interactive mode.
It is basically a module containing many classes and functions that are very common.
Since I'm using IPython with the --pylab=inline option, numpy as well as matplotlib functions are already imported, but when run in batch with a simple python mymodule.py the numpy functions have to be imported specifically.
At the end I'd come up with double imports during the IPython session, a thing I don't like very much.
What is the best practice in this case? Isn't importing modules twice a bad practice?
Repeated imports aren't a problem. No matter how many times a module is imported in a program, Python will only run its code once and only make one copy of the module. All imports after the first will merely refer to the already-loaded module object. If you're coming from a C++ background, you can imagine the modules all having implicit include guards.
Related
I have a project, this project has several modules, let's pick this as an example:
Utility: all modules that use this module have shared functions
Read: This will read files with some configurations provided in the init function, can be called needed.
Process Data 1 (PD1): This will work with some data read with the "Read" module.
Process Data 2: This will work with some data read with the "Read" module and will use Process Data 2.
Manage: This will register any process data and handle them.
First, Utility will be used for all the modules, so, I can't just copy the module folder inside of any module that wants to use it. The dependency of PD1, PD2, and Manage causes that the module Manage needs to can load or access to the PD1, but PD2 needs PD1 too. There is the Read module too, with similar problems.
But, where is the problem?, Usually, if we add a new module, we put them in folders, especially if have a lot of files to don't mix up things, but this doesn't work fine here, excluding symbolic links, there is no way by only sorting the folders to have the right access to any section, well, there are some ways, but here is the point:
If we sort it all now, add a new module that uses the right modules, can break the actual sort.
I try the next thing, I put all the modules folders in a folder, and then add the ".." path, so now I can call any module I need for any existent or new module, but this has several problems... the import function start doing weird things, workaround, first import all external modules, then add the new ones..., but is a bit..., too much messy.
There is a bigger solution, I can handle all the modules in an independent way, install all of them in the system, so now all the modules are exposed to the others, and all works properly, and that is how should works, conceptually and in python terms is right. The major issue here is that installing all in this way is, developing time-consuming. I will test the "system" if I don't have installed everything... but install it every time for every single test.
Another thing to consider is that, if I install all the modules in the system, and I work with too many of them..., there can be a conflict with other modules names.., so, I need a way to can "encapsulate" all these modules with 1 module, and only can expose all of them, "inside" of this module.
Summary, I need a "Super module" that will be exposed to the system, and inside the super module, it needs to contain, Utility, Read, PD1, PD2, and Manage, where all of these modules needs to can call every one of them, without including any new "path" to python variables in any way.
I have a problem with the debugger when some modules in my code call each other.
Practical example:
A file dog.py contains the following code:
import cat
print("Dog")
The file cat.py is the following:
import dog
print("Cat")
When I run dog.py (or cat.py) I don't have any problem and the program runs smoothly.
However, when I try to debug it, the whole spyder freezes and I have to kill the program.
Do you know how can I fix this? I would like to use this circular importing, as the modules use functions that are in the other modules.
Thank you!
When I run dog.py (or cat.py) I don't have any problem and the program runs smoothly.
AFAICT that's mostly because a script is imported under the special name ("__main__"), while a module is imported under it's own name (here "dog" or "cat"). NB : the only difference between a script and a module is actually loaded - passed an argument to the python runtime (python dog.py) or imported from a script or any module with an import statement.
(Actually circular imports issues are a bit more complicated than what I describe above, but I'll leave this to someone more knowledgeable.)
To make a long story short: except for this particular use case (which is actually more of a side effect), Python does not support circular imports. If you have functions (classes, whatever) shared by other scripts or modules, put these functions in a different module. Or if you find out that two modules really depends on each other, you may just want to regroup them into a single module (or regroup the parts that depend on each other in a same module and everything else in one or more other modules).
Also: unless it's a trivial one-shot util or something that only depends on the stdlib, your script's content is often better reduced to a main function parsing command-line arguments / reading config files / whatever, importing the required modules and starting the effective process.
Is there an idiom or suggested style for modules that are never useful when directly executed, but simply serve as components of a larger package — e.g. those containing definitions, etc.?
For example is it customary to omit #!/usr/bin/env python; add a comment; report a message to the user or execute other code (e.g. using a check of whether or not __name__ is '__main__' — or simply do nothing special?
Most of the python code I write is modules that generally don't get directly called as scripts. Sometimes when I'm working on a small project and don't want to set up a more complex testing system, I'll call my tests for the module bellow if __name__ == '__main__':, that way I can quickly test my module just by calling python modulename.py (this sometimes does not play nice with relative imports and such but it works well enough for small projects). Whether or not I do this, I drop the shebang and don't give execute remission to the file because I don't like to make modules executable unless they're meant to be run as scripts.
iPython has this excellent feature of being able to auto-complete modules and this works extremely well in there.
For the longest time, I've had a shell function that allows me to change directories to where a given module is located. It does this by trying to import the name of the module from the first argument and looking into its __file__.
What I would like to do is to find out the importable modules that are available so I can write an auto-complete function for this little helper.
I know that for a give module, I can do something like dir(module_name) and that would give me what I need for that module but I am not sure what to do to find out what I can import, just like iPython when you do something like:
import [TAB]
Or
import St[TAB]
Which would autocomplete all the importable modules that start with St.
I know how to write the auto-completion functionality, I am interesting in just finding the way of knowing what can I import.
EDIT:
Digging through the IPython code I managed to find the exact piece that does the work of getting all of the current modules for the given environment:
from IPython.core.completerlib import module_completion
for module in module_completion('import '):
print module
The reason I'm printing the results is because I am actually using this for ZSH completion so I will need to deal with the output.
As a starting point, you could iterate through the paths in sys.path looking for python modules, as that is exactly what import does. Other than that, Python won’t have an index or something for all modules, as the imports are resolved on run-time, when it’s requested. You could easily provide an index for the built-in modules though and maybe have a cache or something for previously imported modules.
I am working on a project wherein I need to use a third party module in different project files(.py files). The situation is like this.
I have a file "abc.py" which imports third party module "common.py". There are couple of other files which also import "common.py". All these files are also imported in main project file "main.py".
It seems redundant to import same module in your project multiple times in different files since "main.py" is also importing all the project files.
I am also not sure how the size of the project gets affected by multiple import statements.
Can someone pls help me in making things bit simpler.
Importing only ever loads a module once. Any imports after that simply add it to the current namespace.
Just import things in the files you need them to be available and let Python do the heavy-lifting of figuring out loading the modules.
Yes, you are right, this behavior really exists in Python. Namely, if user code tries to import the same module in different ways, for example - import a and import A.a (where a.py file is located into A package and the first import is done from within the A package while the other import comes as from outside).
This can easily happen in real life, especially for multi-level packaged Python projects.
I have experienced a side-effect of such behavior, namely command isinstance does not work when an object is checked against a class that is defined in module that was imported in such way.
The solution I can think about is to redefine the __builtin__. __ import__ function to perform its work more intelligently.