I have a problem with the debugger when some modules in my code call each other.
Practical example:
A file dog.py contains the following code:
import cat
print("Dog")
The file cat.py is the following:
import dog
print("Cat")
When I run dog.py (or cat.py) I don't have any problem and the program runs smoothly.
However, when I try to debug it, the whole spyder freezes and I have to kill the program.
Do you know how can I fix this? I would like to use this circular importing, as the modules use functions that are in the other modules.
Thank you!
When I run dog.py (or cat.py) I don't have any problem and the program runs smoothly.
AFAICT that's mostly because a script is imported under the special name ("__main__"), while a module is imported under it's own name (here "dog" or "cat"). NB : the only difference between a script and a module is actually loaded - passed an argument to the python runtime (python dog.py) or imported from a script or any module with an import statement.
(Actually circular imports issues are a bit more complicated than what I describe above, but I'll leave this to someone more knowledgeable.)
To make a long story short: except for this particular use case (which is actually more of a side effect), Python does not support circular imports. If you have functions (classes, whatever) shared by other scripts or modules, put these functions in a different module. Or if you find out that two modules really depends on each other, you may just want to regroup them into a single module (or regroup the parts that depend on each other in a same module and everything else in one or more other modules).
Also: unless it's a trivial one-shot util or something that only depends on the stdlib, your script's content is often better reduced to a main function parsing command-line arguments / reading config files / whatever, importing the required modules and starting the effective process.
Related
I have a package animals with several thousand modules dog_1.py, cat_3.py, etc. Each module contains functions color(), size(), etc, and these functions often depend on other functions in the package outside the module. In other words, the bark() function in dog_2.py might depend on 5 other functions from different modules within the same animals package. At the bottom of each module, I have if __name__ == '__main__': because I need to be able to run each module as a stand-alone script but I don't want the module to execute if it is imported into another module. I could accomplish what I need by adding a bunch of import statements to each module to satsify the dependencies, but I thought there must be a better way to do this.
How is this type of problem typically handled?
Essentially, I need to be able to run each module separately as a script, and each module has (possibly) hundreds of dependencies within the same package.
I tried adding to the __init__.py a statement: __all__ = ['dog_1.py', 'cat_3.py', ...]. This seems like a good idea. But the problem I run into is that when I include from animals import * at the top of a given module, the import of that current module causes an error. I thought it was possible to import the module to itself, but for some reason, it is not working. If I remove the current module from the __all__ list, then it seems to work fine.
I also tried installing the package locally by creating an outer directory and adding .cfg and setup.py files. I thought it might fix the problem, but I wasn't able to get anywhere.
I feel like I might be just going about this completely wrongly. It seems like it would be an easy problem to handle: Make a bunch of program files in one directory and have them all share the functions between each other. Any help is appreciated. Thank you.
I'm writing a .py file which will be regularly imported at the start of some of my IPython sessions in the first cells but will also be imported from other non-interactive sessions, since it contains functions that can be run in batch in non-interactive mode.
It is basically a module containing many classes and functions that are very common.
Since I'm using IPython with the --pylab=inline option, numpy as well as matplotlib functions are already imported, but when run in batch with a simple python mymodule.py the numpy functions have to be imported specifically.
At the end I'd come up with double imports during the IPython session, a thing I don't like very much.
What is the best practice in this case? Isn't importing modules twice a bad practice?
Repeated imports aren't a problem. No matter how many times a module is imported in a program, Python will only run its code once and only make one copy of the module. All imports after the first will merely refer to the already-loaded module object. If you're coming from a C++ background, you can imagine the modules all having implicit include guards.
Is there an idiom or suggested style for modules that are never useful when directly executed, but simply serve as components of a larger package — e.g. those containing definitions, etc.?
For example is it customary to omit #!/usr/bin/env python; add a comment; report a message to the user or execute other code (e.g. using a check of whether or not __name__ is '__main__' — or simply do nothing special?
Most of the python code I write is modules that generally don't get directly called as scripts. Sometimes when I'm working on a small project and don't want to set up a more complex testing system, I'll call my tests for the module bellow if __name__ == '__main__':, that way I can quickly test my module just by calling python modulename.py (this sometimes does not play nice with relative imports and such but it works well enough for small projects). Whether or not I do this, I drop the shebang and don't give execute remission to the file because I don't like to make modules executable unless they're meant to be run as scripts.
iPython has this excellent feature of being able to auto-complete modules and this works extremely well in there.
For the longest time, I've had a shell function that allows me to change directories to where a given module is located. It does this by trying to import the name of the module from the first argument and looking into its __file__.
What I would like to do is to find out the importable modules that are available so I can write an auto-complete function for this little helper.
I know that for a give module, I can do something like dir(module_name) and that would give me what I need for that module but I am not sure what to do to find out what I can import, just like iPython when you do something like:
import [TAB]
Or
import St[TAB]
Which would autocomplete all the importable modules that start with St.
I know how to write the auto-completion functionality, I am interesting in just finding the way of knowing what can I import.
EDIT:
Digging through the IPython code I managed to find the exact piece that does the work of getting all of the current modules for the given environment:
from IPython.core.completerlib import module_completion
for module in module_completion('import '):
print module
The reason I'm printing the results is because I am actually using this for ZSH completion so I will need to deal with the output.
As a starting point, you could iterate through the paths in sys.path looking for python modules, as that is exactly what import does. Other than that, Python won’t have an index or something for all modules, as the imports are resolved on run-time, when it’s requested. You could easily provide an index for the built-in modules though and maybe have a cache or something for previously imported modules.
I am working on a project wherein I need to use a third party module in different project files(.py files). The situation is like this.
I have a file "abc.py" which imports third party module "common.py". There are couple of other files which also import "common.py". All these files are also imported in main project file "main.py".
It seems redundant to import same module in your project multiple times in different files since "main.py" is also importing all the project files.
I am also not sure how the size of the project gets affected by multiple import statements.
Can someone pls help me in making things bit simpler.
Importing only ever loads a module once. Any imports after that simply add it to the current namespace.
Just import things in the files you need them to be available and let Python do the heavy-lifting of figuring out loading the modules.
Yes, you are right, this behavior really exists in Python. Namely, if user code tries to import the same module in different ways, for example - import a and import A.a (where a.py file is located into A package and the first import is done from within the A package while the other import comes as from outside).
This can easily happen in real life, especially for multi-level packaged Python projects.
I have experienced a side-effect of such behavior, namely command isinstance does not work when an object is checked against a class that is defined in module that was imported in such way.
The solution I can think about is to redefine the __builtin__. __ import__ function to perform its work more intelligently.