Python 3.4: From reading some other SO questions it seems that if a moduleName.py file is outside of your current directory, if you want to import it you must add it to the path with sys.path.insert(0, '/path/to/application/app/folder'), otherwise an import moduelName statement results in this error:
ImportError: No module named moduleName
Does this imply that python automatically adds all other .py files in the same directory to the path? What's going on underneath the surface that allows you to import local files without appending the Python's path? And what does an __init__.py file do under the surface?
Python adds the directory where the initial script resides as first item to sys.path:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.
So what goes on underneath the surface is that Python appends (or rather, prepends) the 'local' directory to sys.path for you.
This simply means that the directory the script lives in is the first port of call when searching for a module.
__init__.py has nothing to do with all this. __init__.py is needed to make a directory a (regular) package; any such directory that is found on the Python module search path is treated as a module.
I have faced same problem when running python script from Intellij Idea.
There is a script in a
C:\Users\user\IdeaProjects\Meshtastic-python\meshtastic
It uses
from meshtastic import portnums_pb2, channel_pb2, config_pb2
and fails.
I have realized that it looks for
C:\Users\user\IdeaProjects\Meshtastic-python\meshtastic\meshtastic
and changed working directory of this script in Run Configuration from
C:\Users\user\IdeaProjects\Meshtastic-python\meshtastic
to
C:\Users\user\IdeaProjects\Meshtastic-python
so it can find this module UNDERNEATH workdir during execution
C:\Users\user\IdeaProjects\Meshtastic-python\meshtastic
Related
Assume I have the following files,
pkg/
pkg/__init__.py
pkg/main.py # import string
pkg/string.py # print("Package's string module imported")
Now, if I run main.py, it says "Package's string module imported".
This makes sense and it works as per this statement in this link:
"it will first look in the package's directory"
Assume I modified the file structure slightly (added a core directory):
pkg/
pkg/__init__.py
plg/core/__init__.py
pkg/core/main.py # import string
pkg/string.py # print("Package's string module imported")
Now, if I run python core/main.py, it loads the built-in string module.
In the second case too, if it has to comply with the statement "it will first look in the package's directory" shouldn't it load the local string.py because pkg is the "package directory"?
My sense of the term "package directory" is specifically the root folder of a collection of folders with __init__.py. So in this case, pkg is the "package directory". It is applicable to main.py and also files in sub- directories like core/main.py because it is part of this "package".
Is this technically correct?
PS: What follows after # in the code snippet is the actual content of the file (with no leading spaces).
Packages are directories with a __init__.py file, yes, and are loaded as a module when found on the module search path. So pkg is only a package that you can import and treat as a package if the parent directory is on the module search path.
But by running the pkg/core/main.py file as a script, Python added the pkg/core directory to the module search path, not the parent directory of pkg. You do have a __init__.py file on your module search path now, but that's not what defines a package. You merely have a __main__ module, there is no package relationship to anything else, and you can't rely on implicit relative imports.
You have three options:
Do not run files inside packages as scripts. Put a script file outside of your package, and have that import your package as needed. You could put it next to the pkg directory, or make sure the pkg directory is first installed into a directory already on the module search path, or by having your script calculate the right path to add to sys.path.
Use the -m command line switch to run a module as if it is a script. If you use python -m pkg.core Python will look for a __main__.py file and run that as a script. The -m switch will add the current working directory to your module search path, so you can use that command when you are in the right working directory and everything will work. Or have your package installed in a directory already on the module search path.
Have your script add the right directory to the module search path (based on os.path.absolute(__file__) to get a path to the current file). Take into account that your script is always named __main__, and importing pkg.core.main would add a second, independent module object; you'd have two separate namespaces.
I also strongly advice against using implicit relative imports. You can easily mask top-level modules and packages by adding a nested package or module with the same name. pkg/time.py would be found before the standard-library time module if you tried to use import time inside the pkg package. Instead, use the Python 3 model of explicit relative module references; add from __future__ import absolute_import to all your files, and then use from . import <name> to be explicit as to where your module is being imported from.
I've seen the following code in a couple Python projects, in __main__.py. Could someone explain the purpose? Of course it puts the directory containing __main__.py at the head of sys.path, but why? And why the tests (__package__ is None and not hasattr(sys, 'frozen')? Also, in the sys.path.insert, why is os.path.dirname called twice?
import sys
if __package__ is None and not hasattr(sys, 'frozen'):
# direct call of __main__.py
import os.path
path = os.path.realpath(os.path.abspath(__file__))
sys.path.insert(0, os.path.dirname(os.path.dirname(path)))
os.path.dirname(os.path.dirname(path)) - Gets the grand-parent directory (the directory containing the directory of the given path variable); this is being added to the system's PATH variable.
os.path.realpath(os.path.abspath(__file__)) - Gets the realpath (resolves symbolic linking) of the absolute path of the running file.
Through this method, the project can now execute binary files that are included in that grandparent directory without needing to prefix the binary executable.
Sidenote: Without context of where you see this code, it's hard to give more of an answer as to why its used.
The test for __package__ lets the code run when package/__main__.py has been run with a command like python __main__.py or python package/ (naming the file directly or naming the package folder's path), not the more normal way of running the main module of a package python -m package. The other check (for sys.frozen) tests if the package has been packed up with something like py2exe into a single file, rather than being in a normal file system.
What the code does is put the parent folder of the package into sys.path. That is, if __main__.py is located at /some/path/to/package/__main__.py, the code will put /some/path/to in sys.path. Each call to dirname strips off one item off the right side of the path ("/some/path/to/package/__main__.py" => "/some/path/to/package" => "/some/path/to").
I have a custom module that I am trying to read from a folder under a hierarchy:
> project-source
/tests
/provider
my_provider.py
settings_mock.py
__init__.py
I am trying to call, from my_provider.py
import tests.settings_mock as settings
Example from command line:
project-source> python tests/provider/my_provider.py
Error:
... ImportError: No module named settings_mock
I keep getting No module named settings_mock as error. I have already exported project_source path to PYTHONPATH. I have made tests into a package by creating a __init__.py file in its root, but no change in the error then.
I can print the settings_mock.py attributes when cd'ing project source
>>> import tests.settings_mock as settings
>>> print settings.storage_provider
correct storage provider value
Is anyone able to point out my mistake here? Thanks!
You only have one small mistake. To use subfolders, you need __init__.py, not init.py as you stated in the question. The difference is that __init__ is a builtin function of python, whereas init is not. Having this file in each subfolder tells the pyhon interpreter that the folder is a "package" that needs to be initialized.
UPDATED: It should be noted that python usually runs from the current directory that the script is located. If your executable main script is my_provider.py, then it's not going to know what to import, since the main script is located in a lower directory than the object it is trying to import. Think of it as a hierarchy. Scripts can only import things that are beneath them. Try separating out the executable from everything else in that file, if there are things that settings_mock needs to import.
I am trying to run some python 3.4 example on visual studio 2013. when I try to import some module from a parent folder and run it from inside visual studio 2013, I always has the error of ImportError: No module named 'foo'
However, when I run it from the console using the python command python boo.py, it executes well.
As an example, this is my project structure
myproject/
foo.py
__init__.py
koo/
boo.py
__init__.py
foo.py content is
def do1():
print('Inside foo module')
boo.py content is
import sys
sys.path.append("..")
import foo
foo.do1()
I guess, this issue is not about Visual Studio and not about why it doesn't work in VS. The real question is why it works in the terminal. Probably it is because the terminal runs under different environmental settings, where the python interpreter can find the parent directory and thus the foo.py:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the
shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The
directory containing the script being run is placed at the beginning
of the search path, ahead of the standard library path. This means
that scripts in that directory will be loaded instead of modules of
the same name in the library directory. This is an error unless the
replacement is intended. See section Standard Modules for more
information.
So, add the parent dir to pythonpath, and it will work.
Or modify sys.path adding the parent dir to it.
Try this it worked for me:
import sys
sys.path.append("Folder PATH"). "Folder PATH" given as...C:\\Working_directory\\VSProject
import <MODULE_NAME>
I'm trying to open a Python script from the Python command line. There's a bug in Python that makes adding Python to my environmental variable's path ineffective. So, I have to run Python command line from the Python directory.
My script is at c:/mydir/myfile.py
How do I open it from c:/python27/python.exe; >>?
access your parent directory by
import sys
sys.path.append("..")
then you access a subdirectory by placing a __init__.py in subdirectory and writing something in it like
__all__ = ['myfile']
then you can import myfile
Or you just want to run it. In that case you can use an absolute path. eg. python c:\mydir\myfile.py