We Keep Coding

In a Python package's __init__.py file, is there a way of detecting if a package was executed directly?

I would like a way to detect if my module was executed directly, as in import module or from module import * rather than by import module.submodule (which also executes module), and have this information accessible in module's __init__.py.
Here is a use case:
In Python, a common idiom is to add import statement in a module's __init__.py file, such as to "flatten" the module's namespace and make its submodules accessible directly. Unfortunately, doing so can make loading a specific submodule very slow, as all other siblings imported in __init__.py will also execute.
For instance:
module/
__init__.py
submodule/
__init__.py
...
sibling/
__init__.py
...
By adding to module/__init__.py:
from .submodule import *
from .sibling import *
It is now possible for users of the module to access definitions in submodules without knowing the details of the package structure (i.e. from module import SomeClass, where SomeClass is defined somewhere in submodule and exposed in its own __init__.py file).
However, if I now run submodule directly (as in import module.submodule, by calling python3 -m module.submodule, or even indirectly via pytest) I will also, unavoidably, execute sibling! If sibling is large, this can slow things down for no reason.
I would instead like to write module/__init__.py something like:
if __???__ == 'module':
from .submodule import *
from .sibling import *
Where __???__ gives me the fully qualified name of the import. Any similar mechanism would also work, although I'm mostly interested in the general case (detecting direct executing) rather than this specific example.

What is being desired is will result in undefined behavior (in the sense whether or not the flattened names be importable from module) when we consider how the import system actually works, if it were actually possible.
Hypothetically, if what you want to achieve is possible, where some __dunder__ that will disambiguate which import statement was used to import module/__init__.py (e.g. import module and from module import *, vs import module.submodule. For the first case, module may trigger the subsequent (slow) import to produce a "flattened" version of the desired imports, while the latter case (import module.submodule) will avoid that and thus module will not contain any assignments of the "flattened" imports.
To illustrate the example a bit more, say one may import SiblingClass from module.sibling.SiblingClass by simply doing from module import SiblingClass as the module/__init__.py file executes from .sibling import * statement to create that binding. But then, if executing import module.submodule resulting in the avoidance of that flatten import, we get the following scenario:
import module.submodule
# module.submodule gets imported
from module import SiblingClass
# ImportError will occur
Why is that? This is simply due to how Python imports a file - the source file is executed in its entirety once to assign imports, function and class declarations to the designated names, and be registered to sys.modules under its import name. Importing the module again will not execute the file again, thus if the from .sibling import * statement was not executed during its initial import (i.e. import module.submodule), it will never be executed again during subsequent import of the same module, as the copy produced by the initial import assigned to its module entry in sys.module is returned (unless the module was reloaded manually, the code for the module will be executed again).
You may verify this fact by putting in a print statement into a file, import the corresponding module to see the output produced, and see that no further output will be produced on subsequent import of that module (related: What happens when a module is imported twice?).
Effectively, the desired functionality as described in the question cannot be implemented in Python.
A related thread on this topic: How to only import sub module without exec __init__.py in the package

This is not a complete solution, but standalone py.test (ignore __init__.py files) proposes setting a global flag to detect when in test. This corrects the problem for tests at least, provided the concerned modules don't call each other.

python module names with same name as existing modules

I have a number of small utils modules that i organize under a 'msa' namespace so I can use in a number of different research projects. Currently I have them organized like this:
# folder structure:
packages <-- in my pythonpath
--msa
----msa_utils.py
----msa_geom.py
----msa_pyglet.py
----msa_math.py
----etc
# imported and used this like
from msa import msa_pyglet
from msa import msa_math
msa_pyglet.draw_rect(msa_math.lerp(...))
However I would like to avoid the 'msa_' in the names and use like this:
# folder structure:
packages <-- in my pythonpath
--msa
----utils.py
----geom.py
----pyglet.py
----math.py
----etc
# imported and used this like
import msa.pyglet
import msa.math
msa.pyglet.draw_rect(msa.math.lerp(...))
This shouldn't cause name conflicts when importing from outside, however there are name conflicts when the modules themselves import modules with conflicting names. E.g. msa/pyglet needs to imports pyglet (the external one), but ends up trying to import itself. Likewise any module which tries to import the standard math library imports only my math module. Which is all understandable. But what is the usual pythonic way of dealing with this? Do I have to give each module file a globally unique name?

In Python 2, imports in packages without a package qualifier indeed first look for package local modules.
Thus, import pyglet will find msa.pyglet before the top-level pyglet is considered.
Switch to absolute imports to make the Python 3 behaviour the default, where unqualified names are always top level names:
from __future__ import absolute_import
Now import pyglet can only ever find the top-level name, never msa.pyglet. To reference other modules within your msa namespace, use from . import pyglet or from msa import pyglet.
See PEP 328 -- Imports: Multi-Line and Absolute/Relative for more details.

Circular function import

I'm trying to do the following in python 2.6.
my_module.py:-
from another_module import another_factory
def my_factory(name):
pass
another_module.py:-
from my_module import my_factory
def another_factory(name):
pass
Both modules in the same folder.
It gives me the error:
Error: cannot import name my_factory

As seen from the comments, you are trying to do a circle import which is impossible.
If in your module A you try to import something from the module B, and when loading the module B (to satisfy this dependency) you are trying to import something from the module A, you are where you started and you got a circle import: A needs B and B needs A!!, it is somehow like saying that A needs A, which is quite unlogic.
For instance:
# moduleA
from moduleB import functionB
...
So the interpreter tries to load the moduleB, which looks like the following:
# moduleB
from moduleA import functionA
...
And goes back to the moduleA, which tries again to import B, and, etc. Therefore python just raises the error and stops the insanity for a greater good.
Dependencies don't work like this. Define what module needs the other one, and just do a simple import. In your example, it seems that another_module needs my_module, so change my_module and eliminate the dependency on another_module.
If both modules actually need each other, it is a clear sign that they belong to the same logical concept, and should be merged.
PD: in some cases to avoid huge files, you can split a logical unit in two, and to avoid the circle dependencies, you write your imports inside of the functions (which are not executed at load time), so that there is not a circle. This is however in general something to avoid.

The real question is... do you consider each file as a module or are they part of a package ?
Trying to import modules outside a package is sometimes painful. You should rather build a package by simply creating an empty __init__.py module in the directory. Though, if you have
__init__.py
my_module.py
another_module.py
If you have te following function in my_module.py,
def my_factory(x):
return x * x
You should be able to access the my_factory() function from another_module.py by writing this :
from my_module import my_factory
But, if you don't have the __init__.py file/module, the import function will be (somehow) lost and will only use the sys.path for searching other modules. You may then add the following lines (before the import) in the another_module.py file :
sys.path.append(os.path.dirname(os.path.expanduser('.')))
You may also use the various packages available to help importing modules, like imp or import_file (see the documentation). Or you can decide to use load_source (also see the doc : https://docs.python.org/2/library/imp.html)

Changes in import statement python3

I don't understand the following from pep-0404
In Python 3, implicit relative imports within packages are no longer
available - only absolute imports and explicit relative imports are
supported. In addition, star imports (e.g. from x import *) are only
permitted in module level code.
What is a relative import?
In what other places star import was allowed in python2?
Please explain with examples.

Relative import happens whenever you are importing a package relative to the current script/package.
Consider the following tree for example:
mypkg
├── base.py
└── derived.py
Now, your derived.py requires something from base.py. In Python 2, you could do it like this (in derived.py):
from base import BaseThing
Python 3 no longer supports that since it's not explicit whether you want the 'relative' or 'absolute' base. In other words, if there was a Python package named base installed in the system, you'd get the wrong one.
Instead it requires you to use explicit imports which explicitly specify location of a module on a path-alike basis. Your derived.py would look like:
from .base import BaseThing
The leading . says 'import base from module directory'; in other words, .base maps to ./base.py.
Similarly, there is .. prefix which goes up the directory hierarchy like ../ (with ..mod mapping to ../mod.py), and then ... which goes two levels up (../../mod.py) and so on.
Please however note that the relative paths listed above were relative to directory where current module (derived.py) resides in, not the current working directory.
#BrenBarn has already explained the star import case. For completeness, I will have to say the same ;).
For example, you need to use a few math functions but you use them only in a single function. In Python 2 you were permitted to be semi-lazy:
def sin_degrees(x):
from math import *
return sin(degrees(x))
Note that it already triggers a warning in Python 2:
a.py:1: SyntaxWarning: import * only allowed at module level
def sin_degrees(x):
In modern Python 2 code you should and in Python 3 you have to do either:
def sin_degrees(x):
from math import sin, degrees
return sin(degrees(x))
or:
from math import *
def sin_degrees(x):
return sin(degrees(x))

For relative imports see the documentation. A relative import is when you import from a module relative to that module's location, instead of absolutely from sys.path.
As for import *, Python 2 allowed star imports within functions, for instance:
>>> def f():
... from math import *
... print sqrt
A warning is issued for this in Python 2 (at least recent versions). In Python 3 it is no longer allowed and you can only do star imports at the top level of a module (not inside functions or classes).

To support both Python 2 and Python 3, use explicit relative imports as below. They are relative to the current module. They have been supported starting from 2.5.
from .sister import foo
from . import brother
from ..aunt import bar
from .. import uncle

Added another case to Michał Górny's answer:
Note that relative imports are based on the name of the current module. Since the name of the main module is always "__main__", modules intended for use as the main module of a Python application must always use absolute imports.

Python: Importing everything from a python namespace / package

all.
I'd think that this could be answered easily, but it isn't. As long as I've been searching for an answer, I keep thinking that I'm overlooking something simple.
I have a python workspace with the following package structure:
MyTestProject
/src
/TestProjectNamespace
__init__.py
Module_A.py
Module_B.py
SecondTestProject
/src
/SecondTestProjectNamespace
__init__.py
Module_1.py
Module_2.py
...
Module_10.py
Note that MyTestProjectNamespace has a reference to SecondTestProjectNamespace.
In MyTestProjectNamespace, I need to import everything in SecondTestProjectNamespace. I could import one module at a time with the following statement(s):
from SecondTestProjectNamespace.Module_A import *
from SecondTestProjectNamespace.Module_B import *
...but this isn't practical if the SecondTestProject has 50 modules in it.
Does Python support a way to import everything in a namespace / package? Any help would be appreciated.
Thanks in advance.

Yes, you can roll this using pkgutil.
Here's an example that lists all packages under twisted (except tests), and imports them:
# -*- Mode: Python -*-
# vi:si:et:sw=4:sts=4:ts=4
import pkgutil
import twisted
for importer, modname, ispkg in pkgutil.walk_packages(
path=twisted.__path__,
prefix=twisted.__name__+'.',
onerror=lambda x: None):
# skip tests
if modname.find('test') > -1:
continue
print(modname)
# gloss over import errors
try:
__import__(modname)
except:
print 'Failed importing', modname
pass
# show that we actually imported all these, by showing one subpackage is imported
print twisted.python
I have to agree with the other posters that star imports are a bad idea.

No. It is possible to set up SecondTestProject to automatically import everything in its submodules, by putting code in __init__.py to do the from ... import * you mention. It's also possible to automate this to some extent using the __import__ function and/or the imp module. But there is no quick and easy way to take a package that isn't set up this way and make it work this way.
It's probably not a good idea anyway. If you have 50 modules, importing everything from all of them into your global namespace is going to cause a proliferation of names, and very likely conflicts among those names.

As other had put it - it might not be a good idea. But there are ways of keeping your namespaces and therefore avoiding naming conflicts - and having all the modules/sub-packages in a module available to the package user with a single import.
Let's suppose I have a package named "pack", within it a module named "a.py" defining some "b" variable. All I want to do is :
>>> import pack
>>> pack.a.b
1
One way of doing this is to put in pack/__init__.py a line that says
import a - thus in your case you'd need fifty such lines, and keep them up to date.
Not that bad.
However, the documentation at http://docs.python.org/tutorial/modules.html#importing-from-a-package - says that if you have a string list named __all__ in your __init__.py file, all module/sub-package names in that list are imported when one does from pack import *
That alone would half-work - but would require users of your package to perform the not-recommended "from x import *" form.
But -- you can do the "... import *" inside __init__.py itself, after defining the __all__ variable - so all you have to do is to keep the __all__ up to date:
With the TestProjectNamespace/__init__.py being like this:
__all__ = ["Module_A", "Module_B", ...]
from TestProjectNamespace import *
your users would have
TestProjectNamespace.Module_A (and others) available upon import of TestProjectNamespace.
And, of course - you could automate the creation of __all__ - it is just a variable, after all - but I would not recommend that.

Does Python support a way to import everything in a namespace / package?
No. A package is not a super-module -- it's a collection of modules grouped together.
At least part of the reason is that it's not trivial to determine what 'everything' means inside a folder: there are problems like network drives, soft links, hard links, ...