As a simple example, in the following code of 2 submodules (a.py and b.py in the same directory). The link to the same submodule function :func:`hook` works but not the link cross-referencing to a different moduel, ie, :func:`foo`. I also tried the syntax of :func:`.a.foo` - still does not work. How can I cross reference to a.foo()?
# script a.py
def foo():
'''foo func'''
# script b.py
def hook():
'''hook func'''
def spam():
'''spam func.
:func:`foo`
:func:`hook`
'''
As described in the docs:
Normally, names in these roles are searched first without any further
qualification, then with the current module name prepended, then with
the current module and class name (if any) prepended. If you prefix
the name with a dot, this order is reversed.
In that case, :func:`.a.foo` means an object named a inside of the module b. It will look for b.a.foo function.
You should try :func:`..a.foo`, which will point to b..a.foo, or just a.foo (cannot check that locally now, sorry; but I remember I was using that syntax before).
But note, that a.py & b.py should be the modules, i.e. importable under their name. If they are just the scripts, and located not in a package (no __init__.py files up to the root of the project), there is no way to cross-reference with these roles.
You can try to use :any: role — :any:`foo` — and hope that it will find the object in the general index of the described objects.
Related
When calling the Python help() function on a package I get the following things:
The content of the docstring specified in __init__.py
A list of the package contents containing all modules
The version of the package
The path of __init__.py
As a provider of a customer-oriented software I would like to restrict this output to the relevant information, i.e. not showing those modules that are not meant to be used by the customer.
So, if my package P contains modules A, B and _c, where A and B are meant to be used as a public interface and _c provides just some utility functionality for A and B then I would like to limit the output of help(P) to:
Help on package P:
Some descriptive text.
PACKAGE CONTENTS
A
B
VERSION
1.0
FILE
/path/to/P/__init__.py
When trying to achieve something similar for modules, I can define my own __dir__() function which interestingly is respected by help(module). But trying to apply the same approach to a package (meaning: defining __dir__() in __init__.py) doesn't achieve the result.
The output of help() comes from the pydoc module.
Looking at the relevant source, the PACKAGE CONTENTS section is generated by pkgutil.iter_modules(), and there's no hook I can see that could hide some modules from there (aside from monkey-patching the whole function to hide the modules you want when the stack indicates it's being called from pydoc, but I would strongly advise against it unless this is an app you're in full control of).
You might use __all__ magic variable to provide list of things which should be described after using help, let mymodule.py be file with content
__all__ = ['func1','func2']
def func1():
'''first function for user'''
return 1
def func2():
'''second function for user'''
return 2
def func3():
'''function for internal usage'''
return 3
then
import mymodule
help(mymodule)
gives output
Help on module mymodule:
NAME
mymodule
FUNCTIONS
func1()
first function for user
func2()
second function for user
DATA
__all__ = ['func1', 'func2']
FILE
<path to mymodule.py here>
This question already has an answer here:
Sphinx linking functions between files
(1 answer)
Closed 10 months ago.
I am building a documentation of my Python package using Sphinx.
I have a API.rst file, in which I list my functions as follows: .. autofunction:: mymodule.foo.bar1
For the different functions that are autodocumented form this page, I can put for instance:
:func:`foo1`
in the docstring of foo2() and it will create a link to the first function.
However, if I have a second file API2.rst, in which I autodoc some other functions, the same syntax doesn't seem to find the links across different pages. Even the See Also functions are not linked.
Is there a way to specify and to link different functions across pages? Thanks
Example
The repo's documentation can be found here
For instance, in the docstring of this function, under the parameter peaks, I am trying to link to another function called ecg_peaks() which is documented here
To sum up what other commenters found, when Sphinx resolve Python cross-references, it can find the cross-reference targets in several ways. Let's get an example.
Say you have the following project structure
my-project/
├── docs/ # Where your docs, and conf.py are
└── my_project/ # where the Python code is, importable from Sphinx
├── __init__.py # empty file
├── foobar.py # defines the foo_func and bar_func functions
└── baz.py # defines the Baz class which as a baz_meth method
Absolute qualified name: Anywhere in your documentation you
should be able to make a fully qualified reference, like so
See the :func:`my_project.foobar.foo_func` function
Relative name: In a given module, you can use the Python
object name directly. For instance, from within the
from foobar.py file:
See the :func:`foo_func` function.
Relaxed qualified name: When referencing a python object
you can use the . character to extend the search space for your target. As an example, from foobar.py file:
See the :func:`.foo_func` function.
Also, from the class docstring of Baz, in baz.py:
See the :meth:`.baz_meth` method.
The risk with the relaxed method is that Sphinx will link
to the first thing it finds, which may of may not be the target
you expected.
Finally, to help prevent these issues go public, use the nitpick option
Here is what Sphinx documentation says on this topic
The name enclosed in this markup can include a module name and/or a
class name. For example, :py:func:filter could refer to a function
named filter in the current module, or the built-in function of that
name. In contrast, :py:func:foo.filter clearly refers to the filter
function in the foo module.
Normally, names in these roles are searched first without any further
qualification, then with the current module name prepended, then with
the current module and class name (if any) prepended. If you prefix
the name with a dot, this order is reversed. For example, in the
documentation of Python’s codecs module, :py:func:open always refers
to the built-in function, while :py:func:.open refers to
codecs.open().
A similar heuristic is used to determine whether the name is an
attribute of the currently documented class.
Also, if the name is prefixed with a dot, and no exact match is found,
the target is taken as a suffix and all object names with that suffix
are searched. For example, :py:meth:.TarFile.close references the
tarfile.TarFile.close() function, even if the current module is not
tarfile. Since this can get ambiguous, if there is more than one
possible match, you will get a warning from Sphinx.
Note that you can combine the ~ and . prefixes:
:py:meth:~.TarFile.close will reference the tarfile.TarFile.close()
method, but the visible link caption will only be close().
Python Double-Underscore methods are hiding everywhere and behind everything in Python! I am curious about how this is specifically working with the interpreter.
import some_module as sm
From my current understanding:
Import searches for requested module
It binds result to the local assignment (if given)
It utilizes the __init__.py . . . ???
There seems to be something going on that is larger than my scope of understanding. I understand we use __init__() for class initialization. It is functioning as a constructor for our class.
I do not understand how calling import is then utilizing the __init__.py.
What exactly is happening when we run import?
How is __init__.py different from other dunder methods?
Can we manipulate this dunder method (if we really wanted to?)
import some_module is going to look for one of two things. It's either going to look for a some_module.py in the search path or a some_module/__init__.py. Only one of those should exist. The only thing __init__.py means when it comes to modules is "this is the module that represents this folder". So consider this folder structure.
foo/
__init__.py
module1.py
bar.py
Then the three modules available are foo (which corresponds to foo/__init__.py), foo.module1 (which corresponds to foo/module1.py), and bar (which corresponds to bar.py). By convention, foo/__init__.py will usually import important names from module1.py and reexport some of them for convenience, but this is by no means a requirement.
I am always confused and never felt confident about relative imports in python. Here I am trying to understand what I don't understand and then seek StackOverflow's help to fix that.
The Reason why PEP366 exists:
Python relative imports are based on the __name__ attribute. __name__ is parsed to determine the relative position of the module in the package hierarchy. The __name__ attribute for foo.py is always __main__ if it is run like python foo.py from anywhere at all. This means there can never be a relative import statement in foo.py since __main__ has no package information. By package information I mean: __name__ is, for instance, set to a.b.foo. This gets parsed as package a, subpackage b and module foo.
This is the problem that gets fixed by PEP366. It included a -m switch and a __package__attribute that will be used to parse the relative position of the module in pakcae hierarchy. One can use -m to run foo like so : python -m foo
Now, here is PEP366.
Questions in bold
The major proposed change is the introduction of a new module level attribute, __package__. When it is present, relative imports will be based on this attribute rather than the module name attribute. Will it ever be the case that it is not present? I know it can be None, '', or __name__.rpartition('.')[0] but, will it ever be that referencing __package__ will throw me an attribute not found error? Is it safe to say that __package__ is always present actually?
As with the current __name__ attribute, setting __package__ will be the responsibility of the PEP 302 loader used to import a module. Loaders which use imp.new_module() to create the module object will have the new attribute set automatically to None. When the import system encounters an explicit relative import in a module without package set (Just to be clear, by module here they mean import foo or python -m foo only? python foo.py means script and not module. Am I correct?)(or with it set to None), it will calculate and store the correct value (name.rpartition('.')[0] for normal modules and name for package initialisation modules). The language suggests that package gets set only if there is a relative import statement present. Is that the case? Will __package__ never be set if there are no relative import statements?. If __package__ has already been set (This means the user manually setting __package__ in code?) then the import system will use it in preference to recalculating the package name from the name and path attributes.
The runpy module will explicitly set the new attribute, basing it off the name used to locate the module to be executed rather than the name used to set the module's name attribute. This will allow relative imports to work correctly from main modules executed with the -m switch.What is being said here? I thought there was just one name attribute.Here they are talking about 2 different names
When the main module is specified by its filename (What is meant by this statement? Is that not always the case? Can you give an example where it is not?), then the __package__ attribute will be set to None. To allow relative imports when the module is executed directly, boilerplate similar to the following would be needed before the first relative import statement:
if __name__ == "__main__" and __package__ is None:
__package__ = "expected.package.name"
Note that this boilerplate is sufficient only if the top level package is already accessible via sys.path. Additional code that manipulates sys.path would be needed in order for direct execution to work without the top level package already being importable. (** Does this boilerplating actually happen under the hood and the user does not have to take care of this explicitly?**)
This approach also has the same disadvantage as the use of absolute imports of sibling modules - if the script is moved to a different package or subpackage, the boilerplate will need to be updated manually. It has the advantage that this change need only be made once per file, regardless of the number of relative imports.Please elucidate this, preferably giving an example of disadvantage and advantage that they are talking about here
Note that setting __package__ to the empty string explicitly is permitted, and has the effect of disabling all relative imports from that module (since the import machinery will consider it to be a top level module in that case). This means that tools like runpy do not need to provide special case handling for top level modules when setting __package__.
The Python documentation for the import statement (link) contains the following:
The public names defined by a module are determined by checking the module’s namespace for a variable named __all__; if defined, it must be a sequence of strings which are names defined or imported by that module.
The Python documentation for modules (link) contains what is seemingly a contradictory statement:
if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered.
It then gives an example where an __init__.py file imports nothing, and simply defines __all__ to be some of the names of modules in that package.
I have tested both ways of using __all__, and both seem to work; indeed one can mix and match within the same __all__ value.
For example, consider the directory structure
foopkg/
__init__.py
foo.py
Where __init__.py contains
# Note no imports
def bar():
print("BAR")
__all__ = ["bar", "foo"]
NOTE: I know one shouldn't define functions in an __init__.py file. I'm just doing it to illustrate that the same __all__ can export both names that do exist in the current namespace, and those which do not.
The following code runs, seemingly auto-importing the foo module:
>>> from foopkg import *
>>> dir()
[..., 'bar', 'foo']
Why does the __all__ attribute have this strange double-behaviour?
The docs seem really unclear on how it is supposed to be used, only mentioning one of its two sides in each place I linked. I understand the overall purpose is to explicitly set the names imported by a wildcard import, but am confused by the additional, seemingly auto-importing behaviour. Is this just a magic shortcut that avoids having to write the import out as well?
The documentation is a bit hard to parse because it does not mention that packages generally also have the behavior of modules, including their __all__ attribute. The behavior of packages is necessarily a superset of the behavior of modules, because packages, unlike modules, can have sub-packages and sub-modules. Behaviors not related to that feature are identical between the two as far as the end-user is concerned.
The python docs can be minimalistic at times. They did not bother to mention that
Package __init__ performs all the module-like code for a package, including support for star-import for direct attributes via __all__, just like a module does.
Modules support all the features of a package __init__.py, except that they can't have a sub-package or sub-module.
It goes without saying that to make a name refer to a sub-module, it has to be imported, hence the apparent, but not really double-standard.
Update: How from M import * actually works?
The __all__ in __init__.py of folder foopkg works the same way as __all__ in foopkg.py
Why it'll auto-import foo you can see here: https://stackoverflow.com/a/54799108/12565014
The most import thing is to look at the cpython implementation: https://github.com/python/cpython/blob/fee552669f21ca294f57fe0df826945edc779090/Python/ceval.c#L5152
It basically loop through __all__ and try to import each element in __all__
That's why it'll auto-import foo and also achieve white listing