We Keep Coding

Sphinx with both Python and Matlab (sphinxcontrib-matlabdomain): Import fail

I have a project including a Python package as well as some Matlab code, which should all go into the same documentation created by Sphinx.
The Python part of the documentation works flawlessly quite some time already - the Matlab part was added now and makes some trouble.
The data structure more or less is as follows:
|- python
|- modules
|- including subpackages
|- some sub-modules
|- matlab
|- scripts
|- functions
|- docs
|- source
|- conf.py
|- matlab.rst
|- python.rst
|- index.rst
|- ...
The most relevant lines in conf.py are prob. the following:
import os
import sys
# for the Python autodoc package
sys.path.insert(0, os.path.abspath('../..'))
extensions = ['sphinx.ext.napoleon', 'sphinxcontrib.matlab', 'sphinx.ext.autodoc']
# Path to the Matlab files
matlab_src_dir = '../../matlab'
#matlab_src_dir = '<<abssolute_path>>/matlab'
As you see, I tried both relative and absolute paths for matlab_src_dir. So far, without any difference.
The python.rst, which was created automatically by apidoc, contains:
python package
==============
.. automodule:: python
:members:
:undoc-members:
:show-inheritance:
Subpackages
-----------
.. toctree::
:maxdepth: 4
python.subpackages
The matlab.rst was created manually and contains:
matlab package
==============
.. mat:automodule:: matlab
:members:
:undoc-members:
:show-inheritance:
Subscripts
----------
.. toctree::
:maxdepth: 4
matlab.function_file
Thus, python.rst and matlab.rst are more or less the same but with mat:automodule for Matlab according to the documentation.
Finally, index.rst contains:
.. toctree::
:maxdepth: 2
python
.. toctree::
:maxdepth: 2
matlab
Now, when I run Sphinx' make html I receive the following error, which results in a nice Python documentation but an empty Matlab documentation:
WARNING: [sphinxcontrib-matlabdomain]: failed to import module 'matlab'; the following exception was raised:
Traceback (most recent call last):
File "C:\Users\<<username>>\AppData\Roaming\Python\Python39\site-packages\sphinxcontrib\mat_documenters.py", line 117, in import_object
obj = self.module = modules[self.modname]
KeyError: 'matlab'
What am I missing here. Why can't Sphinx find the matlab folder with the containing m-files?
And if that should be solved, maybe a secondary question: is there any similar function as apidoc for Matlab files, so that I do not need to create all *.rst files myself?
Thank you for any hints!
BTW: I am running on Sphinx v4.4.0 under Python 3.9.7.

Neither providing a hard-coded absolute path to matlab_src_dir nor providing os.path.abspath(relative_path) lead to success.
Thanks to Steve Piercy, who mentioned an example in the comments, I found that I should use os.path.dirname(os.path.abspath(relative_path)). As such, the errors are gone and Sphinx is working.
This is interesting, because I have already tried os.path.abspath(relative_path to the parent of the package) before, which I would expect is doing the same than os.path.dirname(). Anyway, I got a working solution now.
If anybody would have an idea about the second question ("is there any similar function as apidoc for Matlab files, so that I do not need to create all *.rst files myself?") I would be really happy.

Sphinx: the simplest usage scenario, how to? [duplicate]

I am running Sphinx on a rst file containing automodule but it does not seem to have any effect.
Here are the details: I have a Python project with a file agent.py containing a class Agent in it. I also have a subdirectory apidoc with a file agent.rst in it (generated by sphinx-apidoc):
agent module
============
.. automodule:: agent
:members:
:undoc-members:
:show-inheritance:
I run sphinx with sphinx-build -b html apidoc apidoc/_build with the project's directory as the current working directory.
To make sure the Python files are found, I've included the following in apidoc/conf.py:
import os
import sys
sys.path.insert(0, os.path.abspath('.'))
It runs without errors but when I open the resulting HTML file it only shows "agent module" and everything is blank. Why isn't it showing the class Agent and its members?
Update: the original problem was likely caused by the fact that I had not included sphinx.ext.autodoc in conf.py. Now that I did, though, I get warnings like:
WARNING: invalid signature for automodule ('My Project.agent')
WARNING: don't know which module to import for autodocumenting 'My Project.agent' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)
WARNING: autodoc: failed to import module 'agent'; the following exception was raised:
No module named 'agent'

I'll try answering by putting the "canonical" approach side-by-side with your case.
The usual "getting started approach" follows these steps:
create a doc directory in your project directory (it's from this directory the commands in the following steps are executed).
sphinx-quickstart (choosing separate source from build).
sphinx-apidoc -o ./source ..
make html
This would yield the following structure:
C:\Project
|
| agent.py
|
|---docs
| | make.bat
| | Makefile
| |
| |---build
| |
| |---source
| | conf.py
| | agent.rst
| | index.rst
| | modules.rst
In your conf.py you'd add (after step 2):
sys.path.insert(0, os.path.abspath(os.path.join('..', '..')))
and in index.rst you'd link modules.rst:
Welcome to Project's documentation!
================================
.. toctree::
:maxdepth: 2
:caption: Contents:
modules
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
Now compare the above with what you have - from what you shared in your question:
C:\Project
|
| agent.py
|
|---apidoc
| | agent.rst
| | conf.py
| |
| |-- _build
You ran:
sphinx-build -b html apidoc apidoc/_build
and in your conf.py:
sys.path.insert(0, os.path.abspath('.'))
Your error stacktrace says it couldn't find the module agent. That's probably because you didn't go 1 level down in your conf.py (it's pointing to the path with .rst, not the path with .py), this should work:
sys.path.insert(0, os.path.abspath('..')). Also, if you didn't manually edit/connect your modules.rst in your index.rst you are likely to only see that module.
You may care to notice the signatures of the sphinx commands at play:
sphinx-apidoc [OPTIONS] -o <OUTPUT_PATH> <MODULE_PATH>
sphinx-build [options] <sourcedir> <outputdir> [filenames …]
<sourcedir> refers to where .rst are, and <MODULE_PATH> to where .py are. <OUTPUT_PATH> to where .rst are placed, and <outputdir> to where .html are placed.
Please also notice, you mentioned: "the project's directory as the current working directory." I've seen "working directory" mentioned in sphinx threads on stackoverflow, interchangeably as both the Project base directory, or the docs directory. However, if you search the Sphinx documentation for "working directory" you'll find no mention of it.
Finally, there is an advantage to using the file/directory structure of the "getting started approach". It basically "puts you on the same page" with most threads on the Sphinx tag, and that way alleviates the mental work of mapping the cases to different directory/file structures.
I hope this helps.

How to make Sphinx autodoc extension include an underscore/private Python module

I am autogenerating documentation for a Python package using Sphinx with the autodoc extension. The issue I am facing is that the autodoc skips any modules with an underscore.
Some modules were underscored to discourage users from importing them. However, the classes inside these underscored files have no underscores.
When I add the underscored module manually to the package_name.rst and ran make html, it shows. So my problem is how to automate that from autodoc.
I am trying to avoid parsing the package_name.rst via a script to add them. I am hoping for an autodoc flag or a hack!

It's perhaps a misconception to think the ..automodule :: directive applied to a package will automatically document sub-modules as members (by comparison with classes, variables, etc).
I just tested this but it can not be done using :private-members: and or :special-members:. Not by writing either option in the .. automodule:: directive corresponding to the package. (Trying to set both options in autodoc_default_options gives the same result.)
The following example of package and module layout:
C:.
│
└────your_package
│
│ public_module.py
│ _private_module.py
│ __init__.py
Using a .rst with a single .. automodule:: for the package:
Your package rst
================
.. automodule:: your_package
:members:
:undoc-members:
:private-members:
:special-members:
Minimal example _private_module.py with docstrings (public_module.py is the same except for the title):
"""Private module docstring."""
class PublicClass:
"""Docstring."""
pass
Does indeed give an empty documentation:
But if you remove the underscore from the module you get the exact same result.
I am trying to avoid parsing the package_name.rst via a script to add them
If you are generating the .rst files with sphinx-apidoc if using the -P flag:
-P, --private
Include “_private” modules.
New in version 1.2.
The generated files will include an .. automodule:: directive for the private modules, this does have the side-effect of also including the :private-members: option as noted in another post "Include __main__.py in sphinx-apidoc generated files".
Example explicitly including the .. automodule:: directives:
Your package rst
================
.. automodule:: your_package
:members:
:undoc-members:
.. automodule:: your_package.public_module
:members:
:undoc-members:
.. automodule:: your_package._private_module
:members:
:undoc-members:
The result:

Importing modules from different folder python for unknown path

I have been working on a python project and I am new to it.
I have made a small library for my project in which I have several different modules doing different tasks.
For example: I have 5 modules namely add, subtract, multiply, divide and root.
I call all these .pyc files into my main.py file and my code runs properly if all of them are in the same folder.
Now, I want to store my main.py at: D:\project\main.py
and these 5 .pyc files at : D:\project\Lib\ (In the Lib folder)
I found a solution as to mention the path of the folder Lib into the code but I can not do so as I need to submit the code somewhere and if they try to run this on their PC, it might not import these files.
What would be the possible solution to this?

Try creating a package.
Use a directory structure like this:
.
+-- main.py
+-- lib
+-- __init__.py
+-- add.pyc
+-- substract.pyc
+-- ...
Then, in your main.py file, you can import them like this:
from lib import add
More on packages on Python docs

Inside D:\project\Lib create an __init__.py file. and put all your modules in D:\project\Lib now lib works as a python package.you dir structure should now look like this:
D:\project\Lib
|
+--- __init__.py
+--- add.py
+--- sub.py
+--- multiply.py
Now from any file inside (say for ex main.py) D:\project call any module you want like this.
from Lib.add import something.
final dir structure will roughly look like this.
D:\project
|
+-- main.py
+-- Lib
|
+--- __init__.py
+--- add.py
+--- sub.py
+--- multiply.py

Python Sphinx documenting public interface of a package

I have a Python package which contains submodules. Currently, my intention is to allow usage of functionality what the package exports, e.g.:
package_X
+-- __init__.py
+-- submodule_A.py
+-- submodule_B.py
Submodules are implementation details. Everything the user of the package needs to know is exported in the __init__.py file.
Now when building documentation with Sphinx, I get TOC and documentation as follows:
package_X
Submodules
submodule_A.py
submodule_B.py
Is there any setting or directive I can write to __init__.py or submodule_*.py to make public stuff appear as if it were directly in package_X? I am also ready to modify conf.py to make that work.