I'm trying to document a project using Sphinx, and am running into an issue where only some modules are being imported from a folder. My project structure looks like this:
Project
|
|--Main
| |--Scripts
| __init__.py
| libsmop.py
| conv_table.py
| f_discrim.py
| recipes.py
| ...
When I try to run make html, libsmop and recipes are imported without any issue, however conv_table and f_discrim get the following error:
WARNING: autodoc: failed to import module u'conv_table' from module u'Scripts'; the following exception was raised:No module named conv_table
I don't think it's my config file because it's finding all of the files when I run sphinx-apidoc -o _rst Main/Scripts and I've confirmed that they appear in the resulting Scripts.rst file.
Why is autodoc finding some modules but not others?
Edit:
conv_table.py is of this form:
import re
import numpy as np
"""
conv_table dictionary at the bottom of this file maps from matlab functions
to their python equivalents.
"""
def get_args(line,separator=",", open_char='(', close_char=')'):
"""Returns the arguments of line
>>> get_args('ones(3,1,length(arr))')
...
< a bunch of function definitions>
...
conv_table = {... < a very big dictionary > ...}
Since your autodoc is picking up some of the modules, it may be because the dependencies of the failed modules are either 1) not imported correctly or 2) not installed under your python environment. You will want to check if all the import statements work within your failed modules.
You will want to check the module loading path, according to the Sphinx docs:
For Sphinx (actually, the Python interpreter that executes Sphinx) to find your module, it must be importable. That means that the module or the package must be in one of the directories on sys.path – adapt your sys.path in the configuration file accordingly.
Also it would be useful to know how your __init__.py in Scripts directory looks like and how the conv_table module looks like as well.
I had a similar issue like yours, the fix was to append the path that holds that module inside the ../source/conf.py file using
sys.path.insert(0, os.path.abspath('whatever relative path works for your folder structure'))
sys.path.append('/path/to/the/conv_table/')
installing this library in your environment should resolve the problem as of now:
pip install sphinxcontrib-bibtex
after running the make html command it may warn you about the configuration problems.
Related
I'm trying to use mypy with a package that I've written, but it can't find my stub file.
I have a workspace which looks like this:
/common
/other_dir
/another_dir
I have used a script to add all of these directories to my sys.path.
Inside each directory is a src/ directory, which contains python packages, and is itself a top-level package (has an init.py).
in /common/src/test1 I have a module called components.py, and I've written another file next to it, components.pyi.
This should work as the stub file for components.py.
In /another/src/example.py, I import like this:
from common.src.test1.components.py import x
x is detected and I can use it, but when I run mypy ./another/src/example.py, it says 'Cannot find implementation or library stub for module named 'common.src.test1.components'.
It would be great if anyone who has experience with mypy could help with this.
Many thanks.
try
from common.src.test1.components import x # type: ignore
I have a python project structured like this:
repo_dir/
----project_package/
--------__init__.py
--------process.py
--------config.py
----tests/
--------test_process.py
__init__.py is empty
config.py looks like this:
name = 'brian'
USAGE
I use the library by running python process.py from the project/project/ directory, or by specifying the python file path absolutely. I'm running Python 2.7 on Amazon EC2 Linux.
When process.py looks like below, everything works fine and process.py prints brian.
import config
print config.name
When process.py looks like below, I get the error ImportError: No module named project.config.
import project.config
print config.name
When process.py looks like below, I get the error ImportError: No module named project. This makes sense as the same behavior from the previous example should be expected.
from project import config
print config.name
If I add these lines to process.py to include the library root in sys.path, all configurations above, work fine.
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
MY CONFUSION
Many resources suggest setting up python libraries to import modules using project.module_name, but it doesn't seem like sys.path appending is standard, and seems weird that I need it. I can see that the sys.path append added my library root as a path in sys, but I thought that's what the __init__.py in my library root was supposed to do. What gives? What am I missing? I know Python importing creates lots of headaches so I've tried to simplify this as much as possible to wrap my head around it. I'm going crazy and it's Friday before a holiday. I'm bummed. Please help!!
QUESTIONS
How should I set up my libraries? How should I import packages? Where should I have __init__.py files? Do I need to append my library root to sys.path in every project? Why is this so confusing?
Your project setup is alright. I renamed the directories just for clarity
in this example, but the structure is the same as yours:
repo_dir/
project_package/
__init__.py
process.py
config.py
# Declare your project in a setup.py file, so that
# it will be installable, both by users and by you.
setup.py
When you have a module that wants to import from another module in
the same project, the best approach is to use relative imports. For example:
# In process.py
from .config import name
...
While working on the code on your dev box, do your work in a Python virtualenv,
and pip install your project in "editable" mode.
# From the root of your repo:
pip install -e .
With that approach, you'll never need to muck around with sys.path -- which
is almost always the wrong approach.
I think the problem is how you're running your script. If you want the script to be living in a package (the inner project folder), you should run it with python -m project.process, rather than by filename. Then you can make absolute or explicit relative imports to get config from process.
An absolute import would be from project import config or import project.config.
An explicit relative import would be from . import config.
Python 2 also allows implicit relative imports, but they're a really bad misfeature that you should never use. With implicit relative imports, internal package modules can shadow top-level modules. For instance, a project/json.py file would hide the standard library's json module from all the other modules in the package. You can tell Python you want to forbid implicit relative imports by putting from __future__ import absolute_import at the top of the file. It's the standard behavior in Python 3.
So... I am attempting to teach myself Python.
In such, I am attempting to build something that I appear to have no clue about...
I have a "workingdir" structure such as:
/
-- classes/
-- -- install
-- myfile
In myfile I am simply attempting to "import" the file install by using:
import classes.install
Which fails with: ImportError: No module named 'classes.install'
I have attempted the following as well, and all end the same way, with the same error:
import .classes.install
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import classes.install
As well as putting an empty __init__.py file inside the classes directory
the file install simply contains:
class gyo_install():
inst = False
# check if we have everything we need installed.
def __init__():
print("Hello World")
What am I doing wrong? I've searched and searched and searched, everything I see points to the same solutions I've attempted, and none of them work.
Python looks for files with a .py extension when importing modules. So a file named myfile will not be recognized simply by the command import myfile. The pythonic way to ensure that the interpreter will find the module is to ensure it has a .py extension. Renaming myfile to myfile.py and install to install.py and then changing the import command to
import classes.install
should solve the problem.
Create __init__.py inside install directory.
Explanation: You can import from a file that is in your current directory or from a package. A package is a directory with __init__.py inside. In fact, a package can contain only this single file.
You can read the documentation for further information.
My folder structure in pycharm is as follows.
--python
--concepts
--common
--myds.py
--__init__.py
--data_structures
--test_ds.py
I have the following line in test_ds.py
from common import my_ds
I get the following error.
ImportError: No module named 'common'
I have added common to Settings --> Project Interpreter -> Interpreter Paths
and the folder shows up as library root.
Still why am I getting this error.
Try from ..common import my_ds. Also make sure that it has an __init__.py file in that directory (not required but it's good practice).
As for the .. they indicate that you're importing from the parent package to the one you're currently on.
You need to make your common folder into a python package in order to import it in python. I think you've tried to do it and created init file in your common folder but actually it must be __init__.py. Rename it like this and then your package will be visible to python.
Hope it helps!
I am trying to import the module
import QSTK.qstkutil.qsdateutil as du
But I get the Error
ImportError: No module named QSTK.qstkutil.qsdateutil
My current working directory is
'c:\\Python27\\Lib\\site-packages\\QSTK'
and in the path C:\Python27\Lib\site-packages\QSTK\qstkutil there are the files
qsdateutil.py
qsdateutil.pyc
qsdateutil.pyo
Does importing QSTK work?
import QSTK
How about QSTK.qstkutil?
If not this is most likely a sys.path problem. Please post the result of:
>>>import sys
>>>sys.path
It should look like:
[ [...], 'C:\Python27\Lib\site-packages', [...] ]
Another thing you can check, is if 'C:\Python27\Lib\site-packages\QSTK\qstkutil' contains a file named '__init__.py'. From the module documentation:
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.
try a fresh installation and make sure you run sudo python setup.py install , command after unpack-aging , QSTK. that process links QSTK.qstkutil.qsdateutil.