We Keep Coding

ModuleNotFoundError when importing local packages even though PyCharm autocompletes the name

I am trying to import modules from another Python project on my local machine. The two projects have the following structure
PycharmProjects
|---rlm_project
| |-init.py
| |---fdsm_folder
|
|---spc_project
|---venv
|---data_loader.py
I am trying to load the fdsm_folder from data_loader.py. I have added fdsm_folder to the path for the spc_project interpreter and when I start to import fdsm_folder pycharm suggests it as an autocomplete option. However when I run data_loader.py I get
ModuleNotFoundError: No module named 'fdsm'
What could be causing this issue?

Importing File Into __main__ from Parent Directory

Forgive me for another "relative imports" post but I've read every stack overflow post about them and I'm still confused.
I have a file structure that looks something like this:
|---MyProject
| |---constants.py
| |---MyScripts
| | |---script.py
Inside of script.py I have from .. import constants.py
And of course when I run python3 script.py, I get the error ImportError: attempted relative import with no known parent package.
From what I understand, this doesn't work because when you run a python file directly you can't use relative imports. After hours of searching it seems that the most popular solution is to add "../" to sys.path and then do import constants.py but this feels a little hacky. I've got to believe that there is a better solution than this right? Importing constants has got to be a fairly common task.
I've also seen some people recommend adding __init__.py to make the project into a package but I don't really understand how this solves anything?
Are there other solutions out there that I've either missed or just don't understand?

Your library code should live in a package. For your folder to be a package, it needs to have __init__.py.
Your scripts can live in the top directory in development. Here is how you would structure this layout:
|---MyProject
| |---myproject
| | |---__init__.py
| | |---constants.py
| |---script.py
If you were to productionize this, you could place scripts into a special folder, but then they would rely on the package being installed on your system. The layout could look like this:
|---MyProject
| |---myproject
| | |---__init__.py
| | |---constants.py
| |---scripts
| | |---script.py
In both cases your imports would be normal absolute (non-relative) imports like
from myproject.constants import ...
OR
from myproject import constants
You are correct that attempting a relative import or a path modification in a standalone script is hacky. If you want to use the more flexible second layout, make a setup.py install script in the root folder, and run python setup.py develop.

|---MyProject
| |---__init__.py # File1
| |---constants.py
| |---MyScripts
| | |---__init__.py # File 2
| | |---script.py
And then the content of File1 __init__.py is:
from . import constants
from . import MyScripts
And then the content of File2 __init__.py is:
from . import script
By converting MyProject to a package, you would be able to import constants like:
from MyProject import constants
# Rest of your code
After that, you need to add the root of your MyProject to your python path. If you are using PyCharm, you do not need to do anything. By default the following line is added to your Python Console or Debugging sessions:
import sys
sys.path.extend(['path/to/MyProject'])
If you are not using PyCharm, one is is to add the above script before codes you run or for debugging, add that as a code to run before every session, or you define a setup.py for your MyProject package, install it in your environment and nothing else need to be changed.

The syntax from .. import x is only intended to be used inside packages (thus your error message). If you don't want to make a package, the other way to accomplish what you want is to manipulate the path.
In absolute terms, you can do:
import sys
sys.path.append(r'c:\here\is\MyProject')
import constants
where "MyProject" is the folder you described.
If you want to use relative paths, you need to do a little more work:
import inspect
import os
import sys
parent_dir = os.path.split(
os.path.dirname(inspect.getfile(inspect.currentframe())))[0]
sys.path.append(parent_dir)
import constants

To put it concisely, the error message is telling you exactly what's happening; t's just not explaining why. .. isn't a package. It resolves to MyProject but MyProject isn't a package. To make it a package, you must include an “__init__.py” file directly in the “MyProject” directory. It can even be an empty file if you're feeling lazy.
But given what you've shown us, it doesn't seem like your project is supposed to be a package. Try changing the import in script.py to import constants and running python3 MyScripts/script.py from your project directory.

Sphinx autodoc fails to import module

I'm trying to document a project using Sphinx, and am running into an issue where only some modules are being imported from a folder. My project structure looks like this:
Project
|
|--Main
| |--Scripts
| __init__.py
| libsmop.py
| conv_table.py
| f_discrim.py
| recipes.py
| ...
When I try to run make html, libsmop and recipes are imported without any issue, however conv_table and f_discrim get the following error:
WARNING: autodoc: failed to import module u'conv_table' from module u'Scripts'; the following exception was raised:No module named conv_table
I don't think it's my config file because it's finding all of the files when I run sphinx-apidoc -o _rst Main/Scripts and I've confirmed that they appear in the resulting Scripts.rst file.
Why is autodoc finding some modules but not others?
Edit:
conv_table.py is of this form:
import re
import numpy as np
"""
conv_table dictionary at the bottom of this file maps from matlab functions
to their python equivalents.
"""
def get_args(line,separator=",", open_char='(', close_char=')'):
"""Returns the arguments of line
>>> get_args('ones(3,1,length(arr))')
...
< a bunch of function definitions>
...
conv_table = {... < a very big dictionary > ...}

Since your autodoc is picking up some of the modules, it may be because the dependencies of the failed modules are either 1) not imported correctly or 2) not installed under your python environment. You will want to check if all the import statements work within your failed modules.

You will want to check the module loading path, according to the Sphinx docs:
For Sphinx (actually, the Python interpreter that executes Sphinx) to find your module, it must be importable. That means that the module or the package must be in one of the directories on sys.path – adapt your sys.path in the configuration file accordingly.
Also it would be useful to know how your __init__.py in Scripts directory looks like and how the conv_table module looks like as well.

I had a similar issue like yours, the fix was to append the path that holds that module inside the ../source/conf.py file using
sys.path.insert(0, os.path.abspath('whatever relative path works for your folder structure'))
sys.path.append('/path/to/the/conv_table/')

installing this library in your environment should resolve the problem as of now:
pip install sphinxcontrib-bibtex
after running the make html command it may warn you about the configuration problems.

Python modulenotfounderror from command line

I have looked up solutions online for this issue, but none of them seem to address my specific situation.
I have a pycharm project with multiple directories, subdirs and files. When I invoke the main entry point method from the command line, I get
ModuleNotFoundError on all the imports across ~20 files. The solutions I found online recommend modifying the PYTHONPATH. This is not a viable solution for my use case because
I would have to add the sys.path.append call in all my files. That's the only way that it seems to work.
I cannot use any third party libs.
I will be sharing the project as a zip file. Note: I cannot use github to share the project nor is my intention to create a distributable. So when someone else unzips the project on their PC, they should be able to run it from the command line with out any issues. They should not have to modify their env variables to run it.
What are my options?
EDIT:
Project structure:
Project-
|
|
mod-
|
|
data-
|
|
ClassA.py
ClassB.py
|
config-
|
|
ClassC.py
ClassD.py
...
main.py
Import statements look like this:
from mod.data.ClassA import ClassA
Error:
ModuleNotFoundError: No module named 'ClassA'
This error shows up for every import statement in my python files.
When I add sys.path.append it works, but I have to do it for every import statement and its a hard coded abs path that will need to be updated by anyone with the zip.

If your modules are contained in the same directory as your executable, as you seem to indicate, it's this simple:
File structure:
f1.py
m1
__init__.py
C1.py
C1.py
class C1:
def test(self):
print("Hi!")
f1.py
#!/usr/bin/env python
from m1.C1 import C1
C1().test()
Execution:
> python f1.py
Hi!
You do have __init__.py files in your module directories, right? If not, then that's probably why you're having trouble with this.

Receiving Import Error: No Module named ***, but has __init__.py

I understand that this question has been asked several times but after reading them, and making the suggested fixes, I'm still stumped.
My project structure is as follows:
Project
|
src
|
root - has __init__.py
|
nested - has __init__.py
|
tests - has __init__.py
|
utilities - has __init__.py
|
services - has __init__.py
I've successfully run a unittest regression class from Eclipse without any issues.
As soon as I attempted to run the same class from the command-line (as other users who will be running the suite do not have access to Eclipse) I receive the error:
ImportError: No module named 'root'
As you can see from above, the module root has an __init__.py
All __init__.py modules are completely empty.
And assistance would be gratefully received.

Try adding a sys.path.append to the list of your imports.
import sys
sys.path.append("/Project/src/")
import root
import root.nested.tests

Just a note for anyone who arrives at this issue, using what Gus E showed in the accept answer and some further experience I've found the following to be very useful to ensure that I can run my programs from the command-line on my machine or on another colleague's should the need arise.
import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), "..", ".."))
When I execute the 'main' method, which is located in the 'nested' directory, it ensures that the 'src' directory is added to the PYTHONPATH at the time of execution meaning all following imports will not throw an error.
Obviously, you need to adjust the number of ".." arguments to the os.path.join() method as determined by the location in your program of where your main method is executed from

Yet another way to solve this without the path goes like this: consider the following code where inside your folder name 'app' you have 3 files x.py, y.py and an empty init.py. So to run x.py you have an import from y such that:
x.py
from app.y import say_hi
print ("ok x is here")
say_hi()
And
y.py
print ("Im Y")
def say_hi():
print ("Y says hi")
so the folder structure would look like this:
testpy
app
__init__.py
x.py
y.py
Solution: in the folder BEFORE app do the following:
$ python -m app.x
Note: I did not use x.py (simply app.x)
Result:
Nespresso#adhg MINGW64 ~/Desktop/testpy
$ python -m app.x
Im Y
ok x is here
Y says hi

If anybody lands here:
I encountered this error as well. In my case, I used ~/my/path/ at the path.sys.append(...), and the fix was replacing ~ with the explicit path name (you can inquire it if you type pwd when you are on linux shell, or use os.path.expanduser(..) )

Another way to do this so python searches for modules in the current directory is to add it as an environment variable to your .bash_profile / .zshrc / etc. like so:
export PYTHONPATH="${PYTHONPATH}:."