We Keep Coding

Import error from different directory in python (Err - No module named app)

I have gone through numerous answers and tried most of them as well but still facing the same issue
My structure
-- backend
-- app
-- __init__.py
-- utils.py
-- models.py
-- pytest
-- test_utils.py`
Now I want to import utils into the test_utils to be able to call the function and test it. My init.py is not empty and my test_utils.py looks like this
import sys, os.path
sys.path.insert(0 ,(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')) + '/app/'))
from utils import test_function
def test_test_function():
a = test_function(5)
assert a == 5
I checked my sys path is also pointing to correct directory but keep getting this error and I am using Python 2.7 in Linux
ImportError while importing test module '/home/richie/Documents/Project/backend/pytest/test_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../virenv/local/lib/python2.7/site-packages/six.py:709: in exec_
exec("""exec _code_ in _globs_, _locs_""")
test_utils.py:3: in <module>
from utils import test_function
../app/utils.py:15: in <module>
from models import Users, Accounts
../app/models.py:2: in <module>
from app import db
E ImportError: No module named app

The solution is to create a pytest.ini file in pytest/ folder.
The content of pytest.ini would be:--
[pytest]
python_paths = . ../app

Turns out that have to add path to both app and utils and then it works
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
sys.path.append((os.path.abspath(os.path.join(os.path.dirname(__file__), '..')) + '/app/'))

How to fix appengine ImportError: No module named protobuf?

I have the following folders structure:
myappdir
- libs
- somelib
script1.py
script2.py
- google
- protobuf
__init__.py
message.py
...
__init__.py
...
app.yaml
appengine_config.py
...
And the following files content -
appengine_config.py:
import sys
sys.path.append('libs')
script1.py:
from somelib.script2 import Something
script2.py:
from google.protobuf import message
In result I get:
File "myappdir/libs/somelib/script1.py", line 34, in <module>
from somelib.script2 import Something
File "myappdir/libs/somelib/script2.py", line 38, in <module>
from google.protobuf import message
ImportError: No module named protobuf
What is wrong with my setup?

Change the lines in your appengine_config.py file, from:
import sys
sys.path.append('libs')
to:
import sys
import os.path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'libs'))

I found #Daniel's answer to already be implemented in my setup, but still had the problem. This github comment helped me. Adding the following to appengine_config.py solved the problem for me:
from google.appengine.ext import vendor
vendor.add('lib')
import google.protobuf; print(google.protobuf.__version__)
change lib to libs depending on your project directory naming.

cannot find module in another directory

I have the project structure:
/hdfs-archiver
/logs
/qe
__init__.py
/tests
__init__.py
archiver.py
/utils
__init__.py
HdfsConnector.py
I am trying to run archiver.py but I am getting this error:
Traceback (most recent call last):
File "qe/tests/HdfsArchiver.py", line 8, in <module>
from qe.utils import HdfsConnector
ImportError: No module named qe.utils
I read around and it seemed like most people that come across this issue fixed it with __init__.py
when I pwd:
$ pwd
/Users/bli1/Development/QE/idea/hdfs-archiver
my PYTHONPATH in .bashrc
export PYTHONPATH=$PYTHONPATH:/Users/bli1/Development/QE/idea/hdfs-archiver
I also tried having my PYTHONPATH as
/Users/bli1/Development/QE/idea/hdfs-archiver/qe

You're trying to import HdfsConnector as a function or class. Include the HdfsConnector module as part of your absolute import:
from qe.utils.HdfsConnector import my_function
You can also import the module:
import qe.utils.HdfsConnector
# or
import qe.utils.HdfsConnector as HdfsConnector

Firstly, you could try a relative import such as
from ..utils import HdfsConnector
You'd also need to run the script as a module and not as a simple python script due to the __name__ being different. This wouldn't require you to modify the path.
You can find more info here.

Importing a module into a custom command

Suppose I have the following Django project:
/ # Project dir
/myapp # App dir
/myapp/views.py # views file
/myapp/mymodule.py # a Python module
/myapp/management/commands/mycommand.py # command file
In the views file, I can import mymodule.py by simply writing import mymodule in views.py. However, if I do the same in mycommand.py, I get the error: ImportError: no module named mymodule. I know that to import a model, I can write from myapp.models import mymodel, but mymodule is not a model, it is a separate Python module. So, how do I import this moduel into my command file?

Thats because of where mymodule is in relation to all of your other files.
If you type
from myapp import mymodule
In mycommand.py as long as you have set up your __init__.py files correctly in myapp and are using a setup.py file you should be fine. The modules documentation has more information on this

The fact that you are unable to do this means your myapp folder is not in the python's sys.path. The reason it worked in views.py is because mymodule.py is in the current directory of views.py
Assuming your Project dir (/) is in the python's sys.path:
Add a file __init__.py in myapp directory. This file can be blank.
Then you can do import myapp.mymodule or from myapp import mymodule
The same logic applies for n levels deep. If you have myapp/myappdir1/myappdir2/myfile.py you can do import myapp.myappdir1.myappdir2.myfile assuming each of these have __init__.py file in them.
Check https://docs.python.org/2/tutorial/modules.html#the-module-search-path for python's module search path

How to force import a lib from python site-package?

I am using custom django struct as below:
all settings file in conf, all app in src
and need use below manage.py:
if __name__ == "__main__":
ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
SRC_PATH = os.path.join(ROOT_PATH, 'src')
CONF_PATH = os.path.join(ROOT_PATH, 'conf')
sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
And I also need to use celery in django.
But django-celery need a celery.py file in same directory with settings.py.
When runserver ,it would raise ImportError: cannot import name Celery because below code:
sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)
It import itself! not from site-package, because CONF_PATH is before site-package.
But I can't change that to
sys.path.append(SRC_PATH)
sys.path.append(CONF_PATH)
This way would cause django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
For now, the only way I know is changing celery.py to celery_somename.py, and I have to type this name every time when I start a celery job.
How do I force import a lib from python site-package? Or is there another way to start celery?

You can add at the top of the celery.py module and other modules importing celery:
from __future__ import absolute_import
This will make imports absolute by default, now:
import celery
Will import the installed celery package and not this module.