I know this is yet another question about "No module named x". But I've tried everything I could find from stackoverflow (and also the internet) but it's still not working for me.
Steps I create a project:
Create a project named my_project then cd my_project
Create a virtual environment python -m venv .env and open a VS Code workspace with this folder
This is my folder structure
my_project
├── __init__.py
├── folder_1
│ ├── file_1.py
│ └── folder_2
│ └── file_2.py -- this is where the function my_function is written
However, in file_1.py when I try to import a function in folder_2/file_2.py as below
# file_1.py
from folder_1.folder_2.file_2 import my_function
Then I bump into the following issue:
ModuleNotFoundError: No module named 'folder_1'
Hope someone can help.
UPDATE: CSBigSur is correct. In my case, it's just a problem with Pylance
Try this:
import folder2.file2 as f2
# your code
f2.your_func()
or
from folder2.file2 import your_func
# your code
your_func()
Related
I have a directory structure with 2 basic python files inside seperate directories:
├── package
│ ├── subpackage1
│ │ └── module1.py
└── subpackage2
└── module2.py
module1.py:
def module1():
print('hello world')
module2.py:
from ..subpackage1.module1 import module1
module1()
When running python3 module2.py I get the error: ImportError: attempted relative import with no known parent package
However when I run it with the imports changed to use sys.path.append() it runs successfully
import sys
sys.path.append('../subpackage1/')
from module1 import module1
module1()
Can anyone help me understand why this is and how to correct my code so that I can do this with relative imports?
To be considered a package, a Python directory has to include an __init__.py file. Since your module2.py file is not below a directory that contains an __init__.py file, it isn't considered to be part of a package. Relative imports only work inside packages.
UPDATE:
I only gave part of the answer you needed. Sorry about that. This business of running a file inside a package as a script is a bit of a can of worms. It's discussed pretty well in this SO question:
Relative imports in Python 3
The main take-away is that you're better off (and you're doing what Guido wants you to) if you don't do this at all, but rather move directly executable code outside of any module. You can usually do this by adding an extra file next to your package root dir that just imports the module you want to run.
Here's how to do that with your setup:
.
├── package
│ ├── __init__.py
│ ├── subpackage1
│ │ └── module1.py
│ └── subpackage2
│ └── module2.py
└── test.py
test.py:
import package.subpackage2.module2
You then run test.py directly. Because the directory containing the executed script is included in sys.path, this will work regardless of what the working directory is when you run the script.
You can also do basically this same thing without changing any code (you don't need test.py) by running the "script" as a module.
python3 -m package.subpackage2.module2
If you have to make what you're trying to do work, I think I'd take this approach:
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from subpackage1.module1 import module1
module1()
So you compute in a relative way where the root of the enclosing package is in the filesystem, you add that to the Python path, and then you use an absolute import rather than a relative import.
There are other solutions that involve extra tools and/or installation steps. I can't think why you could possibly prefer those solutions to the last solution I show.
By default, Python just considers a directory with code in it to be a directory with code in it, not a package/subpackage. In order to make it into a package, you'll need to add an __init__.py file to each one, as well as an __init__.py file to within the main package directory.
Even adding the __init__.py files won't be enough, but you should. You should also create a setup.py file next to your package directory. Your file tree would look like this:
├── setup.py
└── package
├── __init__.py
└── subpackage1
│ ├── __init__.py
│ └── module1.py
└── subpackage2
├── __init__.py
└── module2.py
This setup.py file could start off like this:
from setuptools import setup
setup(
name='package',
packages=['package'],
)
These configurations are enough to get you started. Then, on the root of your directory (parent folder to package and setup.py), you will execute next command in you terminal pip install -e . to install your package, named package, in development mode. Then you'll be able to navigate to package/subpackage2/ and execute python module2.py having your expected result. You could even execute python package/subpackage2/module2.py and it works.
The thing is, modules and packages don't work the same way they work in another programming languages. Without the creation of setup.py if you were to create a program in your root directory, named main.py for example, then you could import modules from inside package folder tree. But if you're looking to execute package\subpackage2\module2.py.
If you want relative imports without changing your directory structure and without adding a lot of boilerplate you could use my import library: ultraimport
It gives the programmer more control over their imports and lets you do file system based relative or absolute imports.
Your module2.py could then look like this:
import ultraimport
module1 = ultraimport('__dir__/../subpackage1/module1.py')
This will always work, no matter how you run your code or if you have any init files and independent of sys.path.
I have the following folder structure,
└── project
├── A
│ ├── main.py
│ └── __init__.py
└── B
├── __init__.py
├── C
├── __init__.py
└── module_x.py
I want to import all the methods in module_x.py into main.py. I have tried
from ..B.C.module_x import *
But I get the following error:
ImportError: attempted relative import with no known parent package
I wonder what am I doing wrong? How can this be done using relative import?
from project.B.C import foo
from ...b.c.module_x import foo
However, relative imports are only meant to work within one package. If project is a package, then you can use relative imports here. If project is not a package, you cannot.
However, if you're running a script in / and doing something like import project.A.b.foo, then that relative import will succeed because project is now a package. In that case, the following two would be equivalent:
from ...B.C import foo
from project.B.C import foo
You must use the -m switch to run python modules as scripts:\
$ cd project
$ python -m A.main # note no .py
This tells python that A.main is a module - python will also scan the current working dir (project) and detect package B - this will make your imports work correctly.
I am having problems with importing modules/packages with Python. I noticed this problem when I ran it in my terminal (CMD),
rather than my IDE (I use PyCharm). In PyCharm, I use a virtual enviroment setting with Python 3.7 and everything works as a charm
and as intended.
For reference this is how the imports were done in test_suite.py:
...
from tests.scenarios.test_scenario_01 import TestScenario # They work perfectly fine
from tests.scenarios.test_scenario_02 import TestScenario2 # written like this in PyCharm venv Python 3.7, but why?
...
This is a simplified version of my directory (without the unneccesary files):
QA System/
├── locators/
│ ├── locators.py
│ ├── __init__.py
├── pages/
│ ├── pages.py
│ └── __init__.py
└── tests/
├── reports
├── test_scenarios
├── test_scenario_01.py
├── test_scenario_02.py
├── __init__.py
|── test_suite.py
|── __init__.py
However when running the file test_suite.py manually through my CMD (because I want to integrate it with Jenkins
eventually), I get this error (py -3 test_suite.py):
ModuleNotFoundError: No module named 'tests'
Note: I am using the newest Python 3.7
From what I know about Python imports, for a directory to be treated like a python module, there needs to be a '__init__.py' file
included in the same directory.
After a bit of research I found out that it is possible to do a different type of imports in Python 3 and tried it out (putting a .
before the name of the imports). Like this:
from .scenarios.test_scenario_01 import TestScenario
from .scenarios.test_scenario_02 import TestScenario2
But still, it didn't run successfully and this was the error I've gotten:
ModuleNotFoundError: No module named '__main__.scenarios'; '__main__' is not a package
Could you please help me out on this one?
TLDR: Imports work in a Python3.7 venv, but not outside it
I fixed the issue by setting up a PYTHONPATH in system environment variables with the path to the project. As a value, I put the dir to the project. Thanks for the tips.
In case of directory It is mandate to use init.py. In modules it does not require to include init.py. Please, check what is your main directory and which is your modules..Hope this helps....
I have this code structure in python3:
- datalake
__init__.py
utils
__init__.py
utils.py
lambdas
__init__.py
my-lambdas.py
- tests
__init__.py
demo.py
All init__.py files are empty.
My problem is how I can import datalake module from tests/demo.py?
I tried from datalake.utils import utils in demo.py but when I run python tests/demo.py from command line, I get this error ModuleNotFoundError: No module named 'datalake'.
If I use this code:
from ..datalake.utils import utils
I will get error ValueError: attempted relative import beyond top-level package.
I also tried to import the module utils from my-lambda.py file which also failed. The code in my-lambda.py is from datalake.utils import utils but I get ModuleNotFoundError: No module named 'datalake' error when run python datalake/lambda/my-lambda.py from command line.
How can I import the module?
When you run a command like python tests/demo.py, the folder you are in does not get added to the PYTHONPATH, the script folder does. So a top-level import like import datalake will fail. To get around this you can run your tests as a module:
Python 2:
python -m tests/demo
Python 3:
python -m tests.demo
and any datalake imports in demo.py will work.
It sounds like what you really want to do is have a folder with tests separate to your main application and run them. For this I recommend py.test, for your case you can read Tests Outside Application Code for how to do it. TL;DR is run your tests from your top level project folder with python -m py.test and it will work.
First of all, my-lambdas.py is not importable with the import statement as hyphens are not valid in Python identifiers. Try to follow PEP-8's naming conventions, such as mylambdas.py.
Otherwise the package structure looks good, and it should be importable as long as you are at the level above datalake/, e.g., if you were in the directory myproject/ below:
myproject
├── datalake
│ ├── __init__.py
│ ├── utils
│ │ ├── __init__.py
│ │ └── utils.py
│ └── lambdas
│ ├── __init__.py
│ └── mylambdas.py
└── tests
├── __init__.py
└── demo.py
Then this should work:
~/myproject$ python -c 'from datalake import utils'
Otherwise, setting the environment variable PYTHONPATH to the path above datalake/ or modifying sys.path are both ways of changing where Python can import from. See the official tutorial on modules for more information.
Also some general advice: I've found it useful to stick with simple modules rather than packages (directories) until there is a need to expand. Then you can change foo.py into a foo/ directory with an __init__.py file and import foo will work as before, although you may need to add some imports to the __init__.py to maintain API compatibility. This would leave you with a simpler structure:
myproject
├── datalake
│ ├── __init__.py
│ ├── utils.py
│ └── lambdas.py
└── tests
├── __init__.py
└── demo.py
You can add the module directory into your sys.path:
import sys
sys.path.append("your/own/modules/folder") # like sys.path.append("../tests")
but this is a one-shot method, which is just valid at this time, the added path is not permanent, it will be eliminated after the code completed execution.
One of the ways to import the file directly instead of using from, like import util
you can try run :
python -m datalake.lambda.my-lambda
follow: https://docs.python.org/3.7/using/cmdline.html#cmdoption-m
I have a python 3 package with the following structure:
.
├── package
│ └── bin
└── main_module
│ └── lib
│ ├── __init__.py
│ ├── module1.py
│ ├── module2.py
│ └── module3.py
│ └── test
│ ├── test1.py
│ ├── test2.py
│ └── test3.py
│ └── setup.py
Usually, one runs $ python3 setup.py install and all is good. However, I want to use this package on a cluster server, where I don't have write permissions for /usr/lib/. The following solutions came to my mind.
Somehow install the package locally in my user folder.
Modify the package such that it runs without installation.
Ask the IT guys to install the package for me.
I want to avoid 3., so my question is whether 1. is possible and if not, how I have to modify the code (particularly the imports) in order to be able to use the package without installation. I have been reading about relative imports in python all morning and I am now even more confused than before. I added __init__.py to package and bin and from what I read I assumed it has to be from package.lib import module1, but I always get ImportError: No module named lib.
In order for Python to be able to find your modules you need to add the path of your package to sys.path list. As a general way you can use following snippet:
from sys import path as syspath
from os import path as ospath
syspath.append(ospath.join(ospath.expanduser("~"), 'package_path_from_home'))
os.path.expanduser("~") will give you the path of the home directory and you can join it with the path of your package using os.path.join and then append the final path to sys.path.
If package is in home directory you can just add the following at the leading of your python code that is supposed to use this package:
syspath.append(ospath.join(ospath.expanduser("~"), 'package'))
Also make sure that you have an __init__.py in all your modules.
I had the same problem. I used the first approach
install the package locally in my user folder by running
python setup.py install --user
This will install your module in ~/.local/lib/python3/
Just add the path of your 'package' to environment variable PYTHONPATH. This will get rid of the error you are getting.
OR
programmatically add path of the package to sys.path.append()
you can add this to the "main file" of the package
import sys, os
sys.path.append(os.path.dirname(__file__) + "/..")
you can find the "main file" by looking for this pattern
if __name__ == "__main__":
some_function()