I have the following structure:
│
├── pckg1/
│ └── utils/
│ ├── module11.py
│ └── module12.py
│
├── pckg2/
│ └── utils/
│ ├── module21.py
│ └── module22.py
│
└── main_pckg/
├── utils/
└── main.py
When I run main.py, I import required methods from both module11.py and module21.py under utils directories under from pckg1 and pckg2.
In side those modules there are some local imports. For Example (pckg1/utils/module11.py):
from utils.module21 import X,Y,Z
when I run main_pckg/main.py, the following error naturally raises:
No module named `utils/module21` under the current working dir.
Since there is no module21
A naive solution is to modify those imports to relative import:
from ..utils.module21 import X,Y,Z
But I have many modules and subprojects with are tracked and shall not be modified.
Any workarounds to solve it?
Python is trying to reach those files from main.py.
You can add those folders to path from main file.
import sys
sys.path.append('../pckg1/utils')
sys.path.append('../pckg2/utils')
After that you won't need to specify the folder name
from module21 import x, y, z
should work
Related
i'm experimenting with DDD in Python so i've decided to implement a toy project.
I've created different directories in other to separate shared concepts from specific bounded contexts concepts.
As i try to import these files, i'm facing a No module named error exceptions
For example, with this project structure:
.
└── src/
├── Book/
│ ├── application
│ ├── domain/
│ │ ├── Book.py
│ │ └── __init__.py
│ ├── infrastructure
│ └── __init__.py
└── Shared/
├── application
├── domain/
│ ├── Properties/
│ │ ├── __init__.py
│ │ └── UuidProperty.py
│ ├── ValueObjects/
│ │ ├── __init__.py
│ │ └── BookId.py
│ └── __init__.py
└── infrastructure
On src/Book/domain/Book.py i have:
from Shared.domain.ValueObjects.BookId import BookId
class Book:
bookId: BookId
pages: int
As i've seen in other answer (pretty old ones) it can be fixed by adding these folders to PYTHONPATH or PATH like sys.path.insert(*path to file*) but i'm wondering if there is a more pythonic way to achieve that.
I've also tried to add an __init__.py file to src and import as from src.Shared.domain.ValueObjects.BookId import BookId but none of previous attempts worked for me
On other repos i've saw that they use setuptools to install the src package in order to import it at unit tests (i cant either import them at tests), but i don't know if that is recommended or would work inside package imports
In case someone is facing the same issue as me, i managed to import subpackages and the full package in tests directory.
Just include, in each subpackage, an __init__.py file, and inside the package/subpackages, use relative imports (it looses semantic of imports, when we know where each imports comes from by absolute path from root directory, but works)
from ..Properties import UuidProperty
# inside __init__.py of Properties directory
from .UuidProperty import UuidProperty
And, by including an __init__.py inside src/ we could import them at tests directory like
from src.Book.domain import Book
Hope this helps someone!
While working on my first "bigger" Python project, I'm running into a multitude of issues while trying to debug and import various modules/sub-modules. Here's my tree:
netbox-setup/
├── README.md
├── TODO.md
├── netboxsetup
│ ├── __init__.py
│ ├── constants.py
│ ├── helpers
│ │ ├── __init__.py
│ │ ├── custom_napalm
│ │ │ ├── __init__.py
│ │ │ └── ios.py
│ │ ├── infoblox.py
│ │ ├── ise.py
│ │ ├── netbox.py
│ │ ├── solarwinds.py
│ │ └── utilities.py
│ └── main.py
├── requirements.txt
├── setup.py
└── tests
main.py imports:
from netboxsetup.helpers import utilities
from netboxsetup.helpers import solarwinds
utilities.py imports:
from napalm import get_network_driver
from netboxsetup.constants import USER
From what I've been reading, it's recommended to use absolute imports in a package rather than relative. If I try to run main.py from within the netboxsetup folder, it states that netboxsetup cannot be found. So I removed that and just called from helpers import utilities. Now running main.py works, but when it imports the utilities file, the imports in the utilities file fail. Information I'm finding regarding import use in a package/module seems to be inconsistent on what to do/use.
Finally, if I run a python3 shell from the the netbox-setup folder, and use from netboxsetup.helpers import utilities it imports. If I do the same from the netboxsetup folder, it states that "ModuleNotFoundError: No module named 'netboxsetup'". Going back to where it worked, I then followed https://napalm.readthedocs.io/en/latest/tutorials/extend_driver.html to create the same setup exactly, and it states that my new method isn't found when I run the grab_inventory function I defined in the utilities.py. And I did appropriately create the respective get_inventory function in the class in the ios.py file as the Napalm docs advise.
def grab_inventory(ip, password):
# make less assumption, account for nx-os as well
driver = get_network_driver('ios')
with driver(ip, USER, password) as client:
result = client.get_inventory()
return result
I'm guessing all of my issues are related to pathing - whether absolute or relative, but I'm just having a very difficult time determining what the exact pathing is that works for everything. Is anyone able to point me into a proper source for proper import of modules in custom packages? Thanks.
P.S. Is it possible to debug an individual function in VSCode, giving it arguments at runtime as well (rather than having to run through all of the main sequence code to get to the function)?
Consider you have the following project structure:
FooProject
├── foo_components
│ ├── __init__.py
│ └── foo1.py
├── utils
│ ├── __init__.py
│ └── utils1.py
├── tests
│ ├── __init__.py
│ └── foo_components
│ ├── __init__.py
│ └── test_foo1.py
│ ├── utils
│ └── __init__.py
│ └──test_utils1.py
|
└── __init__.py
My question is as follows:
Consider you are in scope of test_foo1.py file
in order to test the spectacular foo component called :foo1.
You want to import the following:
from foo_components.foo1 import Foo1
But then you get an error because of ambiguity:
Module not found for foo_components.foo1.
This is probably because in your scope foo_components module is referred to as the path of modules:
tests.foo_components. ( meaning it doesn't have foo1, more likely test_foo1).
How would you solve such a case elegantly?
Can I cause python to understand which python module I'm referring to, seems like it automatically make assumptions as relative import, and not to the top-level module.
In your solution please avoid suggesting importing with relative imports, far as I know its not bad practice, meaning something like from ../../xyz import foo1.
I thought of changing foo_component to not be same name also under tests, but I don't like this solution either, I guess there is some reasonable way.
Any suggestions?
You can try importing it specifically from the project root library.
Like this:
From FooProject.foo_components import foo1.py
I’m this case it won’t go to the other foo_components, and if you want to go there than you can do:
From FooProject.tests.foo_components import test_foo1.py
I'm building a website with the bottle framework. I am using nosetests for my unit testing. However there is a problem I can't solve on my own.
I can't seem to find a way that allows me to run the tests without breaking my server. When I run my tests, I have to do relative imports for them to work, but they don't work when I start the server.
This is my folder structure:
├── service
│ ├── __init__.py
│ ├── application.py
│ ├── main.py
│ ├── response_header.py
│ ├── global_data_loader.py
│ ├── renderer
│ │ ├── __init__.py
│ │ ├── header.py
│ │ ├── ....
│ ├── tests
│ │ ├── test_application.py
│ │ ├── ....
Here is how I import so my server works:
application.py --
from response_header import ResponseHeader
from global_data_loader import GlobalDataLoader
from renderer import Application
However, when I run nosetests, I get this message:
ModuleNotFoundError: No module named 'response_header'
So when I want nosetests to work, I have to change the imports to look like this:
application.py --
from .response_header import ResponseHeader
from .global_data_loader import GlobalDataLoader
from .renderer import Application
Then my nosetests work, but when I want to start my server I get this message:
from .response_header import ResponseHeader
ImportError: attempted relative import with no known parent package
I have tried using sys.path.append in my tests, and it works, but I need a solution that doesn't involve PYTHONPATH or using os/sys
edit: i fixed it by moving my main.py I use to start the server to the root folder, above my service and tests module. now it works fine
app structure:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── template.yaml
├── tests
│ ├── __init__.py
│ └── unit
│ └── lambda_application
│ ├── test_handler.py
│ └── test_parent_child_class.py
└── lambda_application
├── __init__.py
├── first_child_class.py
├── lambda_function.py
├── second_child_class.py
├── requirements.txt
└── parent_class.py
4 directories, 14 files
Code sample from lambda_function.py:
import os
import json
from hashlib import sha256
import boto3
from requests import Session
from .first_child_class import FirstChildClass
def lambda_handler(event, context):
# Do some stuff.
As is, I get the error message
Unable to import module 'lambda_function'
but If I comment out the last import, from .first_child_class import FirstChildClass, it is able to get past that part and get the error that I haven't loaded the module for that class.
I only seem to get this error when I run it in the lambci/lambda:python3.7 docker image and when I deploy on AWS. All my tests pass and it is able to import the module with no problems.
Is there something I should load/setup in the __init__.py file?
EDIT I changed the names of some of the files to post it here.
You are using a relative import here which works in case the code you are executing is in a module. However, since your code is being executed not as a module, your AWS Lambda fails.
https://stackoverflow.com/a/73149/6391078
A quick run locally gave the following error:
PYTHON 3.6
Traceback (most recent call last):
File "lambda_function.py", line 4, in <module>
from .first_child_class import FirstChildClass
ModuleNotFoundError: No module named '__main__.first_child_class'; '__main__' is not a package
Your tests pass because your testing suite imports the file as a module from the lambda_application folder which gets treated as a package in the testing module
This got me going in the correct direction but didn't quite give me the answer but did lead me to the answer, so I thought I would update what I found here.
I didn't try it but from what I found, I believe that:
from first_child_class import FirstChildClass
would be the simplest resolution.
What I ended up doing was moving the classes into a sub-directory and essentially did the same as above but with a package name prepended.
So, the file structure changed to:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── template.yaml
├── tests
│ ├── __init__.py
│ └── unit
│ └── lambda_application
│ ├── test_handler.py
│ └── test_parent_child_class.py
└── lambda_application
├── __init__.py
└── lib
├── first_child_class.py
├── second_child_class.py
└── parent_class.py
├── lambda_function.py
└── requirements.txt
and my import became from lib.first_child_class import FirstChildClass