I have the next directory structure:
.
├── README.md
├── my_import
│ ├── __init__.py
│ ├── items.py
│ ├── middlewares.py
│ ├── pipelines.py
│ ├── spiders
│ │ ├── __init__.py
│ │ ├── spider1.py
│ │ ├── spider2.py
│ │ ├── spider3.py
│ ├── settings.py
│ ├── test.py
├── requirements.txt
├── scrapy.cfg
I want to test a one method of spider1 with unittests.
test.py
import unittest
from spiders.spider1 import SpiderA
class TestResult(unittest.TestCase):
def test_return(self):
string1 = '1'
string2 = '1st'
item = SpiderA()
self.assertEqual(item.get_result(string1), string1)
if __name__ == '__main__':
unittest.main()
But I received the error:
ModuleNotFoundError: No module named 'my_import'
However I am able to import settings.py, items.py, pipelines.py files in test.py file.
I think this is due to the reason I am importing class from items.py inside of spider1.py file and this caused this error.
Any ideas how I can overcome this issue?
I think what u have to do is :
from products_spiders.spider1 import SpiderA
Related
Based from a project with the following structure:
.
└── src/
├── main.py
├── PackageA/
│ ├── __init__.py
│ ├── logic.py
│ ├── SubPackageA1/
│ │ ├── __init__.py
│ │ └── util.py
│ └── SubPackageA2/
│ ├── __init__.py
│ └── otherUtil.py
└── PackageB/
├── __init__.py
└── helpers.py
Project structure
It would be possible to import in the file helpers.py the package otherutil.py?
All the combinations I tried until now fail.
If your program is executed from main.py the import in helpers.py should work like this:
from PackageA.SubPackageA2 import otherUtil
Yes, I checked it, main.py:
from PackageB import helpers
print(helpers.HELPERS_UTIL)
otherUtil.py:
OTHER_UTIL = 'test'
helpers.py
from PackageA.SubPackageA2 import otherUtil
HELPERS_UTIL = otherUtil.OTHER_UTIL
My project (written in python 2.7) has a complex structure and most modules are interlinked. There is no direct entry or link to this project to execute. It works as toolbox for other project.
When I tried to use sphinx to create the documentation it is giving error related to "unable to import module.
sample structure:
<workspace>
└── toolbox (main folder)
├── __init__.py
│
├── sub
│ ├── __init__.py
│ ├── sub1.py
│ └── sub2.py
│
├── subpackageA
│ ├── __init__.py
│ ├── submoduleA1.py
│ └── submoduleA2.py
│
└── subpackageB
├── __init__.py
├── submoduleB1.py
└── submoduleB2.py code[from sub import sub1
from subpackageA import submoduleA2 and so on]
Is there a way to configure the index.rst or the config.rst to ignore the import module error and give a output document directory like below:
└── toolbox
│
├── sub
│ ├── index
│ ├── sub1.m
│ └── sub2.m
│
├── subpackageA
│ ├── index
│ ├── submoduleA1.m
│ └── submoduleA2.m
│
└── subpackageB
├── index
├── submoduleB1.m
└── submoduleB2.m
I tried adding system path in config.rst
import os
import sys
sys.path.insert(0, os.path.abspath('../'))
tried ('../..') or ('..')
even hardcoded the project path.
even tried to use the sphinx.ext.autodoc but getting the same import error.
commands used:
sphinx-apidoc -o doc project/toolbox
make html
I have the following folder structure:
...
│
├── src
│ ├── folder_A
│ │ └── file_A.py
│ │ └── __init__.py
│ │
│ ├── folder_B
│ │ └── file_B.py
│ │ └── __init__.py
│ │
│ └── __init__.py
│
│
└── something else
In the file file_A.py I put from folder_B import file_B as fb. But file_A.py works only in debug mode (meaning that the code produces the expected results). If I run file_A.py in the standard way I get the error ModuleNotFoundError: No module named 'folder_B'.
I also changed the configuration before running the code, putting C:\Users\***\***\***\src as the working directory of file_A.py but it still doesn't work.
What can be a solution?
If your current directory is src, then folder_B is in the path because of that. If you want to make a package with sub-packages that can access each other, place everything into a root package:
│
├── src
│ └── root_package
│ ├── folder_A
│ │ ├── file_A.py
│ │ └── __init__.py
│ │
│ ├── folder_B
│ │ ├── file_B.py
│ │ └── __init__.py
│ │
│ └── __init__.py
│
└── something else
Now in file_A, you can do
from ..folder_B import file_B as fb
Since src is not a package, you can't do a relative import through it. By adding root_package, you make it possible to find folder_B in the same package hierarchy (albeit a different branch) as the module doing the import.
Im writing a library for internal use,its called "etllib", and I have the following structure:
etl-lib
├── README.md
├── etllib
│ ├── __init__.py
│ ├── client
│ │ ├── __init__.py
│ │ ├── elastic.py
│ │ └── qradar.py
│ ├── etl
│ │ ├── __init__.py
│ │ └── etl_imperva.py
│ └── util
│ ├── __init__.py
│ ├── config.py
│ ├── daemon.py
│ ├── elastic
│ │ ├── __init__.py
│ │ └── impeva_index_config.py
│ └── imperva
│ ├── __init__.py
│ ├── kpe_config.py
│ └── query_config.py
├── scripts
│ └── etl_imperva
└── setup.py
And I have a script called "etl_imperva" in etllib/scripts. The code inside looks like this :
#!/usr/bin/python3
import sys
from etllib.etl.etl_imperva import ETL
# Run with python3 imperva_run.py start|run|stop|restart
ETL.startup(sys.argv)
If I install this package(etllib) and call this script, it works just fine. But when I need to test stuff, how can I tell python to use the modules that are on my working directory instead the ones are installed? Because each time I make a change on the modules, I need to reinstall the package and this is a little time consuming.
I also tried uninstalling the package for testing, but whe I run this script I get the following error :
Exception has occurred: ModuleNotFoundError
No module named 'etllib'
File "/home/jleonse/etl-lib/scripts/run_imperva", line 3, in <module>
from etllib.etl.etl_imperva import ETL
Is there a better way to do this?
Actually, it is not on the same level in the hierarchy.
from etllib.etl.etl_imperva import ETL
would work only if etllib was in the same directory or in a directory in your system PATH, but the etllib is in the parent directory, hence it can not find it.
so you can make it work if you change the project structure to be:
etl-lib
├── README.md
├── etllib
│ ├── __init__.py
│ ├── client
│ │ ├── __init__.py
│ │ ├── elastic.py
│ │ └── qradar.py
│ ├── etl
│ │ ├── __init__.py
│ │ └── etl_imperva.py
│ └── util
│ ├── __init__.py
│ ├── config.py
│ ├── daemon.py
│ ├── elastic
│ │ ├── __init__.py
│ │ └── impeva_index_config.py
│ └── imperva
│ ├── __init__.py
│ ├── kpe_config.py
│ └── query_config.py
├── etl_imperva
│
└── setup.py
I'm trying to run django unittest using VSCode, not terminal.
my project tree looks like this:
├── db.sqlite3
├── hero
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-37.pyc
│ │ ├── admin.cpython-37.pyc
│ │ ├── apps.cpython-37.pyc
│ │ ├── models.cpython-37.pyc
│ │ ├── tests.cpython-37.pyc
│ │ ├── urls.cpython-37.pyc
│ │ └── views.cpython-37.pyc
│ ├── admin.py
│ ├── apps.py
│ ├── migrations
│ │ ├── 0001_initial.py
│ │ ├── 0002_hero_age.py
│ │ ├── __init__.py
│ │ └── __pycache__
│ │ ├── 0001_initial.cpython-37.pyc
│ │ ├── 0002_hero_age.cpython-37.pyc
│ │ └── __init__.cpython-37.pyc
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── manage.py
└── toh
├── __init__.py
├── __pycache__
│ ├── __init__.cpython-37.pyc
│ ├── settings.cpython-37.pyc
│ ├── urls.cpython-37.pyc
│ └── wsgi.cpython-37.pyc
├── settings.py
├── urls.py
└── wsgi.py
I made tests.py file inside hero directory.
My tests.py code looks like this:
from django.test import TestCase, Client
from .models import Hero
# Create your tests here.
class HeroTestCase(TestCase):
def setUp(self):
Hero.objects.create(name='Superman', age=10)
Hero.objects.create(name='Batman', age=1)
Hero.objects.create(name='Ironman', age=30)
def test_hero_count(self):
self.assertEqual(Hero.objects.all().count(), 3)
def test_hero_id(self):
client=Client()
response=client.get('/hero/1/')
self.assertEqual(response.status_code, 200)
self.assertIn('1', response.content.decode())
def test_hero_visit_count(self):
client = Client()
response = client.get('/hero/hello')
self.assertEqual(response.status_code, 200)
self.assertIn('1', response.content.decode())
response = client.get('/hero/hello')
self.assertEqual(response.status_code, 200)
self.assertIn('2', response.content.decode())
And my .vscode/settings.json looks like this:
{
"python.pythonPath": "/anaconda3/bin/python",
"python.linting.pylintEnabled": true,
"python.linting.enabled": true,
"python.testing.unittestArgs": [
"-v",
"-s",
"./hero",
"-p",
"*test*.py"
],
"python.testing.pytestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.unittestEnabled": true
}
But when I ran test by VSCode this error keeps coming out.
======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests
Traceback (most recent call last):
File "/anaconda3/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/anaconda3/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "toh/hero/tests.py", line 2, in <module>
from .models import Hero
ImportError: attempted relative import with no known parent package
Or:
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
So I checked whether there are something wrong with my test code, but my test code pass when I ran python manage.py test ! How do I resolve this problem?
The relative import problem is because you set -p to hero which changes the top-level directory to that and so it no longer looks like a package to Python.
The configuration problem is because unittest isn't running manage.py. You can go to https://github.com/microsoft/vscode-python/issues/73 and 👍 the issue to vote for it to be prioritized.