Python project structure for selenium test - python

I'm a bit rusted with Python (2.7) modules & packages so I hope to find some help. I am writing some tests using selenium. I would like to organize the tests by "scenario families" and I'd like to implement a helper class to handle some boilerplate that would be my base class for all the tests.
Basically I'd like to have a structure looking like:
.
├── assets
│ └── ressource.ext
├── tests
│ ├── __init__.py
│ ├── user
│ │ ├── __init__.py
│ │ └── upload.py
│ └── visitor
│ ├── __init__.py
│ ├── homepage.py
│ ├── login.py
│ ├── search.py
│ └── signup.py
└── utils
├── __init__.py
└── base.py
I am completely stuck on how to make the utils.base module visible to e.g. the tests.visitor.signup module and still allow to run this specific test using python tests/visitor/signup.py.
How would I do that?
Thanks!
Edit: to make things easier here is a dummy sample of what I'm trying to do:
mkdir -p {utils,tests/user}
touch {utils,tests{,/user}}/__init__.py
echo -e "import unittest\n\nclass Base(unittest.TestCase):\n pass" > utils/base.py
echo -e "from utils.base import Base\n\nclass MyTest(Base):\n pass\n\nif __name__ == '__main__':\n unittest.main()" > tests/user/upload.py
This produces the following tree with empty __init__.py files and the import I'm trying to achieve in tests/user/upload.py:
dummy/
├── tests
│   ├── __init__.py
│   └── user
│   ├── __init__.py
│   └── upload.py
└── utils
├── __init__.py
└── base.py
Now if I am in the dummy/ folder I am getting this:
(env)dummy $ python tests/user/upload.py
Traceback (most recent call last):
File "tests/user/upload.py", line 1, in <module>
from utils.base import Base
ImportError: No module named utils.base
But if I run it interactively there is obviously no problem:
(env) dummy $ python
Python 2.7.6 (default, Jan 16 2014, 16:39:48)
>>> from utils.base import Base
>>>
My problem must be really silly but I can't see what I'm doing wrong. And what I don't get also is that specifying the python path ((env) dummy $ PYTHON_PATH=. python tests/user/upload.py) doesn't fix the issue.

You can set PYTHONPATH to search the current directory:
PYTHONPATH=. python tests/user/upload.py
Or you can use a test runner, like nose:
nosetests tests/user/upload.py
Or you can create your own test runner if you don't want to use nose. For instance a runtests.py file with:
import unittest
from tests.user.upload import *
unittest.main()
Then:
python runtests.py
This test runner could import more tests or could be selective about what tests it imports. Ultimately, I'd recommend using nose over writing your own test runner.
Starting a test runner at the top of your hierarchy works because Python adds the current directory of a script to the list of paths it searches for modules.

Related

ImportError: No module named unable to import toplevel directory of module in python

My Directory structure:
├── common
│   ├── common.py
│   └── __init__.py
├── project1
│   ├── __init__.py
│   └── scripts
│   ├── example_import.py
│   └── __init__.py
└── project2
├── __init__.py
└── scripts
└── __init__.py
I need to import common/common.py module in project1/scripts/example_import.py file
example_import.py:
import sys
sys.path.append("../common")
from common import Test
print("Module Not import error")
Error:
Traceback (most recent call last):
File "project1/scripts/example_import.py", line 3, in <module>
from common import Test
ImportError: No module named common
How to fix a issue?
Understanding how Python imports work is tricky in the beginning but makes sense once you understand how.
There are different way to fix your import issue. I would not recommend messing up with sys.path. Depending on where you are calling your script, you have multiple choice at hand.
Your Directory structure.
├── common
│   ├── common.py
│   └── __init__.py
├── project1
│   ├── __init__.py
│   └── scripts
│   ├── example_import.py
│   └── __init__.py
└── project2
├── __init__.py
└── scripts
└── __init__.py
On the root of directory
python project1/scripts/example_import.py
Will work, assuming that the imports in example_import.py looks like
from common.common import Test
If you want to use from common import Test, you need to add from common.common import Test in __init__.py file in common folder.
You can also use PYTHONPATH environment variable to tell Python where to look for the base module.
PYTHONPATH=/pathto/basefolder python folder/filesx.py
Another way is to create a setup.py on base and do development installation python -m pip install --editable . on environment you are working on.
Example of setup.py
#!/usr/bin/env python
from distutils.core import setup
setup(name='projectX',
version='1.0',
description='The World is not Enoug',
author='James Bond',
author_email='agent#007.net',
url='https://www.python.org/sigs/distutils-sig/',
packages=['common', 'project1', 'project2],
)
See Python Documentation for more setup.py options.

How can I import my own package in python?

I have a project
testci/
├── __init__.py
├── README.md
├── requirements.txt
├── src
│   ├── __init__.py
│   └── mylib.py
└── test
├── __init__.py
└── pow_test.py
When I run python3.6 test/pow_test.py I see an error:
File "test/pow_test.py", line 3, in
import testci.src.mylib as mylib
ModuleNotFoundError: No module named 'testci'
pow_test.py
from testci.src.mylib import get_abs
def test_abs():
assert get_abs(-10) == 10
How can I fix this error?
System details: Ububntu 16.04 LTS, Python 3.6.10
try this
from .src import mylib
from mylib import get_abs
if it won't work then import one by one. But don't import the root folder since the file you are importing to is on the same folder you are trying to import then it will always raise an error
Run Python with the -m argument within the base testsci package to execute as a submodule.
I made a similar mock folder structure:
├───abc_blah
│ │ abc_blah.py
│ │ __init__.py
│
└───def
│ def.py
│ __init__.py
abc_blah.py
print('abc')
def.py
import abc_blah.abc_blah
Execute like such:
python -m def.def
Correctly prints out 'abc' as expected here.
simply add __package__ = "testci" and also it is a good practice to add a try and except block
Your final code should look something like
try:
from testci.src.mylib import get_abs
except ModuleNotFoundError:
from ..testci.src.mylib import get_abs
for running it, type python -m test.pow_test
I think your issue is how the package is installed. The import looks fine to me. As it says CI I'm guessing you're having the package installed remotely with only the test folder somehow.
Try adding a setup.py file where you define that both the test as well as the src packages are part of your testci package.
there are many ways to organize a project, keep things consider in mind, structure should be simple and more scaleable, can differentiate the things in codebase easily.
one of the few good possible ways to structure a project is below
project/
├── app.py
├── dockerfile
├── pipfile
├── Readme.md
├── requiements.txt
├── src_code
│   ├── code
│   │   ├── __init__.py
│   │   └── mylib.py
│   └── test
│   ├── __init__.py
│   └── test_func.py
└── travisfile
here app.py is main file which is responsible to run your entire project

Can't run server and tests without modifying imports

I'm building a website with the bottle framework. I am using nosetests for my unit testing. However there is a problem I can't solve on my own.
I can't seem to find a way that allows me to run the tests without breaking my server. When I run my tests, I have to do relative imports for them to work, but they don't work when I start the server.
This is my folder structure:
├── service
│ ├── __init__.py
│ ├── application.py
│ ├── main.py
│ ├── response_header.py
│ ├── global_data_loader.py
│ ├── renderer
│ │ ├── __init__.py
│ │ ├── header.py
│ │ ├── ....
│ ├── tests
│ │ ├── test_application.py
│ │ ├── ....
Here is how I import so my server works:
application.py --
from response_header import ResponseHeader
from global_data_loader import GlobalDataLoader
from renderer import Application
However, when I run nosetests, I get this message:
ModuleNotFoundError: No module named 'response_header'
So when I want nosetests to work, I have to change the imports to look like this:
application.py --
from .response_header import ResponseHeader
from .global_data_loader import GlobalDataLoader
from .renderer import Application
Then my nosetests work, but when I want to start my server I get this message:
from .response_header import ResponseHeader
ImportError: attempted relative import with no known parent package
I have tried using sys.path.append in my tests, and it works, but I need a solution that doesn't involve PYTHONPATH or using os/sys
edit: i fixed it by moving my main.py I use to start the server to the root folder, above my service and tests module. now it works fine

Trouble loading local modules only with AWS Lambda

app structure:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── template.yaml
├── tests
│ ├── __init__.py
│ └── unit
│ └── lambda_application
│ ├── test_handler.py
│ └── test_parent_child_class.py
└── lambda_application
├── __init__.py
├── first_child_class.py
├── lambda_function.py
├── second_child_class.py
├── requirements.txt
└── parent_class.py
4 directories, 14 files
Code sample from lambda_function.py:
import os
import json
from hashlib import sha256
import boto3
from requests import Session
from .first_child_class import FirstChildClass
def lambda_handler(event, context):
# Do some stuff.
As is, I get the error message
Unable to import module 'lambda_function'
but If I comment out the last import, from .first_child_class import FirstChildClass, it is able to get past that part and get the error that I haven't loaded the module for that class.
I only seem to get this error when I run it in the lambci/lambda:python3.7 docker image and when I deploy on AWS. All my tests pass and it is able to import the module with no problems.
Is there something I should load/setup in the __init__.py file?
EDIT I changed the names of some of the files to post it here.
You are using a relative import here which works in case the code you are executing is in a module. However, since your code is being executed not as a module, your AWS Lambda fails.
https://stackoverflow.com/a/73149/6391078
A quick run locally gave the following error:
PYTHON 3.6
Traceback (most recent call last):
File "lambda_function.py", line 4, in <module>
from .first_child_class import FirstChildClass
ModuleNotFoundError: No module named '__main__.first_child_class'; '__main__' is not a package
Your tests pass because your testing suite imports the file as a module from the lambda_application folder which gets treated as a package in the testing module
This got me going in the correct direction but didn't quite give me the answer but did lead me to the answer, so I thought I would update what I found here.
I didn't try it but from what I found, I believe that:
from first_child_class import FirstChildClass
would be the simplest resolution.
What I ended up doing was moving the classes into a sub-directory and essentially did the same as above but with a package name prepended.
So, the file structure changed to:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── template.yaml
├── tests
│ ├── __init__.py
│ └── unit
│ └── lambda_application
│ ├── test_handler.py
│ └── test_parent_child_class.py
└── lambda_application
├── __init__.py
└── lib
├── first_child_class.py
├── second_child_class.py
└── parent_class.py
├── lambda_function.py
└── requirements.txt
and my import became from lib.first_child_class import FirstChildClass

Import module fails from the Python radish-bdd executable

I am trying to run tests in radish, a Behaviour Driven Development environment for Python, but I am failing to do even the easiest of things.
I have this structure:
.
├── features
│   └── my.feature
└── radish
├── __init__.py
├── harness
│   ├── __init__.py
│   └── main.py
└── steps.py
When I do
python -c "import radish.harness"
from my working dir ".", things are fine.
When I do the same ("import radish.harness" or "import harness") in the file steps.py, I'm getting this when calling the command "radish features" from the same directory:
ModuleNotFoundError: No module named 'radish.harness'
or
ModuleNotFoundError: No module named 'harness'
The radish-bdd quick start guide quick start guide says about this:
How does radish find my python modules? radish imports all python
modules inside the basedir. Per default the basedir points to
$PWD/radish which in our case is perfectly fine.
Indeed a file placed in the radish directory will be imported automatically, but I am unable to import anything from within these files (apart from system libraries).
Can anyone advise me on how to import modules? I'm lost. It seems that my python knowledge on module import isn't helping.
I suggest you to move the 'harness' directory at the same level as 'features' and 'radish' directory.
.
├── features
│ └── my.feature
├── radish
│ ├── __init__.py
│ └── steps.py
└── harness
├── __init__.py
└── main.py
If you call radish from your working dir (".") like this:
radish -b radish features/my.feature
Then you can import your "harness" module from steps.py like this
import harness
That will work because in this case Python will find your "harness" module as it is in the current directory.

Categories