We Keep Coding

Why am i unable to run my project from the command line?

I have created a simple app, doesn't matter what is the purpose but i was trying to run it from the command line but it does not work. I am able to run it and everything from the IDE (PyCharm) but not CLI.
I have init files in every folder and the folder structure looks like this:
Project
├───.idea
│ └───inspectionProfiles
├───files
└───src
├───project
│ ├───Variations
│ │ └───__init__.py
│ │ └───A.py
│ │ └───B.py
│ └───__init__.py
│ └───C.py
│ └───D.py
└────tests
└───__init__.py
└───main.py
When i positioned myself in tests and ran:
python main.py the script returned:
File "G:\Projects\ExampleProject\src\tests\main.py", line 1, in <module>
from project import run
ModuleNotFoundError: No module named 'project'

Python import failure in script

I have a project structure, like so:
└──  auto/
│ ├────  __init__.py
│ └────  __pycache__/
│ ├────  deploy
│ ├────  destroy
│ └────  helpers/
│ │ ├────  __init__.py
│ │ ├────  cli.py
│ │ └────  zip.py
│ └────  synth
└──  src/
│ ├────  __init__.py
│ ├────  main.py
│ └────  models/
│ │ ├────  __init__.py
│ │ ├────  filter.py
│ │ └────  json_extract_config.py
│ └────  utils/
│ │ ├────  __init__.py
│ │ └────  json_extractor.py
I am trying to call the script synth in the ./auto/synth location.
The synth file is a script with a shebang at the top #!/usr/bin/env python3
It imports a function from the ./auto/helpers/zip.py file.
When I run, ./auto/synth from the root folder it throws an error:
from auto.helpers.zip import zip_folder
ModuleNotFoundError: No module named 'auto'
Although when I run PYTHONPATH="$PWD" ./auto/synth the script executes successfully.
Can someone help me understand this behaviour and how to avoid it so that my scripts can be run normally without having to change the PYTHONOPATH?
Thanks in advance.

If you are using python version 3.3+ you do not need to use the __init__.py files anymore, as the namespacing is built into the modules now. link --> Is __init__.py not required for packages in Python 3.3+
I believe you will have to add the {full path}/src and {full path}/auto directories to the PYTHONPATH to provide python scripts the proper pathing, or you can use the python sys module and use sys.path inside your modules to set your pathing for the module you would like to import link --> https://docs.python.org/3/tutorial/modules.html#the-module-search-path
import sys
# example path could be "C:/Users/Bob/Desktop/project/auto"
sys.path.append("{full path to your auto directory}/auto")

Python package not being found on Pytest

I have the following project structure:
foo
│
├── foo
│ ├── cli
│ │ ├── tests
│ │ └── __init__.py
│ ├── core
│ │ ├── tests
│ │ └── __init__.py
│ ├── tests
│ │ ├── __init__.py
│ │ └── test_foo.py
│ ├── __init__.py
│ └── foo.py
├── kube
├── requirements
├── ...any-other-non-source-related
└── README.md
Inside the foo/foo.py file I have the following placeholder code:
import cli
import core
def say_hi():
print('hi')
if __name__ == '__main__':
say_hi()
And on the file foo/tests/test_foo.py I have the following:
from foo import foo
def test_foo():
foo.say_hi()
assert False
Then, in shell, I'm getting the following:
$ python foo/foo.py
hi
$ pipenv run pytest
===================== test session starts =====================
platform linux -- Python 3.7.4, pytest-5.3.1, py-1.8.0, pluggy-0.13.1
rootdir: /xxx/xxx/xxx/xxx/python-project-folder-structures
collected 0 items / 1 error
===================== ERRORS =====================
_____ ERROR collecting foo/tests/test_foo.py _____
ImportError while importing test module '/xxx/xxx/xxx/xxx/python-project-folder-structures/foo/tests/test_foo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
foo/tests/test_foo.py:1: in <module>
from foo import foo
foo/foo.py:1: in <module>
import cli
E ModuleNotFoundError: No module named 'cli'
!!!!!!! Interrupted: 1 error during collection !!!!!!!
===================== 1 error in 0.05s =====================
Can anyone helps? Why Pytest is not founding my imports? I've tried a bunch of approachs, even with tests folder outside the foo folder (on root level) but the error still persists.
The only way that I've found to fix that is putting a conftest.py file on the code level, with that it would be discovered by pytest. I really don't want to do that, since I want to decouple my testing from my source code.

If anyone strugles with this, I was able to find a solution. I removed the __init__.py file from the foo directoty, giving this structure:
foo
│
├── foo
│ ├── cli
│ │ ├── tests
│ │ └── __init__.py
│ ├── core
│ │ ├── tests
│ │ └── __init__.py
│ ├── tests
│ │ ├── __init__.py
│ │ └── test_foo.py
│ └── foo.py
├── kube
├── requirements
├── ...any-other-non-source-related
└── README.md
Them, on mt test, I got the following now:
import foo
def test_foo():
foo.say_hi()
assert False
I noticed that the behaviour of Pytest is to go from the test until the root dir, identifying packages. Since foo was a package (with a __init__.py file) it was adding it as one on my PATH :)
IMPORTANT
Notice that the import foo statement is regarding the foo.py file, not the foo directory. So, if your file is called boo.py inside the foo directory, the import should be: import boo

Python import module/function from below subdirectory or anywhere

I would like to load in a function/module with my working directory as project main directory but the function file is store below the a subdirectory level so the normal
from function_file import function_name
does not work.
This is what the project directory set up looks like:
└───project_main_directory
│ .gitattributes
│ .gitignore
│ README.txt
│
├───code
│ ├───exploration
│ └───scripts
│ │ script1.py
│ │ script2.py
│ │ script3.py
│ │
│ └───functions
│ │ function1.py
│ │ function2.py
│ └───__init__.py
│
├───data
│ └───example_data
│ data.csv
└───documents
So I tried to import functions via
import code.scripts.function.function1 from function1
and that doesn't work. I know it's because the other subdirectories aren't modules, but I want to ask if there is away around that?
-- EDIT
I'm working from .py file in code/scripts/script1.py but working directory is project_main_directory/

Add an empty file __init__.py to every subdirectories to make them as modules.
.
├── code
│ ├── __init__.py
│ └── scripts
│ ├── __init__.py
│ └── script1.py
└── main.py
Then if you have a function called hello in code/scripts/script1.py you can import that function by:
from code.scripts.script1 import hello
hello("yo")

If your current directory is project_main_directory, you can use:
from code.scripts.functions.function1 import function1
Directory of your script doesn't matter. Only your current directory matters (refer top of the IDE)

To import function/module from another python file, you have to do something like below -
from code.scripts.functions.function1 import function1
Above we are loading function1 from function1.py file which is stored in functions directory which is stored in scripts directory and finally in code directory.
EDIT - so you are saying, you want to load a function from function1.py in script1.py? In that case from .functions.function1 import function should work.

How to run this module from this path?

I have the following file structure:
Project
|
├───cambridge_loader
| |
│ ├───tests
│ │
│ ├───__init__.py
│ │
│ ├───__main__.py
│ │
│ ├───CambridgeLoader.py
I'm in the Project directory on my command line, I created a virtual environment in Visual Stdio and executed the activate.bat script.
I've tried running:
python -m CambridgeLoader
And get the following error:
C:\Users\username\Source\Repos\rapdataload\Frauenfeld\env\Scripts\python.exe: No module named CambridgeLoader
I've tried running it in different paths but it doesn't seem to be working. I'm not sure why it's attaching the Scripts path at the end...