We Keep Coding

Why am i unable to run my project from the command line?

I have created a simple app, doesn't matter what is the purpose but i was trying to run it from the command line but it does not work. I am able to run it and everything from the IDE (PyCharm) but not CLI.
I have init files in every folder and the folder structure looks like this:
Project
├───.idea
│ └───inspectionProfiles
├───files
└───src
├───project
│ ├───Variations
│ │ └───__init__.py
│ │ └───A.py
│ │ └───B.py
│ └───__init__.py
│ └───C.py
│ └───D.py
└────tests
└───__init__.py
└───main.py
When i positioned myself in tests and ran:
python main.py the script returned:
File "G:\Projects\ExampleProject\src\tests\main.py", line 1, in <module>
from project import run
ModuleNotFoundError: No module named 'project'

Pycharm configuration difference between running and debugging

I have a folder structure where
project_name
└───folder_level_1
│ └───folder_level_2
│ │ └───folder_level_3
│ │ └─── folder_level_4
│ │ │ └─── ...
| | | main.py
I have marked folder_level_1/2/3 as source roots in Pycharm.
When I try to run main.py in debug - it works, but if I run normally I get an error that
ModuleNotFoundError: No module named folder_level_1.folder_level_2.folder_level_3. In each level I have an init.py as well.
Running it with pipenv environment if that helps.
How should I solve that?

Python import failure in script

I have a project structure, like so:
└──  auto/
│ ├────  __init__.py
│ └────  __pycache__/
│ ├────  deploy
│ ├────  destroy
│ └────  helpers/
│ │ ├────  __init__.py
│ │ ├────  cli.py
│ │ └────  zip.py
│ └────  synth
└──  src/
│ ├────  __init__.py
│ ├────  main.py
│ └────  models/
│ │ ├────  __init__.py
│ │ ├────  filter.py
│ │ └────  json_extract_config.py
│ └────  utils/
│ │ ├────  __init__.py
│ │ └────  json_extractor.py
I am trying to call the script synth in the ./auto/synth location.
The synth file is a script with a shebang at the top #!/usr/bin/env python3
It imports a function from the ./auto/helpers/zip.py file.
When I run, ./auto/synth from the root folder it throws an error:
from auto.helpers.zip import zip_folder
ModuleNotFoundError: No module named 'auto'
Although when I run PYTHONPATH="$PWD" ./auto/synth the script executes successfully.
Can someone help me understand this behaviour and how to avoid it so that my scripts can be run normally without having to change the PYTHONOPATH?
Thanks in advance.

If you are using python version 3.3+ you do not need to use the __init__.py files anymore, as the namespacing is built into the modules now. link --> Is __init__.py not required for packages in Python 3.3+
I believe you will have to add the {full path}/src and {full path}/auto directories to the PYTHONPATH to provide python scripts the proper pathing, or you can use the python sys module and use sys.path inside your modules to set your pathing for the module you would like to import link --> https://docs.python.org/3/tutorial/modules.html#the-module-search-path
import sys
# example path could be "C:/Users/Bob/Desktop/project/auto"
sys.path.append("{full path to your auto directory}/auto")

Questions about packaging the first PyPi project

Few days ago I've created a little project called pyblime and right now I was trying to figure out how to create a proper setup.py that allowed me to upload the "right stuff" to PyPi so users will be able to enjoy the project by using pip without doing anything "too fancy" like calling fancy custom dev scripts, right now the project tree structure looks like this:
│ .gitignore
│ configure.py
│ MANIFEST.in
│ README.md
│ requirements.txt
│ setup.py
│
├───data
│ ├───commands
│ │ comment.py
│ │ fold.py
│ │
│ ├───screenshots
│ │ test_simple.png
│ │ test_themes.gif
│ │
│ ├───st_build_3149
│ │ ├───syntax
│ │ └───themes
│ └───testfiles
├───docs
│ build.md
│ contributing.md
│ guidelines.md
│ usage.md
│
├───examples
│ demo_00.py
│ tutorials.py
│ tutorial_00.py
│ tutorial_01.py
│ tutorial_02.py
│ tutorial_03.py
│ tutorial_04.py
│ tutorial_05.py
│ tutorial_06.py
│
├───pyblime
│ utils.py
│ view.py
│ __init__.py
│
├───sublime_text
│ sublime.py
│ sublime_plugin.py
│
└───tests
run_all.py
test_scopes.py
test_view.py
x.py
Rather than 1 question, I've got few simple doubts:
What'd be the "standard" way to instruct setup.py to copy sublime_text/sublime.py and sublime_text/sublime_plugin.py files into Lib/site-packages root?
How'd you tell setup.py to copy the whole folder pyblime adhoc in Lib/site-packages?
Finally, is it correct to upload tests/examples/tests/docs/data to PyPi? This is, content that won't be necessary to use the SDK/library itself... If it's not, where would you include this type of data... I'm aware there exists the concept of sdist&dist folders and I've already read a bit about it here but the question still remains :)
Right now my setup.py looks something like this:
from pathlib import Path
from setuptools import setup
root_path = Path(__file__).parent
requirements = (root_path / "requirements.txt").read_text()
requirements = [
v for v in requirements.split("\n")
if v.strip() and not v.strip().startswith("#")
]
readme = (root_path / "README.md").read_text()
setup(
author="mcve",
author_email="mcve",
classifiers=["mcve"],
description="mcve",
install_requires=requirements,
keywords=["mcve"],
long_description=(root_path / "README.md").read_text(),
name="mcve",
# package_data = {}, <---- How do i use this?
# packages = [], <---- Do I need to use this?
url="mcve",
version="0.0.1",
)
Ps. And yeah... I've already read the official docs out there about packaging... but if I had understood those docs I wouldn't be asking this on SO ;D . Thanks in advance!

Well, let's step by step.
For the first question. Mostly you shouldn't do that, instead, you should consider sublime_text as a package too. The structure should look like:
├───sublime_text
| __init__.py
│ sublime.py
│ sublime_plugin.py
And you should use it like from sublime_text import sublime in your other packages. That's could be better as you won't pollute the global namespace too much. Or if this is not a common package that you want to share between many other packages, you can directly include it as a submodule in your main package.
Or if you really wanna do this, you can place those two files in the root directory and use:
...
packages = find_packages(),
py_modules=["sublime", "sublime_plugin"],
...
For the second question. As that is a package, you can add that path to packages: packages=[""]. Or for convenience, you can use packages=find_packages(). It will help you to find all packages under the current directory which is "".
For your third question. Mostly it is not correct, you just need to submit what user needs to the PyPi. For docs, you should use readthedocs website. And for other examples and tests, just leave them on your Github. That's enough.

Python import module/function from below subdirectory or anywhere

I would like to load in a function/module with my working directory as project main directory but the function file is store below the a subdirectory level so the normal
from function_file import function_name
does not work.
This is what the project directory set up looks like:
└───project_main_directory
│ .gitattributes
│ .gitignore
│ README.txt
│
├───code
│ ├───exploration
│ └───scripts
│ │ script1.py
│ │ script2.py
│ │ script3.py
│ │
│ └───functions
│ │ function1.py
│ │ function2.py
│ └───__init__.py
│
├───data
│ └───example_data
│ data.csv
└───documents
So I tried to import functions via
import code.scripts.function.function1 from function1
and that doesn't work. I know it's because the other subdirectories aren't modules, but I want to ask if there is away around that?
-- EDIT
I'm working from .py file in code/scripts/script1.py but working directory is project_main_directory/

Add an empty file __init__.py to every subdirectories to make them as modules.
.
├── code
│ ├── __init__.py
│ └── scripts
│ ├── __init__.py
│ └── script1.py
└── main.py
Then if you have a function called hello in code/scripts/script1.py you can import that function by:
from code.scripts.script1 import hello
hello("yo")

If your current directory is project_main_directory, you can use:
from code.scripts.functions.function1 import function1
Directory of your script doesn't matter. Only your current directory matters (refer top of the IDE)

To import function/module from another python file, you have to do something like below -
from code.scripts.functions.function1 import function1
Above we are loading function1 from function1.py file which is stored in functions directory which is stored in scripts directory and finally in code directory.
EDIT - so you are saying, you want to load a function from function1.py in script1.py? In that case from .functions.function1 import function should work.