We Keep Coding

ImportError: No module named unable to import toplevel directory of module in python

My Directory structure:
├── common
│   ├── common.py
│   └── __init__.py
├── project1
│   ├── __init__.py
│   └── scripts
│   ├── example_import.py
│   └── __init__.py
└── project2
├── __init__.py
└── scripts
└── __init__.py
I need to import common/common.py module in project1/scripts/example_import.py file
example_import.py:
import sys
sys.path.append("../common")
from common import Test
print("Module Not import error")
Error:
Traceback (most recent call last):
File "project1/scripts/example_import.py", line 3, in <module>
from common import Test
ImportError: No module named common
How to fix a issue?

Understanding how Python imports work is tricky in the beginning but makes sense once you understand how.
There are different way to fix your import issue. I would not recommend messing up with sys.path. Depending on where you are calling your script, you have multiple choice at hand.
Your Directory structure.
├── common
│   ├── common.py
│   └── __init__.py
├── project1
│   ├── __init__.py
│   └── scripts
│   ├── example_import.py
│   └── __init__.py
└── project2
├── __init__.py
└── scripts
└── __init__.py
On the root of directory
python project1/scripts/example_import.py
Will work, assuming that the imports in example_import.py looks like
from common.common import Test
If you want to use from common import Test, you need to add from common.common import Test in __init__.py file in common folder.
You can also use PYTHONPATH environment variable to tell Python where to look for the base module.
PYTHONPATH=/pathto/basefolder python folder/filesx.py
Another way is to create a setup.py on base and do development installation python -m pip install --editable . on environment you are working on.
Example of setup.py
#!/usr/bin/env python
from distutils.core import setup
setup(name='projectX',
version='1.0',
description='The World is not Enoug',
author='James Bond',
author_email='agent#007.net',
url='https://www.python.org/sigs/distutils-sig/',
packages=['common', 'project1', 'project2],
)
See Python Documentation for more setup.py options.

Importing python modules from a subfolder

I would like to import modules from within the following folder structure by referencing the sub-module app.
├── src
│   ├── __init__.py
│   └── app
│   ├── __init__.py
│   ├── getters
│   │   ├── company_1.py
│   │   └── company_2.py
│   ├── parsers
│   │   ├── company_1.py
│   │   └── company_2.py
│   └── writers
│   ├── company_1.py
│   └── company_2.py
Here is an example of what I want to do:
# src/app/writers/company_1.py
import app.parsers.company_1 as parser_1
def write_data(event, context):
...
but I get the error ModuleNotFoundError: No module named 'app'. I don't understand why this shouldn't work.
NOTE
Why am I including the seemingly redundant sub-folder app? The side story here is that the functions writers/company_*.py will be implemented as Lambda functions on AWS, and in order to get all the necessary dependencies, I need to upload the contents of the src folder to AWS but need to have a top-level folder with a known name so that I can use it for absolute imports. Without app the contents of src are uploaded to a folder with an autogenerated name which I cannot reference.
I first tried relative imports, but received an attempted relative import with no known parent package error, so now I am going for absolute imports, inspo from here.

Did you try having in __init__.py inside your app folder ?
module
|
|--module
| |
| |--__init__.py
| |--module.py

You can add this in the code ,
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), "source"))
Just replace source with the folder where you have saved your modules.
In this case replace source with app

importing python modules and packages in different sub-directories of the same project

I'd like to figure out the cleanest and preferably self contained way to use my packages in scripts that are in a different directory to the package itself.
The example problem is as follows:
The modules in lib need to both be imported, and run as a script.
My project directory is as below and I'm having two issues:
in lib/api.py, I want to read in data_files/key.txt correctly when api.py is called or imported
in testing_script.py I want to import and use lib/get_data.py
I can't seem to find a clean way to do this, does this mean my project is structured in a non-pythonic way?
Thanks for the help.
my-project-git
├── LICENSE
├── README.md
├─── my_project
│   ├── data_files
│   │   ├── key.txt
│   │   ├── mappings.csv
│   ├── lib
│   │   ├── __init__.py
│   │   ├── api.py
│   │   └── get_data.py
│   └── test
│   ├── __init__.py
│   └── testing_script.py
├── requirements.txt
└── setup.py

As far as I know, there's isn't a pythonic way to structure your project.
This is what Kenneth Reitz recommended in 2013 and it's how I use it: https://www.kennethreitz.org/essays/repository-structure-and-python.
README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py
Inside sample (my_project in your case) you can separate into categories as you like. E.g. Utils (common functions), Database (read, write), View (user commands), etc. It depends on your project.
As for calling the modules at the same level, you should define them in the __init__ file of the top hierarchy module which is sample in this case.
For example:
__init__ in _my_project
from sample.core import a_Class
from sample.core import a_function
from sample.core import anything
then from /test/test_basic.py you do:
from sample import a_Class
# or import sample
a = a_Class() # use the class from core.py
# or a = sample.a_Class()
Take a look at the sample module repository: https://github.com/navdeep-G/samplemod

python 101 - best way to access a module from parent folder

So, here's my (okay.. messy) dir:
.
├── app
│   ├── __init__.py
│   ├── analyze_text.py
│   ├── images.py
│   ├── main.py
│   ├── messages.py
│   ├── process_text.py
│   ├── requirements.txt
│   ├── response.py
│   └── tests
│   ├── __init__.py
│   ├── analyze_text_test.py
│   ├── test_process_text.py
│   └── unit_tests.py
└── setup.py # no idea what's going on with this
All I want to do is, simply use
from analyze_text import AnalyzeText
in the analyze_text_test.py file without seeing
"You're an idiot and you don't know what you're doing" in the terminal.. a.k.a:
ImportError: No module named (whatever)
I found this solution:
https://stackoverflow.com/a/11158224/2738183
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
import mymodule
Which works, but it's Janky.
Why?
Because..
I'm using it in a Janky way. I don't wanna have to re-paste this code to every single one of my unittests in the tests folder (which are in different files.)
(So what if i just paste it once in init.py? You get an error that's what. But I did randomly try that just to see what happened)
So what is the most elegant way to approach this problem? ( without repasting code (or just sticking it in a function and calling it multiple times) )
Edit:
The comments so far haven't solved anything, so I'll try to make this a bit more clear. I found a solution that works. So in each file in the tests directory I have to re-paste that solution (or call the same function as many times as there are files.) That's exactly what I'm trying to avoid. I'd like a solution that can apply to every file in the test directory so that I can use imports from the parent directory like normal, instead of appending the parent path inside every single one of those files.

Never mind. I found an elegant solution on stack overflow that I like.
Which amounted to placing
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
inside a file within the tests directory named env.py
and simply adding
import env
before importing as usual
see: https://stackoverflow.com/a/23386287/2738183

Import packages from current project directory in VScode

When I build or debug a particular file in my Python project (which imports a user defined package) I get an import error. How can I solve this problem?
test.py
def sum(a,b):
return a+b
test2.py
from test import sum
sum(3,4)
The above code will give an import error cannot import test.
Directory tree
├── graphs
│   ├── Dijkstra's\ Algorithm.py
│   ├── Floyd\ Warshall\ DP.py
│   ├── Kruskal's\ algorithm.py
│   ├── Prim's\ Algoritm.py
│   ├── __init__.py
│   └── graph.py
├── heap
│   ├── __init__.py
│   ├── heap.py
│   └── priority_queue.py
Trying to import in graphs;
from heap.heap import Heap

About the heap file, make sure that you are running on the project root folder.
If these test.py files are running on the same folder, try to add a __init__.py empty file on this folder.
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later (deeper) on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialisation code for the package or set the __all__ variable, described later.