We Keep Coding

Python ModuleNotFoundError when trying to run main.py

I am pretty new to python and have been working on a data validation program. I am trying to run my main.py file but I am getting the following error.
Traceback (most recent call last):
File "/Users/user/data_validation/main.py", line 5, in <module>
from src.compile_csv import combine_csv_files
File "/Users/user/data_validation/src/compile_csv.py", line 5, in <module>
from helpers.helper_methods import set_home_path
ModuleNotFoundError: No module named 'helpers'
I am using python version 3.94
This is my folder structure
data_validation
- src
- files
validation.csv
- folder_one
combined.csv
- folder_two
combined_two.csv
- helpers
helper_methods.py
compile_csv.py
mysql.py
split.py
main.py
Both compile_csv.py and split.py use methods from helpers.helper_methods.py
My main.py which is throwing the error when being run, looks like the following:
import os
import sys
import earthpy as et
from src.mysql import insert_data, output_non_matches
from src.compile_csv import combine_csv_files
from src.split import final_merged_file
from src.helpers.helper_methods import set_home_path, does_file_exist
home_path = et.io.HOME
file_path = home_path, "data_validation", "src", "files"
folder_name = sys.argv[1]
def configure_file_path():
master_aims_file = os.path.join(
file_path, "validation.csv")
output_csv = os.path.join(
file_path, "output.csv.csv")
gdpr_file_csv = set_home_path(folder_name + "_output.csv")
output_csv = does_file_exist(os.path.join(
file_path, folder_name + "_mismatch_output.csv"))
return output_csv, master_aims_file, gdpr_file_csv
output_csv, master_aims_file, gdpr_file_csv = configure_file_path()
if __name__ == "__main__":
print("🏁 Finding names which do not match name in master file")
if (folder_name == "pd") and (folder_name == "wu"):
final_merged_file()
else:
combine_csv_files()
insert_failures = insert_data(output_csv, master_aims_file)
output_failures = output_non_matches(output_csv, gdpr_file_csv)
if insert_failures or output_failures:
exit(
"⚠️ There were errors in finding non-matching data, read above for more info"
)
os.remove(os.path.join(home_path, "data_validation", "members_data.db"))
exit(
f"✅ mismatches found and have been outputted to {output_csv} in the {folder_name} folder")
From what I understand in python 3 we do not need to use __init__.py and you can use . for defining the path during import, So I am not entirely sure as to what I am doing wrong.
I am executing the file from /Users/user/data_validation and using the following command python main.py pd

There it is. The error is happening in your compile_csv.py file. I'm guessing in that file you have from helpers.helper_methods import blah. But you need to change it to
from .helpers.helper_methods import blah
OR
from src.helpers.helper_methods import blah
Reason being is that imports are relative to cwd not to the file where the code is running. So you need to add the import relative to /Users/user/data_validation.

you can try by setting PYTHONPATH variable. check more about it here

ModuleNotFoundError when using a function from a custom module that imports another custom module

I have a folder structure similar to this (my example has all the necessary bits):
web-scraper/
scraper.py
modules/
__init__.py
config.py
website_one_scraper.py
Where config.py just stores some global variables. It looks a bit like:
global var1
var1 = "This is a test!"
Within website_one_scraper.py it looks like this:
import config
def test_function():
# Do some web stuff...
return = len(config.var1)
if __name__ == "__main__":
print(test_function)
And scraper.py looks like this:
from module import website_one_scraper
print(website_one_scraper.test_function())
website_scraper_one.py works fine when run by itself, and thus the code under if __name__ == "__main__" is run. However, when I run scraper.py, I get the error:
ModuleNotFoundError: No module named 'config'
And this is the full error and traceback (albeit with different names, as I've changed some names for the example above):
Traceback (most recent call last):
File "c:\Users\User\Documents\Programming\Work\intrack-web-scraper\satellite_scraper.py", line 3, in
<module>
from modules import planet4589
File "c:\Users\User\Documents\Programming\Work\intrack-web-scraper\modules\planet4589.py", line 5, in
<module>
import config
ModuleNotFoundError: No module named 'config'
Also note that In scraper.py I've tried replacing from modules import website_one_scraper with import website_one_scraper, from .modules import website_one_scraper, and from . import website_one_scraper, but they all don't work.
What could the cause of my error be? Could it be something to do with how I'm importing everything?
(I'm using Python 3.9.1)

In your website_scraper_one.py, instead of import config.py try to use from . import config
Explanation:
. is the current package or the current folder
config is the module to import

Python wont allow me to import .py files from same directory

So I'm just starting to learn Python, and I am learning classes and imports, but for some reason even when I follow the same code as my book say to do, I get a traceback error.
Traceback (most recent call last):
File "C:/Users/Programming/Desktop/Programs/EX40/main.py", line 1, in <module>
import objects.py
ModuleNotFoundError: No module named 'objects.py'; 'objects' is not a package
Here is both my Python files in which I'm trying to link:
main.py
import objects.py
print(MyStuff.tangerine)
objects.py
class MyStuff(object):
def __init__(self, arm):
self.tangerine = 'And now a thousand years between'
self.arm = arm
def apple(self):
print('I AM CLASSY APPLES!')

Try using
import objects
Because it takes py as a module in the objects file which does not exist

import objects.py is not a valid import
https://docs.python.org/3/reference/import.html
Try
from objects import MyStuff
Instantiating the class:
mystuff = MyStuff("arm value")
print(mystuff.tangerine)

Creating importable Python 3 package / module

I am having trouble creating an importable Python package/library/module or whatever the right nomenclature is. I am using Python 3.7
The file structure I am using is:
Python37//Lib//mypackage
mypackage
__init__.py
mypackage_.py
The code in __init__.py is:
from mypackage.mypackage_ import MyClass
The code in mypackage_.py is:
class MyClass:
def __init__(self, myarg = None):
self.myvar = myarg
And from my desktop I try running the following code:
import mypackage
x = MyClass(None)
But get the following error:
Traceback (most recent call last):
File "C:\Users\***\Desktop\importtest.py", line 3, in <module>
x = MyClass(None)
NameError: name 'MyClass' is not defined

You haven't imported the name MyClass into your current namespace. You've imported mypackage. To access anything within mypackage, you need to prefix the name with mypackage.<Name>
import mypackage
x = mypackage.MyClass(None)

As #rdas says, you need to prefix the name with mypackage.<Name>.
I don't recommend doing this, but you can wildcard import in order to make x = MyClass(None) work:
from mypackage import *
Now, everything from mypackage is imported and usable in the current namespace. However, you have to be careful with wildcard imports because they can create definition conflictions (if multiple modules have the same name for different things).

Error when importing python module from folders

I have a following directory structure:
source
source_1.py
__init__.py
source1.py has class Source defined
source1.py
class Source(object):
pass
I am able to import using this
>>> from source.source1 import Source
>>> Source
<class 'source.source1.Source'>
However when trying to import using the below method it fails.
>>> from source import *
>>> source1.Source
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'source1' is not defined
Please let me know how can we use the 2nd import ?

For importing from a package (unlike importing from a module) you need to specify what * means. To do that, in __init__.py add a line like this:
__all__ = ["source1"]
See the Python documentation for Importing * From a Package.