Consider the following directory structure:
lib/
markov.py
solver.py
test.py
markov.py has the following code: super() throwing an error in Sublime Text, works in PyCharm/Terminal
solver.py only contains a bunch of mathematical functions, here's an example of one:
def mm1_busy_idle(arrival, service):
return round(1 - (arrival / service), 4)
Now, when I try to do import solver in markov.py, PyCharm tells me that there's No module named solver. However, when I do import test it does it just well and I can run the test function from there:
def test():
print("test!")
Also, when I hover over test.test() PyCharm shows me the following tooltip: Cannot find reference 'test' in '__init__.py'.
My question is: Why can I import test and run the function test() but I can't import solver in markov.py?
add one file (__init__.py empty file) in lib,
lib/
__init__.py
markov.py
solver.py
test.py
An ultimate solution:
import sys
sys.path.append("/path/to/lib")
Related
This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed 3 months ago.
I have a directory and module design that looks like this:
MyProject --- - script.py
|
- helpers --- - __init__.py
|
- class_container.py
|
- helper.py
# class_container.py
class MyClass:
pass
# helper.py
from class_container import MyClass
def func():
# some code using MyClass
# script.py
from helpers.helper import func
When I run script.py:
ModuleNotFoundError: No module named 'class_container'
I tried changing the code in helper.py such as:
# helper.py
from helpers.class_container import MyClass
def func():
# some code using MyClass
Then running script.py started working. But when I explicitly run helper.py:
ModuleNotFoundError: No module named 'helpers'
I want to be able run both script.py and helper.py separately without needing to change the code in any module.
Edit: I figured a solution which is changing helper.py such as:
from pathlib import Path
import sys
sys.path.append(str(Path(__file__).parent))
from class_container import MyClass
def func():
# some code using MyClass
Basically I added the directory of helper.py to sys.path by using sys and pathlib modules and __file__ object. And unlike import statement's behaviour, __file__ will not forget it's roots when imported/used from a different module (i.e it won't become the path of script.py when imported into it. It'll always be the path of helper.py since it was initiated there.).
Though I'd appreciate if someone can show another way that doesn't involve messing with sys.path, it feels like an illegal, 'unpythonic' tool.
You have to create a __init__.py file in the MyProject/helpers directory. Maybe you already have created it. If not, create an empty file.
Then in the MyProject/helpers/helper.py, access the module helpers.class_container like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
You can also use a relative import like this.
from .class_container import MyClass
If you want to run the MyProject/helpers/helper.py independently, add test code in helper.py like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
if __name__ == '__main__':
func()
And run like this in the MyProject directory.(I assume a Linux environment.)
$ python3 -m helpers.helper
The point is to differentiate Python modules from Python scripts and treat them differently.
I was working on a project with this type of folder tree:
-main.py
-Message_handlers
-__init__.py
-On_message.py
-Other_modules.py
-Exceptions_handler
-__init__.py
-Run_as_mainException.py
Message_handlers and Exceptions_handler are two packages, now my problem is that from inside the module On_message.py I can't import the class Run_as_mainException (inside of the module Run_as_mainException.py) using this code
# This is On_message.py
from ..Exceptions_handler.Run_as_mainException import Run_as_main_Exception
This line gives the error: ImportError: attempted relative import with no known parent package
p.s. Every file has a class inside with the same name of the file, example:
# This is Run_as_mainExample.py
class Run_as_mainExample:
def __init__(self):
# rest of the code
Can anyone help me please?
You have to assume that you're running everything from the level where main.py is located. This means, imagine that you execute python main.py, and you want to import Run_as_main_Exception from main.py, what should you do?
from Exceptions_handler.Run_as_mainException import Run_as_main_Exception
Try using that line in your On_message.py file, and you shouldn't have any problem when running your script from that exact location (remember, the same level where main.py is located)
Path structure:
base.py
my_module/
__init__.py
mod1.py
mod2.py
base.py:
from my_module.mod2 import *
mod2_func2() # This call will work
my_module/mod1.py:
from mod2 import *
def mod1_func():
mod2_func1("Hello World") # This call will not work.
my_module/mod2.py:
from mod1 import *
def mod2_func1(input_text):
print(input_text)
def mod2_func2():
mod1_func() # This call will work
This code will error out because mod2_func1 is not defined. If I run mod1.py directly (i.e. adding code to an __name__=="__main__" block, it will work fine.
This is a reduced example of my real problem, which involves a bunch of modules inside a package all needing to talk to each other like this. The main thing is ensuring that mod1 and mod2 can access each others contents in the local namespace irrespective of which is called from base. I've looked at a ton of documentation on python namespace stuff, importing, and packages and while I'm relieved to find that it seems frustrating to everyone, I haven't found a solution. Any advice?
Question on unit testing
Goal: The goal is to use pyUnit in testCalc.py to unit test the simpleCalc object in calculator.py.
Problem: I cannot successfully import the simpleCalc object from calculator.py into testCalc.py when testCalc is run from a separate directory in the project.
Background: The unit test in testCalc.py runs perfectly fine when it's included in the same directory as calculator.py, but when I move it into a separate folder and try to import the simpleCalc object defined in calculator.py, I get an error. I am trying to learn how to use the pyUnit unit testing framework in a simple project and I'm clearly missing something basic about how to import modules for unit testing in a hierarchical directory structure. The basic calculator_test project described below is a simple project that I created to practice. You can see all the posts I've gone through already at the end of this post.
Ultimate Question: How can I import the simpleCalc object into testCalc.py with the directory hierarchy described below?
Github: https://github.com/jaybird4522/calculator_test/tree/unit_test
Here's my directory structure:
calculator_test/
calculatorAlgo/
__init__.py
calculator.py
test_calculatorAlgo/
__init__.py
testCalc.py
testlib/
__init__.py
testcase.py
Here's the calculator.py file, which describes the simpleCalc object I want to unit test:
# calculator.py
class simpleCalc(object):
def __init__(self):
self.input1 = 0
self.input2 = 0
def set(self, in1, in2):
self.input1 = in1
self.input2 = in2
def subtract(self, in1, in2):
self.set(in1, in2)
result = self.input1 - self.input2
return result
Here's the testCalc.py file, which contains the unit tests:
# testCalc.py
import unittest
from calculatorAlgo.calculator import simpleCalc
class testCalc(unittest.TestCase):
# set up the tests by instantiating a simpleCalc object as calc
def setUp(self):
self.calc = simpleCalc()
def runTest(self):
self.assertEqual(self.calc.subtract(7,3),4)
if __name__ == '__main__':
unittest.main()
I have been running the unit test file with the simple command:
testCalc.py
What I've attempted so far
First Attempt
I tried simply importing the simpleCalc object based on where it's located in the directory structure:
# testCalc.py
import unittest
from .calculatorAlgo.calculator import simpleCalc
class testCalc(unittest....
And got this error:
ValueError: Attempted relative import in non-package
Second Attempt
I tried importing it without relative references:
# testCalc.py
import unittest
import simpleCalc
class testCalc(unittest....
And got this error:
ImportError: No module named simpleCalc
Third Attempt
Based on this post, http://blog.aaronboman.com/programming/testing/2016/02/11/how-to-write-tests-in-python-project-structure/, I tried creating a separate base class called testcase.py which could do the relative imports.
# testcase.py
from unittest import TestCase
from ...calculator import simpleCalc
class BaseTestCase(TestCase):
pass
And changed my imports in testCalc.py
# testCalc.py
import unittest
from testlib.testcase import BaseTestCase
class testCalc(unittest....
And got this error:
ValueError: Attempted relative import beyond toplevel package
Other Resources
Here are some of the posts I've worked through to no avail:
Import a module from a relative path
python packaging for relative imports
How to fix "Attempted relative import in non-package" even with __init__.py
Python importing works from one folder but not another
Relative imports for the billionth time
Ultimately, I feel that I'm just missing something basic, even after a lot of research. This feels like a common setup, and I'm hoping someone can tell me what I'm doing wrong, and that it might help others avoid this problem in the future.
Inside the testCalc.py file, add the following.
import sys
import os
sys.path.append(os.path.abspath('../calculatorAlgo'))
from calculator import simpleCalc
I am currently doing a personal coding project and I am trying to build a module, but I don't know why my structure doesn't work the way it's supposed to:
\mainModule
__init__.py
main.py
\subModule_1
__init__.py
someCode_1.py
someCode_2.py
\subModule_2
__init__.py
otherCode.py
I want to be able to run the following code from main.py:
>>> from subModule_1 import someCode_1
>>> someCode_1.function()
"Hey, this works!"
>>> var = someCode_2.someClass("blahblahblah")
>>> var.classMethod1()
>>> "blah blah blah"
>>> from subModule2 import otherCode
>>> otherCode("runCode","#ff281ba0")
However, when I try to import someCode_1, for example, it returns an AttributeError, and I'm not really sure why. Is it to do with the __init__.py file?
REVISIONS
Minimal, Complete and verifiable (I hope...)
\mainDir
__init__.py # blank file
main.py
\subDir
__init__.py # blank file
codeFile.py
Using this...
#main.py file
import subDir
subDir.codeFile.function()
And this...
#codeFile.py file
def function():
return "something"
...it returns the same problem mentioned above**.
** The exact error is:
Traceback (most recent call last):
File "C:\...\mainDir\main.py", line 2, in <module>
subDir.codeFile.function()
AttributeError: module 'subDir' has no attribute 'codeFile'
Credits to #jonrsharpe: Thanks for showing me how to use Stack Overflow correctly.
You have two options to make this work.
Either this:
from subdir import codeFile
codeFile.function()
Or:
import subdir.codeFile
subdir.codeFile.function()
When you import subDir, it does three things:
executes the code in mainDir/subDir/__init__.py (i.e. in this case does nothing, because this file is empty)
imports the resulting module under the name subDir locally, which will in turn make it an attribute of the mainDir module;
registers the new import globally in the sys.modules dictionary (because the import is being performed from a parent module mainDir, the name is completed to 'mainDir.subDir' for the purposes of this registration);
What it does not do, because it hasn't been told to, is import subDir.codeFile. Therefore, the code in codeFile.py has not been run and the name codeFile has not yet been imported into the namespace of mainDir.subDir. Hence the AttributeError when trying to access it. If you were to add the following line to mainDir/subDir/__init__.py then it would work:
import codeFile
Specifically, this will:
run the code in codeFile.py
add the resulting module as an attribute of the mainDir.subDir module
store a reference to it as yet another entry in sys.modules, this time under the name mainDir.subDir.codeFile.
You could also achieve the same effect from higher up the module hierarchy, by saying import subDir, subDir.codeFile instead of just import subDir in your mainDir.main source file.
NB: When you test this from the command line or IDE, make sure that your current working directory (queried by os.getcwd(), changed using os.chdir(wherever) ) is neither mainDir nor subDir. Work from somewhere else—e.g. the parent directory of mainDir. Working from inside a module will lead to unexpected results.