We Keep Coding

python file needs to imports a class from another python file that is in the parent folder

i'm struggling with something that feels like it should be simple.
my current dir looks like this:
root/
└─ __init__.py (tried with it and without)
└─ file_with_class.py
└─ tests_folder/
└─ __init__.py (tried with it and without)
└─ unittest_for_class.py
unittest_for_class.py needs to import the class from file_with_class to test it, i tried to import it in various ways i found online but i just keep getting errors like:
(class name is same as file name lets say its called file_with_class)
File "tests_folder/unittest_for_class.py", line 3, in <module>
from ..file_with_class import file_with_class
ValueError: Attempted relative import in non-package
File "tests_folder/unittest_for_class.py", line 3, in <module>
from file_with_class import file_with_class
ImportError: No module named file_with_class
and others..
what is the correct way to import a class from a .py file that is in the parent folder ?

As a short explanation
import * from ..parent works if your program started at the parent level.
You import submodules which can have cross relations to other submodules or files in the package -> They are only relative inside a package not the os structure.
Option 1
you actually enter via a script in your parent folder and import your mentioned file as a submodule. Nicest, cleanest and intended way, but then your file is no standalone.
Option 2 - Add the parent dictionary to your path
sys.path.append('/path/to/parent')
import parent
This is a little bit dirty as you now have an extra path for your imports but still one of most easiest ones without much trickery.
Further Options and theory
There are quite a few posts here covering this topic relative imports covers quite a few good definitions and concepts in the answers.
Option 3 - Deprecated and not future proof importlib.find_loader
import os
import importlib
current = os.getcwd() # for rollback
os.chdir("..") # change to arbitrary path
loader = importlib.find_loader("parent") # load filename
assert loader
parent = loader.load_module() # now this is your module
assert parent
os.chdir(current) # change back to working dictionary
(Half of an) Option 4
When working with an IDE this might work, Spyder allows the following code. Standard python console does NOT.
import os
current = os.getcwd()
os.chdir("..")
import parent
os.chdir(current)

Following up on #Daraan's answer:
You import submodules which can have cross relations to other submodules or files in the package -> They are only relative inside a package not the os structure.
I've written an experimental, new import library: ultraimport which allows to do just that, relative imports from anywhere in the file system. It will give you more control over your imports.
You could then write in your unittest_for_class.py:
import ultraimport
MyClass = ultraimport("__dir__/../file_with_class.py", "MyClass")
# or to import the whole module
file_with_class = ultraimport("__dir__/../file_with_class.py")
The advantage is that this will always work, independent of sys.path, no matter how you run your script and all the other things that were mentioned.

You can add the parent folder to the search path with sys.path.append() like so:
import sys
sys.path.append('/path/to/parentdir')
from file_with_class import file_with_class
...
See also the tutorial for how Python modules and packages are handled

Just keep from file_with_class import file_with_class. Then run python -m test_folder.unittest_for_class. This supports running the script as if it is a module.

Python3 unable to find a package module

I have a file called dns_poison.py that needs to call a package called netscanner. When i try and load the icmpscan module from dns_poison.py I get this message:
ModuleNotFoundError: No module named 'icmpscan'
I've done a sys.path and can confirm that the correct path is in place. The files are located at D:\PythonProjects\Networking\tools and D:\PythonProjects appears when I do a sys.path.
Here is my directory structure:
dns_poison.py
netscanner/
__init__.py
icmpscan.py
Code snippets for the files are as follows:
dns_poison.py
import netscanner
netscanner\__init__.py
from icmpscan import ICMPScan
netscanner\icmpscan.py
class ICMPScan:
def __init__(self, target, count=2, timeout=1):
self.target = target
self.count = count
self.timeout = timeout
self.active_hosts = []
# further code below here....
I don't understand why it cannot find the module, as I've used this exact same method on other python projects without any problems. Any help would be much appreciated.

When you run python dns_poison.py, the importer checks the module path then the local directory and eventually finds your netscanner package that has the following available:
netscanner
netscanner.icmpscan
netscanner.icmpscan.ICMPScan
Now I ask you, where is just icmpscan? The importer cannot find because well, it doesnt exist. The PYTHONPATH exists at wherever dns_poison.py resides, and doesn't append itself to include the absolute path of any imported modules because that simply not how it works. So netscanner can be found because its at the same level as dns_poison.py, but the importer has no clue where icmpscan.py exists because you havent told it. So you have two options to alter your __init__.py:
from .icmpscan import ICMPScan which works with Python 3.x
from netscanner.icmpscan import ICMPScan which works with both Python 2.x/3.x
Couple of references for you:
Python Import System
Python Modules recommend you ref section 6.4.2 Intra-package References

The most simple way to think about this is imports should be handled relative to the program entry-point file. Personally I find this the most simple and fool-proof way of handling import paths.
In your example, I would have:
from netscanner.icmpscan import ICMPScan
In the main file, rather than add it to init.py.

Getting "ImportError: attempted relative import with no known parent package" when running from Python Interpreter

I'm creating a modular app using Flask blueprints feature. As a result, my directory structure is like this:
project
__init__.py
config.py
mould.py
modules
__init__.py
core
__init__.py
core.py
db.py
models.py
The modules directory here is not be confused with Python modules, they are for giving a modular structure to my project (core module, foo module, bar module, etc.). Now each folder in the modules directory (and a module inside it with same name such as core.core) is dynamically imported in my main flask app (mould.py) by doing this:
for item in os.listdir("modules"):
if not os.path.isfile("modules" + os.sep + item) and not item.startswith("__"):
ppath = "modules" + "." + item
fullpath = "modules" + "." + item + "." + item
module = importlib.import_module(fullpath)
app.register_blueprint(module.app)
print("Registered: " + ppath)
As a result of this, I'm unable to do this in the module scripts like db.py:
import models
Since it gives a path error as the entire module is being executed at the project level, so I had to do this:
from . import models
This solves the issue and I'm able to successfully import all modules. However, when I go to the core modules directory for some troubleshooting and start the python interpreter, it doesn't allow me to import the db module:
>>> import db
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "db.py", line 7, in <module>
from . import models
ImportError: attempted relative import with no known parent package
Is there a way around this? So, that I can import the db module successfully in the code as well as interpreter?

I know I'm late to the party, but I think I've found a solution to this problem. Hopefully this will be useful to someone else working on a large Python project.
The trick is to try one import format and fall back to the other format if the first fails.
Approach 1
db.py
try:
# Assume we're a sub-module in a package.
from . import models
except ImportError:
# Apparently no higher-level package has been imported, fall back to a local import.
import models
On the plus side, this approach is pretty simple, but doesn't scale well (module names are duplicated). Scaling can be improved by importing programmatically.
Approach 2 (not recommended)
db.py
import importlib
root = 'project.modules.core'
my_modules = ['core', 'models']
for m in my_modules
try:
globals()[m] = importlib.import_module(root + '.' + m)
except ImportError:
globals()[m] = importlib.import_module(m)
globals() is the global symbol table.
Of course, now this functionality needs to be duplicated in every module. I'm not sure that's actually an improvement over the first approach. However, you can separate this logic out into its own independent package that lives somewhere on pythonpath.
Approach 3
package_importer.py
import importlib
def import_module(global_vars, root, modules):
for m in modules
try:
global_vars[m] = importlib.import_module(root + '.' + m)
except ImportError:
global_vars[m] = importlib.import_module(m)
db.py
import package_importer
root = 'project.modules.core'
my_modules = ['core', 'models']
package_importer.import_module(globals(), root, my_modules)

This may be a bit outdated, but maybe someone else will benefit of my answer. Since python2 und python3 have different default import behavior, you have to determine between these two python versions.
Python 2.X
The default behavior for import models is to look up first the relative and then the absolute search path order. Therefore it should work.
However, in Python 3.X the default behavior for import models is to look for the model only in the absolute paths (called absolute imports). The current package core gets skipped and since the module db cannot be found anywhere else in the sys.path search path, it throws an error. To resolve this issue you have to use the import statement with dots from . import models to make clear that you are trying to import from a relative directory.
If you are interested to learn more about importing python modules, I suggest you to start your research with the following key words: module search path, python package import and relative package imports

Python strange import behavior: import x.y works, from x import y doesn't

I am having a simple directory structure, like that:
MyProject
--main.py
--lib #that's a directory/package
----__init__.py
----view.py
----common_lib.py
----other irrelevant modules...
In main.py:
from lib import view
causes the following error:
ImportError: cannot import name view
If instead, I write it like:
from lib.view import *
This import passes successfully, but next failure happens in view.py, in that:
from common_lib import Comments, Locations, ScreenData, ProgressSignal
causes:
ImportError: No module named 'common_lib'
And as it appears from the directory structure, common_lib.py resides in the same directory as view.py, how can it happen that it cannot be found?
How does it come that 'from x import y' doesn't work, and 'from x.y import *' works?
'__init__.py' is completely empty BTW. And I am using Python 3.3 32-bit.
What is more annoying, this same program worked 2 days ago.I was testing some code in IDLE and when I thought the code was ready to include in the project, pasted it into PyDev, I was shocked by this error. I changed nothing about my directories or modules.
Also, still more strangeness, running view.py and common_lib.py as standalone (without being imported) runs just fine. It should produce the error if any issues really existed.
Thanks. Any advice is highly appreciated.

Since main.py is still at the top level, you need to use lib.common_lib:
from lib.common_lib import Comments, Locations, ScreenData, ProgressSignal
because the previous line from lib import view does not start looking for modules from inside lib.

Given:
+--main.py # from lib import view
+--lib
+--__init__.py
+--common_lib.py # Comments, etc.
+--view.py # from .common_lib import Comments, etc.
This works:
from lib import view
And this works from view.py with a relative import to indicate common_lib is in the same package.
from .common_lib import Comments, Locations, ScreenData, ProgressSignal

Works for me:
danielallan#MacBook:~$mkdir myproject
danielallan#MacBook:~$cd myproject/
danielallan#MacBook:myproject$mkdir lib
danielallan#MacBook:myproject$cd lib
danielallan#MacBook:lib$touch __init__.py
danielallan#MacBook:lib$touch view.py
danielallan#MacBook:lib$touch common_lib.py
danielallan#MacBook:lib$cd ..
In [1]: from lib import view
In [2]: view
Out[2]: <module 'lib.view' from 'lib/view.pyc'>
What happens when you try that on your machine? Are you sitting in the wrong directory, or is your path not configured to find these files?

Python: Importing everything from a python namespace / package

all.
I'd think that this could be answered easily, but it isn't. As long as I've been searching for an answer, I keep thinking that I'm overlooking something simple.
I have a python workspace with the following package structure:
MyTestProject
/src
/TestProjectNamespace
__init__.py
Module_A.py
Module_B.py
SecondTestProject
/src
/SecondTestProjectNamespace
__init__.py
Module_1.py
Module_2.py
...
Module_10.py
Note that MyTestProjectNamespace has a reference to SecondTestProjectNamespace.
In MyTestProjectNamespace, I need to import everything in SecondTestProjectNamespace. I could import one module at a time with the following statement(s):
from SecondTestProjectNamespace.Module_A import *
from SecondTestProjectNamespace.Module_B import *
...but this isn't practical if the SecondTestProject has 50 modules in it.
Does Python support a way to import everything in a namespace / package? Any help would be appreciated.
Thanks in advance.

Yes, you can roll this using pkgutil.
Here's an example that lists all packages under twisted (except tests), and imports them:
# -*- Mode: Python -*-
# vi:si:et:sw=4:sts=4:ts=4
import pkgutil
import twisted
for importer, modname, ispkg in pkgutil.walk_packages(
path=twisted.__path__,
prefix=twisted.__name__+'.',
onerror=lambda x: None):
# skip tests
if modname.find('test') > -1:
continue
print(modname)
# gloss over import errors
try:
__import__(modname)
except:
print 'Failed importing', modname
pass
# show that we actually imported all these, by showing one subpackage is imported
print twisted.python
I have to agree with the other posters that star imports are a bad idea.

No. It is possible to set up SecondTestProject to automatically import everything in its submodules, by putting code in __init__.py to do the from ... import * you mention. It's also possible to automate this to some extent using the __import__ function and/or the imp module. But there is no quick and easy way to take a package that isn't set up this way and make it work this way.
It's probably not a good idea anyway. If you have 50 modules, importing everything from all of them into your global namespace is going to cause a proliferation of names, and very likely conflicts among those names.

As other had put it - it might not be a good idea. But there are ways of keeping your namespaces and therefore avoiding naming conflicts - and having all the modules/sub-packages in a module available to the package user with a single import.
Let's suppose I have a package named "pack", within it a module named "a.py" defining some "b" variable. All I want to do is :
>>> import pack
>>> pack.a.b
1
One way of doing this is to put in pack/__init__.py a line that says
import a - thus in your case you'd need fifty such lines, and keep them up to date.
Not that bad.
However, the documentation at http://docs.python.org/tutorial/modules.html#importing-from-a-package - says that if you have a string list named __all__ in your __init__.py file, all module/sub-package names in that list are imported when one does from pack import *
That alone would half-work - but would require users of your package to perform the not-recommended "from x import *" form.
But -- you can do the "... import *" inside __init__.py itself, after defining the __all__ variable - so all you have to do is to keep the __all__ up to date:
With the TestProjectNamespace/__init__.py being like this:
__all__ = ["Module_A", "Module_B", ...]
from TestProjectNamespace import *
your users would have
TestProjectNamespace.Module_A (and others) available upon import of TestProjectNamespace.
And, of course - you could automate the creation of __all__ - it is just a variable, after all - but I would not recommend that.

Does Python support a way to import everything in a namespace / package?
No. A package is not a super-module -- it's a collection of modules grouped together.
At least part of the reason is that it's not trivial to determine what 'everything' means inside a folder: there are problems like network drives, soft links, hard links, ...