We Keep Coding

Python, absolute/relative import not working as expected

I apologize for the millionth post about this topic.
I thought I had a good grip of the whole absolute/relative import mechanism - I even replied to a couple of questions about it myself - but I'm having a problem with it and I can't figure out how to solve it.
I'm using Python 3.8.0, this is my directory structure:
project_folder
scripts/
main.py
models/
__init__.py
subfolder00/
subfolder01/
some_script.py --> contains def for some_function
I need to import some_function from some_script.py when running main.py, so I tried:
1) relative import
# in main.py
from ..models.subfolder00.subfolder01.somescript import some_function
but when I run (from the scripts/ folder)
python main.py
this fails with error:
ImportError: attempted relative import with no known parent package
This was expected, because I'm running main.py directly as a script, so its _name_ is set to _main_ and relative imports are bound to fail.
However, I was expecting it to work when running (always from within the scripts folder):
python -m main
but I'm getting always the same error.
2) absolute import
I tried changing the import in main.py to:
# in main.py
from models.subfolder00.subfolder01.somescript import some_function
and running, this time from the main project folder:
python scripts/main.py
so that - I was assuming - the starting point for the absolute import would be the project folder itself, from which it could get to models/....
But now I'm getting the error:
ModuleNotFoundError: No module named 'models'
Why didn't it work when using the -m option in the case of relative import, and it's not working when using absolute ones either? Which is the correct way to do this?

I think quite likely you missed python's official doc ( that even come offline )
https://docs.python.org/3/tutorial/modules.html
you'll need a dummy __init__.py within your module, at same level of some_script.py

I think your "absolute" import may not have been absolute in the truest sense.
Prior to running the python scripts/main.py command, you would have needed to setup PYTHONPATH environment variable to include the path to project_folder.
Alternatively I do something like this in main.py:
import sys
import os
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)),'..','models','subfolder00','subfolder01'))
from somescript import some_function
Maybe it is a little pedantic, but it makes sense to me.

Relative imports for the trillionth time

I have the following data structure:
project/
folder_a/
file.py
folder_b/
useful_functions.py
I am running my file.py and attempting to import a set of functions I have written within useful_functions.py.
At first I tried the following:
from ..folder_b.useful_functions import function_a
But got the following error:
ValueError: attempted relative import beyond top-level package
I then removed the two dots at the beginning of the import statement which initially worked. I have since revisited this project with no changes and I am faced with a new error message. The following code:
from folder_b.useful_functions import function_a
Gives me the following error message:
ModuleNotFoundError: No module named 'folder_b'
I find it very strange that one time it works and later with no changes the import fails. I really would like to solve this with relative imports as I would like the code to work other machines with different absolute file paths.
Thanks in advance.

Well for the trillionth time running a script directly inside a package is considered an antipattern. I do not know how spyder runs stuff but presumably does
python file.py # from inside folder_a
This should be changed to
python -m folder_a.file # from inside project
This would resolve relative imports among packages inside project folder alright and will not need any sys.path or PYTHONPATH hacks

How can I import a package from within the package?

I'm writing a little package and I'm trying to include a demo script within it as an example. However, I can't seem to import the package cleanly from within as though I was outside of it.
With a directory structure like:
trainer/
__init__.py
helper.py
trainer.py
[...more files...]
demo.py
In demo.py I can't do from .. import trainer as it complains "Attempted relative import in non-package", despite the __init__.py. If I move the demo up a directory and import trainer it works fine, but I was trying to keep it together with the package.
The hack-looking import __init__ as trainer works, but eeeew.
Importing the various bits from all over the module directly also works, but makes for a messy example. Am I wholly misguided in my attempt or is there a better solution?

If you're trying to run demo.py as python demo.py, the problem that you're having is likely the same as here.
What's happening is that Python's relative import mechanism works by using the __name__ of the current module. When you execute a module directly, the __name__ gets set "__main__" regardless what the actual module name is. Thus, relative (in-package) imports don't work.
To remedy this, you can do the following:
Execute demo.py as a module within a package, like so: python -m trainer.demo. This should fix the error, but you'll still be importing the trainer.py module instead of the package.
Now add from __future__ import absolute_import to demo.py, which will cause your imports to be absolute-only by default, meaning that relative imports have to explicit (as in, from . import (...)). This is force import trainer to import the entire top-level package, instead of the module.

The way you organize the files, demo.py becomes part of the package, which might or might not be what you want. You can organize your files a little differently, moving demo.py outside of the trainer directory:
TopDir/
demo.py
trainer/
__init__.py
helper.py
trainer.py
[... more files ...]
Then, demo.py can do something like:
from trainer import trainer, helper

Cannot Import Python Package

I am having trouble importing python packages only when running python from cmdline/console. However, when using pydev, everything seems to work fine.
I have the following filesystem...
---MarketData
---Parser
---Parser.py
---__init__.py
---IO
---__init__.py
---MarketSocket.py
Currently, Parser and IO are defined as python packages (they have init.py files, although there is no code in the Parser.init.py file.
I am trying to run the following line of code in MarketSocket.py
from Parser import Parser
Which should import the module 'Parser' within the package 'Parser' however, I get the following error.
ImportError: No Module Named Parser
Any help would be appreciated! This should work according to similar issues on stackOverflow, but for some odd reason it isn't.

MarketSocket.py is in the directory IO. Therefore it is not possible to find the package Parser.
The best way to resolve this, are relative imports: from ..Parser import Parser But they might not work, if you start the script like: python MarketSocket.py. To use this, you would also have to add an __init__.py to your MarketData directory.
If it doesn't work extend the sys.path like this:
import sys
sys.path.append('../')
With this addition, Python searches also the paths you want.
If I were you I would also think about restructuring your project. In my opinion executables should be (most of the time) at the top of your working tree, which is also like Python works.

the MarketSocket.py is one level below Parser and thus can't see it
do this:
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
sys.path.append(os.path.dirname(__file__))

Putting an (empty) __init__.py in the MarketData directory will make the whole thing a package (and avoids the ugly path hacks). That should just work then if you call the module from the top level of the package.

You encountered an issue with relative import. Only in the parent directory you may have the access to any child package/module. So in MarketSocket.py, you need
from ..Parser import Parser
Then when you run it with -m option, the trick is you have to run it in the top level directory. So in this case
1) you would go to the parent directory of MarketData
2) in that parent directory, run python -m MarketData.IO.marketSocket

Understanding A Chain of Imports in Python

I know there are several similar questions, but I'm struggling to understand the error I'm getting and browsing the docs and similar questions hasn't helped yet. If anything, the similar questions make me feel like what I'm doing is right.
I have the following files:
src/main.py
from pack import pack
if __name__ == '__main__':
pack.exec("Hello Universe!")
src/pack/pack.py
import util
def exec(text):
util.write(text)
if __name__ == '__main__':
exec("Hello World!")
src/pack/util.py
def write(text):
print(text)
*src/pack/_init_.py*
EMPTY FILE
When I run python pack.py from the src/pack directory, it works (prints "Hello World!"). However when I run python main.py from the src directory I get the following exception:
Traceback (most recent call last):
File ".../src/main.py", line 1, in <module>
from pack import pack
File ".../src/pack/pack.py", line 1, in <module>
import util
ImportError: No module named util
If I change the import line in pack.py to from . import util as suggested, effectively the opposite occours. main.py runs successfully, however now pack.py fails, raising:
Traceback (most recent call last):
File ".../src/pack/pack.py", line 1, in <module>
from . import util
ValueError: Attempted relative import in non-package
I would have thought that imports are relative to the current location, and as such you ought to be able to construct a chain of imports like this. It seems very odd to me that module is supposed to import a sibling file differently depending on where the program starts.
Can someone explain why this error occurs in one way but not the other, and if there is some way to allow this file structure to run whether I want to run from main.py or pack.py?

You will have trouble making the import work in both cases. This is because in one case you are running pack.py as the main file and in another you run it as part of a package.
When you run it as a standalone script python pack.py, the "pack" directory gets added to the PYTHONPATH, meaning that you can import any module in it. Hence, import util will work.
When you run python main.py you add the src directory to your PYTHONPATH. This means that any module or package in src, for example the pack directory, now becomes importable. Hence from pack import pack. However, to access util.py you now need to do from pack import util. You can also do from . import util from within pack.py, as you noticed.
But you can't really do both at the same time. Either src/ is the main directory or src/pack is.
The obvious, but wrong solution, is to let the main.py add the src/pack directory to the PYTHONPATH. That will work, but it's not a good idea. The correct way to do this is to make up your mind. Is src/pack a module that should be imported via import pack or is it just a folder with a bunch of Python scripts? Decide! :-)
I think in this case its' obvious that src/pack should be a module. So then treat it like a module, and make sure it's available like a module. Then you can from pack import util even when running pack.py as a main script.
How do you do that? Well, basically you either install the pack module in your site-packages, or you add the src directory to the PYTHONPATH. That last one is what you want during development. You can either do it manually with export PYTHONPATH=<path> or you can let your testrunners do it for you. You don't have a testrunner? Well you should, but that's another question. :)
For installing it permanently once you don't do development anymore, take a look at Distribute. It includes a testrunner. ;)

Your link is to python 2.7 documentation, but it looks like you are using Python 3.x.
See here: http://docs.python.org/py3k/ for the correct docs.
Python 3 removes implicit relative import. That is, you cannot import packages within the same module in the same way anymore.
You need to use from . import util, this is an explicitly relative import and is allowed. import X no longer checks the current directory. Instead it only check the entries in sys.path. This includes the directory of the script was started and python's standard library.

You are missing __init__.py in your pack directory.
Create an empty file called __init__.py in your pack directory.

Nothing in the correct documentation supports your theory that this form of importing should work.