We Keep Coding

Run Python module from command line like in Pycharm

I have a directory structure like this:
my_project
.idea
src
config.ini
method.py
... other py modules
When I open the project in PyCharm and run with the configuration below, it runs perfectly. It runs src.method as a module using an Anaconda 3 environment. The src.method module uses relative imports from other modules in src and takes config.ini as an argument.
I am trying to run the module in the same way through the Windows command line from src as the working directory
C:\Users\deimos\Anaconda3\envs\cveureka\python.exe -m src.method "config.ini"
but this gives a ModuleNotFoundErrorsaying it could not find src. I tried replacing src.method with just method but that gives ImportError: attempted relative import with no known parent package.
I have also tried setting the PYTHONPATH before running the module with
setlocal
set PYTHONPATH=%2
like in this answer, but to no effect.
It there a way to set up the command line to replicate the way that PyCharm runs the module?

Python - How Do I Make Relative Imports Compatible with Nose

I am working on a package which has a folder structure like:
Root
|---Source
|---Testing
|---Test_Utils
|---test_fixtures.py
|---test_integration.py
|---test_unit.py
There are a fair amount of relative references flying around (e.g. in test_integration.py I need to import classes from the files in the Source folder as well as test harnesses and data from the Test_Utils folder.
So far I've managed this by using complete references e.g.:
from Root.Testing.Test_Utils.test_fixtures import *
Which seemed to work fine until actually trying to run nosetests. This is because nose seems only to find test files in the active directory (not the root working directory), so I have to cd Testing before running nosetests.. at which point the relative references break with:
ModuleNotFoundError: No module named 'Root'
How can I get round this seeming incompatibility (without using pytest, since I am using test generators (i.e. using yield) which I believe are deprecated in pytest)?

PYTHONPATH sets the search path for importing python modules.
If you are using (Mac or GNU/Linux distro), add this to your ~/.bashrc.
# add this line to ~/.bashrc
export PYTHONPATH=PYTHONPATH:/path/to/folder # path where is your Root folder
# after this we need to reload ~/.bashrc
$ source ~/.bashrc
# if source ~/.bashrc don't work simple restart your terminal
# after this we can echo out our PYTHONPATH environment variable to check if it was added successfully
$ echo PYTHONPATH
# it need to output your folder

Run Python plugins in Hiero

I am facing a small problem in executing custom made python code from Hiero. As everybody knows, the first place to look for details is the offical documentation "Nukepedia", the Python Dev Guide and the Hiero User Guide. Well according to the Python Dev Guide, Hiero creates a ".hiero" directory by default which lets people add plugin paths to use them in the software. Well, I can't find that directory and I deduced after several tests that Hiero is using the init.py saved in the ".nuke" directory.
So I thought that maybe I could add the plugin paths there but kept getting the famous Import Error for the _fnpython module (before creating Python/Startup directory).
I added Python/Startup folders in .nuke and added the plugins in Startup, I got the same error, I even tried it by adding the path to the plugins in init.py and got the same error too.
Then I created a ".hiero" folder and did the same thing as before but Hiero never took that folder into consideration, I deduced that by printing some strings in the console, Hiero always took the init.py saved in the ".nuke" folder and kept showing the same error.
Finally, I tried to look into the installation process and try to seperate Nuke and Hiero's folders maybe that would create the ".hiero" directory but everything was automated.
The code that I want to run is given by Nuke and Hiero (in the examples directory), I just can't figure out what to do in order to run it from the program.
What should I do in order to fix this problem ?
Thank you in advance.

The setup for The Foundry HIERO is a little different than for NUKE.
HIERO has a core module. You'll see it in __init__.py file:
import FnRedirect
import FnPythonFixes
import core
import ui
try:
if hasattr(core, "taskRegistry"):
import importers
import exporters
except ImportError as e:
pass
I'm running HIERO and NUKE STUDIO on a Mac, so there's a full path to HIERO's __init__.py file inside package contents:
/Applications/Nuke10.5v5/Contents/MacOs/pythonextensions/site-packages/hiero/__init__.py
You need to import this module using:
import hiero.core
or using a reference to the core package:
from core import *
To find HIERO's current paths you have to run this line in its Script Editor:
print hiero.core.pluginPath()
Click this link for further details: Hiero's Environment Setup
All these instructions are suitable for macOS 10.9 and above. Here are two blocks of steps: first for Terminal Mode and second for UI Mode.
BLOCK 1: setup for Terminal Sessions
You need to manually create .hiero directory in your Home area.
The recommended default location for running Python on startup is:
~/.hiero/Python/Startup
~/.hiero/Python/StartupUI
Type in your bash Terminal (when you're inside your Home user directory) the following line:
mkdir .hiero/
then:
mkdir .hiero/Python/
and then:
mkdir .hiero/Python/StartupUI/
then navigate to Home directory with:
cd ~
and check it with:
ls -a
Also you can specify any number of user-defined paths using the environment variable HIERO_PLUGIN_PATH, just like the standard Unix PATH environment variable.
For that you need to set up an environment variable in .bash_profile. To run in Terminal PICO editor just type (remember you need an administrator's password for sudo command):
sudo pico .bash_profile
and paste these three lines in it (change swift for <yourName> and save this file):
echo HIERO environment var is set...
export HIERO_PLUGIN_PATH=/Users/swift/.hiero/Python/StartupUI/
export PATH=$PATH:$HIERO_PLUGIN_PATH
Write out a file with ctrl o
Exit pico editor with ctrl x
Restart Terminal
In Terminal you could print this environment variable typing:
printenv HIERO_PLUGIN_PATH
You should put inside that StartupUI directory menu.py, any set of xxxx.py or xxxx.pyc files, as well as __init__.py file.
Now you can use /Users/swift/.hiero/Python/StartupUI/ path in Terminal Mode.
BLOCK 2: setup for UI Sessions
To assign the current paths that HIERO searches when loading plug-ins, you need to create __init__.py file with the following lines:
import hiero.core
path='/Users/swift/.hiero/Python/Startup/'
hiero.core.addPluginPath(path)
After that make Python/Startup/ subdirectories under ~/.nuke/ folder.
It's not a mistake: I typed .nuke.
Then place this __init__.py file into /Users/swift/.nuke/Python/Startup/ directory.
Restart HIERO (or NUKE STUDIO) if it works.
After that launch HIERO or NUKE STUDIO and run
print hiero.core.pluginPath()
command in the HIERO's Script Editor or in NUKE STUDIO's Script Editor and you'll see this result:
After that you'll find new __init__.pyc file in /Users/swift/.nuke/Python/Startup/ directory.
Each package or module discovered when you launch HIERO is imported and added to the built-in package hiero.plugins.
TEST 1: custom_guides.py
I do not have a commercial version of HIERO so I tested custom_guides.py script ( found here ) using NUKE STUDIO NC.
I placed custom_guides.py in ~/.nuke/Python/Startup directory and then added two lines to NUKE's init.py file located in ~/.nuke directory.
import nuke
nuke.pluginAddPath("./Python/Startup")
The only thing I could say: "it works" Do the same actions as I did and it'll work for HIERO.
Look at safe_zone and masking_ratio dropdown menus. They are different: before and after.
Before uploading custom_guides.py script:
After uploading custom_guides.py script:
# file custom_guides.py contains these lines:
viewer_masks = [
hiero.ui.guides.MaskGuide("NTSC", 0.91),
hiero.ui.guides.MaskGuide("PAL", 1.09),
hiero.ui.guides.MaskGuide("NTSC_16:9", 1.21),
hiero.ui.guides.MaskGuide("PAL_16:9", 1.46),
hiero.ui.guides.MaskGuide("Cinemascope 2:1", 2.0)
]
TEST 2: web_browser.py
I placed web_browser.py file in ~/.nuke/Python/Startup directory. This Python script creates dockable panel with web browser written with PySide Qt.
I do not have a commercial version of HIERO so I tested web_browser.py script ( found here ) using NUKE STUDIO NC.

How can I use a Python script in the command line without cd-ing to its directory? Is it the PYTHONPATH?

How can I make any use of PYTHONPATH? When I try to run a script in the path the file is not
found. When I cd to the directory holding the script the script runs. So what good is the
PYTHONPATH?
$ echo $PYTHONPATH
:/home/randy/lib/python
$ tree -L 1 '/home/randy/lib/python'
/home/randy/lib/python
├── gbmx_html.py
├── gbmx.py
├── __init__.py
├── __pycache__
├── scripts
└── yesno.py
$ python gbmx.py -h
python: can't open file 'gbmx.py': [Errno 2] No such file or directory
$ cd '/home/randy/lib/python'
After cd to the file directory it runs ..
$ python gbmx.py -h
usage: gbmx.py [-h] [-b]
Why can I not make any use of the PYTHONPATH?

I think you're a little confused. PYTHONPATH sets the search path for importing python modules, not for executing them like you're trying.
PYTHONPATH Augment the default search path for module files. The
format is the same as the shell’s PATH: one or more directory
pathnames separated by os.pathsep (e.g. colons on Unix or semicolons
on Windows). Non-existent directories are silently ignored.
In addition to normal directories, individual PYTHONPATH entries may
refer to zipfiles containing pure Python modules (in either source or
compiled form). Extension modules cannot be imported from zipfiles.
The default search path is installation dependent, but generally
begins with prefix/lib/pythonversion (see PYTHONHOME above). It is
always appended to PYTHONPATH.
An additional directory will be inserted in the search path in front
of PYTHONPATH as described above under Interface options. The search
path can be manipulated from within a Python program as the variable
sys.path.
http://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH
What you're looking for is PATH.
export PATH=$PATH:/home/randy/lib/python
However, to run your python script as a program, you also need to set a shebang for Python in the first line. Something like this should work:
#!/usr/bin/env python
And give execution privileges to it:
chmod +x /home/randy/lib/python/gbmx.py
Then you should be able to simply run gmbx.py from anywhere.

You're confusing PATH and PYTHONPATH. You need to do this:
export PATH=$PATH:/home/randy/lib/python
PYTHONPATH is used by the python interpreter to determine which modules to load.
PATH is used by the shell to determine which executables to run.

PYTHONPATH only affects import statements, not the top-level Python interpreter's lookup of python files given as arguments.
Needing PYTHONPATH to be set is not a great idea - as with anything dependent on environment variables, replicating things consistently across different machines gets tricky. Better is to use Python 'packages' which can be installed (using 'pip', or distutils) in system-dependent paths which Python already knows about.
Have a read of https://the-hitchhikers-guide-to-packaging.readthedocs.org/en/latest/ - 'The Hitchhiker's Guide to Packaging', and also http://docs.python.org/3/tutorial/modules.html - which explains PYTHONPATH and packages at a lower level.

With PYTHONPATH set as in your example, you should be able to do
python -m gmbx
-m option will make Python search for your module in paths Python usually searches modules in, including what you added to PYTHONPATH. When you run interpreter like python gmbx.py, it looks for particular file and PYTHONPATH does not apply.

I think you're mixed up between PATH and PYTHONPATH. All you have to do to run a 'script' is have it's parental directory appended to your PATH variable. You can test this by running
which myscript.py
Also, if myscripy.py depends on custom modules, their parental directories must also be added to the PYTHONPATH variable. Unfortunately, because the designers of python were clearly on drugs, testing your imports in the repl with the following will not guarantee that your PYTHONPATH is set properly for use in a script. This part of python programming is magic and can't be answered appropriately on stackoverflow.
$python
Python 2.7.8 blahblahblah
...
>from mymodule.submodule import ClassName
>test = ClassName()
>^D
$myscript_that_needs_mymodule.submodule.py
Traceback (most recent call last):
File "myscript_that_needs_mymodule.submodule.py", line 5, in <module>
from mymodule.submodule import ClassName
File "/path/to/myscript_that_needs_mymodule.submodule.py", line 5, in <module>
from mymodule.submodule import ClassName
ImportError: No module named submodule

How to fix "ImportError: No module named ..." error in Python?

What is the correct way to fix this ImportError error?
I have the following directory structure:
/home/bodacydo
/home/bodacydo/work
/home/bodacydo/work/project
/home/bodacydo/work/project/programs
/home/bodacydo/work/project/foo
And I am in the directory
/home/bodacydo/work/project
Now if I type
python ./programs/my_python_program.py
I instantly get
ImportError: No module named foo.tasks
The ./programs/my_python_program.py contains the following line:
from foo.tasks import my_function
I can't understand why python won't find ./foo/tasks.py - it's there.
If I do it from the Python shell, then it works:
python
>>> from foo.tasks import my_function
It only doesn't work if I call it via python ./programs/my_python_program.py script.

Python does not add the current directory to sys.path, but rather the directory that the script is in. Add /home/bodacydo/work/project to either sys.path or $PYTHONPATH.

Do you have a file called __init__.py in the foo directory? If not then python won't recognise foo as a python package.
See the section on packages in the python tutorial for more information.

A better fix than setting PYTHONPATH is to use python -m module.path
This will correctly set sys.path[0] and is a more reliable way to execute modules.
I have a quick writeup about this problem, as other answerers have mentioned the reason for this is python path/to/file.py puts path/to on the beginning of the PYTHONPATH (sys.path).

Here is a step-by-step solution:
Add a script called run.py in /home/bodacydo/work/project and edit it like this:
import programs.my_python_program
programs.my_python_program.main()
(replace main() with your equivalent method in my_python_program.)
Go to /home/bodacydo/work/project
Run run.py
Explanation:
Since python appends to PYTHONPATH the path of the script from which it runs, running run.py will append /home/bodacydo/work/project. And voilà, import foo.tasks will be found.

Example solution for adding the library to your PYTHONPATH.
Add the following line into your ~/.bashrc or just run it directly:
export PYTHONPATH="$PYTHONPATH:$HOME/.python"
Then link your required library into your ~/.python folder, e.g.
ln -s /home/user/work/project/foo ~/.python/

In my mind I have to consider that the foo folder is a stand-alone library. I might want to consider moving it to the Lib\site-packages folder within a python installation. I might want to consider adding a foo.pth file there.
I know it's a library since the ./programs/my_python_program.py contains the following line:
from foo.tasks import my_function
So it doesn't matter that ./programs is a sibling folder to ./foo. It's the fact that my_python_program.py is run as a script like this:
python ./programs/my_python_program.py

If you have this problem when using an instaled version, when using setup.py, make sure your module is included inside packages
setup(name='Your program',
version='0.7.0',
description='Your desccription',
packages=['foo', 'foo.bar'], # add `foo.bar` here