We Keep Coding

Executable .py file with shebang path to which python gives error, command not found

I have a self-installed python in my user directory in a corporate UNIX SUSE computer (no sudo privilege):
which python
<user>/bin/python/Python-3.6.1/python
I have an executable (chmod 777) sample.py file with this line at the top of the file:
#!<user>/bin/python/Python-3.6.1/python
I can execute the file like this:
python sample.py
But when I run it by itself I get an error:
/full/path/sample.py
/full/path/sample.py: Command not found
I have no idea why it's not working. I'm discombobulated as what might be going wrong since the file is executable, the python path is correct, and the file executes if I put a python command in the front. What am I missing?
EDIT:
I tried putting this on top of the file:
#!/usr/bin/env python
Now, I get this error:
: No such file or directory
I tried this to make sure my env is correct
which env
/usr/bin/env
EDIT2:
Yes, I can run the script fine using the shebang command like this:
<user>/bin/python/Python-3.6.1/python /full/path/sample.py

Your file has DOS line endings (CR+LF). It works if you run python sample.py but doesn't work if you run ./sample.py. Recode the file so it has Unix line endings (pure LF at the end of every line).

Try using #!/usr/bin/env python as described in this post. Let the OS do the work.

Python external project script execution

After many researches on the web, on many different topics, I wasn't able to find the solution answering my problem.
I am developing a tool (with graphic interface) allowing to process python scripts from different projects.
To not have any problem of dependences, I ask the user to target the project directory as well as the script to process.
Thus to develop this tool I have in my IDE a project "benchmark_tool" which is the project integrating the GUI, and the project "test_project" which groups various testing scripts. Obviously, the final goal is the tool to be compiled and thus that the project "benchmark_tool" disappears from IDE.
Here is the IDE architecture:
benchmark_tool (python project)
__init__.py
main.py
test_project (python project)
__init__.py
module1
__init__.py
test_script.py
module2
__init__.py
imports.py
How it works: The main.py shall call test_script.py.
test_script.py calls imports.py at the first beggining of the script.
UPDATE
I tried some modifications in my main code:
import sys
import subprocess
subprocess.check_call([sys.executable, '-m', 'test_project.module1.test_script'], cwd='D:/project/python')
I got this error
Traceback(most recent call last):
File "C:/Python31/lib/runpy.py", line 110, in run module as main
mod_name, loader, code, fname = _get_module_detail(mod_name)
File "C:/Python31/lib/runpy.py", line 91, in get module details
code = loader.get_code(mod_name)
File "C:/Python31/lib/pkgutil.py", line 272, in get code
self.code = compile(source, self.filename, 'exec')
File "D:/project/python/test_project/module1/test_script.py", line 474
SyntaxError: invalid syntax
Traceback (most recent call last):
File "D:/other_project/benchmark_tool/main.py", line 187, in read
subprocess.check_call([sys.executable, '-m', 'module1.test_script.py'], cwd='D:/project/python/test_project')
File "C:/Python31/lib/subprocess.py", line 446, in check call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:/Python31/python.exe', '-m', 'module1.test_script.py']' returned non-zero exit status 1
Note
It works with subprocess.Popen(['python','D:/project/python/test_project/module1/test_script.py])
What's the main difference between both methods ? I'll also have to pass arguments to test_scripts.py, which one is the best to use to communicate with a python script (input and output datas are exchanged) ?
Thanks by advance

There are 3 problems I see that you have to fix:
In order to import from module2, you need to turn that directory into a package by placing an empty *__init__.py file in it
When executing test_script.py, you need the full path to the file, not just file name
Fixing up sys.path only work for your script. If you want to propagate that to test_script.py, set the PYTHONPATH environment variable
Therefore, my solution is:
import os
import sys
import subprocess
# Here is one way to determine test_project
# You might want to do it differently
script_dir = os.path.abspath(os.path.dirname(__file__))
test_project = os.path.normpath(os.path.join(script_dir, '..', 'test_project'))
python_path = '{}:{}'.format(os.environ.get('PYTHONPATH', ''), test_project)
python_path = python_path.lstrip(':') # In case PYTHONPATH was not set
# Need to determine the full path to the script
script = os.path.join(test_project, 'module1', 'test_script.py')
proc = subprocess.Popen(['python', script], env=dict(PYTHONPATH=python_path))

To run an "external project script", install it e.g.: pip install project (run it in a virtualenv if you'd like) and then run the script as you would with any other executable:
#!/usr/bin/env python
import subprocess
subprocess.check_call(['executable', 'arg 1', '2'])
If you don't want to install the project and it has no dependencies (or you assume they are already installed) then to run a script from a given path as a subprocess (assuming there is module1/__init__.py file):
#!/usr/bin/env python
import subprocess
import sys
subprocess.check_call([sys.executable, '-m', 'module1.test_script'],
cwd='/abs/path/to/test_project')
Using -m to run the script, avoids many import issues.
The above assumes that test_project is not a Python package (no test_project/__init__.py). If test_project is itself a Python package then include it in the module name and start from the parent directory instead:
subprocess.check_call([sys.executable, '-m', 'test_project.module1.test_script'],
cwd='/abs/path/to')
You could pass the path to test_project directory as a command-line parameter (sys.argv) or read it from a config file (configparser,json). Avoid calculating it relative to the installation path of your parent script (if you have to then use pkgutil, setuptools' pkg_resources, to get the data). To find a place to store user data, you could use appdirs Python package.

How do I replicate the way PyCharm is running my Python 3.4 project at the command line?

My project looks like this:
running-pycharm-project-at-cmd
- main.py
- c
- run_project.py
- z
- __init__.py
- the_module.py
- y
- __init__.py
- template.md
- the_module_module.py
- the_support_function.py
The contents of the .py files are shown below:
main.py
from c.run_project import run
print('running main.py...')
run()
c/run_project.py
from c.z.the_module import the_module_function, the_module_write_function
def run():
print('Running run_project.py!')
the_module_function()
# write a file:
the_module_write_function(read_file='./z/y/template.md', write_file='../README.md')
if __name__ == '__main__':
run()
c/z/the_module.py
from c.z.y.the_module_module import the_module_module_function
def the_module_function():
print('the_module_function is running!')
the_module_module_function()
pass
def the_module_write_function(read_file, write_file):
with open(read_file, 'r') as fid:
with open(write_file, 'w') as fid_out:
contents = fid.read()
contents.replace('{}', 'THE-FINAL-PRODUCT!')
fid_out.write(contents)
c/z/y/the_module_module.py
from .the_support_function import this_support_data
def the_module_module_function():
print('The module module function is running!')
print("Here is support data: {}".format(this_support_data))
pass
c/z/y/the_support_function.py
this_support_data = "I am the support data!"
GitHub repo to make replication easier: running-pycharm-project-at-cmd
The problem: in Pycharm load up the project with running-pycharm-project-at-cmd as the root. Then I right click and run the run_project.py file. Everything works fine. I cannot figure out how to run run_project.py from the command line in the same way. Creating main.py was a workaround suggested elsewhere, but it fails due to the relative references to the template file (in actual project, the y folder is a git submodule and therefore cannot have absolute references in it).

If you copy your main.py file into the c folder and rename it __init__.py, then you could run:
$ python -m c
The -m argument tells python to run a module or package (in this case c). Python will look in the c's folder for an __init__.py file and run that. You just need to ensure that the folder c is either in your PYTHONPATH or is a subfolder of the current working directory.

I circled back to this and was able to get it working. Note that I am on Windows 7, and some of this might be platform dependent. In order to get this working without changing any code, the key is setting PYTHONPATH environment variable before running. PyCharm does this automatically (by default, but it is configurable in the Run Configuration options).
Steps:
Recreating the issue:
Clone the github repo to G:\TEST.
Navigate to G:\TEST\running-pycharm-project-at-cmd-master\c
python run_project.py
Traceback (most recent call last):
File "run_project.py", line 1, in <module>
from c.z.the_module import the_module_function, the_module_write_function
ImportError: No module named 'c'
Simplest Solution (if able to run Python from the script location):
set the PYTHONPATH environment variable before running. e.g.
SET PYTHONPATH=G:\TEST\running-pycharm-project-at-cmd-master
Now python run_project.py
Running run_project.py!
the_module_function is running!
The module module function is running!
Here is support data: I am the support data!
To run from a remote location (this is definitely Windows specific):
Create a .bat file that navigates to correct working directory before running Python. i.e.:
START cmd /K "cd G:\TEST\running-pycharm-project-at-cmd-master\c & python run_project.py"
G:\TEST\myotherlocation
run_it.bat
Running run_project.py!
the_module_function is running!
The module module function is running!
Here is support data: I am the support data!

run a python script in terminal without the python command

I have a python script let's name it script1.py. I can run it in the terminal this way:
python /path/script1.py
...
but I want to run like a command-line program:
arbitraryname
...
how can i do it ?

You use a shebang line at the start of your script:
#!/usr/bin/env python
make the file executable:
chmod +x arbitraryname
and put it in a directory on your PATH (can be a symlink):
cd ~/bin/
ln -s ~/some/path/to/myscript/arbitraryname

There are three parts:
Add a 'shebang' at the top of your script which tells how to execute your script
Give the script 'run' permissions.
Make the script in your PATH so you can run it from anywhere.
Adding a shebang
You need to add a shebang at the top of your script so the shell knows which interpreter to use when parsing your script. It is generally:
#!path/to/interpretter
To find the path to your python interpretter on your machine you can run the command:
which python
This will search your PATH to find the location of your python executable. It should come back with a absolute path which you can then use to form your shebang. Make sure your shebang is at the top of your python script:
#!/usr/bin/python
Run Permissions
You have to mark your script with run permissions so that your shell knows you want to actually execute it when you try to use it as a command. To do this you can run this command:
chmod +x myscript.py
Add the script to your path
The PATH environment variable is an ordered list of directories that your shell will search when looking for a command you are trying to run. So if you want your python script to be a command you can run from anywhere then it needs to be in your PATH. You can see the contents of your path running the command:
echo $PATH
This will print out a long line of text, where each directory is seperated by a semicolon. Whenever you are wondering where the actual location of an executable that you are running from your PATH, you can find it by running the command:
which <commandname>
Now you have two options: Add your script to a directory already in your PATH, or add a new directory to your PATH. I usually create a directory in my user home directory and then add it the PATH. To add things to your path you can run the command:
export PATH=/my/directory/with/pythonscript:$PATH
Now you should be able to run your python script as a command anywhere. BUT! if you close the shell window and open a new one, the new one won't remember the change you just made to your PATH. So if you want this change to be saved then you need to add that command at the bottom of your .bashrc or .bash_profile

Add the following line to the beginning script1.py
#!/usr/bin/env python
and then make the script executable:
$ chmod +x script1.py
If the script resides in a directory that appears in your PATH variable, you can simply type
$ script1.py
Otherwise, you'll need to provide the full path (either absolute or relative). This includes the current working directory, which should not be in your PATH.
$ ./script1.py

You need to use a hashbang. Add it to the first line of your python script.
#! <full path of python interpreter>
Then change the file permissions, and add the executing permission.
chmod +x <filename>
And finally execute it using
./<filename>
If its in the current directory,

Error "No such file or directory" when running Django ./manage.py

In my django project, the command ./manage.py [command] results in this error message:
: No such file or directory
The command python manage.py [command] works well. I tried with syncdb and runserver.
I tried chmod a+x manage.py, but the problem persists.
My manage.py:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
I use django 1.4.1 in a virtualenv.
How can I fix this to use manage.py [command]?

Likely, the reason is because your line endings in the manage.py file are \n instead of \r\n. As a result the #! hash-bang is misinterpreted.
This happens to me when I use a Windows based text-editor for my linux connection.

The #! hash-bang line doesn't point to your virtualenv python; replace the first line with:
#!/path/to/virtualenv/bin/python

In my django project, the command ./manage.py [command] results in
this error message:
: No such file or directory
The command python manage.py [command] works well
If specifying the interpreter makes it work, then it is the first line that must be wrong:
#!/usr/bin/env python
Try:
#!/usr/bin/python
(or wherever the interpreter is. Find it with: which python).

In my case, I was erroneously changing the sys.path in my manage.py.

In my case on Windows 7, every else it seems to be, but I've accidentally added an import in a views.py file:
from Scripts.pilprint import description
My software doesn't need this import, maybe with some wrong short-kut, my Eclipse wrote it for me, but removed this line, the problem disappear.
I suppose that description contain some painful character or have a wrong encoding for Windows.