Python script: problems with shebang line (unix) - python

I am trying to get a feel for the Flask microframework by launching a test application to local server. When trying to run my code, app.py, I keep getting the error message:
-bash: ./app.py: /flask/bin/python: bad interpreter: No such file or directory
Here is the basic code (taken from here) for app.py, which lives in my todo-api directory:
#!/flask/bin/python
from flask import Flask
app = Flask(__name__)
#app.route('/')
def index():
return "Hello, World!"
if __name__ == '__main__':
app.run(debug=True)
I've checked the file path to the python interpreter, and it should exist:
:bin $ pwd python
Users/me/Documents/Python/todo-api/flask/bin
I have followed the tutorial to the T; I've tried changing the shebang line to:
#!/flask/bin/python2.x
#!flask/bin/python
#!/flask/bin/env python
But to no avail. I am not that knowledgeable about bash, and have tried looking up what is going on, but the solutions to folks with similar problems have not worked for me; is there something going on behind the scenes that I am not understanding?

Bash shebangs expect an absolute path to the interpreter. So in your case you need to specify the full path to your Python interpreter i.e.:
#!/Users/me/Documents/Python/todo-api/flask/bin
You might want to investigate the use of /usr/bin/env python to be able to use the interpreter that is available in your user's $PATH environment variable. See https://unix.stackexchange.com/questions/12736/how-does-usr-bin-env-know-which-program-to-use/12751#12751

pwd tells you the current directory. It doesn't tell you where a command is located. The output from that command is a red herring.
You may be looking for which python. Put that path into your shebang line. Note that this will give you the Python interpreter from your $PATH, which may or may not be the right one.
The standard shebang line for Python scripts is
#!/usr/bin/env python
or
#!/usr/bin/python

I was having a similar issue with trying to setup a python script as an executable for testing some things and realized that bash was getting in the way more than it was helping. I ended up setting up pyinstaller (which is incredibly easy) and then making my script a stand alone executable without bash being in the mix.
Here's what I did (only takes a couple of minutes and no config):
First; pyinstaller needs: build-essential & python-dev
apt-get install build-essential python-dev
(or yum, etc... depending on your OS)
Then use the built in python package manager to setup pyinstaller:
pip install pyinstaller
That's it. Run: pyinstaller --onefile myapp.py (or pyinstaller.exe if your OS needs exe)
If it's successful (and it usually is), your new executable will be in a folder "Dist" in the same area you ran pysinstaller.

Related

Python Path For Local Import Git Bash

How should git bash be set up to run an non pip installed python library?
I am trying to run a Python library that I have git cloned (not pip installed) on Windows using git bash. I keep getting an error that the module cannot be found even though I have set the $PYTHONPATH to point to this folder.
This occurs both when using a venv and the base environment.
I've activated the venv using the activate command in the venv folder, and then added the folder to the PYTHONPATH as is shown here (excuse the blanking):
The first line shows the error I get when running the script which calls in other modules. The second shows my PYTHONPATH, the third my PATH
Adding the local folder to the path is how this functionality works on mac.
This the particular file that is meant to run, datagen.py:
#!/usr/bin/env python
from msq_data_gen.cli import cli
if __name__ == '__main__':
cli.datagen()
The code fails on the first import of msq_data_gen.
The structure of the program is as so:
Curiously, the integrated running of Pycharm works correctly, but as the code is based off a CLI, this isn't the intended usecase. (I could also use the unit testing mode for click, but I think an answer for this could be useful for other people!)

How can I setup a python CLI application so that I can use it without directly referring to the interpreter?

I want to build an application, say it's called helloworld, with a command line interface in python. My application as multiple nested modules and a top level module main.py.
Say the layout is :
helloworld/
helloworld/
__init__.py
module1/
module2/
module3/
setup.py
main.py
At this point, once my project is installed I can run it using
#> python path/to/helloworld/main.py arg1 arg2 arg3
I want to be able to interact with the program using commands such as :
#> helloworld arg1 arg2 arg3, in a similar manner as for example, the Flask framework comes with a command line application I can use with #> flask run.
I looked around and found questions suggesting using a shebang. As far as I can tell, this is not what I want.
I want a self contained application I can install using pip and then launch directly without any reference to the python interpreter.
How should I go about this ?
EDIT
It is an important requirement to me that the package can be installed using pip and the command line interface is immediately available. Shebangs/fiddling manually with the path/creating an additional shell script do not solve my problem.
You should add main.py to your PATH. What happens when you are running flask run is that your teminal looks up the command flask in PATH and runs the program that it is pointing to. You could see it as a kind of shortcut to the program Flask.
By adding your program to your PATH, you can tell the computer that if you type helloworld in your terminal, the terminal should run /my/path/to/helloworld.py.
I don't know what OS you are on, so here are links for most common OS on how to add a PATH variable.
Windows
Linux
Mac OSX
Edit: After you gave more explanation, setuptools is what you are looking for. Please look at this reference to see how to use it.
There's an explanation here of how to tell setuptools to add a script to the Python Scripts-folder during the pip installation. You can then make that script be for example a bat-file that calls your Python-script. In that way you achieve both that you do not need to write .py for your script to run, and that it happens automatically.
I know this has already been answered, but just in case someone comes across this in the future..
Here's what worked for a new package I'm developing after reading up on this from many different sources.
tldr; use setuptools and add your command as a console_scripts option under [options.entry_points] in setup.cfg.
You will also need some other fields in setup.cfg or setup.py for package discovery to work. Check out the setuptools docs for that.
Example entry point:
[options.entry_points]
console_scripts =
mycmd = mypackage.mymodule:myfunc
For local development, you can install the package in editable (development) mode with pip install -e . which will allow you to test the command as if you pip intalled it.
Alternatively, you could create a __main__.py that initializes your module when you run python -m mypackage from the root.
from mypackage import app
if __name__ == '__main__':
app.main()
I personally opted for both. Usually your main.py (or app.py in my example) belongs in your package not project level dir. Then you would create __main__.py there as well.

shebang not working on WSL (Windows Subsystem Linux)

I installed WSL (Ubuntu 18.04) on Windows 10 to use instead of using parallel 2 OS. However, when I was using shebang, I couldn't run the script named test_file.py successfully. First, I created a folder named test_project. In this folder, I created:
An src folder that contains a simple Python script is shown as follows
#!/usr/bin/env python3
print("Hello world")
A Python environment folder named environment through python3 -m venv environment/ (I'm using Python 3.6)
The problem is when I stepped into the environment and typed some commands like in the attached picture Run python file, the program ran "command not found" although I already used shebang at the beginning of the code.
Apart from that, I also changed #!/usr/bin/env to #!/usr/bin/python3 and escape the environment, it didn't work as well. Only when I type Python3 test_file.py it worked.
Has anyone faced this problem before? Can anyone explain to me why this happens? I'm wondering whether it is different between using WSL compared to Ubuntu in this case, or I was missing some steps during coding. I know that this problem is debated common previously, however I couldn't find any source that can tackle my problem. I really appreciate all your help.
I had the same issue and resolved it by the changing the line endings from CRLF which is native to Windows to LF which is what the shell is expecting.
You can do this with the dos2unix command:
dos2unix -b test_file.py
You'll probably need to install dos2unix on some newer distributions.
There are other methods that can be used as well.

Script working in ipython but not from the command line

I have a script that functions from within ipython but when I try and run the same script from the command line I receive import errors for a local module that I am trying to import:
from helper_functions.email_from_server import send_email
Error:
ImportError: No module named helper_functions.email_from_server
This script imports from within Ipython without any issues.
Comparatively, I have code that runs without any issues within ipython I can run another script using the command:
run script.py
From the command line I can run the same script:
python /dir/script.py
However this python /dir/script.py doesn't work with the script with local imports (from above) and I can't figure out if its a pythonpath issue or some local env issue? I have been reading through stack to find it but haven't been able to thus far. It feels like its just around the corner
One attempted solution:
PYTHONPATH=/dir/ python /dir/script.py
EDIT (to help clarify):
I am using an anaconda distribution on a linux machine.
Mucking about with PYTHONPATH is a recipe for sadness. You can do it, but you shouldn't. The correct thing to do is install your package in your correct environment. If you don't know how to create a package here's a super simple example. There may be some differences in your path when running via ipython vs command line.
You can find out what the differences are by using sys.executable and sys.path:
import sys
print(sys.executable)
print(sys.path)
Run that from IPython, and then run that from the python on your command line. You will undoubtedly get two different results. Since you're running Anaconda, you want to follow their guide for installing non-conda packages to install the one that you build.
Though of course that assumes that you've got the anaconda python on your path - you can check that out with which python since you're on Linux.
I resolved it via creating a wrapper shell script. Ugly in that i'm exporting the python path each time, but it works.
#!/bin/bash
export PYTHONPATH="${PYTHONPATH}:/my/dir"
source ~/.bash_profile
cd /my/dir && my/anaconda/location/bin/python /my/dir/to/script/cript.py

Why can't python find some modules when I'm running CGI scripts from the web?

I have no idea what could be the problem here:
I have some modules from Biopython which I can import easily when using the interactive prompt or executing python scripts via the command-line.
The problem is, when I try and import the same biopython modules in a web-executable cgi script, I get a "Import Error"
: No
module named Bio
Any ideas here?
Here are a couple of possibilities:
Apache (on Unix) generally runs as a different user, and with a different environment, to python from the command line. Try making a small script that just prints out sys.version and sys.prefix, and compare the result through apache and via the command line, to make sure that you're running from the same installation of python in both environments.
Is Biopython installed under your home directory, or only readable just for your normal user? Again, because apache generally runs as a different user, perhaps you don't have access to that location, so can't import it.
Can you try doing import site before trying to import Biopython? Perhaps something is preventing site packages from being imported when you run through apache.
In the cgi script, you could try to add the path to this package before any import.
sys.path.insert(0, 'path to biopython package')
If you are using Apache, you should be able to set the PYTHONPATH in conf file with directive SetEnv
SetEnv PYTHONPATH "path to biopython package"
I had same problem. I solved this problem by changing user of Apache in Linux Ubuntu by command in terminal:
sudo gedit /etc/apache2/envvars
Please change www-data on export APACHE_RUN_USER and export APACHE_RUN_GROUP to your current user or user that can run python script.
Have a good time ;)

Categories