I have a python file download.py in which I am importing a module opc
Now, this python file invoked from a shell script build.sh like:
python3 $(pwd)/scripts/download.py -d $dir_name
When I am running the shell script, I am getting the following error:
Traceback (most recent call last):
File "/home/ray/Desktop/repo/image-builder/scripts/download.py", line 14, in <module>
from opc.utils import file_utils, logger
ModuleNotFoundError: No module named 'opc'
This is how shell script is invoked:
sudo sh scripts/src/build.sh
I have a virtual environment active where I have installed this package. And this package is there as well
>pip list
Package Version
--------------------------------- -------
attrs 21.2.0
.
.
opc 0.8.0
Interestingly , when I run the python code directly like
python3 scripts/download.py -d dir_path
it works fine. I don't not get ModuleNotFoundError error.
Also, this works fine as well:
❯ python3
Python 3.8.10 (default, Sep 28 2021, 16:10:42)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import opc.utils.global_vars as g
>>> print(g.LOCAL_HOST)
127.0.0.1
>>>
Directory structure:
/home/ray/Desktop/repo/image-builder/scripts/download.py
/home/ray/Desktop/repo/image-builder/scripts/src/build.sh
and i am running commands from:
/home/ray/Desktop/repo/image-builder
This is likely to be because when you invoke the shell script, the python path is different from when you're running the Python interpreter. If that is the case, there are several ways to address that.
What happens if you manually change into the same directory as download.py and run your python script from there? Something like:
cd scripts
python download.py -d <some-dir>
Does that work, or do you still hit the same problem?
In shell scripts like this I first make sure the script will always be invoking the python app from the exact same place in the file system, so that I don't hit these python path issues. If you know for sure that python download.py -d <some-dir> works from inside the same directory as download.py, then you can add something like this to your shell script (assuming your shell script and download.py are in the same directory):
pushd "${0%/*}" # this changes directory to the directory where the shell script itself lives
# do your stuff here - run your python app, etc.
popd # returns to whatever directory we were originally in
In this way, I can always invoke the shell script from whatever directory I am in and I know it will behave the same way - the current directory is no longer relevant.
I wouldn't use $(pwd), because that will give the current directory, which varies, so your shell script will only ever work properly when invoked from inside the right directory if you do that.
Related
I want to run a python file in the Git bash but it is not executing without adding py before the file name. Python is installed but when I use the shortcut to execute the python file, an error is shown as below-
user#WinPC MINGW64 /d/git_basic/scripts (main)
$ python
Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
user#WinPC MINGW64 /d/git_basic/scripts (main)
$ ./all_checks.py
Python was not found; run without arguments to install
from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
Please guide me. Thanks
Edit 1
What I want is to run the python file in the terminal without using python or py in the command (py file.py or python file.py). I want to run it like this ./file.py
This is the all_checks.py code-
#! /usr/bin/env python3
import os
import sys
def check_reboot():
# returns true if pc has a pending reboot
return os.path.exists("/run/reboot-required")
def main():
if check_reboot():
print("pending reboot.")
sys.exit(1)
print("No pending reboot.")
sys.exit(0)
main()
Edit 2
I was able to run the file with ./ by just changing the python3 to python in the shebang line.
I was able to run the file with ./ by just changing the python3 to python in the shebang line.
Python 3 Docs - Shebang lines
I was not clear on exactly what you wanted but I have 2 options here.
Just running the file and making the file an executable.
If you just want to run it then use the terminal command, python file-name or python3 file-name if you have any python 2 installations.
You do not run a python file with this command: ./file-name
However, if you want to make it an executable file keep on reading.
If you are trying to make an executable, pyinstaller will do the job for you. If you have pip, then run the command pip install pyinstaller. Then, once you have pyinstaller installed, open terminal or command prompt and locate the directory of the python file using cd. Then once you are in the directory in terminal/command prompt run the command pyinstaller --onefile file-name include -w after --onefile if you do not want the command prompt/terminal/command-line to open up when run.
(Pyinstaller works for all operating systems)
I'm currently working on Pycharm with remote python Interpreter(miniconda3/bin/python).
So when I type echo $PATH in remote server, it prints
/home/woosung/bin:/home/woosung/.local/bin:/home/woosung/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
I created project in Pycharm and set remote python Interpreter as miniconda3 python, it works well when I just run some *.py files.
But when I typed some os.system() lines, weird things happened.
For instance, in test.py from Pycharm project
import os
os.system('echo $PATH')
os.system('python --version')
Output is
ssh://woosung#xxx.xxx.xxx.xxx:xx/home/woosung/miniconda3/bin/python -u /tmp/pycharm_project_203/test.py
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
Python 2.7.12
Process finished with exit code 0
I tried same command in remote server,
woosung#test-pc:~$ echo $PATH
/home/woosung/bin:/home/woosung/.local/bin:/home/woosung/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
woosung#test-pc:~$ python --version
Python 3.6.6 :: Anaconda, Inc.
PATH and the version of python are totally different! How can I fix this?
I've already tried add os.system('export PATH="$PATH:$HOME/miniconda3/bin"') to test.py. But it still gives same $PATH.(/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games)
EDIT
Thanks to the comment of #Dietrich Epp, I successfully add interpreter path to the shell $PATH.
(os.environ["PATH"] += ":/home/woosung/miniconda3/bin")
But I stuck the more basic problem. When I add the path and execute command the some *.py file including import library which is only in miniconda3, the shell gives ImportError.
For instance, in test.py
import matplotlib
os.environ["PATH"] += ":/home/woosung/miniconda3/bin"
os.system("python import_test.py")
and import_test.py
import matplotlib
And when I run test.py,
Traceback (most recent call last):
File "import_test.py", line 1, in <module>
import matplotlib
ImportError: No module named matplotlib
Looks like the shell doesn't understand how to utilize modified $PATH.
I find the solution.
It is not direct but quite simple.
I changed os.system("python import_test.py") to os.system(sys.executable + ' import_test.py').
This makes the shell uses the Pycharm remote interpreter(miniconda3), not original.
Hi My python installation is in different directory and i am using a docker image which is mac based and it is referring shebang line as /user/local/bin/python3 from other folder in shell script .
my python installation path
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/myuser/project', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
>>>
so is there a way without changing the shebang line i can redirect or link to my installation of python3 to get out of this error.
is it recommended to install python3 in given path. ?
please advice.
If you can't modify the shebang of the file, and you have access to the Dockerfile that creates your docker image, you can add a command directive to create a symbolic link: ln -s /usr/bin/python3 /usr/local/bin/.
If you don't have access to the Dockerfile. Then you can run the above command from within the running docker instance. That should solve your issue without having to modify the file.
https://docs.docker.com/engine/reference/builder/#cmd
You could set you shebang to "/usr/bin/env python" as usual, then set your path appropriately so that the correct version of python is on the path for your executable. In bash you can set the path on the command line using:
PATH=python/path:$PATH app
I will sometimes ignore the shebang and type python/path/python $(which app) in order to control which python interpreter is running.
When using Python on an interactive shell I'm able to import the cx_Oracle file with no problem. Ex:
me#server~/ $ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
>>>
As you can see, importing works without a hitch. However, when I try to run a Python script doing the same thing, I get an error:
me#server~/ $ sudo script.py
Traceback (most recent call last):
File "/usr/local/bin/script.py", line 19, in <module>
import cx_Oracle
ImportError: No module named "cx_Oracle'
Here is the important section from script.py:
# 16 other lines above here
# Imports
import sys
import cx_Oracle
import psycopg2
...
I'm befuddled here. Other pertinent information is the server I'm running is Ubuntu 14.04.1 LTS (upgraded from 12.04) 64bit. which python and sudo which python both point to the same location. Also, doing this as root via sudo su - gets the same results; import OK from interactive but error from script.
Nothing other than the OS upgrade happened between when this worked and when it stopped working.
Sorry, all. This was a silly on my part. Turns out the script in question was using Python3, and when the server upgraded, Python3 went from being 3.2 version to being 3.4 version.
Once the cx_Oracle module was set up in the 3.4 version, everything worked as expected.
Phil, your final note talking about the shebang was what lead me to discover this, so kudos to you! The reason I didn't mark your response as the answer was because technically it wasn't but led me on the right path.
Cheers!
sudo starts a new bash environment which is then pointing to a different python executable (different installed modules).
You can verify this with which python and sudo which python
EDIT: so if they point to the same executable, then you should look at sys.path to find differences. In both environemnts you can:
python -c "import sys; print('\n'.join(sys.path))"
sudo python -c "import sys; print('\n'.join(sys.path))"
Look for differences. If there are none:
A common error in import situations like this is that python will first look at the local dir. So if you happen to be running python and importing something what is found locally (i.e. cx_Oracle is a subdir of your current location), you will get an import error if you change directories.
Final note: I have assumed here that the shbang of the script.py points to the same executable as which python. That is, that python script.py and script.py return the same error.
Here's my setup: a Mac, running OS X Tiger. Windows XP running in a virtual machine (Parallels). Windows XP has my Mac home directory mapped as a network drive.
I have two files in a directory of my Mac home directory:
foo.py
pass
test.py
import foo
If I run test.py from within my virtual machine by typing 'python test.py', I get this:
Traceback (most recent call last):
File "test.py", line 1, in <module>
import foo
ImportError: No module named foo
If I try to import foo from the console (running python under Windows from the same directory), all is well:
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import foo
>>>
If I run test.py with Mac python, all is well.
If I copy test.py and foo.py to a different directory, I can run test.py under Windows without problems.
There is an init.py in the original directory, but it is empty. Furthermore, copying it with the other files doesn't break anything in the previous paragraph.
There are no python-related environment variables set.
Any ideas?
Add import sys; print sys.path to the start of test.py. See what it prints out in the failing case. If "." isn't on the list, that may be your problem.
As a random guess: are the permissions on foo.py accessable from the windows client? (eg try opening with notepad from the virtual machine).
If that's OK, try running:
python -v -v test.py
and looking at the output (alternatively, set PYTHONVERBOSE=2). This should list all the places it tries to import foo from. Comparing it with a similar trace on the working machine may give some further clues.