I am executing python script with multiple command line parameter, but using shell script.
command i execute for shell script execution is:
./scripts/run_qa.sh data/questions/questions.txt data/lexicons/paralex data/weights/paralex.txt data/database > output.txt
run_qa.sh files looks like below (please explain how it works):
#!/bin/bash
set -u
set -e
if [ $# != 4 ]; then
echo "Usage: run.sh questions lexicon weights db"
exit 1
fi
questions=$1
lexicon=$2
weights=$3
db=$4
PYTHONPATH=$PWD/python python -m lex.gearman_worker $lexicon $weights $db < $questions
I tried to execute python command as below in Command line :
python -m python/lex/gearman_worker.py data/lexicons/paralex data/weights/paralex.txt data/database > output.txt
which gives error :
/usr/bin/python: Import by filename is not supported.
Update1 :
gearman_worker.py file import other files like ths:
import lex.parse
import lex.semantics
from collections import namedtuple
from collections import defaultdict
import line gives error like this:
ImportError: No module named lex.lexicon
Update2 (executed on linux terminal):
export PYTHONPATH=$/mnt/paralex-evaluation-gearman/python
PYTHONPATH = ./python python -m python/lex/gearman_worker data/lexicons/paralex data/weights/paralex.txt data/database > output.txt
gives:
PYTHONPATH: command not found
Then
python -m python/lex/gearman_worker data/lexicons/paralex data/weights/paralex.txt data/database > output.txt
gives:
File "/mnt/paralex-evaluation-gearman/python/lex/gearman_worker.py", line 3, in <module>
import lex.lexicon
ImportError: No module named lex.lexicon
You just need to execute the following command:
PYTHONPATH=./python python -m lex.gearman_worker ARGUMENT_2 ARGUMENT_3 ARGUMENT_4 < ARGUMENT_1
If that doesn't work then you may have to export the PYTHONPATH setting:
export PYTHONPATH=${PWD}/python
python -m lex.gearman_worker ARGUMENT_2 ARGUMENT_3 ARGUMENT_4 < ARGUMENT_1
The original arguments that you would pass to the script are listed as ARGUMENT_N.
The script just:
sets some sensible defaults (see the documentation for set)
tests the the right number of arguments have been supplied
invokes the command above
Your attempt to invoke it:
misses the PYTHONPATH setting which is present in the script
passes the gearman_worker module as a file rather than a python module import
Related
I'm trying to run a python script at start for my Raspberry Pi 3B.
My shell script "atboot.sh" looks like this:
#!/bin/bash
cd /home/pi/pythoncodefolder
date >> logs
sudo python3 test.py >> logs
When I try to run it on cmd with sh atboot.sh command, I get an import error which is:
Traceback (most recent call last):
File "test.py", line 5, in <module>
import cv2
ModuleNotFoundError: No module named 'cv2'
But when I run the program on cmd without shell script, using python3 test.py, I get no errors.
Thanks.
The use of sudo causing it. When you run python3 program.py you invoke it in your $USER environment setup.
You can remove Defaults !env_reset from sudoers. Or to add Defaults env_keep += "PYTHONPATH".
But I would assert that you can do it without sudo in the first place.
#!/bin/bash
cd /home/pi/pythoncodefolder
date >> logs
python3 test.py >> logs
Also, that's probably a (not exact) duplicate.
I have a python project where I execute the app as a module using the -m flag. So something like:
python -m apps.validate -i input.mp4
Now, I want to profile it using the command line. So the inline examples suggest invoking cProfile itself a module. However, I cannot do something like:
python -m cProfile apps.validate -i input.mp4
However, this results in the error "No such file or directory". I cannot just go to the apps directory and launch validate.py due to relative imports.
Is there a way to profile a module on the command line?
Instead of running cProfile in shell, maybe you can use cProfile in your python script by adding some code in apps.validate or creating a new script and import apps.validate like this. Maybe some typo below :)
import cProfile
import sys
def run_validate(args):
# run your apps.validate code with shell arguments args here
pass
if __name__ == '__main__':
pr = cProfile.Profile()
pr.enable()
run_validate(*sys.argv)
pr.disable()
pr.print_stats()
then just run the original script: python -m apps.validate -i input.mp4
I am trying to do some scripting with IPython, but I am finding that it behaves very differently in a script to when I run an interactive shell.
For example, I can run the following interactively:
In [1]: %profile
default
In [2]: ls /
bin/ cdrom/ etc/ initrd.img# lib/ lib64/ media/ opt/ root/ sbin/ sys/ usr/ vmlinuz#
boot/ dev/ home/ initrd.img.old# lib32/ lost+found/ mnt/ proc/ run/ srv/ tmp/ var/ vmlinuz.old#
In [3]: mkdir tmpdir
In [4]: cd tmpdir
/home/alex/tmp/tmpdir
No problem.
However, none of these commands works when I run them in a script:
#!/usr/bin/ipython3
%profile
ls /
mkdir tmpdir
cd tmdir
I get an error:
$ ./tmp.py
File "/home/alex/tmp/tmp.ipython", line 3
%profile
^
SyntaxError: invalid syntax
I have tried running this by:
calling the file directly as above,
calling it explicitly with ipython: `ipython3 tmp.py'
passing the -i or --profile=sh arguments to ipython when calling it with ipython
changing the file extension to .ipython and .ipy
My question:
Why is it behaving differently in a script to the shell? How can I get IPython to run these commands in a script?
They are working due to IPython magic but they are shell commands and do not work in Python. To get them consider the subprocess library. Where you would have spaces in a shell command instead have comma-separated values in the list.
import subprocess
subprocess.check_call(['ls'])
subprocess.check_call(['ls', '-a'])
I have several command execution in python on Windows using subprocess.call(), but for each one I need to execute batch file with environmet setup before calling proper command, it looks like this
subprocess.call(precommand + command)
Is there way to "create" shell in python that will have batch file executed only once and in that shell command will be executed several times?
Write commands to a bat-file (tempfile.NamedTemporaryFile())
Run the bat-file (subprocess.check_call(bat_file.name))
(not tested):
#!/usr/bin/env python
from __future__ import print_function
import os
import subprocess
import tempfile
with tempfile.NamedTemporaryFile('w', suffix='.bat', delete=False) as bat_file:
print(precommand, file=bat_file)
print(command, file=bat_file)
rc = subprocess.call(bat_file.name)
os.remove(bat_file.name)
if rc != 0:
raise subprocess.CalledProcessError(rc, bat_file.name)
Do you need to get output from every command separately? If no - you can convey these commands using &&, || or ;
cd dir && cp test1 test2 && cd -
I am trying to pipe output from a command written in the terminal to a Python script.
For example:
ls | ./foo.py
I wrote a Python script to do the same:
#foo.py
import fileinput
with fileinput.input() as f_input :
for line in f_input :
print(line,end='')
But this does not seem to work,
when I run the following command:
$ ls | sudo ./foo.py
I get an error that says:
$ ./foo.py: command not found
I have checked the working directory and I can see the foo.py when I use the ls command, so what am I doing wrong here?
It seems like you forgot the Shebang:
#!/usr/bin/env python3
import fileinput
with fileinput.input() as f_input :
for line in f_input :
print(line,end='')
Also remember make it as executable via command:
chmod +x foo.py
Then run your command again.
You have to pipe it to the Python executable, not to the name of a file. As the error says, that filename doesn't represent a command it knows.
ls | py ./foo.py
Use py or python or however you run the Python interpreter on your particular system.