I am trying to call a perl script from my python program with the following line:
subprocess.call(r'/path/to/compute_lexrank.pl /path/to/11sent',shell=True)
when I run the same perl script from the shell (just typing /path/to/compute_lexrank.pl /path/to/11sent) it works fine as expected, but when I run it from the python program, the perl script is executed, but gives a mysterious error:
Math::MatrixReal::new(): number of rows must be integer > 0 at /Users/filippo/Downloads/clairlib-core-1.08/lib//Clair/Network.pm line 1628
now because I havent wrote the perl script myself I dont know how to fix this, but why the same script behaves differently when I run it from the shell or from subprocess.call?
I am using MacOsX, python 2.6 (but I have tried also with 2.5, same stuff) and perl 5.10.
Anyone can help?
In Perl, there's a good rule: if possible, use list forms of popen and system. Python seems to have those, too. I wonder what happens if you try this out:
helper = "/path/to/compute_lexrank.pl"
helper_input = "/path/to/11sent"
subprocess.call([helper, helper_input])
You may need to pass the working directory to the subprocess.call
subprocess.call(r'/path/to/compute_lexrank.pl /path/to/11sent',shell=True,cwd="/path/to")
If cwd is not None, the child’s current directory will be changed to cwd before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd.
Related
I need to run a MATLAB script from a Python script. I don't care about the output of it nor do I need to give it any arguments.
However, MATLAB R2016B's "engine" does not support Python 3.7 (Upgrading Matlab or down-grading python is not an option at this time)
So, I decided to make a shell script that runs it:
#!matlab -nodisplay -nodesktop -r 'try; myMatlabScript; catch; end; quit'
Now I need to run a bash script from Python. To do so, I did:
import subprocess
subprocess.call("./mybashscript.sh")
(And yes, the python script is at the same level as the shell script)
The python script does not complain directly. However, I do get the following:
'.' is not recognized as an internal or external command, operable program or batch file.
Which to me means that since Windows doesn't directly have bash, it doesn't know what to do with this shell script. I am not sure how to handle this. Some way to tell Python to use MSYS instead of Windows for the shell?
And thus the MATLAB script does not appear to run at all.
When I attempt under Linux (just for testing, I can't run it here for performance reasons), I get:
./mybashscript.sh: matlab: bad interpreter: No such file or directory
Is it possible this is because I didn't do the command addpath(genpath('.'))? If so, I'm not sure how I would do that in the shell script, and some help would be appreciated.
Or some other better solution would also be great.
1: Needed to re-name mybashscript.sh to mybashscript.bat
2: Needed to change the sub-process call to subprocess.call("mybashscript.bat") (as ./ was confusing the windows shell)
3: Needed to add the path properly. Here is what the batch script looked like:
matlab -nodisplay -nodesktop -r "addpath(genpath('C:/path/to/myscript')); myMatlabScript"
The double quotes are neccesary so the single quotes inside genpath do not cause it to end early.
And that was it!
EDIT: You can add -wait in the batch file to get the script to wait until it is complete before handing back to the Python script.
Excuse the awkward question wording.
I've made a script. I would like for others to download it from github, and run it by typing programName argument1 argument2, similar to any other popular app used through the terminal such as Jupyter or even opening Atom/Sublime/etc. (ex:jupyter notebook, atom .). However, unlike Jupyter or sublime, my script isn't launching another app, it's a small app meant to be used in the shell.
Currently, to use my script, one must type into the command line python programName.py arg1 etc from within the file's directory.
How do I allow others to dl it and use it from anywhere (not having to be within the directory), without having to type out the whole python programName.py part, and only having to type programName arg1?
This blog post explains step by step how to create a distribution that you can install and it would turn into an executable.
You can refer to this github repo for a sample application.
The full documentation of setuptools is available here.
In general, you should configure your setup.py in order to use the command in the entry-point option:
setup(
name = "your_app_name",
packages = ["package_name"],
entry_points = {
"console_scripts": ['cmd_name = package_name.package_name:main']
},
....
)
This solution would work on every OS where you have installed python.
Your script may need to have an interpreter, "shebang", besides being "reachable" by the $PATH
#!interpreter [optional-arg]
For example, you could have something like
#!/usr/bin/env python
or to force a specific version
#!/usr/local/bin/python2.7
Next, your script needs to be available within the $PATH, check this answer that covers that part: https://unix.stackexchange.com/q/29608/53084
You can simply add your script to PATH variable in order to launch it from anywhere.
In Linux distros, you can simply do it by using a bash command PATH=$PATH:/path/to/your/script.
Make sure you don't have the space around the "=" operator.
Now, the second thing is you don't want your script to be named as pythonProgram.py.You can simply remove the extension .py from PythonProgram.py by adding a single line to the starting of your script.
Open up your script and at the very begining type #!/usr/bin/python.This should be the first line of your code.This line is called shebang and is used to tell the bash which interpreter to be used for compiling the script.
If everything went right, you will be able to run your script as pythonProgram arg1.
In addition to mabe02: Python is a scripting language and usually not compiled, which means you will need an interpreter to run your program.
Programms made in C f.e. can be run on its own because they were compiled into machine code by a compiler.
An interpreter is similar to a compiler as it reads your script and interprets it at runntime. That is why you need to write python before your programm, to tell your computer to interpret your script on runntime, using python. This doesn't mean that there are no possibilities to compile python as can be seen in the other answer and in this link Can a python program be run on a computer without Python? What about C/C++? (py2exe and py2app).
Is it possible to use a "more complex" shell than just a single command shell? We have written a python shell that is a command loop, and it works fine in /etc/passwd like this:
user:x:1000:1000::/home/user:/usr/bin/ourshell.py
Of course the Python file has the shebang line for /usr/bin/python in it. However, we'd like to compile the Python shell into a .pyc file to save a bit of time on execution in login. So, after compiling, I've been trying to "quote" the shell line in /etc/passwd as "python ourshell.pyc", and I even tried making the shell a bash script which simply executes that same command (with the initial arguments).
Of course none of this has worked. When we SSH in, there is always some kind of error. Is there any special trick to what I am trying to do?
CPython's .pyc files are not text, and do not allow use of a shebang line. The traditional method is to have your called script be tiny; it would simply import a module with the rest of the program, which can then be precompiled. For instance, here is the main script of xonsh:
#!/usr/bin/env python3 -u
from xonsh.main import main
main()
This script takes negligible time to compile. It is also possible to run installed modules using -m, but that takes module names, not filenames, so is not suitable for a shebang script.
I suggest to code a small C wrapper program running your python shell.
(notice that execve(2) forbids nested shebang interpreters; I don't know if that applies for your case)
Look into your log files, probably /var/log/messages and /var/log/auth.log
You may also need to explicitly add (the compiled C executable for the wrapper) to /etc/shells; see shells(5)
Look also into scsh.
Your sshd daemon is probably using Linux Plugin Authentification Modules. So read more about PAM.
Create a file /usr/bin/shell_wrapper that contains this one line:
#!/usr/bin/python /usr/bin/ourshell.pyc
The compiled bytecode ourshell.pyc has to live in /usr/bin, or else change the path accordingly. The python path should go to the same version that compiled the bytecode.
Then make sure to have your /etc/passwd use /usr/bin/shell_wrapper for the shell executable:
user:x:1000:1000::/home/user:/usr/bin/shell_wrapper
The blow command works well in the shell/terminal, but something goes wrong when it is called in my python script using subprocess.call() method.
-- command in shell/terminal
$ th neural_style.lua -gpu 0 -style_image input/style.jpg -content_image input/img.jpg
-- subprocess.call() in python script
# this works
subprocess.call(["th", "neural_style.lua", "-gpu", "0"])
# this goes wrong - Error during read_image: Read Error
-- subprocess.call in the python script
subprocess.call(["th", "neural_style.lua", "-gpu", "0", "-style_image" "input/style.jpg" "-content_image" "input/img.jpg"])
How should I use subprocess.call ?
As the error message says, it failed to read an image. The error is (presumably) coming from the th program you're calling. I'd guess that there's additional information in the error message you haven't shared, but the most likely explanation is that you're running your Python script from a different directory than where you're running th directly. For example are you running your Python script from an IDE? It's likely running commands relative to the workspace or project directory.
The first thing to try is swapping the image arguments for absolute paths (e.g. /home/username/input.style.jpg or wherever they're located). This will work around the scripts running from different directories.
Once you've verified that's the issue how you fix it is up to you. You could simply run your Python script from the correct directory, you could specify the paths relative to where your script runs, or you could simply always use absolute paths your Python script. Which you chose really depends on your use case.
I use TextMate to debug python script, as I like the feature of using 'Command-R' for running python from TextMate, and I learned that emacs provide similar feature.
I need to know if the python is run from command line or from TextMate/emacs. How can I do that?
ADDED
I use TextMate for python coding/debugging, and it's pretty useful. But, sometimes I need to run the test using command line. I normally turn on debugging/logging mode with TextMate, and off with command line mode. This is the reason I asked the question. Also, I plan to use emacs for python debugging, so I wanted to ask the case for emacs.
I got an answer in the case with emacs, and I happen to solve this issue with TextMate.
Set variables in Preferences -> Advanced -> Shell Variables, and I found that TM_ORGANIZATION_NAME is already there to be used. So, I'll just use this variable.
Use this variable, if os.environ['TM_ORGANIZATION_NAME']: return True
I guess the shell variable from TextMate disappear when I'm done using it.
For Emacs: If python is run as an inferior process, then the environment variable INSIDE_EMACS will be set.
From docs:
Emacs sets the environment variable
INSIDE_EMACS in the subshell to a
comma-separated list including the
Emacs version. Programs can check this
variable to determine whether they are
running inside an Emacs subshell.
sys.argv will tell you how Python was invoked. I don't know about TextMate, but when I tell Emacs to eval buffer, its value is ['-c']. That means it's executing a specified command, according to the man page. If Python's run directly from the command line with no parameters, sys.argv will be []. If you run a python script, it will have the script name and whatever arguments you pass it. You might want to set up your python-mode in Emacs and whatever the equivalent in TextMate is to put something special like -t in the command line.
That's pretty hackish though. Maybe there's a better way.
From the docs for sys.path:
As initialized upon program startup,
the first item of this list, path[0],
is the directory containing the script
that was used to invoke the Python
interpreter. If the script directory
is not available (e.g. if the
interpreter is invoked interactively
or if the script is read from standard
input), path[0] is the empty string,
which directs Python to search modules
in the current directory first. Notice
that the script directory is inserted
before the entries inserted as a
result of PYTHONPATH.
So
if sys.path[0]:
# python was run interactively
else:
# python is running a script.
Or, for example, from the IPython prompt (inside Emacs):
In [65]: sys.path
Out[65]:
['', <-------------------- first entry is empty string
'/usr/bin',
'/usr/local/lib/python2.6/dist-packages/scikits.statsmodels-0.2.0-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/pyinterval-1.0b21-py2.6-linux-i686.egg',
... ]
Use Command-R to run the script directly
Use Shift-Command-R to run the script from terminal.