I am working on a machine learning program and attempting to perform experiments on the variables of my neural network. Due to Matlab's prowess with matrices, the learning is being performed in Matlab but for ease of use, the use of the learned outputs is in Python.
I need to make repeated calls to a Matlab file (currently not a function) and change a certain variable each time without me doing it manually. The Matlab script then outputs to a file read by the Python files. The Python file is responsible for the final output.
The big ideas is being able to set up a line of experiments and walk away from the computer. I could also see this perhaps being more easily done in another script if it is easy to make calls from one program to the other. I haven't done any work in Matlab until this summer and have only ever used the GUI shell.
Thank you for any help you can offer and let me know if more information is necessary.
There are several possible approaches for running MATLAB without a GUI and even without a MATLAB command line input:
start MATLAB in command line mode and call your script manually: matlab -nodesktop -nosplash
use a MATLAB to set the parameter and call your MATLAB script. this can be done by
using in terminal: matlab -nodesktop -nosplash -nodisplay -r myScript
use a Bash script to set the parameter and the call: matlab -nodesktop -nosplash -nodisplay -r "myscript"
(on Linux machines, for Windows you need to use the Power Shell)
You could then call Python from within your MATLAB scripts using the system() command to fetch your results.
Here is an example using a Bash script:
#!/bin/bash
# set parameter:
myParameter = 1
# call MATLAB with no interface at all and send it to background
# run MATLAB function with parameter, exit afterwards
# print command window output to a textfile
matlab -nodesktop -nosplash -nodisplay -r "myMatlabFunction(${myParameter});exit" > matlab_output.txt &
Related
I'm facing a slight problem with a hidden MPIRUN call inside a program called ORCA during an external call from python using subprocess.run(). I would appreciate some assistance troubleshooting.
I am using Python 3.8.3 within Jupyter Notebook 6.0.3 on Windows 10. My workflow performs an external call from python to the ORCA orca.exe program using:
fileHandler = open(outputFileStr, "w")
result = subprocess.run([orcaLocationStr, inputFileStr], cwd=cwdStr, shell=True, stdout=fileHandler)
NOTE: I've tried tweaking the subprocess.run() parameters, what you see is just one iteration of many. I think the fundamental issue I face is the one I describe below.
The external call essential recreates the Windows command terminal call:
c:\ORCA\orca my_inputfile.inp
(Note: I would have added > output.out if I was executing ORCA from the terminal. However, via python, I've redirected the output to a file).
The variable orcaLocationStr contains c:\ORCA\orca. The absolute path to orca.exe is required (even if it's in your PATH env variable) to use its multiprocessing capabilities.
The variable inputFileStr points to a file that contains the parameters required by ORCA. One of those parameters is the number of processes e.g., nproc 16.
All was going well until ORCA crashed. orca.exe turns out to be a wrapper to several other .exe files within the ORCA homedirectory. The nproc parameters I mentioned earlier is used so the orca.exe wrapper can execute n child processes using mpirun. That is when executing orca.exe within Python crashes and returns a non-zero value. I know this having performed some basic checks:
N processes specified in the ORCA parameter file. Calls MPIRUN.
The command will execute from the Windows command line using N processes.
The command fails when executing from Python using subprocess.run().
1 process specified in the ORCA parameter file. Does not call MPIRUN.
The command will execute from Windows command line using 1 process.
The command will execute from Python using subprocess.run().
I think it's important to be clear, it is not Python executing MPIRUN. It is an external binary called by Python that executes MPIRUN.
How can I make a Python process ready to accept multiple child processes via an external-embedded MPIRUN call?
EDIT #1 I am expecting my software to (a) perform all the preprocessing steps such as performing IO tasks etc., then (b) I want to use subprocess.run() 1600 times - as I have 1600 input files to execute. I am wondering if all else fails, whether to move just this part across to something like Bash and execute independent of Python.
I need to run a MATLAB script from a Python script. I don't care about the output of it nor do I need to give it any arguments.
However, MATLAB R2016B's "engine" does not support Python 3.7 (Upgrading Matlab or down-grading python is not an option at this time)
So, I decided to make a shell script that runs it:
#!matlab -nodisplay -nodesktop -r 'try; myMatlabScript; catch; end; quit'
Now I need to run a bash script from Python. To do so, I did:
import subprocess
subprocess.call("./mybashscript.sh")
(And yes, the python script is at the same level as the shell script)
The python script does not complain directly. However, I do get the following:
'.' is not recognized as an internal or external command, operable program or batch file.
Which to me means that since Windows doesn't directly have bash, it doesn't know what to do with this shell script. I am not sure how to handle this. Some way to tell Python to use MSYS instead of Windows for the shell?
And thus the MATLAB script does not appear to run at all.
When I attempt under Linux (just for testing, I can't run it here for performance reasons), I get:
./mybashscript.sh: matlab: bad interpreter: No such file or directory
Is it possible this is because I didn't do the command addpath(genpath('.'))? If so, I'm not sure how I would do that in the shell script, and some help would be appreciated.
Or some other better solution would also be great.
1: Needed to re-name mybashscript.sh to mybashscript.bat
2: Needed to change the sub-process call to subprocess.call("mybashscript.bat") (as ./ was confusing the windows shell)
3: Needed to add the path properly. Here is what the batch script looked like:
matlab -nodisplay -nodesktop -r "addpath(genpath('C:/path/to/myscript')); myMatlabScript"
The double quotes are neccesary so the single quotes inside genpath do not cause it to end early.
And that was it!
EDIT: You can add -wait in the batch file to get the script to wait until it is complete before handing back to the Python script.
I have one Maya scene and a Python script where import obj files into it. I need to create a batch render file which calls the maya file and applies the script without opneing maya.
I have this code in a .sh file:
#!/bin/bash
"/Applications/Autodesk/maya2016/Maya.app/Contents/bin/Render" -r file -s 1 -e 4 -cam camera1 -rd "/Users/MyComp/Documents/maya/projects/default/images" "/Users/MyComp/Documents/maya/projects/default/Scenes/test1.mb"
But I have this code into the script which can be an issue or maybe not:
def renderFile(i):
cmds.setAttr("defaultRenderGlobals.imageFilePrefix", i, type="string")
cmds.render(batch=True)
If I execute this .sh file it renders without the python script. How can I add the python script?
I need that file for a renderfarm purposes
I know it's an old thread but thought I'd jump in just incase someone finds this thread in a search.
The comments seem a little confused. This comes from the fact that there are two different Python interpreters being talked about. The first is the system level one, which the original question seems to be talking about. In that case, you can use any of the various shell command launchers (like, subprocess/Popen) that suit your need. Here you are looking to run the render command like you would any other command in in the shell.
In the responses, people there are referring to the other interpreter, the custom Maya Python interpreter (mayapy.exe). In that case you are working with actual Maya libraries and it's the same as working with Python in it's shell, with the added Maya libraries/environment.
The two have different uses, the first is to control things like they were in the shell and the second is controlling things inside of a Maya context. Hope that clarifies things.
I have 2 scripts:
python script
matlab script
I need to run this two scripts in parallel (no output for both of them). I was thinknig to call to the python scirpt, from the matlab script.
I know it is possible to run python script from matlab like:
systemCommand='my_script.py'
system(systemCommand)
however in this way, the matlab script will wait to the return of the python script, and the rest of my matlab script will not be executed.
any ideas?
As mentioned near the end of MATLAB's system documentation in the "Tips" section, to run a system command in the background (on *nix), you can append an ampersand(&) to the end of your command to tell it to run in the background.
system('my_script.py &')
If you're on Windows, you'll want to use the following to prevent a command window from opening.
system('start /b my_script.py');
Is there a way to run a Windows command line from within SPSS?
I'm working with a program that is not supported by SPSS BEGIN PROGRAM procedures and I would like to run a command line from my SPSS syntax (I'm not interested in the output and I don't need to import/export any SPSS data I just want to run a cmd line from within SPSS).
To take a simpler example using freeware, let's say I want to run my R script at the end of my SPSS syntax but this would only work with a version not supported by SPSS (e.g. at the moment SPSS 21 supports R 2.14, let's say I absolutely need to run it on R 2.15 installed on my Windows PC), is there a way to run
R CMD BATCH C:\Files\MyRcode.R
within the SPSS syntax?
This is equivalent to running a program (in this case R) in batch mode. The program I need to use is not R, it's just an example.
As far as I'm aware SPSS can't do this but maybe using python (through SPSS BEGIN PROGRAM PYTHON) one can run a command line by using Python?
Thanks
P.S. For Python users who may not have used SPSS before one should in theory be able to run any Python code from within SPSS by using
BEGIN PROGRAM PYTHON.
Run some Python code
END PROGRAM.
Yes, you can do it with Python code:
BEGIN PROGRAM PYTHON.
import subprocess
subprocess.call(['C:\\R\\R.exe', 'CMD', 'BATCH', 'C:\\Files\\MyRcode.R'])
END PROGRAM.
Or, directly, with this:
HOST COMMAND=['R CMD BATCH C:\Files\MyRcode.R'].
Alternatives:
Put it in a batch file then execute it, for example using os.system().
Execute the Python or Basic script through the SCRIPT command.