I have a MATLAB file that currently saves its variables into a .mat workspace. The python script uses SciPy.io to read these variables from the workspace. The python script performs some operations & resaves variables into a MATLAB workspace(agin using Scipy.io) which matlab should then reopen. I'm using MATLABR2013a and I dont think there's an easy way to run the python script from within the .m file itself.
There may be an easier way then the method I'm going about doing it but my current plan is to create a bash script that runs the matlab file and only proceeds to the latter section if a random variable (stored in another file) is of a certain value. The script then calls the python script, sets the random variable to a different (can view as a sort of boolean). The matlab script will then execute the second section but not the first section. I need to have about 5 or 6 such exclusive sections however and it's easier to have them all in the same .m file than it is to separate them
This seems tedious however when all I really want is a way to have the system pause the matlab script, run the python script and come back to that spot in the matlab script.
Appreciate all creative suggestions to make this workflow as efficient as possible and easy to modify
MATLAB code detailed below
I saved the workspace using MATLAB's save function
Used MATLAB's system() function to execute the python script.
Within python, used scipy.iosavemat to save variables I wanted to access in matlab
Used MATLAB's load function to load the variables from python back into matlab's workspace
writeto=['insert path to save to here']
save(writeto)
first_Pypath=['insert path of python script here']
py_call=horzcat('python ',first_Pypath);
system(py_call);
Related
I have a collection of expert advisor (EA) scripts written in the MQL5 programming language for the stock/forex trading platform, MetaTrader5. The extension of these files is mq5. I am looking for a way to programatically run these MQL5 files from my Python script on a regular basis. The EAs do some price transformations, eventually saving a set of csv files that will later be read by my Python script to apply Machine Learning models on them.
My first natural choice was the Python API for MetaTrader5. However, according to its documentation, it "is designed for convenient and fast obtaining of exchange data via interprocessor communication directly from the MetaTrader 5 terminal" and as such, it doesn't provide the functionality I need to be able to run MQL scripts using Python.
I have found some posts here on SO (such as #1, #2) about executing non-python files using Python but those posts seemed to always come with the precondition that they already had Python code written in them, only the extension differed - this is different from my goal.
I then came across Python's subprocess module and started experimenting with that.
print(os.path.isfile(os.path.join("path/to/dir","RSIcalc.mq5")))
with open(os.path.join("path/to/dir","RSIcalc.mq5")) as f:
subprocess.run([r"C:\Program Files\MetaTrader 5\terminal64.exe", f], capture_output=True)
The print statement returns True, so the mq5 file exists in the specified location. Then the code opens the MetaTrader5 terminal but nothing else happens, the EA doesn't get executed, process finishes immediately after that.
Am I even on the right track for what I'm trying to achieve here? If yes, what might be the solution for me to run these MQL5 scripts programatically from Python?
Edit:
I use Windows 10 64-bit.
subprocess is indeed the right module for what you want to achieve. But let's look at what you're doing here:
with open(os.path.join("path/to/dir","RSIcalc.mq5")) as f
You're creating a file descriptor handle called f, which is used to write or read contents from a file. If you do print(f) you'll see that it's a python object, that converted to string looks like <_io.TextIOWrapper name='RSIcalc.mq5' mode='r' encoding='UTF-8'>. It is extremely unlikely that such a string is what you want to pass as a command-line parameter to your terminal executable, which is what happens when you include it in your call to subprocess.run().
What you likely want to do is this:
full_path = os.path.abspath(os.path.join("path/to/dir","RSIcalc.mq5"))
result = subprocess.run([r"C:\Program Files\MetaTrader 5\terminal64.exe", full_path], capture_output=True)
Now, this assumes your terminal64 can execute arbitrary scripts passed as parameters. This may or may not be true - you might need extra parameters like "-f" before passing the file path, or you might have to feed script contents through the stdin pipe (unlikely, on Windows, but who knows). That's for you to figure out, but my code above should probably be your starting point.
I don’t think you need to be passing a file object to your sub process statement. In my experience. A program will run a file when the path to the file is provided as a command line argument. Try this:
subprocess.run([r"C:\\Program Files\\MetaTrader 5\\terminal64.exe", os.path.join(“path/to/dir”, “RSIcalc.mq5”], capture_output=True)
This is the same as typing C:\Program Files\MetaTrader 5\terminal64.exe path\to\dir\RSIcalc.mq5 in your terminal.
I am using VBA-macros in order to automate serveral data processing steps in excel, such as data reduction and visualization. But since excel has no appropriate fit for my purposes I use a python script using scipy's least squares cubic b-spline function. The in and output is done via .txt files. Since I adjusted the script from a manual script I got from a friend.
VBA calls Python
Call common.callLSQCBSpline(targetrng, ThisWorkbook) #calls python which works
Call common.callLoadFitTxt(targetrng, ThisWorkbook) # loads pythons output
Now the funny business:
This works in debug mode but does not when running "full speed". The solution to the problem is simply wait for the directory were the .txt is written to refresh and to load the current and not the previous output file. My solution currently looks like this:
Call common.callLSQCBSpline(targetrng, ThisWorkbook)
Application.Wait (Now + 0.00005)
Call common.callLoadFitTxt(targetrng, ThisWorkbook)
This is slow and anoying but works. Is there a way to speed this up? The python script works fine and writes the output.txt file properly. VBA just needs a second or two before it can load it. The txts are very small under 1 kB.
Thanks in advance!
There are some ways to execute python code from within vim. E.g.:
typing ":! python %" on the command line! or
using the marvellous python-mode plugin!
These approaches invoke a new python instance, transfer the code, execute the code, feed back the output and close the instance. Thus, the python workspace will always be empty at start of execution. No possibility to access previously calculated results. When I want to execute just an incremental subset (some lines) of my code, I would have to take care of all the prerequisites as well (cf.: working with MATLAB provides an interactive mode where the workspace is always persistent).
Is there a way to keep a python instance alive (keep the objects within the workspace) and feed incremental code portions from vim into the open python instance and execute them with respect to the retained workspace?
One could think of storing / retrieving the python workspace on each exit / entry of a new python instance, but this might not be the best way to keep the workspace alive.
Background:
My program currently assembles arrays in Python. These arrays are connected to a front-end UI and as such have interactive elements (i.e. user specified values in array elements). These arrays are then saved to .txt files (depending on their later use). The user must then leave the Python program and run a separate Fortran script which simulates a system based on the Python output files. While this only takes a couple of minutes at most, I would ideally like to automate the process without having to leave my Python UI.
Assemble Arrays (Python) -> Edit Arrays (Python) -> Export to File (Python)
-> Import File (Fortran) -> Run Simulation (Fortran) -> Export Results to File (Fortran)
-> Import File to UI, Display Graph (Python)
Question:
Is this possible? What are my options for automating this process? Can I completely remove the repeated export/import of files altogether?
Edit:
I should also mention that the fortran script uses Lapack, I don't know if that makes a difference.
You do not have to pass arrays to Fortran code using text files. If you create an entry point to the Fortran code as a subroutine, you can pass all the numpy arrays using f2py. You should be aware of that if you added the f2py tag yourself. Just use any of the numerous tutorials, for example https://github.com/thehackerwithin/PyTrieste/wiki/F2Py or http://www.engr.ucsb.edu/~shell/che210d/f2py.pdf .
The way back is the same, the Fortran code just fills any of the intent(out) or intent(inout) arrays and variables with the results.
I love the Python+Fortran stack. :)
When needing close communication between your Python front-end and Fortran engine, a good option is to use the subprocess module in Python. Instead of saving the arrays to a text file, you'll keep them as arrays. Then you'll execute the Fortran engine as a subprocess within the Python script. You'll pipe the Python arrays into the Fortran engine and then pipe the results out to display.
This solution will require changing the file I/O in both the Python and Fortran codes to writing and reading to/from a pipe (on the Python side) and from/to standard input and output (on the Fortran side), but in practice this isn't too much work.
Good luck!
So I have the following problem: I have a folder at a location known from a config.py file, in my case externals/bct. Now this needs to be added to matlab's path. Now I've searched for some examples to edit matlab path but from what I can see let's say here: matlab path the changes are done from matlab. My question would then be: is there any way I could change the matlab path from python?
Best regards,
Bogdan
Thanks for the inputs. The project already has an adapter that allows runnig matlab code from python using from scipy.io import loadmat, savemat. The problem was that we are using BCT and that need to be added to matlab path dinamically at startup. The solution that worked for me was to use the method already defined to execute matlab code and just send at startup:
addpath(PATH_TO_BCT); savepath;
Your source says:
path displays the MATLAB search path, which is stored in pathdef.m
I believe your best bet is to find this pathdef.m file in the Matlab install folder, then open and change it from python.
On my Windows Machine with Matlab 2008a, it's in C:\MATLAB\R2008a\toolbox\local\pathdef.m. There are two emptied-out versions of the same file at \local\ja and \local\template, but that first one seems to be the one that counts.
There's a big warning in it saying not to edit, but it's plain text Matlab language, really easy to reverse-engineer, you should be fine. Just don't forget the semicolon at the end of each path string (unless they changed synthax in the new version you might have... just take a look at your file.)