Can't pass data from octave to python through command line arguments - python

I am calculating HoG feature descriptors in Octave and then I am trying to cluster those data in Python using scikit-learn.
For testing my code in Python I am trying to pass a 4000x2 data to Python.
I am calling the Python script from Octave using
system('python filename.py data')
and then trying to get the data using
sys.argv
but I am getting the second argument as a string 'data' and not the 4000x2 data that I am passing from Octave
What should I do so that I can get the original data in Python and not just the string 'data'

There's a python command built into octave.
Alternatively, I would save as a .mat file, and open this in your python script using scipy.io.loadmat.
There's also eval_py and python_cmd from the symbolic package, but I'm not sure if this is appropriate for your particular use-case. The most general, matlab-compatible, and recommended way to do this would be the .mat one.

I am not an expert regarding Octave but the answer is most likely something like that:
command=sprintf("python filename.py %s",data)
system(command)
Be careful that the amount of command line arguments is limited in most operating systems.

Related

Python command to execute non-Python (MQL5) files?

I have a collection of expert advisor (EA) scripts written in the MQL5 programming language for the stock/forex trading platform, MetaTrader5. The extension of these files is mq5. I am looking for a way to programatically run these MQL5 files from my Python script on a regular basis. The EAs do some price transformations, eventually saving a set of csv files that will later be read by my Python script to apply Machine Learning models on them.
My first natural choice was the Python API for MetaTrader5. However, according to its documentation, it "is designed for convenient and fast obtaining of exchange data via interprocessor communication directly from the MetaTrader 5 terminal" and as such, it doesn't provide the functionality I need to be able to run MQL scripts using Python.
I have found some posts here on SO (such as #1, #2) about executing non-python files using Python but those posts seemed to always come with the precondition that they already had Python code written in them, only the extension differed - this is different from my goal.
I then came across Python's subprocess module and started experimenting with that.
print(os.path.isfile(os.path.join("path/to/dir","RSIcalc.mq5")))
with open(os.path.join("path/to/dir","RSIcalc.mq5")) as f:
subprocess.run([r"C:\Program Files\MetaTrader 5\terminal64.exe", f], capture_output=True)
The print statement returns True, so the mq5 file exists in the specified location. Then the code opens the MetaTrader5 terminal but nothing else happens, the EA doesn't get executed, process finishes immediately after that.
Am I even on the right track for what I'm trying to achieve here? If yes, what might be the solution for me to run these MQL5 scripts programatically from Python?
Edit:
I use Windows 10 64-bit.
subprocess is indeed the right module for what you want to achieve. But let's look at what you're doing here:
with open(os.path.join("path/to/dir","RSIcalc.mq5")) as f
You're creating a file descriptor handle called f, which is used to write or read contents from a file. If you do print(f) you'll see that it's a python object, that converted to string looks like <_io.TextIOWrapper name='RSIcalc.mq5' mode='r' encoding='UTF-8'>. It is extremely unlikely that such a string is what you want to pass as a command-line parameter to your terminal executable, which is what happens when you include it in your call to subprocess.run().
What you likely want to do is this:
full_path = os.path.abspath(os.path.join("path/to/dir","RSIcalc.mq5"))
result = subprocess.run([r"C:\Program Files\MetaTrader 5\terminal64.exe", full_path], capture_output=True)
Now, this assumes your terminal64 can execute arbitrary scripts passed as parameters. This may or may not be true - you might need extra parameters like "-f" before passing the file path, or you might have to feed script contents through the stdin pipe (unlikely, on Windows, but who knows). That's for you to figure out, but my code above should probably be your starting point.
I don’t think you need to be passing a file object to your sub process statement. In my experience. A program will run a file when the path to the file is provided as a command line argument. Try this:
subprocess.run([r"C:\\Program Files\\MetaTrader 5\\terminal64.exe", os.path.join(“path/to/dir”, “RSIcalc.mq5”], capture_output=True)
This is the same as typing C:\Program Files\MetaTrader 5\terminal64.exe path\to\dir\RSIcalc.mq5 in your terminal.

How do I specify mesh parameters when using OpenSCAD from the Command Line?

I am trying to export a ton of simple shapes generated in OpenSCAD and exported as STL files with varying mesh coarse-ness. I'm using a combination of solidpython and subprocesses to call openscad from the command line but I'm not sure how to use the arguments to specify any of the mesh parameters. I think I have to use the -d argument but I am unsure.
Anything you can offer would be a huge help for me to avoid having to manually save each STL through the GUI.
I have a script and system for doing something similar to this at https://github.com/tinkerology/buildmodels.
It uses a bash script and not python. It could be modified easily to handle your needs.

Convert data values from Python to C

For this project I am working with libsvm.
I have a python file that is able to output a list of feature vectors and I have a C executable that takes 2 csv files, a list of feature vectors and the svm model, as arguments and outputs a prediction in the form of a csv file.
Now, I would like to change the C file such that it takes in the list output from the python file as its input arguments to make a prediction. This is because I am having to run the python code and C in real time. Therefore, latency will be an issue if I am having to write to a csv file with python and read the file in C.
I have tried searching things like cython, subprocess module and argparse. However, it seems like these are used to execute Python functions in C and C in Python. Can someone please help understand how to transfer data from python to C? Thank you.
It will depends the amount of latency you are accepting as input.
I am not familiar with libsvm so I admit you are able to read your feature vector in my solutions:
First solution (easier but slower) would be to make a little Python to C library as follow:
#/usr/bin/python
def print_vector(vec):
first = true;
print("[")
for i in vec:
if first:
first = false
else:
print(',')
print(i)
print("]")
and to parse it from the standard input in C with <string.h> library and using atoi.
You will then execute your command as follow:
./my_python_program | ./my_c_program
Second solution (way faster but way harder) which is implement pipe connection between you python program and C. Or even TCP socket connections.
You can, for instance, have a look to the Linux Documentation if you are under Linux.

How to pass multiple images as input to python script

I use nodejs to call a python script that runs object detection for some jpg images reading from the hard disk. These images are written to the disk by nodejs prior to calling the script.
To make it dynamic and faster, now I want to send multiple images as multidimensional array from nodejs to the python script. This saves me writing and reading images from disk. Is this the best way to do this? If so how do I pass images as multidimensional array to python script? Or Is there any better solution?
Your question leaves out a lot of specifics, but if you literally mean "pass input to a python script," you have two options: command line arguments or standard input.
While it is technically possible to read a series of images from standard input, it is certainly more complicated with no benefit to make it worth while.
Command line arguments can only be strings, read from sys.argv So, you while you could try to use multiple command line arguments, it would be more trouble than it's worth to translate an array of strings into a multidimensional array.
So, tl;dr create a data file format that lets you represent a set of images as a multidimensional array of file paths (or URLs, if you wanted). The easiest by far would be simply to use a CSV file and read it in with the csv Python module.
import csv
image_path_array = list(csv.reader(open('filename.csv','r')))

Alternate between Executing a MATLAB file and a Python script

I have a MATLAB file that currently saves its variables into a .mat workspace. The python script uses SciPy.io to read these variables from the workspace. The python script performs some operations & resaves variables into a MATLAB workspace(agin using Scipy.io) which matlab should then reopen. I'm using MATLABR2013a and I dont think there's an easy way to run the python script from within the .m file itself.
There may be an easier way then the method I'm going about doing it but my current plan is to create a bash script that runs the matlab file and only proceeds to the latter section if a random variable (stored in another file) is of a certain value. The script then calls the python script, sets the random variable to a different (can view as a sort of boolean). The matlab script will then execute the second section but not the first section. I need to have about 5 or 6 such exclusive sections however and it's easier to have them all in the same .m file than it is to separate them
This seems tedious however when all I really want is a way to have the system pause the matlab script, run the python script and come back to that spot in the matlab script.
Appreciate all creative suggestions to make this workflow as efficient as possible and easy to modify
MATLAB code detailed below
I saved the workspace using MATLAB's save function
Used MATLAB's system() function to execute the python script.
Within python, used scipy.iosavemat to save variables I wanted to access in matlab
Used MATLAB's load function to load the variables from python back into matlab's workspace
writeto=['insert path to save to here']
save(writeto)
first_Pypath=['insert path of python script here']
py_call=horzcat('python ',first_Pypath);
system(py_call);

Categories