Run Python app (script on PATH) from Java - python

I'm facing quite simple problem, yet still not able to figure out what in particular causes that, and - more importantly - how to solve it.
I'm currently on Linux, and I would like to run an python app (script) from a Java application. The script is on PATH, thus I really would like to utilise that (avoid using absolute path to the script), if possible.
However, all I tried resulted in various forms of "File does not exist".
Sample
As a demonstration, I've tried to run one python3-based app (meld)
and one binary-built app (ls) for comparison:
$ which ls
/usr/bin/ls
$ file /usr/bin/ls
/usr/bin/ls: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=[...], for GNU/Linux 3.2.0, stripped
$ which meld
/usr/bin/meld
$ file /usr/bin/meld
/usr/bin/meld: Python script, UTF-8 Unicode text executable
$ head -n1 /usr/bin/meld
#!/usr/bin/python3
Nextly, I've created simple Java main, which tries several ways how to start theese:
package cz.martlin.processes;
import java.io.File;
import java.io.IOException;
import java.lang.ProcessBuilder.Redirect;
import java.util.List;
import java.util.stream.Collectors;
public class ProcessPlaying {
public static void main(String[] args) {
List<List<String>> commands = List.of(
List.of("ls"),
List.of("/usr/bin/ls"),
List.of("meld"),
List.of("/usr/bin/meld"),
List.of("python3", "/usr/bin/meld"),
List.of("/usr/bin/python3", "/usr/bin/meld"),
List.of("sh", "-c", "meld"),
List.of("sh", "-c", "/usr/bin/meld"),
List.of("sh", "-c", "python3 /usr/bin/meld")
);
for (List<String> command : commands) {
run(command);
}
}
private static void run(List<String> command) {
System.out.println("Running: " + command);
//String executable = command.get(0);
//boolean exists = new File(executable).exists();
//System.out.println("Exists the " + executable + " ? " + exists);
try {
ProcessBuilder pb = new ProcessBuilder(command);
pb.redirectError(Redirect.INHERIT);
Process proc = pb.start();
// Process proc = Runtime.getRuntime().exec(command.stream().collect(Collectors.joining(" ")));
int code = proc.waitFor();
System.out.println("OK, return code: " + code);
} catch (IOException e) {
System.out.println("Failed to start: " + e.toString());
} catch (InterruptedException e) {
System.out.println("Failed to await: " + e.toString());
}
System.out.println();
}
}
Here are the results:
Running: [ls]
OK, return code: 0
Running: [/usr/bin/ls]
OK, return code: 0
Running: [meld]
Failed to start: java.io.IOException: Cannot run program "meld": error=2, Directory or file doesn't exist
Running: [/usr/bin/meld]
Failed to start: java.io.IOException: Cannot run program "/usr/bin/meld": error=2, Directory or file doesn't exist
Running: [python3, /usr/bin/meld]
python3: can't open file '/usr/bin/meld': [Errno 2] No such file or directory
OK, return code: 2
Running: [/usr/bin/python3, /usr/bin/meld]
/usr/bin/python3: can't open file '/usr/bin/meld': [Errno 2] No such file or directory
OK, return code: 2
Running: [sh, -c, meld]
sh: line 1: meld: command not found
OK, return code: 127
Running: [sh, -c, /usr/bin/meld]
sh: line 1: /usr/bin/meld: Directory or file doesn't exist
OK, return code: 127
Running: [sh, -c, python3 /usr/bin/meld]
python3: can't open file '/usr/bin/meld': [Errno 2] No such file or directory
OK, return code: 2
To sum it up:
all of the commands tried works when executed directly from the shell (either sh or bash)
executing the binary works either by the full path (/usr/bin/ls) or by just the name (ls)
executing the python script the same way doesn't work neither way
when trying to run the python3 interpreter and populate the script path as an argument to the python, now the python yields the script file doesn't exist
trying to populate it as a command to the brand new shell didn't help either
I've tried to use the Runtime#exec (based on this comment: https://stackoverflow.com/a/36783743/3797793) to start the process (both the exec(String) and exec(String[] forms), but no sucess (none of the listed commands did actually execute).
Thus, my question is/are:
What do I understand wrong?
How to start the Python script from Java?
Would some small (ba)sh script wrapper do the job?
Since it's python3-based, Jython wouldn't help here (because the latest one is 2.7.*), would it?
Further requirements:
As mentioned, I would like to be able to avoid using full path to the Python script
Also, I would like to have platform independant solution (at least Linux and Windows compatible)
Thanks in advance

Related

Python Subprocess [Errno 2] No such file or directory when calling subprocess.run()

I have a simple python web application made with flask. It compiles and runs the given c++ files on given input. Upon receiving the inputs from front-end, I use this function to compile and run the given file.
exec_id = get_unique_identifier(data["user_id"])
code = data["code"]
input_data = data["input"]
code_path = file_ops.write_code(code, data["lang"], exec_id)
logging.info("Code written to "+code_path)
#compile the code
logging.info("Compiling the code")
cmd = ['g++',code_path,'-o',exec_id]
logging.info("Command is : "+" ".join(cmd))
try:
compile_code=subprocess.run(cmd, stderr=PIPE)
has_compiled = compile_code.returncode
except Exception as e:
return {"status": "server Error", "output" : e}
compiled_file_path = './'+exec_id
file_ops.write_code is a utility function which takes input code, extension as input and writes the code in a file in the same directory. The code runs alright in local environment, However, In production server with nginx, The code never gets compiled and I get the following error(After a big traceback)
[Errno 2] No such file or directory: 'g++'
I tried the following things
Giving Absolute Path in code
passing shell=True in subprocess.run
Does this have something to do with Configuration with Nginx ? I'm running this application with gunicorn and nginx on Ubuntu VPS. If I run the code in development environment on vps, it runs without any error.

Build gradle assemble apk using laravel function

I'm currently calling the main python file in larval function, and inside that main python file I'm calling another 2 files ( PowerShell and sub python file) the problem is when the Laravel function is triggered it only call the main python file, however when I call the main python file using terminal all the files are executed like below:
Laravel function:
public function initialize(Request $request)
{
$store_name = $request->get('store_name', 1);
if (empty($store_name)) {
return 'Missing store name';
} else {
$processes = File::get("/root/flask/android_api/processes.txt");
File::put('/root/flask/android_api/url.txt', $store_name );
$process = new Process(['python3.6', '/root/flask/android_api/buildAPK.py']);
$process->run();
if (!$process->isSuccessful()) {
throw new ProcessFailedException($process);
} else {
return 'Starting the processes to build';
}
}
}
and within the main python file I have:
try:
p = subprocess.Popen(["/usr/bin/pwsh",
"/root/flask/android_api/set_apk_builder.ps1", '-ExecutionPolicy',
'Unrestricted',
'./buildxml.ps1'],
stdout=sys.stdout)
p.communicate()
except:
file = open ("/root/flask/android_api/log.txt", "w")
file.write ("fail")
file.close()
import slack
// call(["python", "/root/flask/flask.py"])
os.system('python3.7 /root/flask/flask.py')
Edit:
now I changed the build to be direct from laravel function to generate the apk
using this command:
public function initialize(Request $request)
{
$store_name = $request->get('store_name', 1);
if (empty($store_name)) {
return 'Missing store name';
} else {
return shell_exec('cd /var/www/html/androidProject && chmod +x gradlew && ./gradlew assembledemoDebug');
}
}
however, the command line returns the Gradle command build is starting but it doesn't create a folder and generate the apk
the Current folder structure /var/www/html and inside html is the project folder and laravel project
note: before I call Gradle build command inside Laravel function, I used to call python file and that python file is calling Gradle command but I had the same issue the apk is not created, but when I run the same python file from bash command it works fine
There are 2 ways you can accomplish this:
The first is to include your commands into a .sh file that you should make it executable using this command:
chmod +x file.sh
Then you should call that file from laravel using Symfony process so you can get details of the log process and errors:
use Symfony\Component\Process\Process;
use Symfony\Component\Process\Exception\ProcessFailedException;
$process = new Process('sh /folder_name/file_name.sh');
$process->run();
if (!$process->isSuccessful()) {
throw new ProcessFailedException($process);
}
echo $process->getOutput();
Then include all commands in that file you wish to run.
Second you can run those commands using
$output = shell_exec('ls -lart');
then echo "<pre>$output</pre>";
You'll need to move all your writable files into public directory in Laravel, because that's where everything should be editable.'=
I actually suggest the first one as you don't need to change owner of some folders to www-data to give write permissions.

Node JS spawn result: python3: can't open file './test': [Errno 2] No such file or directory

So I'm trying to run a simple python file called test.py through the Node JS child process, however it keeps saying that python3: can't open file './test': [Errno 2] No such file or directory. I tried everything to fix this error. For example, the python script ran successfully from the same directory the typescript file is present, I even tried the full path, the relative path for the python file and it still didn't work. I kindly request someone too look into this, thank you!
test.py
import sys
print("hello world")
sys.stdout.flush()
var spawn = require("child_process").spawn;
var child = await spawn("python3", ['./test.py']);
let dataMain = "";
child.stdout.on("data", (data: Buffer) => {
// Do something with the data returned from python script
console.log(data); // buffer data
dataMain += data.toString();
});
child.stdout.on("end", () => {
console.log("end");
console.log(dataMain);
});
child.stderr.pipe(process.stderr); // prints out the error
Worked with ${process.cwd()}/test.py

SSIS Execute Process Task Python script

I'm trying to execute a python scrip from SSIS Execute Process Task. I followed all the tutorials of how to do this an still the script is failing from the start. when i execute the python script out of SSIS it runs perfectly.
This is my Python scrip:
import sys
import gender_guesser.detector as gender
import xml.etree.cElementTree as ET
from xml.etree.ElementTree import ParseError
try:
input("Press Enter to continue...")
except SyntaxError:
pass
tree = ET.parse('user.xml')
root = tree.getroot()
for child_of_root in root:
for attr in child_of_root:
if attr.tag == 'first_name':
upperName = "%s%s" % (attr.text[0].upper(), attr.text[1:])
print attr.tag,upperName
d = gender.Detector()
gen = d.get_gender(upperName)
print gen
attr.text= gen
tree = ET.ElementTree(root)
tree.write("user1.xml")
this is an image of the SSIS Execute Process Task:
error message:
[Execute Process Task] Error:
In Executing "C:\Python27\python.exe" "C:\Users\bla\blalba\bla\gender-guesser-0.4.0\test\genderTest.py " at "", The process exit code was "1" while the expected was "0".
Did you have spaces in your file path to your python script? I had the same error when trying to pass a path with spaces as my argument in Execute Script Process. The resolution was to enter the argument with quotes.
When the process is launched from Execute Process Task step of SSIS Package, it's not being run from the same folder as the executable file (.bat, .py, .exe and so on) located.
What is different from the direct file execution.
And it can be especial critical in case when your executable file working with some other files in the same folder.
So, it is necessary additionally specify working folder property of Execute Process Task step of SSIS Package.
On your screenshot Working directory property value is empty. Put there the
C:\Users\bla\blalba\bla\gender-guesser-0.4.0\test\

Running xlwings on a server

Solved the problem by making sure that python27.dll and win32api into the PYTHON_WIN directory. The two files with an arrow were missing in the server folder (P:/)
I am using udf module of xlwings and setting xlwings.bas configuration into a server. (I have my personal disk C:/ however other users need to run this macro and the file must be stored in P:/)
I have python, xlwings and other modules installed in P:/, but I get some strange error about importing os lib. Does any one have ever passed through this ?
Thanks in advance.
I got this from python:
"Python process exited before it was possible to create the interface object.
Command: P:\Python 27\python.exe -B -c ""import sys, os;sys.path.extend(os.path.normcase(os.path.expandvars(r'P:\Risco\Python Scripts')).split(';'));import xlwings.server; xlwings.server.serve('{2bb649ad-487e-48d5-ab31-24adaeefca59}')""
Working Dir: "
Xlwings.bas is set like this:
PYTHON_WIN = "P:\Python 27\python.exe"
PYTHON_MAC = ""
PYTHON_FROZEN = ThisWorkbook.Path & "\build\exe.win32-2.7"
PYTHONPATH = "P:\Risco\Python Scripts"
UDF_MODULES = "FetchPL"
UDF_DEBUG_SERVER = False
LOG_FILE = ""
SHOW_LOG = True
OPTIMIZED_CONNECTION = False

Categories