We Keep Coding

How I can execute commands using subprocess function

I want to execute the commands in a python script :
Open cmd as administrator
cd C:\elastic_stack\logstash-7.6.2
.\bin\logstash -f C:/Users/Asus/Desktop/flask_project_part2/project/logstash_file.conf
This what I'm trying to do but it doesn't execute the last config file :
import os, subprocess
from subprocess import *
os.chdir("C:\\Users")
cmd = subprocess.Popen(["runas", "/noprofile", "/user:Administrator", "|", "cd", "C:/elastic_stack/logstash-7.6.2"], shell=True)
cmd.subprocess.run(["./bin/logstash", "-f", "C:/Users/Asus/Desktop/flask_project_part2/project/logstash_file.conf"], shell=True)

You have a combination of forward and backward slashes, but I am assuming that you have installed ELK on a windows machine:
Unfortunately I do not have access to a windows machine, so I didn't get a chance to test the code. But mainly it should be like this:
import os
import subprocess
# desired path
target_dir = os.path.join("C:",os.sep,"elastic_stack",os.sep,"logstash-7.6.2")
# small check
if os.path.isdir(target_dir):
os.chdir(target_dir)
else:
print(" pathname does not refer to an existing directory")
# current working directory
print(os.getcwd())
# start logstash directly os.system will return the return code of the command if it's 0 means OK
os.system(".\\bin\\logstash -f C:\\Users\\Asus\\Desktop\\flask_project_part2\\project\\logstash_file.conf")
# if you need the output after you started logstash it will work ONLY in Python3
process = subprocess.run([".\\bin\\logstash", "-f", "C:\\Users\\Asus\\Desktop\\flask_project_part2\\project\\logstash_file.conf"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = process.stdout
errors = process.stderr
PS:
You can os.sep when constructing absolute or relative paths because in this way the separator is system agnostic.
shell argument default is set to False and it means there is no system shell started up, and if shell=True means system shell will first spin up.

python variables as argument in subprocess

I am running below Python code & using subprocess to call one Python script. It is not able to substitute the value of ${ENVIRONMENT}.
import sys
import subprocess
#ENVIRONMENT=sys.argv[1]
ENVIRONMENT='test'
url=subprocess.check_output("python env.py ${ENVIRONMENT}", shell=True)

Use string formatting:
url = subprocess.check_output(
"python env.py {ENVIRONMENT}".format(ENVIRONMENT=ENVIRONMENT), shell=True)
or pass your command as a list:
url = subprocess.check_output(["python", "env.py", ENVIRONMENT])

redircet stdout to file using variable python 3

I want to redirect o/p of shell commands to file using variable "path" but it is not working
import os, socket, shutil, subprocess
host = os.popen("hostname -s").read().strip()
path = "/root/" + host
if os.path.exists(path):
print(path, "Already exists")
else:
os.mkdir("Directory", path , "Created")
os.system("uname -a" > path/'uname') # I want to redirect o/p of shell commands to file using varibale "path" but it is not working
os.system("df -hP"> path/'df')

I think the problem is the bare > and / symbols in the os.system command...
Here is a python2.7 example with os.system that does what you want
import os
path="./test_dir"
command_str="uname -a > {}/uname".format(path)
os.system(command_str)

Here's a very minimal example using subprocess.run. Also, search StackOverflow for "python shell redirect", and you'll get this result right away:
Calling an external command in Python
import subprocess
def run(filename, command):
with open(filename, 'wb') as stdout_file:
process = subprocess.run(command, stdout=subprocess.PIPE, shell=True)
stdout_file.write(process.stdout)
return process.returncode
run('test_out.txt', 'ls')

How can I use an executable jar file with a mainClass in python? [duplicate]

I have been looking for an answer for how to execute a java jar file through python and after looking at:
Execute .jar from Python
How can I get my python (version 2.5) script to run a jar file inside a folder instead of from command line?
How to run Python egg files directly without installing them?
I tried to do the following (both my jar and python file are in the same directory):
import os
if __name__ == "__main__":
os.system("java -jar Blender.jar")
and
import subprocess
subprocess.call(['(path)Blender.jar'])
Neither have worked. So, I was thinking that I should use Jython instead, but I think there must a be an easier way to execute jar files through python.
Do you have any idea what I may do wrong? Or, is there any other site that I study more about my problem?

I would use subprocess this way:
import subprocess
subprocess.call(['java', '-jar', 'Blender.jar'])
But, if you have a properly configured /proc/sys/fs/binfmt_misc/jar you should be able to run the jar directly, as you wrote.
So, which is exactly the error you are getting?
Please post somewhere all the output you are getting from the failed execution.

This always works for me:
from subprocess import *
def jarWrapper(*args):
process = Popen(['java', '-jar']+list(args), stdout=PIPE, stderr=PIPE)
ret = []
while process.poll() is None:
line = process.stdout.readline()
if line != '' and line.endswith('\n'):
ret.append(line[:-1])
stdout, stderr = process.communicate()
ret += stdout.split('\n')
if stderr != '':
ret += stderr.split('\n')
ret.remove('')
return ret
args = ['myJarFile.jar', 'arg1', 'arg2', 'argN'] # Any number of args to be passed to the jar file
result = jarWrapper(*args)
print result

I used the following way to execute tika jar to extract the content of a word document. It worked and I got the output also. The command I'm trying to run is "java -jar tika-app-1.24.1.jar -t 42250_EN_Upload.docx"
from subprocess import PIPE, Popen
process = Popen(['java', '-jar', 'tika-app-1.24.1.jar', '-t', '42250_EN_Upload.docx'], stdout=PIPE, stderr=PIPE)
result = process.communicate()
print(result[0].decode('utf-8'))
Here I got result as tuple, hence "result[0]". Also the string was in binary format (b-string). To convert it into normal string we need to decode with 'utf-8'.

With args: concrete example using Closure Compiler (https://developers.google.com/closure/) from python
import os
import re
src = test.js
os.execlp("java", 'blablabla', "-jar", './closure_compiler.jar', '--js', src, '--js_output_file', '{}'.format(re.sub('.js$', '.comp.js', src)))
(also see here When using os.execlp, why `python` needs `python` as argv[0])

How about using os.system() like:
os.system('java -jar blabla...')
os.system(command)
Execute the command (a string) in a subshell. This is implemented by calling the Standard C function system(), and has the same limitations. Changes to sys.stdin, etc. are not reflected in the environment of the executed command.

Malformed environment variables detection in python

I am trying to source a bash script containing some environment variables in python. I followed one other thread to do it. But, there seems that one of the variable is malformed, as can be seen in the given snippet.
COLORTERM=gnome-terminal
mc=() { . /usr/share/mc/mc-wrapper.sh
}
_=/usr/bin/env
I am using the following code to set up the current environment.
import os
import pprint
import subprocess
command = ['bash', '-c', 'source init_env && env']
proc = subprocess.Popen(command, stdout = subprocess.PIPE)
for line in proc.stdout:
(key, _, value) = line.partition("=")
os.environ[key] = value
proc.communicate()
'
If I change the above code a little like putting a condition:
for line in proc.stdout:
(key, _, value) = line.partition("=")
if not value:
continue
os.environ[key] = value
then things are working but the environment is corrupted because of one missing bracket as can be seen from the snippet of environment variable that the bracket is appearing on new line. Because of this corruption, If I run some other command like
os.system("ls -l")
it gives me the following error
sh: mc: line 1: syntax error: unexpected end of file
sh: error importing function definition for `mc'
What could be the possible solutions for this problem?
Thanks alot

Probably the best way to do this is to create a separate program that writes out the environment variables in a way that is easily and unambiguously processed by your own program; then call that program instead of env. Using the standard pickle module, that separate program can be as simple as this:
import os
import sys
import pickle
pickle.dump(os.environ, sys.stdout)
which you can either save into its own .py file, or else put directly in a Bash command:
python -c 'import os, sys, pickle; pickle.dump(os.environ, sys.stdout)'
In either case, you can process its output like this:
import os
import pprint
import subprocess
import pickle
command = [
'bash',
'-c',
'source init_env && ' +
'python -c "import os, sys, pickle; ' +
'pickle.dump(os.environ, sys.stdout)"'
]
proc = subprocess.Popen(command, stdout = subprocess.PIPE)
for k, v in pickle.load(proc.stdout).iteritems():
os.environ[k] = v
proc.communicate()