We Keep Coding

Python/Linux/Streamlit: executing subprocess for linux script

I'm a bit turned around on how to execute a shell script file in a Linux environment via Python's subprocess command in Streamlit. Any assistance on what I'm missing is appreciated.
I'm using a shell script called 0_texts.sh to run Pylanguagetool for a grammar check of one text file and return corrections in another text file, like so:
cd /home/user/dir/TXTs/
pylanguagetool text_0.txt > comments_0.txt
This script runs correctly in the Linux terminal, writing a comments_0.txt with appropriate grammar checks from text_0.txt.
I need to create a Python/Streamlit app that runs these shell scripts. In attempting to run this shell script, I've written script.py below:
import os
import subprocess
import sys
subprocess.run(['bash','/home/user/dir/Scripts/0_texts.sh'])
I then run script.py in Streamlit via the code below, keeping with Streamlit's documentation on using subprocess here.
import streamlit as st
import os
import subprocess
import sys
def app():
button1 = st.button("Click me")
if button1:
p = subprocess.run([f"{sys.executable}", "/home/user/dir/pages/script.py"])
st.write(p)
When I execute the script.py via Streamlit, the 0_txts.sh script executes, writing comments_0.txt in the correct directory and providing the following traceback: CompletedProcess(args=['/usr/bin/python3', '/home/user/dir/pages/script.py'], returncode=0). However, the comments_0.txt output contains the error input file is required, as if it can't properly access or read text_0.txt. I've tinkered around trying to find the problem, but have hit a brick wall.
Any suggestions on what I'm missing, or paths forward? Any help greatly appreciated.

Import on shared object fails with ModuleNotFound only when script is executed from command line

ran into a weird problem where there is a shared-object import error only when I run the script from command line. It succeed if i run the script in the python console using exec(...)
I have a class that needs a shared object: foo.py:
import os
cur_dir = os.curdir()
os.chdir('/tmp/dir_with_shared_object/')
import shared_object_class
os.chdir(cur_dir)
class Useful:
... # use the shared object import
Then there is a script like this:
from foo import Useful
If I enter python console and run:
exec(open('script.py').read())
Everything works fine.
If I run this on command line:
python script.py
I will get
ModuleNotFoundError: No module named 'shared_object_class'
The python is the same. It is 3.7.3, GCC 7.3.0 Anaconda. Anyone knows what is causing this discrepancy in behavior for shared object import?

A standard way of importing from a custom directory would be to include it in the PYTHONPATH environmental variable, with export PYTHONPATH=/tmp/dir_with_shared_object/.
update
It could also be done dynamically with
import sys
p = '/tmp/dir_with_shared_object/'
sys.path.insert(0, p)
PS
I think I have an explanation for why OP's original code didn't work. According to this python reference page, the import system searches, inter alia, in "[t]he directory containing the input script (or the current directory when no file is specified)." So the behavior in the REPL loop is different from how it is when running a script. Apparently the current directory is evaluated each time an import statement is encountered, while the directory containing the input script doesn't change.

How to read different configuration file based on option - python & command line

I have multiple configuration files and I want to read a different config file based on which option I type. For example if I type in the terminal
python test.py -60min
I want to read the python script to read the config file '/home/matt/config_60min.ini'
Similarly, python test.py -30min would read '/home/matt/config_30min.ini'
I'm not sure if this would be done using conditional logic within the script or with a simple option parser. Maybe there's a better way to go about it such as python test.py -f 60min
Thanks in advance.

You could do it like this:
import sys
config_path = '/home/matt/config_{}.ini'.format(sys.argv[1])
and run the script using:
python test.py 60min
If it gets more complicated than this, consider using the argparse library

how to run python script from the console as if called from command prompt?

The python script I would use (source code here) would parse some arguments when called from the command line. However, I have no access to the Windows command prompt (cmd.exe) in my environment. Can I call the same script from within a Python console? I would rather not rewrite the script itself.

%run is a magic in IPython that runs a named file inside IPython as a program almost exactly like running that file from the shell. Quoting from %run? referring to %run file args:
This is similar to running at a system prompt python file args,
but with the advantage of giving you IPython's tracebacks, and of
loading all variables into your interactive namespace for further use
(unless -p is used, see below). (end quote)
The only downside is that the file to be run must be in the current working directory or somewhere along the PYTHONPATH. %run won't search $PATH.
%run takes several options which you can learn about from %run?. For instance: -p to run under the profiler.

If you can make system calls, you can use:
import os
os.system("importer.py arguments_go_here")

You want to spawn a new subprocess.
There's a module for that: subprocess
Examples:
Basic:
import sys
from subprocess import Popen
p = Popen(sys.executable, "C:\test.py")
Getting the subprocess's output:
import sys
from subprocess import Popen, PIPE
p = Popen(sys.executable, "C:\test.py", stdout=PIPE)
stdout = p.stdout
print stdout.read()
See the subprocess API Documentation for more details.

Run python script from python using different python

I have a software that has python 2.5.5. I want to send a command that would start a script in python 2.7.5 and then proceed with the script.
I tried using
#!python2.7.5
and http://redsymbol.net/articles/env-and-python-scripts-version/
But I cant get it to work...
In my python 2.5.5 I can execute script as
execfile("c:/script/test.py")
The problem is that the 2.7.5 has a module comtypes + few other. I dont know how to install it for my 2.5.5 so I'm trying to start a separate script and run it under python27. Now another reason why I want it its because I want to take the load off program. I have 2 heavy tasks to perform. The second task is the one that need comptypes so sending it to external shell/app would do perfect trick. Is there a way to do it ?
I wish I could just type run("C:/Python27/python.exe % C:/script/test,py")
Thanks, bye.
Little update. I try to run
import os
os.system("\"C:\Python27\python.exe\" D:\test\runTest.py")
But I'm getting a quick pop up and close window saying that
Import Error : no module named site...
This works if I run from external shell but not from here :(
So I've tried another approach this time to add modules to python... in any case I run this :
import os
import sys
sys.path.append("C:/python27")
sys.path.append("C:/Python27/libs")
sys.path.append("C:/Python27/Lib")
sys.path.append("C:/Python27/Lib/logging")
sys.path.append("C:/Python27/Lib/site-packages")
sys.path.append("C:/Python27/Lib/ctypes")
sys.path.append("C:/Python27/DLLs")
import PyQt4
print PyQt4
import comtypes
import logging
but it crashes with C error...
Runtime Error :
Program: c:\Pr...
R6034
An application has made attempt to load the C runtime library incorectly.
blablabla....
How can I import it ? Maybe if I can import it I can run it directly from my app rather than starting separate python...
Traceback (most recent call last):
File "<string>", line 18, in <module>
File "C:\Python27\Lib\site-packages\comtypes\__init__.py", line 22, in <module>
from ctypes import *
File "C:\Python27\Lib\ctypes\__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
Another update to isseu
so I run now
import os
os.system("start cmd {D:\test\runTest.py}")
now this works and he open CMD with c:\Python27 as directory but he dont run the file... any hitns how to fix it?

Use "raw" strings so that you don't need to escape as much; I think the backslashes are what was breaking your code since backslash is considered an escape character except in raw strings.
Also, use the subprocess module. It makes it easy to avoid manually making a safe command string (the module takes care of that for you). All you need to do is pass it a list of arguments.
Your code would then look something like this:
import subprocess
proc = subprocess.Popen([r"C:\Python27\python.exe", r"D:\test\runTest.py"])
# then either do this
proc.wait() # wait until the process finishes
# or this
while True:
# NOTE: do something else here
# poll the process until it is done
if proc.poll() is not None:
break # break out of loop
See subprocess docs for Python 2 here. Be sure to check if a feature was added after Python 2.5 (the 2.5 docs aren't available online anymore AFAIK).
UPDATE:
I just noticed that you tried to use the Python 2.7 libraries and modules in your 2.5 code. This probably won't work due to new features added after 2.5. But it got me thinking how you might be able to make 2.7 work.
It may be that your Python2.7 install can't find its libraries; this is probably why you get the error Import Error : no module named site. You can do something like the above and modify the PYTHONPATH environment variable before starting the subprocess, like this:
import os
import subprocess
paths = [r"C:\python27", r"C:\python27\libs", r"C:\python27\Lib\site-packages", r"C:\python27\DLLs"]
paths += os.environ.get('PYTHONPATH', '').split(os.pathsep)
env27 = dict(os.environ)
env27['PYTHONPATH'] = os.pathsep.join(paths)
proc = subprocess.Popen([r"C:\Python27\python.exe", r"D:\test\runTest.py"], env=env27)