I have web application on Flask and user can send some request which my script is processing and then, running another script in python in the console with some parameters like this:
import sys
os.system('python start.py -u 100 -p 122224')
All works good, but now I want controlling all running copies of my script like start, stop and pause.
How i can do this without crutches?
Check subprocess and multiprocessing modules. The first one allows you to execute external application. To use the second one, you'll be required to call some python code, but the management capabilities should be much wider.
Related
I have a small Flask API that is receiving requests from a remote server. Whenever a request is received, a subprocess is started. This subprocess is simply executing a second Python file that is in the same folder. This subprocess can run for several hours and several of these subprocesses can run simultaneously. I am using stdout to write the output of the python file into a text file.
All of this is working fine, but every couple of weeks it happens that the Flask API becomes unresponsive and needs to be restarted. As soon as I stop the Flask server, all running subprocesses stop. I would like to avoid this and run each subprocess independently from the flask API.
This is a small example that illustrates what I am doing (this code is basically inside a method that can be called through the API)
import subprocess
f = open("log.txt","wb")
subprocess.Popen(["python","job.py"],cwd = "./", stdout = f, stderr = f)
I would like to achieve that the subprocess keeps running after I stop the Flask API. This is currently not the case. Somewhere else I read that the reason is that I am using the stdout and stderr parameters, but even after removing those the behavior stays the same.
Any help would be appreciated.
Your sub-processes stop because their parent process dies when you restart your Flask server. You need to completely separate your sub-processes from your Flask process by running your Python call in a new shell:
from subprocess import call
# On Linux:
command = 'gnome-terminal -x bash -l -c "python job.py"'
# On Windows:
# command = 'cmd /c "python job.py"'
call(command, shell=True)
This way your Python call of job.py will run in a separate terminal window, unaffected by your Flask server process.
Use fork() to create a child process of the process in which you are calling this function. On successful fork(), it returns a zero for the child id.
Below is a basic example of fork, which you can easily incorporate in your code.
import os
pid = os.fork()
if pid == 0: # new process
os.system("nohup python ./job.py &")
Hope this helps!
I am currently writing a program to help me with some database administration. It's supposed to execute commands in MariaDB. I thought since this is a CLI application I can just do that with os.system, but I'm having problems doing so. So lets say I have the following code
import os
os.system('mysql --user=%s --password=%s' %(user, password)
os.system('USE database;')
This code logs me into MariaDB, but it does not execute the second command to choose the database. Is this possible with os.system, and if not, what are my alternatives? Thanks.
Edit: I tried using subprocess instead, which gives me another problem: It immediately exits MariaDB after logging in.
I recomand you to use Subprocess instead of os python module, using Popen
try to check this link out :
Calling an external command in Python
Normally, I would use "blender -P script.py" to run a python script. In this case, a new blender process is started to execute the script. What I am trying to do now is to run a script using a blender process that is already running, instead of starting a new one.
I have not seen any source on this issue so far, which makes me concern about the actual feasibility of this approach.
Any help would be appreciated.
Blender isn't designed to be started from the cli and to then keep receiving more commands from the cli as it is running. It does however include a text editor that can open text files and run the text block as a python script, it also includes a python console that can be used to interactively type in commands while blender is running. You may also find this addon useful as it lets you to run a text block in the python console, this leaves you with an interactive session that contains the variables as they exist at the end of the scripts execution.
There is a cli option to run blender as a python console blender --python-console - the gui does not get updated while this console is running, so you could open and exec several scripts and then when you exit the console, blender will update it's gui and allow interactive use, or if you start in background mode -b then it will quit when you exit the console.
My solution was to launch Blender via console with a python script (blender --python script.py) that contains a while loop and creates a server socket to receive requests to process some specific code. The loop will prevent blender from opening the GUI, and the socket will handle the multiple requests inside the same blender process.
I have what I believe to be a fairly unique problem for a script I use to stand up webservers on remote machines.
I have a controller script which after checking a ledger initiates a "builder" script on a remote machine. Part of this builder script calls a bash script which starts a process I want to continue running after both scripts are finished.
My only problem is that the builder script seems to finish (gets to the last line) but doesn't seem to return control to the controller script.
For the record I am using subprocess.call in the controller script (to initiate a ssh call) to start the builder script on the remote machine. I have toyed with various ways of initiating the bash script in the builder script but it seems the builder won't return control to the controller until kill the processes spawned by the bash script.
Things I have tried:
pid=os.spawnl(os.P_NOWAIT,dest+'/start_background_script.sh')
pid=subprocess.Popen([dest+'/start_background_script.sh'])
os.system(dest+'/start_background_script.sh &')
pid=os.spawnl(os.P_NOWAIT,dest+'/start_background_script.sh')
The bash script is written to that you execute it and it backgrounds two processes and then returns control.
Any recommendations?
Sound like a job for fabric to me.
Fabric wraps the handling of shell-calls on remote (and also local) machines for you.
Pretty much what the title says, I would like to be able to connect to a python process running under paster or uwsgi and utilize pdb functionality.
Using winpdb, you can attach to a running process like this:
Insert
import rpdb2; rpdb2.start_embedded_debugger('mypassword')
inside your script.
Launch your script (through paster or uwsgi) as usual.
Run winpdb
Click File>Attach
Type in password (e.g. "mypassword"), select the process.
To detach, click File>Detach. The script will continue to run, and can be attached to again later.