We Keep Coding

Use python to connect to sqlplus in a remote host and execute sql commands

Here is my situation :
We have sqlplus set up in a remote machine and I want to connect to that remote machine and then run sqlplus to execute sql queries. I am trying to write a python script to do that.
Here is my code:
import sys
import getpass
import paramiko
import time
user=raw_input('Enter User Name :')
#host_name=raw_input('Enter Host Name:')
psswd=getpass.getpass()
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('xxx.hostname.xxx',port=22, username=user, password=psswd)
command='export ORACLE_HOME=/opt/app/oracle/product/10.2.0.2/client export LD_LIBRARY_PATH=$ORACLE_HOME/lib \
sudo -S -H /XX/XX/XX/bin/sqlplus'
print 'running remote command'
print(command)
stdin, stdout, stderr=ssh.exec_command(command)
stdin.write(psswd+'\n')
stdin.flush()
for out in stdout.readlines():
print out
ssh.close()
I have two issues here
First is if i pass command like this
'export ORACLE_HOME=/opt/app/oracle/product/10.2.0.2/client export LD_LIBRARY_PATH=$ORACLE_HOME/lib \
sudo -S -H /XX/XX/XX/bin/sqlplus' +' echo $ORACLE_HOME'
I get an empty response even if I have added echo that means that variable is not set correctly.
Secondly, I can't figure out what next to do here. How to provide username/password to sqlplus to allow executing sql queries and then how to supply sql statements.

I had a similar issue as you and eventually wrote a library to do this. Here's a snippet of 'psuedo psuedo code' that should point you in the right direction.
Keep in mind these are methods of a class and you'll need to adapt this pseudo code to your needs. Keep in mind you'll already need a SSHConnection from paramiko here.
def sqlplus_cmd(self, command):
# Create string which exports environmental variables from OracleEnv class ()
if 'CYGWIN' not in <return from 'uname' on the host>:
# If NOT Cygwin, concatenate environmental variable exports
oracle_exports = 'export PATH={0}:$PATH;' \
'export ORACLE_HOME={1};' \
'export ORACLE_SID={2}'.format(<oracle_path>, <oracle_home>, <oracle_sid>)
else:
# If Cygwin, need to source environmental variables for shell session from script
# TODO: May need to get Oracle Home and Path as well for some systems.
self.cmd('echo "export ORACLE_SID={0}" > /tmp/sid'.format(<oracle_sid>))
oracle_exports = 'source /tmp/sid'
# Issue concatinated one line command which exports variables, opens sqlplus, and issues a sqlplus statement
# final_command = oracle_exports + ';' + 'echo "' + command + '" | sqlplus -S / as sysdba'
final_command = '{0};echo "{1}" | sqlplus -S / as sysdba'.format(oracle_exports, command)
stdout, stderr, rc = <paramiko_SSHConnection.exec_command>(final_command)
That should do it. Have fun parsing the output and catching the ORA-xxx and SP2-xxx errors in stdout.

Why don't you split your command into a function, and use subprocess.Popen() to execute it in a subprocess?
from subprocess import *
def run_sql_query(sql_command, connection_string):
session = Popen(['sqlplus', '-S', connection_string], stdin=PIPE, stdout=PIPE, stderr=PIPE)
session.stdin.write(sql_command)
return session.communicate()
Then you can pass your connection string and command as arguments to your function:
con_str = 'xxx.hostname.xxx',port=22, username=user, password=psswd'
cmd = ''export ORACLE_HOME=/opt/app/oracle/product/10.2.0.2/client export LD_LIBRARY_PATH=$ORACLE_HOME/lib sudo -S -H /apollo/env/envImprovement/bin/sqlplus'
print(run_sql_query(con_str, cmd))

How can I make my custom shell work with ssh?

I'm making a custom shell in Python for a very limited user on a server, who is logged in via ssh with a public key authentication. They need to be able to run ls, find -type d, and cat in specific directories with certain limitations. This works fine if you run something like ssh user#server -i keyfile, because you see the interactive prompt, and can run those commands. However, something like ssh user#server -i keyfile "ls /var/log" doesn't. ssh simply hangs, with no response. By using the -v switch I've found that the connection is succeeding, so the problem is in my shell. I'm also fairly certain that the script isn't even being started, since print sys.argv at the beginning of the program does nothing. Here's the code:
#!/usr/bin/env python
import subprocess
import re
import os
with open(os.devnull, 'w') as devnull:
proc = lambda x: subprocess.Popen(x, stdout=subprocess.PIPE, stderr=devnull)
while True:
try:
s = raw_input('> ')
except:
break
try:
cmd = re.split(r'\s+', s)
if len(cmd) != 2:
print 'Not permitted.'
continue
if cmd[0].lower() == 'l':
# Snip: verify directory
cmd = proc(['ls', cmd[1]])
print cmd.stdout.read()
elif cmd[0].lower() == 'r':
# Snip: verify directory
cmd = proc(['cat', cmd[1]])
print cmd.stdout.read()
elif cmd[0].lower() == 'll':
# Snip: verify directory
cmd = proc(['find', cmd[1], '-type', 'd'])
print cmd.stdout.read()
else:
print 'Not permitted.'
except OSError:
print 'Unknown error.'
And here's the relevant line from ~/.ssh/authorized_keys:
command="/path/to/shell $SSH_ORIGINAL_COMMAND" ssh-rsa [base-64-encoded-key] user#host
How can I make the shell script when the command is passed on the command line so it can be used in scripts without starting an interactive shell?

The problem with ssh not responding is related to the fact that ssh user#host cmd does not open a terminal for the command being run. Try calling ssh user#host -t cmd.
However, even if you pass the -t option, you'd still have another problem with your script: it only works interactively and totally ignores the $SSH_ORIGINAL_PROGRAM being passed. A naive solution would be to check sys.argv and if its bigger than 1 you don't loop forever, and instead only execute whatever command you have in it.

Paramiko "Invalid command" when trying non-standard command for Vyatta

I've just tried to connect my Python/Django app with Vyatta server using Paramiko for SSHing. Unfortunately, when I try to run show interfaces it throws "Invalid command". However, if try to SSH manually from that server, it works fine. I tried also '/vbash -c "show interfaces"' - the same result.
ssh = paramiko.SSHClient()
ssh.connect('10.0.0.1','vyatta','vyatta')
stdin, stdout, stderr = ssh.exec_command('show interfaces')
# or stdin, stdout, stderr = ssh.exec_command('vbash -c "show interfaces"')
print '-'.join(stdout)
print '-'.join(stderr)

As mentioned earlier you can use vyatta-cfg-cmd-wrapper and set any configuration node:
<import stuff>
command = """
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper begin
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper set system host-name newhostname
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper commit
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper save
"""
sshobj = paramiko.SSHClient()
sshobj.set_missing_host_key_policy(paramiko.AutoAddPolicy())
sshobj.connect(IP,username=login,password=sshpass)
stdin,stdout,stderr=sshobj.exec_command(command)
print ''.join(stdout)
sshobj.close()
And the reult as follow:
user#hostname$ python vyatta.py
Saving configuration to '/config/config.boot'...

Vyatta commands are done by templates in vbash. There are a bunch of environment variables that need to be set in order for templates to work. You have to either use a remote envrionment that sources .profilerc or there is an undocumented script vyatta-cfg-command wrapper to setup more complex state necessary to commit changes.

In my case, I solved the problem by using this command:
vbash -c -i "restart vpn"

execute local python script over sshClient() with Paramiko in remote machine

This is my first post in StackOverflow, so I hope to do it the right way! :)
I have this task to do for my new job that needs to connect to several servers and execute a python script in all of them. I'm not very familiar with servers (and just started using paramiko), so I apologize for any big mistakes!
The script I want to run on them modifies the authorized_keys file but to start, I'm trying it with only one server and not yet using the aforementioned script (I don't want to make a mistake and block the server in my first task!).
I'm just trying to list the directory in the remote machine with a very simple function called getDir(). So far, I've been able to connect to the server with paramiko using the basics (I'm using pdb to debug the script by the way):
try_paramiko.py
#!/usr/bin/python
import paramiko
from getDir import get_dir
import pdb
def try_this(server):
pdb.set_trace()
ssh = paramiko.SSHClient()
ssh.load_host_keys("pth/to/known_hosts")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
my_key = paramiko.RSAKey.from_private_key_file("pth/to/id_rsa")
ssh.connect(server, username = "root", pkey = my_key)
i, o, e = ssh.exec_command(getDir())
This is the function to get the directory list:
getDir.py
#!/usr/bin/python
import os
import pdb
def get_dir():
pdb.set_trace()
print "Current dir list is:"
for item in os.listdir(os.getcwd()):
print item
While debugging I got the directory list of my local machine instead of the one from the remote machine... is there a way to pass a python function as a parameter through paramiko? I would like to just have the script locally and run it remotely like when you do it with a bash file from ssh with:
ssh -i pth/to/key username#domain.com 'bash -s' < script.sh
so to actually avoid to copy the python script to every machine and then run it from them (I guess with the above command the script would also be copied to the remote machine and then deleted, right?) Is there a way to do that with paramiko.sshClient()?
I have also tried to modify the code and use the standard output of the channel that creates exec_command to list the directory leaving the scripts like:
try_paramiko.py
#!/usr/bin/python
import paramiko
from getDir import get_dir
import pdb
def try_this(server):
pdb.set_trace()
ssh = paramiko.SSHClient()
ssh.load_host_keys("pth/to/known_hosts")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
my_key = paramiko.RSAKey.from_private_key_file("pth/to/id_rsa")
ssh.connect(server, username = "root", pkey = my_key)
i, o, e = ssh.exec_command(getDir())
for line in o.readlines():
print line
for line in e.readlines():
print line
getDir.py
def get_dir():
return ', '.join(os.listdir(os.getcwd()))
But with this, it actually tries to run the local directory list as commands (which actually makes sense they way I have it). I had to convert the list to a string because I was having a TypeError saying that it expects a string or a read-only character buffer, not a list... I know this was a desperate attempt to pass the function... Does anyone know how I could do such thing (pass a local function through paramiko to execute it on a remote machine)?
If you have any corrections or tips on the code, they are very much welcome (actually, any kind of help would be very much appreciated!).
Thanks a lot in advance! :)

You cannot just execute python function through ssh. ssh is just a tunnel with your code on one side (client) and shell on another (server). You should execute shell commands on remote side.
If using raw ssh code is not critical, i suggest fabric as library for writing administration tools. It contains tools for easy ssh handling, file transferring, sudo, parallel execution and other.

I think you might want change the paramaters you're passing into ssh.exec_command Here's an idea:
Instead of doing:
def get_dir():
return ', '.join(os.listdir(os.getcwd()))
i, o, e = ssh.exec_command(getDir())
You might want to try:
i, o, e = ssh.exec_command('pwd')
o.printlines()
And other things to explore:
Writing a bash script or a Python that lives on your servers. You can use Paramiko to log onto the server and executing the script with ssh.exec_command(some_script.sh) or ssh.exec_command(some_script.py)
Paramiko has some FTP/SFTP utilities so you can actually use it to put the script on the server and then execute it.

It is possible to do this by using a here document to feed a module into the remote server's python interpreter.
remotepypath = "/usr/bin/"
# open the module as a text file
with open("getDir.py", "r") as f:
mymodule = f.read()
# setup from OP code
ssh = paramiko.SSHClient()
ssh.load_host_keys("pth/to/known_hosts")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
my_key = paramiko.RSAKey.from_private_key_file("pth/to/id_rsa")
ssh.connect(server, username = "root", pkey = my_key)
# use here document to feed module into python interpreter
stdin, stdout, stderr = ssh.exec_command("{p}python - <<EOF\n{s}\nEOF".format(p=remotepypath, s=mymodule))
print("stderr: ", stderr.readlines())
print("stdout: ", stdout.readlines())

subprocess.popen seems to fail when run from crontab

I'm running a script from crontab that will just ssh and run a command and store the results in a file.
The function that seems to be failing is subprocess.popen.
Here is the python function:
def _executeSSHCommand(sshcommand,user,node):
'''
Simple function to execute an ssh command on a remote node.
'''
sshunixcmd = '/usr/bin/ssh %s#%s \'%s\'' % (user,node,sshcommand)
process = subprocess.Popen([sshunixcmd],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
process.wait()
result = process.stdout.readlines()
return result
When it's run from the command line, it executes correctly, from cron it seems to fail with the error message below.
Here are the crontab entries:
02 * * * * /home/matt/scripts/check-diskspace.py >> /home/matt/logs/disklog.log
Here are the errors:
Sep 23 17:02:01 timmy CRON[13387]: (matt) CMD (/home/matt/scripts/check-diskspace.py >> /home/matt/logs/disklog.log)
Sep 23 17:02:01 timmy CRON[13386]: (CRON) error (grandchild #13387 failed with exit status 2)
I'm going blind trying to find exactly where I have gone so wrong. Any ideas?

The cron PATH is very limited. You should either set absolute path to your ssh /usr/bin/ssh or set the PATH as a first line in your crontab.

You probably need to pass ssh the -i argument to tell ssh to use a specific key file. The problem is that your environment is not set up to tell ssh which key to use.
The fact that you're using python here is a bit of a red herring.

For everything ssh-related in python, you might consider using paramiko. Using it, the following code should do what you want.
import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.connect(node, username=user)
stdout = client.exec_command(ssh_command)[0]
return stdout.readlines()

When running python scripts from cron, the environment PATH can be a hangup, as user1652558 points out.
To expand on this answer with example code to add custom PATH values to the environment for a subprocess call:
import os
import subprocess
#whatever user PATH values you need
my_path = "/some/custom/path1:/some/custom/path2"
#append the custom values to the current PATH settings
my_env = os.environ.copy()
my_env["PATH"] = my_path + ":" + my_env["PATH"]
#subprocess call
resp = subprocess.check_output([cmd], env=my_env, shell=True)