We Keep Coding

Executing password-protected psql from python

I need to execute the following command from Python on Windows:
psql -h localhost -p 5432 -U postgres -f script.sql db_name
The above script works fine when ran from git bash / powershell. After entering the script in a terminal, I need to provide a password to confirm it (similar to when using sudo).
How can I do that? I keep finding solutions that I think are linux-based.
How do I do it on Windows? I have tried many variations of solutions involving subprocess, i.e:
import subprocess
p2 = subprocess.Popen(
'psql -h localhost -p 5432 -U postgres -f script.sql db_name',
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True)
print('this will print')
sudo_prompt = p2.communicate('THE_PASSWORD' + '\n')[1]
print('this will not')

A better option (more secure) than invoking psql with explicit mention of your password is to have a .pgpass file as described in the docs file (and keep it protected e.g. chmod 600 ~/.pgpass). This keeps your password out of the list of running processes.
On Windows:
On Microsoft Windows the file is named %APPDATA%\postgresql\pgpass.conf (where %APPDATA% refers to the Application Data subdirectory in the user's profile).

SCP Through python is not transferring file

I have two Raspberry Pi's. I am trying to transfer files from one Pi to the other using scp. I am trying to do this through Python because the program that will be transferring files is a python file.
below is the shell script I have for the SCP part (Blurred out the pass and IP):
#!/bin/sh
sshpass -p ######## scp test.txt pi#IP:/home/pi
and below is the Python Script that launches that Shell script.
import subprocess
subprocess.call(['./ssh.sh'])
print("DONE")
For some reason the python script doesnt kick back any errors and hits the print line but the file is not transferred. When i run the scp command outside of python the file transfers just fine. Am I doing something incorrect here?
****EDIT****
I cant even get Subprocess to work with this which is why i ended up using na shell script. Here is my attempt with Subprocess:
import subprocess
subprocess.call("sshpass -p ######## scp test.txt pi#IP:/home/pi")
print"DONE"
Again I get no errors, but the file is not transferred
****EDIT #2****
So I found out that because sshpass is being used, scp isnt prompting me to add the IP to known hosts, as a result the file simply isnt trnasferred at all. I need a way to add this acceptance into the script IE I ge the following if I launch the command without sshpass:
The authenticity of host 'IP (IP)' can't be established.
ECDSA key fingerprint is 13:91:24:8e:6f:21:98:1f:5b:3a:c8:42:7a:88:e9:91.
Are you sure you want to continue connecting (yes/no)?
I want to communicate to pass "yes\n" to this prompt as well as the password afterwards. Is this possible?

For the first query
You can use 'subprocess.popen' to get output(STDOUT) and error(STDERR) for the executed command.
import subprocess
cmd = 'sshpass -p ****** scp dinesh.txt root#256.219.210.135:/root'
p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
print "Output is ",out
print "Error is ",err
If you execute above code with wrong password, the you will get below output:
[root#centos /]# python code.py
Output is
Error is Permission denied, please try again.
In this case, if the file is successfully transferred, then there is no output.
If you execute command like 'ls -l' then output will be printed.
For your second query (****EDIT #2****)
Options are :
Password less SSH. Check this.
Pexpect

I found a much easier way of tackling all of this
sshpass -p ###### scp -o StrictHostKeyChecking=no test.txt pi#IP:/home/pi
The -o switch allows me to auto store the IP into known hosts thus I do not need to communicate with the shell at all. The interaction from Python to Shell works with that addition; Doing this solely through subprocess also works.

If you don't mind to try other approaches it worth to use SCPClient from scp import.

Connecting to remote linux machine via python subprocess using PuTTY or plink throws error

I am trying to execute few scripts in remote linux machine from windows host machine. I am hoping to achieve this using python subprocess +putty/plink.
When I try Putty or plink commands from windows cmd, it works fine. But if I try the same command using python subprocess, I get a lot of errors.
C:\Users\username>plink.exe username#machinename -pw password
Works fine. But when I try from python,
process = subprocess.Popen('plink.exe username#machinename -pw password'.split(),
env={'PATH':'C:\\Program Files (x86)\\PuTTY\\'},
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
Throws the following error.
Unable to open connection:
gethostbyname: unknown error'
process = subprocess.Popen("putty.exe -ssh -2 -l username -pw password -m C:\\script.sh machinename",
env={'PATH':'C:\\Program Files (x86)\\PuTTY\\'},
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT
,shell=True);
Unable to open connection:
gethostbyname: unknown error'
I tried subprocess.check_ouput too with no luck.
output = subprocess.check_output("putty.exe -ssh -2 -l username -pw password -m C:\\script.sh machinename", stderr=subprocess.STDOUT,shell=True)
Throws the following error
CalledProcessError: Command 'putty.exe -ssh -2 -l username -pw
password -m C:\script.sh machinename' returned non-zero exit status 1
Could this be a firewall issue?

I highly advise against using PuTT or in general every external program to connect to shh and then interface with pipes.
Using the python library paramiko this can be done much better.
For example:
# ... connect like one of the examples on github
stdin, stdout, stderr = client.exec_command('ls')
for line in stdout:
print '... ' + line.strip('\n')

python subprocess output on nohup

Trying to monitor the available physical disc space of a remote machine using a python script, which executes the df -h . command using subprocess.popen.
import subprocess
import time
command = 'ssh remoteserver "df -h ."'
while True:
proc = subprocess.Popen(command,shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output,err=proc.communicate()
print output
print err
time.sleep(60)
The script runs fine and prints the output to the terminal when run from command line
$> python2.7 script.py
Filesystem Size Used Avail Use% Mounted on
remoteserver:/home/user
555G 447G 109G 81% /home
The scripts does not produce any output or seems to be blocking when the script is started with nohup command.
$> nohup python2.7 script.py &
Would like the script to work and fetch the disc space of remote machine using the above script when started in nohup.

I'm not 100% sure of the underlying issue here, but when you invoke NOHUP in the shell, it's disconnected some of the STDIN/STDOUT from the terminal process, which I suspect it causing some of this interactions you're seeing.
Given that you're doing this from a remote machine, I'd actually recommend you look at using something like Fabric as a library to do what you're after. It's pretty straightforward, and does most of the handling of terminal sessions as well as closing things down nicely for you when you're complete.
something like:
from fabric import api
from fabric.api import env
import fabric
env.host_string = '%s#%s' % (username, remote_host)
env.disable_known_hosts = True
env.password = password
fabric.state.output['stdout'] = False
fabric.state.output['stderr'] = False
results = api.run('df -h')

You might try sending stdin=subprocess.PIPE to the subprocess command, then calling proc.stdin.close() on the next line, before the communicate() call. Or you can try changing the command to 'ssh remoteserver "df -h ." </dev/null'. Others report using FNULL = open(os.devnull, 'r') and passing in FNULL to the stdin= argument, but I'm not sure if you need to call FNULL.close() after or not.
SSH is most likely waiting for input for some reason when it is run from nohup. Perhaps it is unable to authenticate in the nohup environment and is asking for password input?
To make sure SSH is not waiting for input, try adding -o "BatchMode yes" to the ssh command and see if there are some clues in the output/error from the subprocess communicate call.

Python - String deformats when running via SSH

I need to run some bash commands via Fabric API (ssh).
I have the following String in my Python module:
newCommand = command + "'`echo -ne '\\015'"
When I print this string directly in Python the output is the expected:
command'`echo -ne '\015'
However, if I try to run this command via the Fabric API the command is somehow modified into this:
/bin/bash -l -c "command'\`echo -ne '\015'"
Notice the '\' before 'echo'. Why is this happenning? The '\' is breaking my command and I can't successfuly run the command.
ps: The prefix "/bin/bash -l -c" is expected since that's how Fabric works with SSH

This is not a valid shell command:
command'`echo -ne '\015'
Even if you add the missing backtick and single quote, it's nothing like writing "command" and pressing enter.
The context your command will be run in is basically what you'd get if you'd ssh and paste a command:
clientprompt$ ssh host
Welcome to Host, User
hostprompt$ <COMMAND HERE>
You should focus your efforts on finding a single command that does what you want, and not a series of keypresses that you could write to do it (that's not how ssh works).