We Keep Coding

Executing password-protected psql from python

I need to execute the following command from Python on Windows:
psql -h localhost -p 5432 -U postgres -f script.sql db_name
The above script works fine when ran from git bash / powershell. After entering the script in a terminal, I need to provide a password to confirm it (similar to when using sudo).
How can I do that? I keep finding solutions that I think are linux-based.
How do I do it on Windows? I have tried many variations of solutions involving subprocess, i.e:
import subprocess
p2 = subprocess.Popen(
'psql -h localhost -p 5432 -U postgres -f script.sql db_name',
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True)
print('this will print')
sudo_prompt = p2.communicate('THE_PASSWORD' + '\n')[1]
print('this will not')

A better option (more secure) than invoking psql with explicit mention of your password is to have a .pgpass file as described in the docs file (and keep it protected e.g. chmod 600 ~/.pgpass). This keeps your password out of the list of running processes.
On Windows:
On Microsoft Windows the file is named %APPDATA%\postgresql\pgpass.conf (where %APPDATA% refers to the Application Data subdirectory in the user's profile).

Paramiko sed find and replace wont pass through

I am currently working on a script where when I launch an EC2 instance, I send a paramiko command to rename the host name. Because this is a custome AMI, I cannot use the AWS Boto3 CLI to do it, so I need to do it via an SSH command.
The problem I am running into, is Paramiko seems to fail at passing my specific command. It will pass other commands just fine, but I am assuming I am running into some sort of limitation of either paramiko or python and cannot seem to troubleshoot it. This is for a RHEL instance, so renaming the Network file is the only way I can think to do this.
If I run the command, as is, through the terminal of the host, it works. So something between paramiko and this command seems to be the blocker.
Here is my sample script t hat should work, but seems to fail at running the command.
#!/usr/bin/env python
import boto3
import time
import subprocess
import paramiko
import StringIO
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
c.connect(hostname = '12.34.56.78', username = "username", key_filename='''/Users/mallachar/Downloads/testkey.pem''' )
stdin , stdout, stderr = c.exec_command('sudo sed -i -E "s/^HOSTNAME.*/HOSTNAME=testhost.company/" /etc/sysconfig/network')
print stdout.read()
print stderr.read()
c.close
Here is me printing stdout and stderr
sudo: sorry, you must have a tty to run sudo

Pretty simple, I had to add this to the command.
get_pty=True
so
stdin , stdout, stderr = c.exec_command('sudo sed -i -E "s/^HOSTNAME.*/HOSTNAME=testhost.company/" /etc/sysconfig/network',get_pty=True)

Connecting to remote linux machine via python subprocess using PuTTY or plink throws error

I am trying to execute few scripts in remote linux machine from windows host machine. I am hoping to achieve this using python subprocess +putty/plink.
When I try Putty or plink commands from windows cmd, it works fine. But if I try the same command using python subprocess, I get a lot of errors.
C:\Users\username>plink.exe username#machinename -pw password
Works fine. But when I try from python,
process = subprocess.Popen('plink.exe username#machinename -pw password'.split(),
env={'PATH':'C:\\Program Files (x86)\\PuTTY\\'},
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
Throws the following error.
Unable to open connection:
gethostbyname: unknown error'
process = subprocess.Popen("putty.exe -ssh -2 -l username -pw password -m C:\\script.sh machinename",
env={'PATH':'C:\\Program Files (x86)\\PuTTY\\'},
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT
,shell=True);
Unable to open connection:
gethostbyname: unknown error'
I tried subprocess.check_ouput too with no luck.
output = subprocess.check_output("putty.exe -ssh -2 -l username -pw password -m C:\\script.sh machinename", stderr=subprocess.STDOUT,shell=True)
Throws the following error
CalledProcessError: Command 'putty.exe -ssh -2 -l username -pw
password -m C:\script.sh machinename' returned non-zero exit status 1
Could this be a firewall issue?

I highly advise against using PuTT or in general every external program to connect to shh and then interface with pipes.
Using the python library paramiko this can be done much better.
For example:
# ... connect like one of the examples on github
stdin, stdout, stderr = client.exec_command('ls')
for line in stdout:
print '... ' + line.strip('\n')

How to execute remote pysftp commands under a specific shell

I use python module pysftp to connect to remote server. Below you can see python code :
import pysftp
import sys
import sqr_common
srv = pysftp.Connection(host="xxxxxx", username="xxxx",
password="xxxxx")
command = "/usr/bin/bash"
command2="APSHOME=/all/aps/msc_2012; export APSHOME; "
srv.execute(command)
srv.execute(command2)
srv.close()
Problem is that command /usr/bin/bash is an infinite process , so my script will never be executed. Can anyone help me how to choose shell on remote server for example bash and execute command in bash on remote server?? Is there any pysftp function that allows me chosing shell??

try this
/usr/bin/bash -c "APSHOME=/all/aps/msc_2012; export APSHOME; "

This problem is not specific to Python, but more like how to execute commands under specific shell.
If you need to run only single command you can run using bash -c switch
bash -c "echo 123"
You can run multiple commands ; separated
bash -c "echo 123 ; echo 246"
If you need to many commands under a specific shell, remotely create a shell script file (.bash file) an execute it
bash myscript.bash

How to continue the execution of python script as a different user

I want to execute a python script which executes the following command in order:
sudo su - postgres #change user to postgres
psql #enter the psql command promt from
create user user_name with password 'mypassword'; #
create database voylla_development with encoding = 'utf8'; #all the 3 commands are to be executed in psql command prompt
grant all on database voylla_development to user_name; #
exit #psql prompt
exit #postgres user
cat <backup_file_name> | zcat - | PGPASSWORD=mypassword psql -d voylla_development -h localhost -p 5432 -U user_name
I tried using subprocess and os.system():
cmd='sudo -u postgres psql'
args = shlex.split(cmd)
p=subprocess.Popen(args)
p.wait()
cmd1='psql'
args1 = shlex.split(cmd1)
p=subprocess.Popen(args1)
p.wait()
##and so on for each command
But the script stops after I login as postgres user. How can I continue the script after user change?
Thanks
EDIT: using psycopg2 helped the cause

When you create a new Popen() object, you start a new program. You are not communicating with an open psql shell.
You either have to drive psql directly by setting stdin to subprocess.PIPE, or, much easier, use pexpect to drive the psql shell:
import pexpect
psql = pexpect.spawn('psql')
psql.expect('=>') # wait for the prompt
psql.send('create user user_name with password 'mypassword';')
# etc.