We Keep Coding

Python Postgres query with transaction block over single connection

Currently I have two separate statements being passed through to Postgres (Greenplum).
1. Truncates a table
2. loads data using \copy
myStr="export PGPASSWORD=" + dbPass + "; psql -h " + dbHost + " -p " + dbPort + " -d " + dbName + " -U " + dbUser + " -c " + "\"" + "truncate table " + dbTable + ";\""
print(myStr)
subprocess.call(myStr,shell=True)
myStr="export PGPASSWORD=" + dbPass + "; psql -h " + dbHost + " -p " + dbPort + " -d " + dbName + " -U " + dbUser + " -c " + "\"" + "\\" + "copy " + dbTable + " from " + "'" + csvfile + "' with " + copyOpts + ";" + "select count(*) from " + dbTable + ";\""
print(myStr)
subprocess.call(myStr,shell=True)
Sometimes the load has errors but the truncate already happened, so I'm trying to run the two statements in one connection so I can put a transcation block (BEGIN ... COMMIT;) that way if the data load fails it will rollback to before the truncate happens.
I tried the below method:
myStr="export PGPASSWORD=" + dbPass + "; psql -h " + dbHost + " -p " + dbPort + " -d " + dbName + " -U " + dbUser + " -c " + "\"" + "truncate table " + dbTable + ";" + " \\" + "copy " + dbTable + " from " + "'" + csvfile + "' with " + copyOpts + ";" + "select count(*) from " + dbTable + ";\""
print(myStr)
Which resolves to the command:
export PGPASSWORD=abcde;
psql -h abcde.testserver.corp
-p 5432 -d namem -U username -c
"truncate table schema.example;
\copy schema.example from
'/home/testing/schema/schema.example_export.csv'
with header null as '' escape 'off' delimiter E',' ;
select count(*) from schema.example;"
However I am getting the error:
ERROR: syntax error at or near "\"
I believe this is due to the \ commands have to be on a separate line.
Is there a way to split the command into separate lines so I can execute ll the commands in a single connection?

The problem is that you can't separate backslash commands from other commands if you are using the -c option. You can send your commands via STDIN to psql using echo:
export PGPASSWORD=abcde;
echo "truncate table schema.example;
\copy schema.example from '/home/testing/schema/schema.example_export.csv' with header null as '' escape 'off' delimiter E',' ;
select count(*) from schema.example;" | psql -h abcde.testserver.corp -p 5432 -d namem -U username
That's a little bit clumsy. It's better to use subprocess.Popen
theCommand = """truncate table schema.example;
\copy schema.example from
'/home/testing/schema/schema.example_export.csv'
with header null as '' escape 'off' delimiter E',' ;
select count(*) from schema.example;"""
theProcess = subprocess.Popen("psql -h abcde.testserver.corp -p 5432 -d namem -U username",
stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
theOutput, theErrors = theProcess.communicate(input = theCommand)
But the best way should be avoiding shell commands and using a database adaptor like PyGreSQL.

python IndexError: list index out of range on arp_scan script

I'm getting this error when running my code:
prefix = ip.split('.')[0] + '.' + ip.split('.')[1] + '.' + ip.split('.')[2] + '.'
IndexError: list index out of range
I didn't write the script myself. I need to perform an arp scan on intranet and get as output every IP-addr that responds. My code is this:
#!/usr/bin/python
import logging
import subprocess
logging.getLogger("scapy.runtime").setLevel(logging.ERROR)
from scapy.all import *
if len(sys.argv) != 2:
print("Usage - ./arp_disc.py [interface]")
print("Example - ./arp_disc.py eth0")
print("Example will perform an ARP scan of the local subnet to which eth0 is assigned")
sys.exit()
interface = str(sys.argv[1])
ip = subprocess.check_output("ifconfig " + interface + " | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1", shell=True).strip()
prefix = ip.split('.')[0] + '.' + ip.split('.')[1] + '.' + ip.split('.')[2] + '.'
for addr in range(0,254):
answer=sr1(ARP(pdst=prefix+str(addr)),timeout=1,verbose=0)
if answer == None:
pass
else:
print prefix+str(addr)

Use the following instead:
ip = subprocess.check_output("ifconfig " + interface + " | grep 'inet' | cut -d ':' -f 2 | cut -d '' -f 1", shell=True).strip()
ip= ip[5:]

subprocess, cannot concatenate 'str' and ... objects

I have code like:
if len(sys.argv)>1:
host = sys.argv[1]
number = sys.argv[2] if len(sys.argv)>2 else "1"
size = sys.argv[3] if len(sys.argv)>3 else "56"
timeout = sys.argv[4] if len(sys.argv)>4 else "1"
proc = subprocess.Popen(["ping \"" + host + "\" -c \"" + number + "\" -s \"" + size + "\" -W \"" + timeout + "\" 2>/dev/null | grep packets | sed 's/[^0-9,%]*//g'"], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
print "address=\"" + host + "\" data=" + out
I need to split out to list. How I can achieve this?
Everything I try causes error: cannot concatenate 'str' and ... objects
Like when I tried:
...
res=list(out)
print "address=\"" + host + "\" data=" + res
I got error:
TypeError: cannot concatenate 'str' and 'list' objects

To unite strings and lists you first must make a string out of the list:
res=list(out)
print "address=\"" + str(host) + "\" data=" + str(res)

Can't quit while running long Popen command

I am running the below code which runs a command line application which runs for about 40 mins. While this is running my QUIT button is not accessible so I am unable to quit the running application. The below code and the button are both seated in their own def. Any ideas as to how I can get a working quit button while my application is running?
command1 = transporterLink + " -m verify -f " + indir1 + " -u " + username + " -p " + password + " -o " + logPath + " -s " + provider1 + " -v eXtreme"
master, slave = pty.openpty()
process = Popen(command1, shell=True, stdin=PIPE, stdout=slave, stderr=slave, close_fds=True)
stdout = os.fdopen(master)
global subject
subject = "Test"
while True:
wx.Yield()
line = stdout.readline()
line = line.rstrip()
print line
if "Returning 1" in line:
result1 = "Verify FAILED!"
subject = "FAILED! - "
self.sendEmail(self)
break
if "Returning 0" in line:
result1 = "Verify PASSED!"
subject = "PASSED! - "
self.sendEmail(self)
break

stdout.readline is blocking until there is something in stdout. You could use select module's poll
command1 = transporterLink + " -m verify -f " + indir1 + " -u " + username + " -p " + password + " -o " + logPath + " -s " + provider1 + " -v eXtreme"
master, slave = pty.openpty()
process = Popen(command1, shell=True, stdin=PIPE, stdout=master, stderr=slave, close_fds=True)
stdout = os.fdopen(master)
import select
q = select.poll()
q.register(stdout,select.POLLIN)
global subject
subject = "Test"
while True:
wx.Yield()
events = q.poll(0)
for fd, flags in events:
assert(fd == stdout.fileno())
line = stdout.readline()
print line
if "Returning 1" in line:
result1 = "Verify FAILED!"
subject = "FAILED! - "
self.sendEmail(self)
sys.exit(0)
if "Returning 0" in line:
result1 = "Verify PASSED!"
subject = "PASSED! - "
self.sendEmail(self)
sys.exit(0)

Python print adds new line and 0

Here's my issue. I'm trying to ssh to Cisco devices and pull information off. When I run my code, the print statement adds a new line with a 0 in it to the bottom of the output. Here is the output of the code followed by the output of the plink CLI input:
C:\Python30>python PLINKSSHtest.py
Enter your username: josh
Password:
plink -pw nowayjose -ssh nope#1.1.1.1 "show run | inc hostname"
hostname net-R2
0 <------------MY ISSUE
C:\Python30>plink -pw nowayjose -ssh nope#1.1.1.1 "show run | inc hostname"
hostname net-R2
<------------WHAT I EXPECT
Here is my code:
def read_dev():
# Print statement here for debugging
print ("plink -pw " + password + " -ssh " + user + "#" + HOST + " " + command)
cur_dev = os.system("plink -pw " + password + " -ssh " + user + "#" + HOST + " " + command)
return(cur_dev)
HOST = None
user = input("Enter your username: ")
password = getpass.getpass()
command = '"show run | inc hostname"'
HOST = '1.1.1.1'
print (read_dev())

cur_dev is getting the result code returned by the plink command, which is 0. Your read_dev function returns this code, so print(read_dev()) prints the 0.
Just say read_dev() instead of print(read_dev()).

It doesn't "print zero". It prints cur_dev which is returned by read_dev function, which happens to be zero. And it does so, because you told it to. Remove print function and it won't print anything."

If you want to explicitly set the exit code use sys.exit(cur_dev). Simply using a return value from a function does not do what you want it to.