I have to connect to a sybase database and run a simple select query using python script
On my server isql command can run only from sybase bin directory, so i have to cd to that directory before firing the query.
---------------------------Edited-----------------------------
Uptill now i'm able to do this:-
#!/usr/bin/python
import subprocess
path = "path/to/sybase/bin"
os.chdir(path)
arguments = ['./isql',"-S server_name", "-U user", "-P password", "-D database","""<<EOF
SELECT * FROM sometable
go
EOF"""]
ps = subprocess.Popen(arguments)
out = ps.communicate()
print out
The errors are just out of my understanding capability :(
Traceback (most recent call last):
File "./test_db.py", line 8, in ?
ps = subprocess.Popen(arguments)
File "/usr/lib64/python2.4/subprocess.py", line 542, in __init__
errread, errwrite)
File "/usr/lib64/python2.4/subprocess.py", line 975, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I'm able to do this outside my python script using isql command on my unix terminal
how can i use isql in python subprocess module?
There is a particular Popen argument for that: cwd, as mentioned here. Provide your command using an array and then the cwd parameter with where the command must be executed:
subprocess.Popen(['ls', '-l'], cwd="/path/to/folder")
Popen only takes one args argument, for the command to run. You could try calling a shell with both the cd and isql commands as arguments, but changing the working directory from python is probably simpler
For the former approach:
subprocess.Popen('/bin/sh -c "cd /path/to/... && isql -arg1..'...)
for the latter:
os.chdir('/path/to...')
subprocess.Popen('isql -arg1..'...)
Try:
import os
import subprocess
os.chdir('/path/to/sybase/bin')
if os.path.exists('isql') or os.path.exists(os.path.join('/path/to/sybase/bin', 'isql')):
ps = subprocess.Popen('isql -S %s -U %s -P %s -D %s <<EOF SELECT * FROM sometable EOF' % (server,user,passwd,database), stdout=subprocess.PIPE, shell=True)
out, err = ps.communicate()
else:
print "isql does not exists in this folder"
I am not super experienced with subprocess but this is how I generally use it on the odd occasion. Hopefully someone else can give a better answer/explanation.
Edit: removed the square brackets to remove confusion.
i know it's been long but just wanted to close this question
from subprocess import Popen, PIPE
from textwrap import dedent
isql = Popen(['./isql', '-I', '/app/sybase/...',
'-S', mdbserver,
'-U', muserid,
'-P', password, ...,
'-w', '99999'], stdin=PIPE, stdout=PIPE, cwd=sybase_path)
output = isql.communicate(dedent("""\
SET NOCOUNT ON
{}
go
""".format(User_Query)))[0]
Related
I would like to run a command in python using subprocess.run
I would like to switch the working directory JUST for the execution of this command.
Also, I need to record the output and the return code.
Here is the code I have:
import subprocess
result = subprocess.run("echo \"blah\"", cwd=directory, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
but this only returns
FileNotFoundError: [Errno 2] No such file or directory: 'echo "Running ls -la" && ls -la'
I also tried using the following arguments:
subprocess.run(["echo", "\"blah\""], cwd=directory, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Like Jean-François Fabre said, the solution is to add "shell=True" to the call
import subprocess
result = subprocess.run("echo \"blah\"", cwd=directory, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
shell=True seems to tell subprocess to use the string as a command.
I am invoking shell script using os.execvp() in python. my shell script has some echo statements whcih I want to redirect in file.
Here is what I am trying:
cmd = "/opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log"
cmdline = ["/bin/sh", cmd]
os.execvp(cmdline[0], cmdline)
Below is the error I am getting:
Error: /bin/sh: /opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log: No such file or directory
Can any one help?
This is happening because you are passing this entire string as if it were the program name to execute:
"/opt/rpm/rpm_upgrade.sh >& /opt/rpm/upgrader.log"
The easy way to fix this is:
cmdline = ["/bin/sh", "/opt/rpm/rpm_upgrade.sh",
">&", "/opt/rpm/upgrader.log"]
os.execvp(cmdline[0], cmdline)
Now sh will receive three arguments rather than one.
Or you can switch to the more full-featured subprocess module, which lets you redirect output in Python:
import subprocess
with open("/opt/rpm/upgrader.log", "wb") as outfile:
subprocess.check_call(["/opt/rpm/rpm_upgrade.sh"], shell=True,
stdout=outfile, stderr=subprocess.STDOUT)
I'm trying to execute a command using Popen.
The command uses some PostGIS/Postgresql utility programs to upload a raster file to a database and works when executed from the command line. It uses unix style pipes to chain 2 commands and looks like this:
"C:\\Program Files\\PostgreSQL\\9.2\\bin\\raster2pgsql.exe" -d -I -C -e -Y -F -t 128x128 "C:\\temp\\SampleDTM\\SampleDTM.tif" test | "C:\\Program Files\\PostgreSQL\\9.2\\bin\\psql.exe" -h localhost -p 5432 -d adr_hazard -U postgres
When using within Python, I make it a string with the ' codes:
command = '"C:\\Program Files\\PostgreSQL\\9.2\\bin\\raster2pgsql.exe" -d -I -C -e -Y -F -t 128x128 "C:\\temp\\SampleDTM\\SampleDTM.tif" test | "C:\\Program Files\\PostgreSQL\\9.2\\bin\\psql.exe" -h localhost -p 5432 -d adr_hazard -U postgres'
attempting to execute it results in an error:
p = subprocess.Popen(command)
ERROR: Unable to read raster file: test
The error seems like the command was not parsed correctly (it is interpreting the wrong argument as the raster file)
Am I using Popen wrong?
Your command uses pipe |. It requires a shell:
p = subprocess.Popen(command, shell=True)
The command itself as far as I can tell looks ok.
It's not necessary to use shell=True to achieve this with pipes. This can be done programmatically with pipes even where concern about insecure input is an issue. Here, conn_params is a dictionary with PASSWORD, NAME (database name), USER, and HOST keys.
raster2pgsql_ps = subprocess.Popen([
'raster2pgsql', '-d', '-I', '-C', '-e', '-Y', '-F', '-t', '128x128',
'C:\\temp\\SampleDTM\\SampleDTM.tif',
'test'
], stdout=subprocess.PIPE)
# Connection made using conninfo parameters
# http://www.postgresql.org/docs/9.0/static/libpq-connect.html
psql_ps = subprocess.check_output([
'psql',
'password={PASSWORD} dbname={NAME} user={USER} host={HOST}'.format(**conn_params),
], stdin=raster2pgsql_ps.stdout)
The following worked for me on Windows, while avoiding shell=True
One can make use of Python's fstring formatting to make sure the commands will work in windows.
Please note that I used shp2pgsql but it should be a very similar process for raster2pgsql.
Parameters for the shp2pgsql: srid is the coordinate system of the shape file, filename is the path to the shape file to be imported, tablename is the name you'd like to give your table.
import os
import subprocess
shp2pgsql_binary = os.path.join(pgsql_dir, "bin", "shp2pgsql")
psql_binary = os.path.join(pgsql_dir, "bin", "psql")
command0 = f'\"{shp2pgsql_binary}\" -s {srid} \"{filename}\" {tablename}'
command1 = f'\"{psql_binary}\" \"dbname={databasename} user={username} password={password} host={hostname}\"'
try:
shp2pgsql_ps = subprocess.Popen(command0, stdout=subprocess.PIPE)
psql_ps = subprocess.check_output(command1, stdin=shp2pgsql_ps.stdout)
except:
sys.stderr.write("An error occurred while importing data into the database, you might want to \
check the SQL command below:")
sys.stderr.write(command)
raise
To adpat to raster2pgsql, you just need to modify the string in command0, e.g. -s {srid} becomes -d -I -C -e -Y -F -t 128x128. The string for command1 can remain the same.
PIPE = subprocess.PIPE
pd = subprocess.Popen(['"C:\\Program Files\\PostgreSQL\\9.2\\bin\\raster2pgsql.exe", '-d', '-I', '-C', '-e', '-Y', '-F', '-t', '128x128', "C:\\temp\\SampleDTM\\SampleDTM.tif", 'test'],
stdout=PIPE, stderr=PIPE)
stdout, stderr = pd.communicate()
It will be better to use subprocess.Popen in this way:
proc = subprocess.Popen(['"C:\\Program Files\\PostgreSQL\\9.2\\bin\\raster2pgsql.exe"', '-d', '-I', '-C', '-e', '-Y', '-F', '-t', '128x128', '"C:\\temp\\SampleDTM\\SampleDTM.tif"', 'test', '|', '"C:\\Program Files\\PostgreSQL\\9.2\\bin\\psql.exe"', '-h', 'localhost', '-p', '5432', '-d', 'adr_hazard', '-U', 'postgres'], shell = True, stdout = subprocess.pipe, stderr = subprocess.STDOUT)
proc.wait()
result = proc.stdout.readlines()#if you want to process the result of your command
proc.kill()
B.T.W, it's good to format the path first, use:
path = os.path.normalpath("C:\\Program Files\\PostgreSQL\\9.2\\bin\\raster2pgsql.exe")
this will avoid some path problems for different OS platform.
The shell = True is important if you want to execute your command just like executing it in local shell.
Hope will help you.
I want to call a subprocess to backup mysql database.
A command line which ran fine in the terminal (and created a file named mydatabase.sql) is:
mysqldump -uroot -ppassword --add-drop-database --database mydatabase > mydatabase.sql
Now the code to be ran by python to call a subprocess:
args = shlex.split('mysqldump -uroot -ppassword --add-drop-database --database mydatabase > mydatabase.sql')
subprocess.check_call(args)
The exeption is raised (no file created):
Traceback (most recent call last):
File "<pyshell#29>", line 1, in <module>
subprocess.check_call(args)
File "/usr/lib/python3.2/subprocess.py", line 485, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['mysqldump', >'-uroot', '-ppassword', '--add-drop-database', '--database', >'mydatabase', '>', 'mydatabase.sql']' returned non-zero exit status 2
I have tried different ways, but they still don't work:
args = shlex.split('/opt/lampp/bin/mysqldump -uroot -ppassword --add-drop-database --database mydatabase > mydatabase.sql')
subprocess.check_call(args)
or
args = shlex.split('/opt/lampp/bin/mysqldump -uroot -ppassword --add-drop-database --database mydatabase > mydatabase.sql')
subprocess.Popen(args)
I also tried with shell=True or or shell=False. In both cases, they still don't work.
I have read the docs, google for the answer for my problem, but I haven't got a clue how to show my problem. stackoverflow is probably my last hope.
the problem here is the way you're redirecting the output.
if you pass the command as a list of arguments, then ">" will always be interpreted as a literal >, no matter if you use shell=True or shell=False
if you pass the command as a single string, then it hould work, but only if you have shell=True.
the best way do do what you want would be to redirect the output to a file directly from python:
args = shlex.split('/opt/lampp/bin/mysqldump -uroot -ppassword --add-drop-database --database mydatabase')
output = open("mydatabase.sql", "w")
subprocess.Popen(args, stdout=output)
The problem may be the shell redirection. If you run with shell=True don't use shlex.split. In other words, try:
args = '/opt/lampp/bin/mysqldump -uroot -ppassword --add-drop-database --database mydatabase > mydatabase.sql'
subprocess.Popen(args,shell=True)
Of course, the safer solution would be to remove the shell redirection, use shlex.split on the arguments (without shell=True), and use subprocess.PIPE to capture the output (which you can then send to a file or do whatever you want with in your program)
For example:
args = '/opt/lampp/bin/mysqldump -uroot -ppassword --add-drop-database --database mydatabase'
p=psubprocess.Popen(shlex.split(args),shell=False,stdout=subprocess.PIPE)
p.wait()
returnvalue=p.returncode
data=p.stdout.read()
with open('mydatabase.sql','w') as f:
f.write(data)
...
If you don't want to do anything with the data in your program, you can do the redirection a little more easily as described by mata.
I'm setting up a program to connect my computer to our schools proxy and currently have something like this:
import subprocess
import sys
username = 'fergus.barker'
password = '*************'
proxy = 'proxy.det.nsw.edu.au:8080'
options = '%s:%s#%s' % (username, password, proxy)
subprocess.Popen('export http_proxy=' + options)
But upon running I get:
Traceback (most recent call last):
File "school_proxy_settings.py", line 19, in <module>
subprocess.Popen('export http_proxy=' + options)
File "/usr/lib/python2.6/subprocess.py", line 621, in __init__
errread, errwrite)
File "/usr/lib/python2.6/subprocess.py", line 1126, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Why is this happening please guys?
The problem is that export is not an actual command or file. It is a built-in command to shells like bash and sh, so when you attempt a subprocess.Popen you will get an exception because it can not find the export command. By default Popen does an os.execvp() to spawn a new process, which would not allow you to use shell intrinsics.
You can do something like this, though you have to change your call to Popen.
http://docs.python.org/library/subprocess.html
You can specify shell=True to make it use shell commands.
class subprocess.Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)
On Unix, with shell=True: If args is a string, it specifies the command string to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:
Popen(['/bin/sh', '-c', args[0], args[1], ...])
export is not a separate binary on your system, it is actually just a command within the shell itself. For example, try using which rm on your system. You'll probably see something like:
[21:16:28] ~ $ which rm
/bin/rm
Now try using which export. You'll get something like:
[21:16:37] ~ $ which export
/usr/bin/which: no export in (/usr/lib/qt-3.3/bin:/usr/kerberos/sbin:
/usr/kerberos/bin:/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:
/usr/local/sbin:/usr/sbin:/sbin:/home/carter/bin)
So you can't actually invoke an export process/subprocess by default. You may want to look at os.putenv() and os.environ() instead.