I want to call a python script, script1.py, which takes a regex string as argument. This script then connects to another server and calls a script2.py.
script1.py:
regex = sys.argv[1]
log = os.popen('ssh otherserver python /home/log/scripts/script2.py \"%s\"' % regex)
for line in log:
print line.strip()
log.close()
script2.py:
regex = re.compile(sys.argv[1])
for line in ['[1','[ 2]',' a ']:
if regex.search(line):
print line.strip()
The problem I'm running into is that parentheses are breaking the script when run through SSH.
This works:
[foo#bar scripts]$ python script1.py ".*"
[1
[ 2]
a
This doesn't:
[foo#bar scripts]$ python script1.py "(.*)"
bash: -c: line 0: syntax error near unexpected token ('
bash: -c: line 0:python /home/log/scripts/script2.py (.*)'
Why isn't the argument in the second example escaped? This does not occur if I call script2.py locally without SSH.
Update with solution:
Turns out the solution was using subprocess, but passing arguments didn't work, so I had to format the last argument to include the escaped regex:
log = subprocess.check_output(['ssh', 'eu1', 'nice', 'python', '/home/log/scripts/script2.py \"%s\"' % regex])
print log.strip()
Try this:
import subprocess
output = subprocess.check_output(['ssh', 'otherserver', 'python', '/home/log/scripts/script2.py', regex])
Related
I have completed a py file as follow:
import re
file = open('/Path/text1.txt')
word = 'summer flowers'
try :
flag = 0
for line in file :
lines = line.lower()
x=re.findall('.*'+word+'\s.*',lines)
if len(x)>0:
flag =1
print line
else:
flag = flag
if flag == 0:
print 'No match!'
except:
print 'No enough arguments!'
I have saved above py file as test1.py. My question is that: How to type above code in the cmd line?
e.g. I hope to input the code:
" test1.py 'summer flowers', text1.txt "
in the cmd to execute above code
where test1.py is the file name of the py file, 'summer flowers' is the key word I want to search and match in the txt file and text1.txt is the txt file name.
import re
import sys
file = sys.argv[2]
word = sys.argv[1]
try :
flag = 0
for line in file :
lines = line.lower()
x=re.findall('.*'+word+'\s.*',lines)
if len(x)>0:
flag =1
printline
else:
flag = flag
if flag == 0:
print 'No match!'
except:
print 'No enough arguments!'
Use sys to deal with command line stuff. sys.argv returns a list of the arguments from the command line and since you wanted to run it in cmd line use python script_name.py keyword filename don't need quotes for single words or commas. Also note sys.argv[0] is reserved for the file name so sys.argv returns ['script_name.py', 'keyword', 'filename'] if you ran the command above.
Edit:
Like the comment says if you want a phrase instead of a word you can use quotes in the commend line so:
python script_name.py "my phrase" filename
will return in sys.argv as ['script_name.py', 'my phrase', 'filename']
Using the sys module. Specifically, sys.argv
From the docs:
sys.argv
The list of command line arguments passed to a Python script. argv[0] is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'. If no script name was passed to the Python interpreter, argv[0] is the empty string.
Following is the code which I am trying to execute using python
from subprocess import Popen, PIPE
cmd = 'if (-e "../a.txt") then \n ln -s ../a.txt . \n else \n echo "file is not present " \n endif'
ret_val = subprocess.call(cmd,shell="True")
When executed gives following error message
/bin/sh: -c: line 5: syntax error: unexpected end of file
In sh scripts, if is terminated with fi, not endif.
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html
Or just write the darn code in Python:
import os
if os.path.exists('../a.txt'):
print 'Exists'
os.symlink('../a.txt', 'a.txt')
else:
print 'Does not exist'
os.path.exists ()
os.symlink()
If you really want to run tcsh commands, then:
import shlex
import subprocess
args = ['tcsh', '-c'] + shlex.split(some_tcsh_command)
ret = suprocess.call(args)
I am having hard time parsing the arguments to subprocess.Popen. I am trying to execute a script on my Unix server. The script syntax when running on shell prompt is as follows:
/usr/local/bin/script hostname = <hostname> -p LONGLIST. No matter how I try, the script is not running inside subprocess.Popen
The space before and after "=" is mandatory.
import subprocess
Out = subprocess.Popen(['/usr/local/bin/script', 'hostname = ', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
The above does not work.
And when I use shell=False, I get OSError: [Errno 8] Exec format error
OSError: [Errno 8] Exec format error can happen if there is no shebang line at the top of the shell script and you are trying to execute the script directly. Here's an example that reproduces the issue:
>>> with open('a','w') as f: f.write('exit 0') # create the script
...
>>> import os
>>> os.chmod('a', 0b111101101) # rwxr-xr-x make it executable
>>> os.execl('./a', './a') # execute it
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/os.py", line 312, in execl
execv(file, args)
OSError: [Errno 8] Exec format error
To fix it, just add the shebang e.g., if it is a shell script; prepend #!/bin/sh at the top of your script:
>>> with open('a','w') as f: f.write('#!/bin/sh\nexit 0')
...
>>> os.execl('./a', './a')
It executes exit 0 without any errors.
On POSIX systems, shell parses the command line i.e., your script won't see spaces around = e.g., if script is:
#!/usr/bin/env python
import sys
print(sys.argv)
then running it in the shell:
$ /usr/local/bin/script hostname = '<hostname>' -p LONGLIST
produces:
['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']
Note: no spaces around '='. I've added quotes around <hostname> to escape the redirection metacharacters <>.
To emulate the shell command in Python, run:
from subprocess import check_call
cmd = ['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']
check_call(cmd)
Note: no shell=True. And you don't need to escape <> because no shell is run.
"Exec format error" might indicate that your script has invalid format, run:
$ file /usr/local/bin/script
to find out what it is. Compare the architecture with the output of:
$ uname -m
I will hijack this thread to point out that this error may also happen when target of Popen is not executable. Learnt it hard way when by accident I have had override a perfectly executable binary file with zip file.
Have you tried this?
Out = subprocess.Popen('/usr/local/bin/script hostname = actual_server_name -p LONGLIST'.split(), shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Edited per the apt comment from #J.F.Sebastian
It wouldn't be wrong to mention that Pexpect does throw a similar error
#python -c "import pexpect; p=pexpect.spawn('/usr/local/ssl/bin/openssl_1.1.0f version'); p.interact()"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/pexpect.py", line 430, in __init__
self._spawn (command, args)
File "/usr/lib/python2.7/site-packages/pexpect.py", line 560, in _spawn
os.execv(self.command, self.args)
OSError: [Errno 8] Exec format error
Over here, the openssl_1.1.0f file at the specified path has exec command specified in it and is running the actual openssl binary when called.
Usually, I wouldn't mention this unless I have the root cause, but this problem was not there earlier. Unable to find the similar problem, the closest explanation to make it work is the same as the one provided by #jfs above.
what worked for me is both
adding /bin/bash at the beginning of the command or file you are
facing the problem with, or
adding shebang #!/bin/sh as the first line.
for ex.
#python -c "import pexpect; p=pexpect.spawn('/bin/bash /usr/local/ssl/bin/openssl_1.1.0f version'); p.interact()"
OpenSSL 1.1.0f 25 May 2017
If you think the space before and after "=" is mandatory, try it as separate item in the list.
Out = subprocess.Popen(['/usr/local/bin/script', 'hostname', '=', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
I'm attempting to execute a command over SSH, but bash on the other end doesn't think it's escaped properly.
Here, self._client is a paramiko.SSHClient object; args is a list of arguments, the command to execute.
def run(self, args, stdin=None, capture_stdout=False):
"""Runs a command.
On success, returns the output, if requested, or None.
On failure, raises CommandError, with stderr and, if captured, stdout,
as well as the exit code.
"""
command = ' '.join(_shell_escape(arg) for arg in args)
print('About to run command:\n {}'.format(command))
print('About to run command:\n {!r}'.format(command))
channel = self._client.get_transport().open_session()
channel.exec_command(command)
_shell_escape:
_SHELL_SAFE = _re.compile(r'^[-A-Za-z0-9_./]+$')
def _shell_escape(s):
if _SHELL_SAFE.match(s):
return s
return '\'{}\''.format(s.replace('\'', '\'\\\'\''))
I'm attempt to run some Python through this. On stderr, I get back:
bash: -c: line 5: unexpected EOF while looking for matching `''
bash: -c: line 6: syntax error: unexpected end of file
The output from the two print statements:
About to run command:
python -c 'import os, sys
path = sys.argv[1]
if sys.version_info.major == 2:
path = path.decode('\''utf-8'\'')
entries = os.listdir(path)
out = b'\'''\''.join(e.encode('\''utf-8'\'') + b'\'''\'' for e in entries)
sys.stdout.write(out)
' .
About to run command:
"python -c 'import os, sys\npath = sys.argv[1]\nif sys.version_info.major == 2:\n path = path.decode('\\''utf-8'\\'')\nentries = os.listdir(path)\nout = b'\\'''\\''.join(e.encode('\\''utf-8'\\'') + b'\\''\x00'\\'' for e in entries)\nsys.stdout.write(out)\n' ."
If I copy and paste the output of command, and paste it into bash, it executes, so it really does appear to be properly escaped. My current understanding is that SSH, on the other end, will take command, and run [my_shell, '-c', command].
Why is bash erroring on that command?
The input contains an embedded nul character, which bash appears to treat as the end of the string. (I'm not sure there's any way it couldn't!). This is visible in my question, where I output command:
About to run command:
"python -c 'import os, sys [SNIP…] + b'\\''\x00'\\'' for [SNIP…]"
That's a repr output, but notice the single slash before the x in \x00: that's an actual \x00 that made it through. My original code has this Python embedded as a snippet, which I didn't include (I didn't believe it was relevant):
_LS_CODE = """\
import os, sys
path = sys.argv[1]
if sys.version_info.major == 2:
path = path.decode('utf-8')
entries = os.listdir(path)
out = b''.join(e.encode('utf-8') + b'\x00' for e in entries)
sys.stdout.write(out)
"""
Here, Python's """ is still processing \ as an escape character. I need to double up, or look into raw strings (r""")
You need to escape newlines as well. A better option is to put the program text in a here document.
Make the output of "About to run command:" to look like
python -c << EOF
import os, sys
path = sys.argv[1]
if sys.version_info.major == 2:
path = path.decode('\''utf-8'\'')
entries = os.listdir(path)
out = b'\'''\''.join(e.encode('\''utf-8'\'') + b'\'''\'' for e in entries)
sys.stdout.write(out)
.
EOF
Maybe you wouldn't need to escape anything at all.
I have a list of IP's that I want to run a whois (using the linux tool whois) against and only see the Country option.
Here is my script:
import os
import time
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print os.system("whois -h whois.arin.net + {0} | grep Country ".format(i))
So I want to display what IP is being ran, then I just want to see the Country info using grep. I see this error when I run it and the grep is not ran:
sh: -c: line 1: syntax error near unexpected token `|'
sh: -c: line 1: ` | grep Country '
this code below works so it must be an issue with my for loop:
print os.system("whois -h whois.arin.net + {0} | grep Country ".format('8.8.8.8'))
What am I doing wrong? Thank you!!!!
You're not stripping trailing newlines from the lines you read from the file. As a result, you are passing to os.system a string like "whois -h whois.arin.net + a.b.c.d\n | grep Country". The shell parses the string as two commands and complains of "unexpected token |" at the beginning of the second one. This explains why there is no error when you use a hand-made string such as "8.8.8.8".
Add i = i.strip() after the sleep, and the problem will go away.
user4815162342 is correct about the issue you are having, but might I suggest you replace os.system with subprocess.Popen? Capturing the output from the system call is not intuitive.. should you want to result to go anywhere but your screen, you'll likely going to have issues
from subprocess import Popen, PIPE
server = 'whois.arin.net'
def find_country(ip):
proc = Popen(['whois', '-h', server, ip], stdout = PIPE, stderr = PIPE)
stdout, stderr = proc.communicate()
if stderr:
raise Exception("Error with `whois` subprocess: " + stderr)
for line in stdout.split('\n'):
if line.startswith('Country:'):
return line.split(':')[1].strip() # Good place for regex
for ip in [i.strip() for i in open('ips.txt').readlines()]:
print find_country(ip)
Python is awesome at string handling- there should be no reason to create a grep subprocess to pattern match the output of a separate subprocess.
Try sh:
import os
import time
import re
import sh
iplist = open('ips.txt').readlines()
for i in iplist:
time.sleep(2)
print "Country: IP {0}".format(i)
print sh.grep(sh.whois(i, h="whois.arin.net"), "Country")