We Keep Coding

sh: 1: Syntax error: redirection unexpected Python/Bash

I have problem with my python cmd script.
I don't know why it does not work. Maybe something wrong with my code.
Im trying to run the program in cmdline through my python script.
And Im getting error in bash "sh: 1: Syntax error: redirection unexpected"
pls help Im just biologist :)
Im using spyder (anaconda)/Ubuntu
#!/usr/bin/python
import sys
import os
input_ = sys.argv[1]
output_file = open(sys.argv[2],'a+')
names = input_.rsplit('.')
for name in names:
os.system("esearch -db pubmed -query %s | efetch -format xml | xtract -pattern PubmedArticle -element AbstractText >> %s" % (name, output_file))
print("------------------------------------------")

output_file is a file object. When you do "%s" % output_file, the resulting string is something like "<open file 'filename', mode 'a+' at 0x7f1234567890>". This means that the os.system call is running a command like
command... >> <open file 'filename', mode 'a+' at 0x7f1234567890>
The < after the >> causes the "Syntax error: redirection unexpected" error message.
To fix that, don't open the output file in your Python script, just use the filename:
output_file = sys.argv[2]

I got similar error on following line:
os.system('logger Status changed on %s' %s repr(datetime.now())
Indeed, as nomadictype stated the problem is in running plain OS command. The command may include special characters. In my case this was <.
So instead of changing OS command significantly, I just added quotes and this works:
os.system('logger "Status changed on %s"' %s repr(datetime.now())
Quotes make content of passed parameter invisible for shell.

awk calling in python script responding as file cannot be open/read

I am trying to open a file via python script by calling awk by giving folder and file name as %s.
import commands
genotype = 'rice'
filename = 'model.txt'
dear = commands.getoutput('''awk '{print $0}' /home/angad/Desktop/Python_result/%s/%s''') % (genotype,filename)
print dear
But it is returning as:
awk: cmd. line:1: fatal: cannot open file '/home/angad/Desktop/Python_result/rice/model.txt' for reading (No such file or directory)
The concerned file is indeed present there.
As if, I write input file path directly instead of raw_input(%s), it successfully open and print the file content.

One of your parentheses is in the wrong place. Try:
dear = commands.getoutput('''awk '{print $0}' /home/angad/Desktop/Python_result/%s/%s''' % (genotype,filename))
Or better yet:
cmd_str = '''awk '{print $0}' /home/angad/Desktop/Python_result/%s/%s''' % (genotype,filename)
dear = commands.getoutput(cmd_str)
The reason the error message shows the correct file path in your original example is that the result of getoutput, which included the error message with the %s formats still present, was being substituted before being printed. Very misleading!

Path not being found in Windows via bash/python script

This is .csv file containing target,path:
[root#server ~]# head /usr/local/CustomAppResults/App/targets.txt
server1.com,C:\\Program Files\\App\\
I loop through the .csv, line by line in a Bash script ..
IFS=","
while read target path
do
# Windows target
result=$(python /root/apache-tomcat-7.0.37/webapps/App/tools/remoteStafCommand.py "$target" "dir $path /B")
resultCode=$?
echo "$result"
done
The bash script passes the contents of the csv. file into a Python script, it's contents are below.
The Python script runs a staf subprocess and parses the results, which is returned and stored inside result of the Bash script.
[root#server ~]# cat App/tools/remoteStafCommand.py
import subprocess,re,sys
from subprocess import CalledProcessError, check_output
print "1st arg: %s" % (sys.argv[1])
print "2nd arg: %s" % (sys.argv[2])
try:
output = subprocess.check_output(["staf", "%s" % (sys.argv[1]) , "PROCESS", "START", "SHELL", "COMMAND", "%s" % (sys.argv[2]), "WAIT", "RETURNSTDOUT", "STDERRTOSTDOUT"])
result = re.findall(r'Data\s*:\s*((?:[^\n]*(?:[\r\n]+(?!\s*}))?)+)', output, re.DOTALL)[0]
print result
except CalledProcessError as e:
print(e.returncode)
sys.exit(e.returncode)
This is the output I get, why can it not find the path? It exists.
1st arg: server1.com
2nd arg: dir C:\Program Files\App\ /B
The system cannot find the path specified.

The read command tokenizes on whitespace, not comma. You can override this by changing the value of IFS. Or if the text file isn't used for anything else, change its format to have a space instead of a comma between the fields. Or better still, have your Python script read and parse the file directly -- the simple Bash script begs to be factored away.

Wrapping path with double quotes directly in .csv file for escaping ..
[root#server ~]# head /usr/local/CustomAppResults/App/targets.txt
server1.com,"C:\\Program Files\\App\\"

Using wildcards in filename in scp in python

I want to execute a simple scp command in a python script, copying files following a certain name pattern.
I'm executing the following command:
filename = '\*last_processed_date\*.txt'
command = ''' scp test#100.41.14.27:/home/test/test2/test3/%s %s '''\
% (filename,self.unprocessed_file_dir)
os.system(command)
I understand that I have to escape the wildcard '*', which I'm doing..but still I get:
scp: /home/test/test2/test3/*last_processed_date*.txt: No such file or directory
I'm wondering what I'm doing wrong..
EDIT:
It was a careless mistake from my side. I should have done:
command = ''' scp 'test#100.41.14.27:/home/test/test2/test3/%s' %s '''
instead of:
command = ''' scp test#100.41.14.27:/home/test/test2/test3/%s %s '''\
% (filename,self.unprocessed_file_dir)

This works on my system:
host = 'test#100.41.14.27'
filename = '*last_processed_date*.txt'
rpath = '/home/test/test2/test3'
lpath = self.unprocessed_file_dir
command = 'scp %s:%s/%s %s' % (host, rpath, filename, lpath)
os.system(command)
If it gives you an error, try this on from the terminal first:
ssh test#100.41.14.27 ls /home/test/test2/test3/*last_processed_date*.txt

What did I do wrong in my python script to get output?

I think that's should work. I want to get output of this command
"grep -l %s *.py" % tag for each tag,
on created file with tag name.
import os
import sys
results_dir = '/home/ks/one/work/tn/format-description 15852/results'
converters_dir = '/home/ks/one/work/cvr'
export_dir = '/home/ks/one/work/epr'
all_dirs = {'cvr':cvrs_dir,
'epr':eprs_dir}
tags = [tag.strip() for tag in open('new file').readlines()]
for coe, directory in all_dirs.items(): # coe - type of our file
os.chdir(results_dir)
for tag in tags:
tag_file = open(coe + ' ' + tag, 'w')
sys.stdout = tag_file
os.chdir(directory)
os.system("grep -l %s *.py" % tag)
tag_file.close()
But all what I see when script is runned - it's output in my console.

Use the subprocess module of Python
http://docs.python.org/library/subprocess.html
The documentation contains extensive documentation and examples.
The other (poor) option with os.system() is redirecting the output to a file
os.system('do_something >/tmp/myoutput')
and then later reading the output file from within Python. However this is ugly
and likely not very portable.

The most minimal change I can think of so you can get at the output in python is to change:
os.system("grep -l %s *.py" % tag)
for:
output = commands.getoutput("grep -l %s *.py" % tag)
This way, the output of the command will end up in the output variable.
I'm not sure why other people dogmatically tell you to change your code to use subprocess; it will need more rewriting to do so.