I have a bash script that receives data from json. I'd like to delegate json parsing to python and operate other things with bash. So I tried the following and it worked:
$cat json.txt | python -c "import sys, json; app_data=json.load(sys.stdin); print app_data['item'][0]['id'];"
I decided to check the list size:
$cat json.txt | python -c 'import sys, json; app_data=json.load(sys.stdin); if len(app_data['item'])==1: print '1 item:'; print app_data['item'][0]['id']'
It failed with SyntaxError: invalid syntax.
List size check (the code above) works from a separate .py file in general.
I'd prefer to use one-liner to keep it simple and store together in shell script.
Is it possible to run python one-liner with some logic (like import json) and if block?
A similar question has already been answered here:
Executing Python multi-line statements in the one-line command-line. In short, using the funny $'' quoting (which interprets escapes like \n) should work, at least in bash:
$ cat json.txt
{"item": [{"id": 1}]}
$ cat json.txt | python -c $'import sys, json;\nd=json.load(sys.stdin)\nif len(d["item"])==1: print("""1 item:\n%s""" % d["item"][0]["id"])'
1 item:
1
From a syntactic POV, the problem is that Python allows to use ; only as a separator of so called simple_stmt. But an if_stmt is not a simple statement. See https://docs.python.org/2/reference/simple_stmts.html#grammar-token-simple_stmt.
Related
I'm trying to convert over a Bash script that includes the following commands:
PYCODE=$(cat << EOF
#INSERT_PYTHON_CODE_HERE
EOF
)
RESPONSE=$(COLUMNS=999 /usr/bin/env python3 -c "$PYCODE" $#)
The idea being that a sed find/replace is then used to inject an arbitrary Python script where #INSERT_PYTHON_CODE_HERE is, creating the script that is then ran.
The corresponding Fish command would seem to be something like this
set PYCODE "
#INSERT_PYTHON_CODE_HERE
"
set RESPONSE (COLUMNS=999 /usr/bin/env python3 -c "$PYCODE" $argv)
but this falls apart when you have a Python script that can include both ' and " (and any other valid) characters.
What is the correct way to handle translate this use of EOF?
As a side note, I would prefer not to modify the sed command that is injecting the python code, but for reference here it is:
set FISH_SCRIPT (sed -e "/#INSERT_PYTHON_CODE_HERE/r $BASE_DIR/test_sh.py" $BASE_DIR/../src/mfa.fish)
If I look for a filename pattern in perl, you can do this simple:
ls -l | perl -n -e'if(/.*180205.*/){ print "$_\n"; }'
-n
causes Perl to assume the following loop around your program, which makes it iterate over filename arguments somewhat like sed -n or awk:
LINE:
while (<>) {
... # your program goes here
}
How can I code this in python3? (python3 --help shows not such option)
Python oneliners with python -c '...' are extremely restricted, as the Python grammar assumes that each statement sits on its own line. You can combine some statements with a semicolon “;”, but some statements need to have their own line, notably compound statements like loops. If you want to write this on the command line, we have to express the loop over all lines as a list comprehension:
python3 -c 'import re, fileinput; [print(line, end="") for line in fileinput.input() if re.search("180205", line)]'
This is of course fairly unreadable, because Python is not well-suited for one-liners. Looping over the fileinput.input() is similar to Perl's -n option.
If you want to use Python, consider writing a script. This is much more readable:
import re, fileinput
for line in fileinput.input():
if re.search("180205", line):
print(line, end="")
I've got the following bash script:
#!/bin/bash
while read line
do
ORD=`echo $line | cut -c 7-21`
if [[ -r ../FASTA_SEC/${ORD}.fa ]]
then
WCR=`fgrep -o N ../FASTA_SEC/$ORD.fa | wc -l`
WCT=`wc -m < ../FASTA_SEC/$ORD.fa`
PER1=`echo print $WCR/$WCT.*100 | python`
WCTRIN=`fgrep -o N ../FASTA_SEC_EDITED/$ORD"_Trimmed.fa" | wc -l`
WCTRI=`wc -m < ../FASTA_SEC_EDITED/$ORD"_Trimmed.fa"`
PER2=`echo print $WCTRIN/$WCTRI.*100 | python`
PER3=`echo print $PER1-$PER2 | python`
echo $ORD $PER1 $PER2 $PER3 >> Log.txt
if [ $PER2 -ge 30 -a $PER3 -lt 10 ]
then
mv ../FASTA_SEC/$ORD.fa ./TRASH/$ORD.fa
mv ../FASTA_SEC_EDITED/$ORD"_Trimmed.fa" ./TRASH/$ORD"_Trimmed.fa"
fi
fi
done < ../READ/Data.txt
$PER variables are floating numbers as u might have noticed so I cannot use them normaly in the nested if conditional. I'd like to do this conditional iteration in python but I have no clue how do it whithin a bash script also I dont know how to import the value of the variables $PER2 and $PER3 into python. Could I write directly python code in the same bash script invvoking python somehow?
Thank you for your help, first time facing this.
You can use python -c CMD to execute a piece of python code from the command line. If you want bash to interpolate your environment variables, you should use double quotes around CMD.
You can return a value by calling sys.exit, but keep in mind that true and false in Python have the reverse meaning in bash.
So your code would be:
if python -c "import sys; sys.exit(not($PER2 > 30 and $PER3 < 10 ))"
It is possible to feed Python code to the standard input of python executable with the help of here document syntax:
variable=$(date)
python2.7 <<SCRIPT
print "The current date: %s" % "${variable}"
SCRIPT
In order to avoid parameter substitution (interpretation within the block), quote the first limit string: <<'SCRIPT'.
If you want to assign the output to a variable, use command substitution:
output=$(python2.7 <<SCRIPT
print "The current date: %s" % "${variable}"
SCRIPT
)
Note, it is not recommended to use back quotes for command substitution, as it is impossible to nest them, and the form $(...) is more readable.
maybe this helps?
$ X=4; Y=7; Z=$(python -c "print($X * $Y)")
$ echo $Z
28
python -c "str" takes "str" as input and runs it.
but then why not rewrite all in python? bash commands can nicely be executed with subprocess which is included in python or (need to install that) sh.
If I use import and a for follows afterwards I get an invalid syntax error. I have no idea why this happens.
> python3 -c 'import os; for a in range(1,5): print(a)'
File "<string>", line 1
import os; for a in range(1,5): print(a)
^
Removing the import works fine:
> python3 -c 'for a in range(1,5): print(a)'
1
2
3
4
or totally removing the for loop:
> python3 -c 'import os; print(10)'
10
So what's going on??
It's an error because it's not in the Python grammar.
If you check out the syntax specification for compound statements, you'll see that a statement list (i.e. what you're making with the semicolon) is defined as:
stmt_list ::= simple_stmt (";" simple_stmt)* [";"]
and the for construct is not a simple_stmt, but instead is a compound_stmt.
The print(10), however, is a simple_stmt and, as such, is just fine.
As you see Python does not allow you to put a block opening statement (like for) into a line with an other statement.
In general you should not use the ; inside a script in some situations it (like the command execution of python code with -c) it is the only solution, but inside a script just use a line break.
For details about coding style standards in Python you can read the PEP 8: http://legacy.python.org/dev/peps/pep-0008/
Well in 2022 with python3 you can do this:
python3 -c 'import os; [print(a) for a in range(1,5)]'
I'm trying to use Python (through a Linux terminal) to replace the word 'example' in following line of a text file text_file.txt:
abcdefgh example uvwxyz
What I want is:
abcdefgh replaced_example uvwxyz
Can I do this with a one-liner in Python?
EDIT:
I have a perl one-liner perl -p -i -e 's#example#replaced_example#' text_file.txt but I want to do it in Python too
You can do it:
python -c 'print open("text_file.txt").read().replace("example","replaced_example")'
But it's rather clunky. Python's syntax isn't designed to make nice 1-liners (although frequently it works out that way). Python values clarity above everything else which is one reason you need to import things to get the real powerful tools python has to offer. Since you need to import things to really leverage the power of python, it doesn't lend to creating simple scripts from the commandline.
I would rather use a tool that is designed for this sort of thing -- e.g. sed:
sed -e 's/example/replace_example/g' text_file.txt
Incidentally the fileinput module supports inplace modification just like sed -i
-bash-3.2$ python -c '
import fileinput
for line in fileinput.input("text_file.txt", inplace=True):
print line.replace("example","replace_example"),
'