If I look for a filename pattern in perl, you can do this simple:
ls -l | perl -n -e'if(/.*180205.*/){ print "$_\n"; }'
-n
causes Perl to assume the following loop around your program, which makes it iterate over filename arguments somewhat like sed -n or awk:
LINE:
while (<>) {
... # your program goes here
}
How can I code this in python3? (python3 --help shows not such option)
Python oneliners with python -c '...' are extremely restricted, as the Python grammar assumes that each statement sits on its own line. You can combine some statements with a semicolon “;”, but some statements need to have their own line, notably compound statements like loops. If you want to write this on the command line, we have to express the loop over all lines as a list comprehension:
python3 -c 'import re, fileinput; [print(line, end="") for line in fileinput.input() if re.search("180205", line)]'
This is of course fairly unreadable, because Python is not well-suited for one-liners. Looping over the fileinput.input() is similar to Perl's -n option.
If you want to use Python, consider writing a script. This is much more readable:
import re, fileinput
for line in fileinput.input():
if re.search("180205", line):
print(line, end="")
Related
I have a bash script that calls a python script with parameters.
In the bash script, I'm reading a file that contains one row of parameters separated by ", and then calls the python script with the line I read.
My problem is that the python gets the parameters separated by the space.
The line looks like this: "param_a" "Param B" "Param C"
Code Example:
Bash Script:
LINE=`cat $tmp_file`
id=`python /full_path/script.py $LINE`
Python Script:
print sys.argv[1]
print sys.argv[2]
print sys.argv[3]
Received output:
"param_a"
"Param
B"
Wanted output:
param_a
Param B
Param C
How can I send the parameters to the Python script the way I need?
Thanks!
What about
id=`python /full_path/script.py $tmp_file`
and
import sys
for line in open(sys.argv[1]):
print(line)
?
The issue is in how bash passes the arguments. Python has nothing do to with it.
So, you have to solve all these stuff before sending it to Python, I decided to use awk and xargs for this. (but xargs is the actual MVP here.)
LINE=$(cat $tmp_file)
awk -v ORS="\0" -v FPAT='"[^"]+"' '{for (i=1;i<=NF;i++){print substr($i,2,length($i)-2)}}' <<<$LINE |
xargs -0 python ./script.py
First $(..) is preferred over backticks, because it is more readable. You are making a variable after all.
awk only reads from stdin or a file, but you can force it to read from a variable with the <<<, also called "here string".
With awk I loop over all fields (as defined by the regex in the FPAT variable), and print them without the "".
The output record separator I choose is the NULL character (-v ORF='\0'), xargs will split on this character.
xargs will now parse the piped input by separating the arguments on NULL characters (set with -0) and execute the command given with the parsed arguments.
Note, while awk is found on most UNIX systems, I make use of FPAT which is a GNU awk extension and you might not be having GNU awk as default (for example Ubuntu), but gnu awk is usually just a install gawk away.
Also, the next command would be a quick and easy solution, but generally considered as unsafe, since eval will execute everything it receives.
eval "python ./script "$LINE
This can be done using bash arrays:
tmp_file='gash.txt'
# Set IFS to " which splits on double quotes and removes them
# Using read is preferable to using the external program cat
# read -a reads into the array called "line"
# UPPERCASE variable names are discouraged because of collisions with bash variables
IFS=\" read -ra line < "$tmp_file"
# That leaves blank and space elements in "line",
# we create a new array called "params" without those elements
declare -a params
for((i=0; i < ${#line[#]}; i++))
do
p="${line[i]}"
if [[ -n "$p" && "$p" != " " ]]
then
params+=("$p")
fi
done
# `backticks` are frowned upon because of poor readability
# I've called the python script "gash.py"
id=$(python ./gash.py "${params[#]}")
echo "$id"
gash.py:
import sys
print "1",sys.argv[1]
print "2",sys.argv[2]
print "3",sys.argv[3]
Gives:
1 param_a
2 Param B
3 Param C
I have a bash script that receives data from json. I'd like to delegate json parsing to python and operate other things with bash. So I tried the following and it worked:
$cat json.txt | python -c "import sys, json; app_data=json.load(sys.stdin); print app_data['item'][0]['id'];"
I decided to check the list size:
$cat json.txt | python -c 'import sys, json; app_data=json.load(sys.stdin); if len(app_data['item'])==1: print '1 item:'; print app_data['item'][0]['id']'
It failed with SyntaxError: invalid syntax.
List size check (the code above) works from a separate .py file in general.
I'd prefer to use one-liner to keep it simple and store together in shell script.
Is it possible to run python one-liner with some logic (like import json) and if block?
A similar question has already been answered here:
Executing Python multi-line statements in the one-line command-line. In short, using the funny $'' quoting (which interprets escapes like \n) should work, at least in bash:
$ cat json.txt
{"item": [{"id": 1}]}
$ cat json.txt | python -c $'import sys, json;\nd=json.load(sys.stdin)\nif len(d["item"])==1: print("""1 item:\n%s""" % d["item"][0]["id"])'
1 item:
1
From a syntactic POV, the problem is that Python allows to use ; only as a separator of so called simple_stmt. But an if_stmt is not a simple statement. See https://docs.python.org/2/reference/simple_stmts.html#grammar-token-simple_stmt.
I am trying to execute the following command in python using plumbum:
sort -u -f -t$'\t' -k1,1 file1 > file2
However, I am having issues passing the -t$'\t' argument. Here is my code:
from plumbum.cmd import sort
separator = r"-t$'\t'"
print separator
cmd = (sort["-u", "-f", separator, "-k1,1", "file1"]) > "file2"
print cmd
print cmd()
I can see problems right away after print separator and print cmd() executes:
-t$'\t'
/usr/bin/sort -u -f "-t\$'\\t'" -k1,1 file1 > file2
The argument is wrapped in double quotes.
An extra \ before $ and \t is inserted.
How should I pass this argument to plumbum?
You may have stumbled into limitations of the command line escaping.
I could make it work using subprocess module, passing a real tabulation char litteraly:
import subprocess
p=subprocess.Popen(["sort","-u","-f","-t\t","-k1,1","file1",">","file2"],shell=True)
p.wait()
Also, full python short solution that does what you want:
with open("file1") as fr, open("file2","w") as fw:
fw.writelines(sorted(set(fr),key=lambda x : x.split("\t")[0]))
The full python solution doesn't work exactly the same way sort does when dealing with unicity. If 2 lines have the same first field but not the same second field, sort keeps one of them, whereas the set will keep both.
EDIT: unchecked but you just confirmed that it works: just tweak your plumbum code with:
separator = "-t\t"
could just work, although out of the 3 ones, I'd recommend the full python solution since it doesn't involve an external process and therefore is more pythonic and portable.
I'm trying to use Python (through a Linux terminal) to replace the word 'example' in following line of a text file text_file.txt:
abcdefgh example uvwxyz
What I want is:
abcdefgh replaced_example uvwxyz
Can I do this with a one-liner in Python?
EDIT:
I have a perl one-liner perl -p -i -e 's#example#replaced_example#' text_file.txt but I want to do it in Python too
You can do it:
python -c 'print open("text_file.txt").read().replace("example","replaced_example")'
But it's rather clunky. Python's syntax isn't designed to make nice 1-liners (although frequently it works out that way). Python values clarity above everything else which is one reason you need to import things to get the real powerful tools python has to offer. Since you need to import things to really leverage the power of python, it doesn't lend to creating simple scripts from the commandline.
I would rather use a tool that is designed for this sort of thing -- e.g. sed:
sed -e 's/example/replace_example/g' text_file.txt
Incidentally the fileinput module supports inplace modification just like sed -i
-bash-3.2$ python -c '
import fileinput
for line in fileinput.input("text_file.txt", inplace=True):
print line.replace("example","replace_example"),
'
I'm using Windows, and I would like to extract certain columns from a text file using a Perl, Python, batch etc. one-liner.
On Unix I could do this:
cut -d " " -f 1-3 <my file>
How can I do this on Windows?
Here is a Perl one-liner to print the first 3 whitespace-delimited columns of a file. This can be run on Windows (or Unix). Refer to perlrun.
perl -ane "print qq(#F[0..2]\n)" file.txt
you can download GNU windows and use your normal cut/awk etc..
Or natively, you can use vbscript
Set objFS = CreateObject("Scripting.FileSystemObject")
Set objArgs = WScript.Arguments
strFile = objArgs(0)
Set objFile = objFS.OpenTextFile(strFile)
Do Until objFile.AtEndOfLine
strLine=objFile.ReadLine
sp = Split(strLine," ")
s=""
For i=0 To 2
s=s&" "&sp(i)
Next
WScript.Echo s
Loop
save the above as mysplit.vbs and on command line
c:\test> cscript //nologo mysplit.vbs file
Or just simple batch
#echo off
for /f "tokens=1,2,3 delims= " %%a in (file) do (echo %%a %%b %%c)
If you want a Python one liner
c:\test> type file|python -c "import sys; print [' '.join(i.split()[:3]) for i in sys.stdin.readlines()]"
That's rather simple Python script:
for line in open("my file"):
parts = line.split(" ")
print " ".join(parts[0:3])
The easiest way to do it would be to install Cygwin and use the Unix cut command.
If you are dealing with a text file that has very long lines and you are only interested in the first 3 columns, then splitting a fixed number of times yourself will be a lot faster than using the -a option:
perl -ne "#F = split /\s/, $_, 4; print qq(#F[0..2]\n)" file.txt
rather than
perl -ane "print qq(#F[0..2]\n)" file.txt
This is because the -a option will split on every whitespace in a line, which potentially can lead to a lot of extra splitting.