We Keep Coding

linux command pipe with python "-c" flag

I am trying to do a string printing with python -c flag, e.g.
python3 -c "print('Hello World')"
So now I wanna substitute an argument with pipe, e.g. echo "Hello World" | python3 -c "print($1)"
the pipe is to take output from previous command and take it as input to next command, if I am not wrong, this is possible? But I think I got syntax error which I cannot find any source of this
I also bumped into question previously asked, but the solution required python imports and .py file depends on how we run this, I understand but I just wanna get it in a line of command in linux shell

If your input is always single line then you should be able to harness input function for example
echo "Hello World" | python3 -c "print(input().upper())"
would output
HELLO WORLD

How should I embed a python script in a Makefile?

I want to embed a python script in a Makefile
I built a python -c script (below), and it works well in my MacBook Hyper terminal:
% python -c $'from subprocess import getstatusoutput\noutput=getstatusoutput("open --background -a Docker")\nif int(output[0])>0:\n print("Docker desktop failed to launch: exit-code:{}".format(output[0]))'
For reasons I can't yet figure out, this seems to fail if I build a Makefile with it (note: a tab is four spaces... I used tab indentation in the Makefile).
all:
$(shell python -c $'from subprocess import getstatusoutput\noutput=getstatusoutput("open --background -a Docker")\nif int(output[0])>0:\n print("Docker desktop failed to launch: exit-code:{}".format(output[0]))')
Running the make all target...
% make all
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `python -c from subprocess import getstatusoutput\noutput=getstatusoutput("open --background -a Docker")\nif int(output[0])>0:\n print("Docker desktop failed to launch: exit-code:{}".format(output[0]))''
make: `all' is up to date.
%
I have been struggling with this for a while...
Can someone help explain why make all fails and the best way to fix this python -c command? My shell CLI python -c ... command successfully launches Docker desktop on my MacBook.
I understand there are non-python ways to solve this specific problem... I need a general python Makefile solution.

Using python -c in a Makefile is tricky because of Python's indentation requirements. One simple solution is to use SHELL=/bin/bash if you want to use a Bash "C-style" string:
SHELL=/bin/bash
all:
# python command must be wrapped in single quotes _and_ have double dollar sign in front
python -c $$'from subprocess import getstatusoutput\noutput=getstatusoutput("open --background -a Docker")\nif int(output[0])>0:\n print("Docker desktop failed to launch: exit-code:{}".format(output[0]))'
(Notice how the dollar sign needs to be doubled to escape it. And obviously, this restricts the portability of your Makefile to systems where Bash is available. The $'...' syntax lets you use escape codes like \n and \t within a string, and have them expanded to newline and tab, respectively. This construct specifically requires a leading dollar sign and single quotes around the string - merely '...' does someting slightly different, and $"..." does something entirely different.)
You could also define a make multi-line variable. But in this isolated case, Python is not playing any useful role anyway.
all:
open --background -a Docker
make will terminate with an error message if open fails; printing essentinally the same message from Python seems superfluous. If you want to proceed in spite of the error, you can do
all:
open --background -a Docker || \
echo "Docker desktop failed to launch: exit-code: $$?"
... though I assume failing to fail (sic) from the Python script was just a mistake.

I found a fairly simple way to embed a multiline python script in a Makefile...
Save this as Makefile...
define MULTILINE_PYTHON_SCRIPT
###########################################
# Start multiline python string here...
###########################################
from subprocess import getstatusoutput as gso
print("Here we go:")
for hello_int in [1, 2, 3,]:
print(' Hello World %i' % hello_int)
retval, _ = gso("ls -la")
assert retval==0, "ls command execution failed"
###########################################
# End of multiline python string...
###########################################
endef
export MULTILINE_PYTHON_SCRIPT
EMBEDDED_PY := python -c "$$MULTILINE_PYTHON_SCRIPT"
.PHONY: nothing
nothing:
echo "raw makefile command"
.PHONY: test
test:
$(EMBEDDED_PY)
.PHONY: all
all:
open --background -a Docker
Testing the output:
% make nothing
echo "raw makefile command"
raw makefile command
%
% make test
python -c "$MULTILINE_PYTHON_SCRIPT"
Here we go:
Hello World 1
Hello World 2
Hello World 3
%
%

How to use awk command in python

I have to write a python script to find available space in /var/tmp folder in linux. So I am using awk command to filter out only available space.
import subprocess
subprocess.call("$(awk 'NR==2 {print $4}' file1.txt)")
The output should be 4.7G but it comes
/bin/sh: 1: 4.7G: not found
and the return value is 127 and not 0.

Apart from the fact that python should be sufficient to do what awk does here, what you are doing with
$(awk 'NR==2 {print $4}' file1.txt)
is to run the awk command and expand its output as part of the command line by $(...). As this is the only part of the command line, the shell tries to execute awk's output as a command.
If you really want to run awk from your python script, remove $( and ) from your command.

Unable to dowload file using os.system

I am trying to download a file using os.system in python and it never completely downloads the file
Here is the code
import os
url = 'wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O- | sed -rn "s/.*confirm=([0-9A-Za-z_]+).*/\1\n/p")&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O cnn_stories_tokenized.zip && rm -rf /tmp/cookies.txt'
os.system(url)
On trying to download the file with that with the same command on the terminal works just fine, are there any escape characters that I should be handling?

are there any escape characters that I should be handling?
Short answer: Yes.
There are \1 and \n in the string and Python tries to interpret it like a normal escape sequence.
You can either escape them manually by doubling each backslash or make it into raw string.
To make a raw string, add r just at the opening quote ' (making it r'wget...). "Raw" means Python will use it as-is, and not try to interpret things that look like escape codes (e.g. r'\n' == '\n). Anywhere you have a path to file or regex, just use raw strings to not worry about escaping backslashes by yourself and just paste what you wrote somewhere else!

There is one way, you can ran this command. I think you might be already knowing the answer.
Save the linux command as shell script:
e.g.: vi downloader.sh
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O- | sed -rn "s/.*confirm=([0-9A-Za-z_]+).*/\1\n/p")&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O cnn_stories_tokenized.zip && rm -rf /tmp/cookies.txt
save the file. call this file from python.
from subprocess import call
call(["bash", "downloader.sh"])
This is one way which can solve your problem, other alternatives using python libraries is also possible like
requests package

python 2.3 how to run a piped command

I want to run a command like this
grep -w 1 pattern <(tail -f mylogfile.log)
basically from a python script i want to monitor a log file for a specific string and continue with the python script as soon as i found that.
I am using os.system(), but that is hanging. The same command in bash works good.
I have a very old version of python (v2.3) and so don't have sub-process module.
do we have a way to acheive this

In Python 2.3, you need to use subprocess from SVN
import subprocess
import shlex
subprocess.call(shlex.split("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'"))
To be explicit, you need to install it from the SVN link above.
You need to call this with /bin/bash -c due to the shell redirection you're using
EDIT
If you want to solve this with os.system(), just wrap the command in /bin/bash -c since you're using shell redirection...
os.system("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'")

First of all, the command i think you should be using is grep -w -m 1 'pattern' <(tail -f in)
For executing commands in python, use the Popen constructor from the subprocess module. Read more at
http://docs.python.org/library/subprocess.html

If I understand correctly, you want to send the output to python like this -
tail -f mylogfile.log | grep -w 1 pattern | python yourscript.py
i.e., read all updates to the log file, and send matching lines to your script.
To read from standard input, you can use the file-like object: sys.stdin.
So your script would look like
import sys
for line in sys.stdin.readlines():
#process each line here.