We Keep Coding

linux command pipe with python "-c" flag

I am trying to do a string printing with python -c flag, e.g.
python3 -c "print('Hello World')"
So now I wanna substitute an argument with pipe, e.g. echo "Hello World" | python3 -c "print($1)"
the pipe is to take output from previous command and take it as input to next command, if I am not wrong, this is possible? But I think I got syntax error which I cannot find any source of this
I also bumped into question previously asked, but the solution required python imports and .py file depends on how we run this, I understand but I just wanna get it in a line of command in linux shell

If your input is always single line then you should be able to harness input function for example
echo "Hello World" | python3 -c "print(input().upper())"
would output
HELLO WORLD

How to use python -c "code here" with newlines?

The command
python -c "print('hello')"
runs the code inside the quotes successfully, both in Linux (bash) and Windows (cmd.exe).
But how to pass code with newlines, with python -c?
Example: both
python -c "for i in range(10): if i % 2 == 0: print('hello')"
python -c "for i in range(10):\n if i % 2 == 0:\n print('hello')"
fail.
Example use case: I need to send a SSH command (with paramiko) to execute a short Python code on a remote server, so I need to pass one command like
ssh.exec_command('python -c "..."').

You can use bash's $'foo' string syntax to get newlines:
python -c $'for i in range(10):\n if i % 2 == 0:\n print("hello")'
(I'm using single space indents here)
For windows, you really should be using powershell, which has `n as a newline:
python -c "for i in range(10):`n if i % 2 == 0:`n print('hello')"
In cmd.exe, it seems that you can use ^ to escape a newline, however I'm unable to test this currently so you should refer to this question's answers.

You can use a bash "heredoc" (also available ksh, and POSIX shells):
python <<EOF
import numpy as np
print(dir(np))
EOF

While not using -c it is worth mentioning that, if using Bash, you can pipe echoed code directly to python.
echo -e 'for i in range(10):\n if i % 2 == 0:\n print("hello")' | python
Following Aplet123's lead and using using single space indents.

One way could be with exec(), what make strings executable. It looks bad, but it works.
python -c "exec(\"for i in range(10):\n if i % 2 == 0:\n print('hello')\")"

Can we use a variable containing linux command result in the python code

I am working with shell scripting and trying to learn Python scripting, any suggestion is welcome.
I want to achieve something like below:
Usage1:
ps_result=`ps -eaf|grep -v "grep" | grep "programmanager"`
then can we straight away use ps_result variable in python code; if yes, how?
Usage2:
matched_line=`cat file_name |grep "some string"`
can we use matched_line variable in python code as a list, if yes how?
PS: If possible assume I am writing the bash and python code in the one file, if not possible request you to please suggest a way. TIA

Yes, you can do it via environment variables.
First, define the environment variable for the shell using export:
$ export ps_result=`ps -eaf|grep -v "grep" | grep "programmanager"`
Then, import the os module in Python and read the environment variable:
$ python -c 'import os; ps_result=os.environ.get("ps_result"); print(ps_result)'

Second question first, if you run python -, python will run the script piped on stdin. There are several functions in python subprocess that let you run other programs. So you could write
test.sh
#!/bin/sh
python - << endofprogram
import subprocess as subp
result = subp.run('ps -eaf | grep -v "grep" | grep "python"',
shell=True, capture_output=True)
if result.returncode == 0:
matched_lines = result.stdout.decode().split("\n")
print(matched_lines)
endofprogram
In this example we pipe via the shell but python can chain stdout/stdin also, albeit more verbosely.

Unable to dowload file using os.system

I am trying to download a file using os.system in python and it never completely downloads the file
Here is the code
import os
url = 'wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O- | sed -rn "s/.*confirm=([0-9A-Za-z_]+).*/\1\n/p")&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O cnn_stories_tokenized.zip && rm -rf /tmp/cookies.txt'
os.system(url)
On trying to download the file with that with the same command on the terminal works just fine, are there any escape characters that I should be handling?

are there any escape characters that I should be handling?
Short answer: Yes.
There are \1 and \n in the string and Python tries to interpret it like a normal escape sequence.
You can either escape them manually by doubling each backslash or make it into raw string.
To make a raw string, add r just at the opening quote ' (making it r'wget...). "Raw" means Python will use it as-is, and not try to interpret things that look like escape codes (e.g. r'\n' == '\n). Anywhere you have a path to file or regex, just use raw strings to not worry about escaping backslashes by yourself and just paste what you wrote somewhere else!

There is one way, you can ran this command. I think you might be already knowing the answer.
Save the linux command as shell script:
e.g.: vi downloader.sh
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O- | sed -rn "s/.*confirm=([0-9A-Za-z_]+).*/\1\n/p")&id=0BzQ6rtO2VN95cmNuc2xwUS1wdEE" -O cnn_stories_tokenized.zip && rm -rf /tmp/cookies.txt
save the file. call this file from python.
from subprocess import call
call(["bash", "downloader.sh"])
This is one way which can solve your problem, other alternatives using python libraries is also possible like
requests package

python 2.3 how to run a piped command

I want to run a command like this
grep -w 1 pattern <(tail -f mylogfile.log)
basically from a python script i want to monitor a log file for a specific string and continue with the python script as soon as i found that.
I am using os.system(), but that is hanging. The same command in bash works good.
I have a very old version of python (v2.3) and so don't have sub-process module.
do we have a way to acheive this

In Python 2.3, you need to use subprocess from SVN
import subprocess
import shlex
subprocess.call(shlex.split("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'"))
To be explicit, you need to install it from the SVN link above.
You need to call this with /bin/bash -c due to the shell redirection you're using
EDIT
If you want to solve this with os.system(), just wrap the command in /bin/bash -c since you're using shell redirection...
os.system("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'")

First of all, the command i think you should be using is grep -w -m 1 'pattern' <(tail -f in)
For executing commands in python, use the Popen constructor from the subprocess module. Read more at
http://docs.python.org/library/subprocess.html

If I understand correctly, you want to send the output to python like this -
tail -f mylogfile.log | grep -w 1 pattern | python yourscript.py
i.e., read all updates to the log file, and send matching lines to your script.
To read from standard input, you can use the file-like object: sys.stdin.
So your script would look like
import sys
for line in sys.stdin.readlines():
#process each line here.