We Keep Coding

Python: Read data from STDIN, unless its not provided

I have a python program which reads from STDIN:
#!/usr/bin/env python
def my_func(data):
print (data)
if __name__ == '__main__':
import sys
data = sys.stdin.read()
my_func(data)
I see the expected results when I execute this with:
cat file.txt | ./app.py
I want to add some other functionality to the program:
if __name__ == '__main__':
import sys
data = sys.stdin.read()
if data:
my_func(data)
else:
print ('I am some other functionality')
However when I execute this with:
./app.py
... the program just hangs, as if it is waiting for STDIN input.
What's the correct way to write this, so it will handle both methods of executing.

As #Klaus D. suggested, using different command line arguments should be the most straightforward option.
sys.argv[i] returns the argument at index i used when launching the script. Index 0 is always the script name while any other index represents subsequent arguments passed to the script. With that in mind:
if __name__ == '__main__':
import sys
objective = sys.argv[1]
if objective == 'read':
data = sys.stdin.read()
if data:
my_func(data)
else:
print ('I am some other functionality')

Many Unix-style programs look for a dash - to indicate input from stdin. So,
import sys
if sys.argv[1] == '-':
read_from_stdin()
else:
other_action()
$ ./app.py - < /path/to/input

How to print the first N lines of a file in python with N as argument

How would I go about getting the first N lines of a text file in python? With N have to give as argument
usage:
python file.py datafile -N 10
My code
import sys
from itertools import islice
args = sys.argv
print (args)
if args[1] == '-h':
print ("-N for printing the number of lines: python file.py datafile -N 10")
if args[-2] == '-N':
datafile = args[1]
number = int(args[-1])
with open(datafile) as myfile:
head = list(islice(myfile, number))
head = [item.strip() for item in head]
print (head)
print ('\n'.join(head))
I wrote the program, can let me know better than this code

Assuming that the print_head logic you've implemented need not be altered, here's the script I think you're looking for:
import sys
from itertools import islice
def print_head(file, n):
if not file or not n:
return
with open(file) as myfile:
head = [item.strip() for item in islice(myfile, n)]
print(head)
def parse_args():
result = {'script': sys.argv[0]}
args = iter(sys.argv)
for arg in args:
if arg == '-F':
result['filename'] = next(args)
if arg == '-N':
result['num_lines'] = int(next(args))
return result
if __name__ == '__main__':
script_args = parse_args()
print_head(script_args.get('filename', ''), script_args.get('num_lines', 0))
Running the script
python file.py -F datafile -N 10
Note: The best way to implement it would be to use argparse library

You can access argument passed to the script through sys
sys.argv
The list of command line arguments passed to a Python script. argv[0] is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'. If no script name was passed to the Python interpreter, argv[0] is the empty string.
So in code it would look like this:
import sys
print("All of argv")
print(sys.argv)
print("Last element every time")
print(sys.argv[-1])
Reading the documentation you'll see that the first values stored in the sys.argv vary according to how the user calls the script. If you print the code I pasted with different types of calls you can see for yourself the kind of values stored.
For a basic first approach: access n through sys.argv[-1] which returns the last element every time, assuming. You still have to do a try and beg for forgiveness to make sure the argument passed is a number. For that you would have:
import sys
try:
n = int(sys.argv[-1])
except ValueError as v_e:
print(f"Please pass a valid number as argument, not ${sys.argv[-1]}")
That's pretty much it. Obviously, it's quite basic, you can improve this even more by having the users pass values with flags, like --skip-lines 10 and that would be your n, and it could be in any place when executing the script. I'd create a function in charge of translating sys.argv into a key,value dictionary for easy access within the script.

Arguments are available via the sys package.
Example 1: ./file.py datafile 10
#!/usr/bin/env python3
import sys
myfile = sys.argv[1]
N = int(sys.argv[2])
with open("datafile") as myfile:
head = myfile.readlines()[0:args.N]
print(head)
Example 2: ./file.py datafile --N 10
If you want to pass multiple optional arguments you should have a look at the argparse package.
#!/usr/bin/env python3
import argparse
parser = argparse.ArgumentParser(description='Read head of file.')
parser.add_argument('file', help='Textfile to read')
parser.add_argument('--N', type=int, default=10, help='Number of lines to read')
args = parser.parse_args()
with open(args.file) as myfile:
head = myfile.readlines()[0:args.N]
print(head)

Python's sh module - is it at all possible for a script to request input?

Using Python's sh, I am running 3rd party shell script that requests my input (not that it matters much, but to be precise, I'm running an Ansible2 playbook with the --step option)
As an oversimplification of what is happening, I built a simple bash script that requests an input. I believe that if make this simple example work I can make the original case work too.
So please consider this bash script hello.sh:
#!/bin/bash
echo "Please input your name and press Enter:"
read name
echo "Hello $name"
I can run it from python using sh module, but it fails to receive my input...
import errno
import sh
cmd = sh.Command('./hello.sh')
for line in cmd(_iter=True, _iter_noblock=True):
if line == errno.EWOULDBLOCK:
pass
else:
print(line)
How could I make this work?

After following this tutorial, this works for my use case:
#!/usr/bin/env python3
import errno
import sh
import sys
def sh_interact(char, stdin):
global aggregated
sys.stdout.write(char)
sys.stdout.flush()
aggregated += char
if aggregated.endswith(":"):
val = input()
stdin.put(val + "\n")
cmd = sh.Command('./hello.sh')
aggregated = ""
cmd(_out=sh_interact, _out_bufsize=0)
For example, the output is:
$ ./testinput.py
Please input your name and press Enter:arod
Hello arod

There are two ways to solve this:
Using _in:
using _in, we can pass a list which can be taken as input in the python script
cmd = sh.Command('./read.sh')
stdin = ['hello']
for line in cmd(_iter=True, _iter_noblock=True, _in=stdin):
if line == errno.EWOULDBLOCK:
pass
else:
print(line)
Using command line args if you are willing to modify the script.

Quoted string as optional argument to Python script

I have simple utility script that I use to download files given a URL. It's basically just a wrapper around the Linux binary "aria2c".
Here is the script named getFile:
#!/usr/bin/python
#
# SCRIPT NAME: getFile
# PURPOSE: Download a file using up to 20 concurrent connections.
#
import os
import sys
import re
import subprocess
try:
fileToGet = sys.argv[1]
if os.path.exists(fileToGet) and not os.path.exists(fileToGet+'.aria2'):
print 'Skipping already-retrieved file: ' + fileToGet
else:
print 'Downloading file: ' + fileToGet
subprocess.Popen(["aria2c-1.8.0", "-s", "20", str(fileToGet), "--check-certificate=false"]).wait() # SSL
except IndexError:
print 'You must enter a URI.'
So, for example, this command would download a file:
$ getFile http://upload.wikimedia.org/wikipedia/commons/8/8e/Self-portrait_with_Felt_Hat_by_Vincent_van_Gogh.jpg
What I want to do is permit an optional second argument (after the URI) that is a quoted string. This string will be the new filename of the downloaded file. So, after the download finishes, the file is renamed according to the second argument. Using the example above, I would like to be able to enter:
$ getFile http://upload.wikimedia.org/wikipedia/commons/8/8e/Self-portrait_with_Felt_Hat_by_Vincent_van_Gogh.jpg "van-Gogh-painting.jpg"
But I don't know how to take a quoted string as an optional argument. How can I do this?

Just test the length of sys.argv; if it is more than 2 you have an extra argument:
if len(sys.argv) > 2:
filename = sys.argv[2]

The shell will pass it as a second argument (normally) if you provide spaces between them.
For example, here is test.py:
import sys
for i in sys.argv:
print(i)
And here is the result:
$ python test.py url "folder_name"
test.py
url
folder_name
The quotes doesn't matter at all, as it's handled in the shell, not python. To get it, just take sys.argv[2].
Hope this helps!

Python: -bash: syntax error near unexpected token

I have written a little script in Python that I use to append text to a work log. I have placed the script in a directory in my $PATH
#!/usr/bin/python
# import necessary modules
import sys
import os
import datetime
# main() function
def main():
now = datetime.datetime.now()
tmp = ' '.join(sys.argv[1:])
outfile = '/path/to/output/done.log'
outstr = now.strftime("%Y-%m-%d %H:%M:%S") + ' - ' + tmp + '\n'
f=open(outfile,'a')
f.write(outstr)
f.close()
# print sys.argv[0:1]
print 'Adding ' + outstr
# Call main()
if __name__ == '__main__':
main()
When I run the script as in example 1, I get an error.
Example 1:
host:scripts user$ done this is a test
-bash: syntax error near unexpected token `done'
If I run the script as in example 2, it behaves as expected.
Example 2:
host:scripts user$ python done this is a test
Adding 2012-11-15 09:57:44 - this is a test
How do I get this to work in the first example?

done is a bash keyword, so can't be used in certain places like "the place Bash expects a command name". You could use ./done (or /path/to/done, or python /path/to/done), or re-name the command.