This question already has answers here:
Read from File, or STDIN
(9 answers)
Closed 21 days ago.
When writing text-oriented command line programs in Python, I often want to read either all the files passed on the command line, or (XOR) standard input (like Unix cat does, or Perl's <>). So, I say
if len(args) == 0: # result from optparse
input = sys.stdin
else:
input = itertools.chain(*(open(a) for a in args))
Is this the Pythonic way of doing this, or did my miss some part of the library?
You need fileinput.
A standard use case is:
import fileinput
for line in fileinput.input():
process(line)
In Python 3, argparse handles filetype objects very nicely. It's an extremely powerful module and the docs come with many examples, so it's easy to quickly write the code you want. (How Pythonic!)
You may also benefit from this StackOverflow question about using argparse to optionally read from stdin.
Related
This question already has answers here:
read subprocess stdout line by line
(10 answers)
Closed 16 days ago.
I just want to use os.system("dir") and also be able to save the text outputted to a variable. I tried using sys.stdout.read() but running sys.stdout.readable() returns False. Do you know how I can read from the terminal?
using os library:
info = os.popen('dir').read()
You can use the subprocess.check_output method
Example
import subprocess as sp
stdout = sp.check_output("dir")
print(stdout)
There is a bit of confusion here about the different streams, and possibly a better way to do things.
For the specific case of dir, you can replace the functionality you want with the os.listdir function, or better yet os.scandir.
For the more general case, you will not be able to read an arbitrary stdout stream. If you want to do that, you'll have to set up a subprocess whose I/O streams you control. This is not much more complicated than using os.system. you can use subprocess.run, for example:
content = subprocess.run("dir", stdout=subprocess.PIPE, check=True).stdout
The object returned by run has a stdout attribute that contains everything you need.
If you want to read just go with
x = input()
This reads a one line from the terminal. x is a string by default, but you can cast it, say to int, like so
x = int(x)
This question already has answers here:
How can I make one python file run another? [duplicate]
(8 answers)
How to redirect output with subprocess in Python?
(6 answers)
Closed 3 years ago.
I found a script online which converts a file from one specific extension to another, I am trying to utilize subprocess.call and have had no luck so far.
I am trying to use subprocess.call to convert a file in a directory.
I successfully managed to perform this in the terminal and am now trying to execute this in python as part of a program.
For the arguments, I am giving 'python' the name of the script, the file I want to convert and then the new name once it has been converted.
subprocess.call(["python", "converter.py", "file1.txt", ">", "file1converted.xml"])
Am I using subprocess.call in the correct manner? I can't seem to find anywhere on how to perform what I'm trying to do, also if possible I would like to find out a way on how I can convert a file without specifying the new name after conversion, but instead using change the 'file1.txt' to '.xml' once done.
There are two ways to do this, the first is the way you did it but you should add shell=True among the parameter. Do not use shell=True if the shell command is coming from an outside source, cause that would a huge security risk
subprocess.call(["python", "converter.py", "file1.txt", ">", "file1converted.xml"], shell=True)
The other way do this is to make use of Popen which I PERSONALLY prefer
subprocess.Popen(["python", "converter.py", "file1.txt", ">", "file1converted.xml"])
Using Python 3.5.x, not any greater version than that.
https://stackoverflow.com/a/30254551/257924 is the right answer, but doesn't provide a solution that is built into Python, but requires writing code from scratch:
I need to have a string that has a value of "-" to represent stdin, or its value is a path to a text file I want to read from. I want to use the with operator to open up either type of those files, without using conditional logic to check for "-" in my scripts. I have something that works, but it seems like it should be something that is built into Python core and not requiring me to roll my own context-manager, like this:
from contextlib import contextmanager
#contextmanager
def read_text_file_or_stdin(path):
"""Return a file object from stdin if path is '-', else read from path as a text file."""
if path == '-':
with open(0) as f:
yield f
else:
with open(path, 'r') as f:
yield f
# path = '-' # Means read from stdin
path = '/tmp/paths' # Means read from a text file given by this value
with read_text_file_or_stdin(path) as g:
paths = [path for path in g.read().split('\n') if path]
print("paths", paths)
I plan to pass in the argument to a script via something like -p - to mean "read from standard-input" or -p some_text_file meaning "read from some_text_file".
Does this require me to do the above, or is there something built into Python 3.5.x that provides this already? This seems like such a common need for writing CLI utilities, that it could have already been handled by something in the Python core or standard libraries.
I don't want to install any module/package from repositories outside of the Python standard library in 3.5.x, just for this.
The argparse module provides a FileType factory which knows about the - convention.
import argparse
p = argparse.ArgumentParser()
p.add_argument("-p", type=argparse.FileType("r"))
args = p.parse_args()
Note that args.p is an open file handle, so there's no need to open it "again". While you can still use it with a with statement:
with args.p:
for line in args.p:
...
this only ensures the file is closed in the event of an error in the with statement itself. Also, you may not want to use with, as this will close the file, even if you meant to use it again later.
You should probably use the atexit module to make sure the file gets closed by the end of the program, since it was already opened for you at the beginning.
import atexit
...
args = p.parse_args()
atexit.register(args.p.close)
Check https://docs.python.org/3/library/argparse.html#filetype-objects.
Where you can do like this
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('infile', nargs='?', type=argparse.FileType('r'),
... default=sys.stdin)
>>> parser.add_argument('outfile', nargs='?', type=argparse.FileType('w'),
... default=sys.stdout)
So infile & outfile will support read & write streams for stdin & stdout by default.
Or using my favorite library click
Check more details at the library docs website
https://click.palletsprojects.com/en/7.x/api/#click.File
https://click.palletsprojects.com/en/7.x/arguments/#file-arguments
File Arguments
Since all the examples have already worked with filenames, it makes sense to explain how to deal with files properly. Command line tools are more fun if they work with files the Unix way, which is to accept - as a special file that refers to stdin/stdout.
Click supports this through the click.File type which intelligently handles files for you. It also deals with Unicode and bytes correctly for all versions of Python so your script stays very portable.
The library is the most "BASH/DASH" features friendly I believe.
This question already has answers here:
How do I read from stdin?
(25 answers)
Closed 6 years ago.
I am doing small project in which I have to read the file from STDIN.
I am not sure what it means, what I asked the professor he told me,
there is not need to open the file and close like we generally do.
sFile = open ( "file.txt",'r')
I dont have to pass the file as a argument.
I am kind of confused what he wants.
The stdin takes input from different sources - depending on what input it gets.
Given a very simple bit of code for illustration (let's call it: script.py):
import sys
text = sys.stdin.read()
print text
You can either pipe your script with the input-file like so:
$ more file.txt | script.py
In this case, the output of the first part of the pipeline - which is the content of the file - is assigned to our variable(in this case text, which gets printed out eventually).
When left empty (i.e. without any additional input) like so:
$ python script.py
It let's you write the input similar to the input function and assigns the written input to the defined variable(Note that this input-"window" is open until you explicitly close it, which is usually done with Ctrl+D).
import sys, then sys.stdin will be the 'file' you want which you can use like any other file (e.g. sys.stdin.read()), and you don't have to close it. stdin means "standard input".
Might be helpful if you read through this post, which seems to be similar to yours.
'stdin' in this case would be the argument on the command line coming after the python script, so python script.py input_file. This input_file would be the file containing whatever data you are working on.
So, you're probably wondering how to read stdin. There are a couple of options. The one suggested in the thread linked above goes as follows:
import fileinput
for line in fileinput.input():
#read data from file
There are other ways, of course, but I think I'll leave you to it. Check the linked post for more information.
Depending on the context of your assignment, stdin may be automatically sent into the script, or you may have to do it manually as detailed above.
This question already has answers here:
Formatting a command in python subprocess popen
(2 answers)
Closed 7 years ago.
I have a bunch of different text files, and am trying to sort the texts into one file. I am using python's subprocess, and I wrote the following code
command_line = "sort -m 1.txt 2.txt > a.txt"
args = shlex.split(command_line)
subprocess.call(args)
and the subprocess.call(args) returned 2 as a result, and nothing was written in a.txt. Anything wrong with my code?
If you want to use the shell redirection operator > in your command line, you have to pass shell=True to subprocess.call. Otherwise, '>' and 'a.txt' are passed as command line arguments to sort. With shell=True, the command line is passed to and interpreted by an actual shell, and you should therefore not shlex.split it. It may be easier to use os.system instead of subprocess.call, which uses a shell by default.