How does one structure their command line processing block in such a way as to allow naming multiple files in any order AND to discover those file types by their suffixes?
In this Python program, I need to pass both a binary file and a .vhdr file to my command line. The .vhdr file will be read to memory, while the (large) binary file will be processed incrementally. I would like to build this in a way such that the user can pass the file names in any order. It seems to me that an intelligent way to deal with this would be to iterate over each item in argv, check if it has a ".vhdr" suffix, and use whichever item has this to save to my file object.
Do any libraries have this functionality, or should I write this from scratch? I was not able to find something like this in the argparse library, but I am new so I easily could have looked right at it and not understood.
Use well known argparse library. Simple example
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--vhdr", dest="vhdr_file")
parser.add_argument("--bin", dest="bin_file")
args = parser.parse_args()
print(args)
output:
$ python demo.py --vhdr 1 --bin 2
Namespace(bin_file='2', vhdr_file='1')
$ python demo.py --bin 1 --vhdr 2
Namespace(bin_file='1', vhdr_file='2')
Related
Using Python 3.5.x, not any greater version than that.
https://stackoverflow.com/a/30254551/257924 is the right answer, but doesn't provide a solution that is built into Python, but requires writing code from scratch:
I need to have a string that has a value of "-" to represent stdin, or its value is a path to a text file I want to read from. I want to use the with operator to open up either type of those files, without using conditional logic to check for "-" in my scripts. I have something that works, but it seems like it should be something that is built into Python core and not requiring me to roll my own context-manager, like this:
from contextlib import contextmanager
#contextmanager
def read_text_file_or_stdin(path):
"""Return a file object from stdin if path is '-', else read from path as a text file."""
if path == '-':
with open(0) as f:
yield f
else:
with open(path, 'r') as f:
yield f
# path = '-' # Means read from stdin
path = '/tmp/paths' # Means read from a text file given by this value
with read_text_file_or_stdin(path) as g:
paths = [path for path in g.read().split('\n') if path]
print("paths", paths)
I plan to pass in the argument to a script via something like -p - to mean "read from standard-input" or -p some_text_file meaning "read from some_text_file".
Does this require me to do the above, or is there something built into Python 3.5.x that provides this already? This seems like such a common need for writing CLI utilities, that it could have already been handled by something in the Python core or standard libraries.
I don't want to install any module/package from repositories outside of the Python standard library in 3.5.x, just for this.
The argparse module provides a FileType factory which knows about the - convention.
import argparse
p = argparse.ArgumentParser()
p.add_argument("-p", type=argparse.FileType("r"))
args = p.parse_args()
Note that args.p is an open file handle, so there's no need to open it "again". While you can still use it with a with statement:
with args.p:
for line in args.p:
...
this only ensures the file is closed in the event of an error in the with statement itself. Also, you may not want to use with, as this will close the file, even if you meant to use it again later.
You should probably use the atexit module to make sure the file gets closed by the end of the program, since it was already opened for you at the beginning.
import atexit
...
args = p.parse_args()
atexit.register(args.p.close)
Check https://docs.python.org/3/library/argparse.html#filetype-objects.
Where you can do like this
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('infile', nargs='?', type=argparse.FileType('r'),
... default=sys.stdin)
>>> parser.add_argument('outfile', nargs='?', type=argparse.FileType('w'),
... default=sys.stdout)
So infile & outfile will support read & write streams for stdin & stdout by default.
Or using my favorite library click
Check more details at the library docs website
https://click.palletsprojects.com/en/7.x/api/#click.File
https://click.palletsprojects.com/en/7.x/arguments/#file-arguments
File Arguments
Since all the examples have already worked with filenames, it makes sense to explain how to deal with files properly. Command line tools are more fun if they work with files the Unix way, which is to accept - as a special file that refers to stdin/stdout.
Click supports this through the click.File type which intelligently handles files for you. It also deals with Unicode and bytes correctly for all versions of Python so your script stays very portable.
The library is the most "BASH/DASH" features friendly I believe.
I'm confused on how exactly we should use the python sh library, specifically the sh.Command(). Basically, I wish to pass input_file_a to program_b.py and store its output in a different directory as output_file_b, how should I achieve this using the sh library in python?
If you mean input and output redirection, then see here (in) and here (out) respectively. In particular, looks like to "redirect" stdin you need to pass as argument the actual bytes (e.g. read them beforehand), in particular, the following should work according to their documentation (untested, as I don't have/work with sh - please let know if this works for you / fix whatever is missing):
import sh
python3 = sh.Command("python3")
with open(input_file_a, 'r') as ifile:
python3("program_b.py", _in=ifile.read(), _out=output_file_b)
Note that may need to specify argument search_paths for sh.Command for it to find python. Also, may need to specify full path to program_b.py file or os.chdir() accordingly.
This question already has answers here:
How do I read from stdin?
(25 answers)
Closed 6 years ago.
I am doing small project in which I have to read the file from STDIN.
I am not sure what it means, what I asked the professor he told me,
there is not need to open the file and close like we generally do.
sFile = open ( "file.txt",'r')
I dont have to pass the file as a argument.
I am kind of confused what he wants.
The stdin takes input from different sources - depending on what input it gets.
Given a very simple bit of code for illustration (let's call it: script.py):
import sys
text = sys.stdin.read()
print text
You can either pipe your script with the input-file like so:
$ more file.txt | script.py
In this case, the output of the first part of the pipeline - which is the content of the file - is assigned to our variable(in this case text, which gets printed out eventually).
When left empty (i.e. without any additional input) like so:
$ python script.py
It let's you write the input similar to the input function and assigns the written input to the defined variable(Note that this input-"window" is open until you explicitly close it, which is usually done with Ctrl+D).
import sys, then sys.stdin will be the 'file' you want which you can use like any other file (e.g. sys.stdin.read()), and you don't have to close it. stdin means "standard input".
Might be helpful if you read through this post, which seems to be similar to yours.
'stdin' in this case would be the argument on the command line coming after the python script, so python script.py input_file. This input_file would be the file containing whatever data you are working on.
So, you're probably wondering how to read stdin. There are a couple of options. The one suggested in the thread linked above goes as follows:
import fileinput
for line in fileinput.input():
#read data from file
There are other ways, of course, but I think I'll leave you to it. Check the linked post for more information.
Depending on the context of your assignment, stdin may be automatically sent into the script, or you may have to do it manually as detailed above.
Can my Python script take its arguments from a file, rather than the command line? I don't mind passing the file containing the arguments on the command line.
I am using argparse.
The reason is that I have a very complex argument list. I suppose that I could just wrap the call in a batch file, or another Python script, but I wondered if this is possible and thought that I would ask and maybe learn something.
So, instead of myScript.py --arg_1=xxx --arg2_=xxx ... --arg_n=xxx, can I
myScript.py --file args.txt where args.txt contains
--arg_1=xxx
--arg_2=xxx
...
--arg_n=xxx
You can tell the parser that arguments beginning with certain characters are actually names of files containing more arguments. From the documentation:
>>> with open('args.txt', 'w') as fp:
... fp.write('-f\nbar')
>>> parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
>>> parser.add_argument('-f')
>>> parser.parse_args(['-f', 'foo', '#args.txt'])
Namespace(f='bar')
The parser reads from args.txt and treats each line as a separate argument.
You can do this by taking command line argument as filename and then opening it.
like
file_name=sys.argv[1]
f=open(file_name)
arguments= f.read()
user_input=arguments.split()
user_argument=[]
for i in range():
user_argument.append(user_input[i])
Here you get the list of user argument in the list user_argument. Perheps you will get what you want!
I want to pass an input fasta file stored in a variable say inp_a from python to bowtie and write the output into another out_a. I want to use
os.system ('bowtie [options] inp_a out_a')
Can you help me out
Your question asks for two things, as far as I can tell: writing data to disk, and calling an external program from within Python. Without more detailed requirements, here's what I would write:
import subprocess
data_for_bowtie = "some genome data, lol"
with open("input.fasta", "wb") as input_file:
input_file.write(data_for_bowtie)
subprocess.call(["bowtie", "input.fasta", "output.something"])
There are some fine details here which I have assumed. I'm assuming that you mean bowtie, the read aligner. I'm assuming that your file is a binary, non-human-readable one (which is why there's that b in the second argument to open) and I'm making baseless assumptions about how to call bowtie on the command line because I'm not motivated enough to spend the time learning it.
Hopefully, that provides a starting point. Good luck!