from importlib.resources import path
import os, shutil, pathlib, fnmatch
import argparse
import time
parser = argparse.ArgumentParser()
parser.add_argument('--src', type=str)
parser.add_argument('--dst', type=str)
parser.add_argument('--pattern', type=str)
args = parser.parse_args()
def move_dir(src: str, dst: str, pattern: str = '*'):
if not os.path.isdir(dst):
pathlib.Path(dst).mkdir(parents=True, exist_ok=True)
for f in fnmatch.filter(os.listdir(src), pattern):
shutil.move(os.path.join(src, f), os.path.join(dst, f))
move_dir(args.dst(), args.src(), args.pattern())
time.sleep(5)
move_dir(r"C:\Users\user\Downloads", r"C:\Users\user\Documents\pdf", r"*.pdf")
Creating a script to move files based on pattern in a specific directory. Initially ran into the raw string error when I hard coded and solved by making the output raw as suggested here : How to move a file in Python?
PROBLEM
As I tested with the hard code and had to put r before the string I wanted to test moving back from to the old directory and then recopying all over again. (Code was ran correctly once without the 2nd to last 2 lines)
I think I'm now failing to do that when calling the argument string as the error returned shows double backslashes '\'
C:\Users\user\Documents\GitHub\personalUtilities>python downloadOrganizer.py --src "C:\Users\user\Documents\pdf" --dst "C:\Users\user\Downloads" --pattern "*.pdf"
usage: downloadOrganizer.py [-h] [--src SRC] [--dst DST] [--pattern PATTERN]
downloadOrganizer.py: error: argument --src: invalid path value: 'C:\\Users\\user\\Documents\\pdf'
I see a few articles on making a class to check if the directory exists but that wouldn't solve my problem if I'm passing the wrong string type and any article about this in the subject seems to be answering a different question or solving something else.
So whats the proper way to pass the source and destination directory strings from argparse?
Tried changing the type to path for src and dst but didn't change anything. tried adding r to the arg in the cli before the quote string that just made the directory extra wrong!
Related
Using Python 3.5.x, not any greater version than that.
https://stackoverflow.com/a/30254551/257924 is the right answer, but doesn't provide a solution that is built into Python, but requires writing code from scratch:
I need to have a string that has a value of "-" to represent stdin, or its value is a path to a text file I want to read from. I want to use the with operator to open up either type of those files, without using conditional logic to check for "-" in my scripts. I have something that works, but it seems like it should be something that is built into Python core and not requiring me to roll my own context-manager, like this:
from contextlib import contextmanager
#contextmanager
def read_text_file_or_stdin(path):
"""Return a file object from stdin if path is '-', else read from path as a text file."""
if path == '-':
with open(0) as f:
yield f
else:
with open(path, 'r') as f:
yield f
# path = '-' # Means read from stdin
path = '/tmp/paths' # Means read from a text file given by this value
with read_text_file_or_stdin(path) as g:
paths = [path for path in g.read().split('\n') if path]
print("paths", paths)
I plan to pass in the argument to a script via something like -p - to mean "read from standard-input" or -p some_text_file meaning "read from some_text_file".
Does this require me to do the above, or is there something built into Python 3.5.x that provides this already? This seems like such a common need for writing CLI utilities, that it could have already been handled by something in the Python core or standard libraries.
I don't want to install any module/package from repositories outside of the Python standard library in 3.5.x, just for this.
The argparse module provides a FileType factory which knows about the - convention.
import argparse
p = argparse.ArgumentParser()
p.add_argument("-p", type=argparse.FileType("r"))
args = p.parse_args()
Note that args.p is an open file handle, so there's no need to open it "again". While you can still use it with a with statement:
with args.p:
for line in args.p:
...
this only ensures the file is closed in the event of an error in the with statement itself. Also, you may not want to use with, as this will close the file, even if you meant to use it again later.
You should probably use the atexit module to make sure the file gets closed by the end of the program, since it was already opened for you at the beginning.
import atexit
...
args = p.parse_args()
atexit.register(args.p.close)
Check https://docs.python.org/3/library/argparse.html#filetype-objects.
Where you can do like this
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('infile', nargs='?', type=argparse.FileType('r'),
... default=sys.stdin)
>>> parser.add_argument('outfile', nargs='?', type=argparse.FileType('w'),
... default=sys.stdout)
So infile & outfile will support read & write streams for stdin & stdout by default.
Or using my favorite library click
Check more details at the library docs website
https://click.palletsprojects.com/en/7.x/api/#click.File
https://click.palletsprojects.com/en/7.x/arguments/#file-arguments
File Arguments
Since all the examples have already worked with filenames, it makes sense to explain how to deal with files properly. Command line tools are more fun if they work with files the Unix way, which is to accept - as a special file that refers to stdin/stdout.
Click supports this through the click.File type which intelligently handles files for you. It also deals with Unicode and bytes correctly for all versions of Python so your script stays very portable.
The library is the most "BASH/DASH" features friendly I believe.
the string that contains a file looks like this in the console:
>>> target_file
'src//data//annual_filings//ABB Ltd//ABB_ar_2015.pdf'
I got the target_file from a call to os.walk
The goal is to build a command to run in subprocess.call
Something like:
from subprocess import call
cmd_ = r'qpdf-7.0.0/bin/qpdf --password=%s --decrypt %s %s' %('', target_file, target_file)
call([cmd_])
I tried different variations, setting shell to either True or False.
Replacing the // with /,\ etc.
The issue seems to be with the space in the folder (I can not change the folder name).
The python code needs to run on Windows
you have to define cmd_ as a list of arguments not a list with a sole string in it, or subprocess interprets the string as the command (doesn't even try to split the args):
cmd_ = ['qpdf-7.0.0/bin/qpdf','--password=%s'%'','--decrypt',target_file, target_file]
call(cmd_)
and leave the quoting to subprocess
As a side note, no need to double the slashes. It works, but that's unnecessary.
I'm trying to make a python script that search words in files.
If I pass txt it will only look in files with .txt extension, but I want to pass * as argument to search in every files.
if sys.argv[4] == "*"
Don't work and if I try
print sys.argv[4]
It print the name of the script
find.py
But not the same way as
print sys.argv[0]
As it will return
./find.py
So, someone already had this problem and, of course, solved it ?
Your shell attaches meaning to * as well. You need to escape it when calling your script to prevent the shell from expanding it:
python find.py \*
sys.argv[0] is the exact name passed used to run the script. That can be a relative path (./find.py, ../bin/find.py) or an absolute path, depending on how it was invoked. Use os.path.abspath() to normalize it.
I am a beginner at Python. Below is the testing code for Python's command line args. If executing from command line with different parameter formats, I get different results, but it feels strange, can anyone help me understand why?
1, $test.py d:\ --> this seems ok for os.walk call
2, $test.py 'd:\' --> this will cause nothing output
BTW: I used Python 2.7.3
Test code:
import os
import sys
if __name__ == '__main__':
argMock = 'D:\\'
path = len(sys.argv) > 1 and sys.argv[1] or argMock
for root, dirs, files in os.walk(path):
for name in files:
print name
Maresh and Jakub's answers are incorrect.
The command line argument d:\ will become the string "d:\".
The command line argument 'd:\' will become the string "'d:\'".
Running the following with input 'D:\':
print sys.argv[1] # $test.py 'D:\\'
print argMock
yields:
'D:\\'
D:\
The issue is that putting quote marks around something that is already considered a string will just include the quote marks as part of the string.
The problem doesn't come from your program, it comes from the shell interpretation.
When you write 'd:\' your shell interprets the backslash as an escaping command for the next caracter. Therefore you must escape the backslash like this: 'd:\\'
How do you get a file name from command line when you run a Python code? Like if your code opens a file and reads the line, but the file varies whenever you run it, how to you say:
python code.py input.txt
so the code analyzes "input.txt"? What would you have to do in the actual Python code? I know, this is a pretty vague question, but I don't really know how to explain it any better.
A great option is the fileinput module, which will grab any or all filenames from the command line, and give the specified files' contents to your script as though they were one big file.
import fileinput
for line in fileinput.input():
process(line)
More information here.
import sys
filename = sys.argv[-1]
This will get the last argument on the command line. If no arguments are passed, it will be the script name itself, as sys.argv[0] is the name of the running program.
Using argparse is quite intuitive:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--file", "-f", type=str, required=True)
args = parser.parse_args()
Now the name of the file is located in:
args.file
You just have to run the program a little differently:
python code.py -f input.txt
Command line parameters are available as a list via the sys module's argv list. The first element in the list is the name of the program (sys.argv[0]). The remaining elements are the command line parameters.
See also the getopt, optparse, and argparse modules for more complex command line parsing.
If you're using Linux or Windows PowerShell you could pipe " | " it after using cat on input.txt file, suppose you have input.txt file and your code.py file in same directory you could use:
cat input.txt | python code.py
This will provide python input from STDIN. for example: if for example you're trying get names from input.txt file
input.txt has
john,matthew,peter,albert
code.py has
print(" is not ".join(input().rstrip().split(',')))
would give
john is not matthew is not peter is not albert
I also like argparse, it's clean, simple, fairly standard, gives free error handling, and add a [-h] option to help the user.
Here is a version that do not need the named parameters, which may be annoying for a very simple script:
#!/usr/bin/python3
import argparse
arg_parser = argparse.ArgumentParser( description = "Copy source_file as target_file." )
arg_parser.add_argument( "source_file" )
arg_parser.add_argument( "target_file" )
arguments = arg_parser.parse_args()
source = arguments.source_file
target = arguments.target_file
print( "Copying [{}] to [{}]".format(source, target) )
Example of how it handles errors and help for you:
>my_test.py
usage: my_test.py [-h] source_file target_file
my_test.py: error: the following arguments are required: source_file, target_file
>my_test.py my_source.cpp
usage: my_test.py [-h] source_file target_file
my_test.py: error: the following arguments are required: target_file
>my_test.py -h
usage: .py [-h] source_file target_file
Copy source_file as target_file.
positional arguments:
source_file
target_file
optional arguments:
-h, --help show this help message and exit
>my_test.py my_source.cpp my_target.cpp
Copying [my_source.cpp] to [my_target.cpp]
In addition to what is mentioned by the already existing answers, there is an other alternative relying on the use of Command Line Interface Creation Kit (Click). Its latest stable version by the time I posted this answer is version 6. The official documentation has examples on how to deal with files and pass them as command line arguments.
Just use the basic command raw_input
declare input file name as string
inFile = ""
inFile = raw_input("Enter the input File Name: ")
Now you can open the file by using with open(inFile,'w')