I'm trying to import a json file into firebase using a python script. Then I came across a similar script in Github. I tried running it after the replacing the required parameters. I'm getting a too few arguments error. This is the Github project if you would like to refer
https://github.com/firebase/firebase-streaming-import
Snippet of the code I'm running is given below. I'm using IDLE to run the program. Thanks in advance
argParser = argparse.ArgumentParser(description="Import a large json file into a Firebase via json Streaming.\
Uses HTTP PATCH requests. Two-pass script, run once normally,\
then again in --priority_mode.")
argParser.add_argument('firebase_url', help="Specify the Firebase URL (e.g. https://test.firebaseio.com/dest/path/).")
argParser.add_argument('json_file', help="The JSON file to import.")
argParser.add_argument('-a', '--auth', help="Optional Auth token if necessary to write to Firebase.")
argParser.add_argument('-t', '--threads', type=int, default=8, help='Number of parallel threads to use, default 8.')
argParser.add_argument('-s', '--silent', action='store_true',
help="Silences the server response, speeding up the connection.")
argParser.add_argument('-p', '--priority_mode', action='store_true',
help='Run this script in priority mode after running it in normal mode to write all priority values.')
main(argParser.parse_args())
Since you cannot parse arguments with parameters like in terminal/cmd in IDLE. Try parsing the args as such : args = argParser.parse_args(['URL','FILE_PATH'])
import argparse
argParser = argparse.ArgumentParser(description="Import a large json file into a Firebase via json Streaming.\
Uses HTTP PATCH requests. Two-pass script, run once normally,\
then again in --priority_mode.")
argParser.add_argument('firebase_url', help="Specify the Firebase URL (e.g. https://test.firebaseio.com/dest/path/).")
argParser.add_argument('json_file', help="The JSON file to import.")
argParser.add_argument('-a', '--auth', help="Optional Auth token if necessary to write to Firebase.")
argParser.add_argument('-t', '--threads', type=int, default=8, help='Number of parallel threads to use, default 8.')
argParser.add_argument('-s', '--silent', action='store_true',
help="Silences the server response, speeding up the connection.")
argParser.add_argument('-p', '--priority_mode', action='store_true',
help='Run this script in priority mode after running it in normal mode to write all priority values.')
args = argParser.parse_args(['URL','FILE_PATH'])
print(args)
main(args)
Output:
Namespace(auth=None, firebase_url='URL', json_file='FILE_PATH', priority_mode=False, silent=False, threads=8)
Related
I am testing a job. For that, I run the script via notebook using the following code.
I want to be able to retrieve the value of the arg
"dataset" via "argparse". I don't understand what I am doing wrong here:
The notebook that runs the job:
from sagemaker.processing import Processor, ProcessingInput, ProcessingOutput
from sagemaker.network import NetworkConfig
netconfig = NetworkConfig(XXX)
dataset= 'A'
processor = Processor(image_uri='XXX',
role='XXX',
instance_count=1,
instance_type="XXX",
network_config=netconfig)
processor.run(inputs=[ProcessingInput(
source='s3://input-bucket/settings_dataset_A.json',
destination='/opt/ml/processing/input')],
outputs=[ProcessingOutput(
source='/opt/ml/processing/output',
destination='s3://output-bucket')],
arguments=["--verbose", "--dataset", dataset],
wait=True,
logs=True)
and the code in the script:
parser = argparse.ArgumentParser()
parser.add_argument('--verbose', help="verbose",action="store_true")
parser.add_argument("--dataset", type=str, required =False)
args = parser.parse_args()
print('Arguments:', sys.argv)
print(args.dataset)
It gives an error: "unrecognized argument" but it is able to print the correct value ! And then it just stops running the rest of the code.
Do you see anything wrong with the code above ? Thanks a lot
I am trying to set up training arguments and parse but I got this error could anyone help please!
parser = argparse.ArgumentParser(description='Explore pre-trained AlexNet')
parser.add_argument(
'--image_path', type=str,
help='Full path to the input image to load.')
parser.add_argument(
'--use_pre_trained', type=bool, default=True,
help='Load pre-trained weights?')
args = parser.parse_args()
got this error
usage: ipykernel_launcher.py [-h] [--image_path IMAGE_PATH]
[--use_pre_trained USE_PRE_TRAINED]
ipykernel_launcher.py: error: unrecognized arguments: -f /root/.local/share/jupyter/runtime/kernel-ff8e2476-e39b-4e40-b8f9-6b8113fe8f1f.json
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
In a Jupyter notebook cell:
import sys
sys.argv
I get get
['/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py',
'-f',
'/home/paul/.local/share/jupyter/runtime/kernel-7923bfd2-9f96-45cf-8b44-1859a2185715.json']
The Jupyter server is using the sys.argv to set up the communication channel with your kernel. argparse parses this list too.
So commandline and argparse cannot be used to provide arguments to your notebook when run this way.
How did you start this script? Did you even try to provide the commandline values that the script expected?
'--image_path'
'--use_pre_trained'
If you did, you probably would have gotten a different parser's error, about 'unexpected arguments'. That's coming from the server.
If you use a Colab may this solution help you, this solution suggest to you to write argparse in the other python file introduction-argparse-colab
%%writefile parsing.py
import argparse
parser = argparse.ArgumentParser()
parser.parse_args()
In a Jupyter notebook code works like this:
parser = argparse.ArgumentParser()
parser.add_argument('--image_folder', type=str, default='/content/image', help='path to image folder')
parser.add_argument("-f", "--file", required=False)
You need to add in the end of " parser.add_argument " line ( This is the reason you are getting 'SystemExit: 2' error ):
parser.add_argument("-f", required=False)
in your case this should work:
parser = argparse.ArgumentParser(description='Explore pre-trained AlexNet')
parser.add_argument('--image_path', type=str, default='your_path', help='Full path to the input image.')
parser.add_argument('--use_pre_trained', type=bool, default=True, help='Load pre-trained weights?')
parser.add_argument("-f", "--file", required=False)
args = parser.parse_args()
Now you can call:
image = args.image_path
Or
from PIL import Image
image = Image.open(args.image_path)
Tested in Google colab
Recently I downloaded pypdfocr, however, in the documentation there are no examples of how to call pypdfocr as a library, could anybody help me to call it just to convert a single file?. I just found a terminal command:
$ pypdfocr filename.pdf
If you're looking for the source code, it's normally under the directory site-package of your python installation. What's more, if you're using a IDE (i.e. Pycharm), it would help you find the directory and file. This is extremly useful to find class as well and show you how you can instantiate it, for example :
https://github.com/virantha/pypdfocr/blob/master/pypdfocr/pypdfocr.py
this file has a pypdfocr class type you can re-use and, possibly, do what a command-line would do.
In that class, the developper has put a lot of argument to be parsed :
def get_options(self, argv):
"""
Parse the command-line options and set the following object properties:
:param argv: usually just sys.argv[1:]
:returns: Nothing
:ivar debug: Enable logging debug statements
:ivar verbose: Enable verbose logging
:ivar enable_filing: Whether to enable post-OCR filing of PDFs
:ivar pdf_filename: Filename for single conversion mode
:ivar watch_dir: Directory to watch for files to convert
:ivar config: Dict of the config file
:ivar watch: Whether folder watching mode is turned on
:ivar enable_evernote: Enable filing to evernote
"""
p = argparse.ArgumentParser(description = "Convert scanned PDFs into their OCR equivalent. Depends on GhostScript and Tesseract-OCR being installed.",
epilog = "PyPDFOCR version %s (Copyright 2013 Virantha Ekanayake)" % __version__,
)
p.add_argument('-d', '--debug', action='store_true',
default=False, dest='debug', help='Turn on debugging')
p.add_argument('-v', '--verbose', action='store_true',
default=False, dest='verbose', help='Turn on verbose mode')
p.add_argument('-m', '--mail', action='store_true',
default=False, dest='mail', help='Send email after conversion')
p.add_argument('-l', '--lang',
default='eng', dest='lang', help='Language(default eng)')
p.add_argument('--preprocess', action='store_true',
default=False, dest='preprocess', help='Enable preprocessing. Not really useful now with improved Tesseract 3.04+')
p.add_argument('--skip-preprocess', action='store_true',
default=False, dest='skip_preprocess', help='DEPRECATED: always skips now.')
#---------
# Single or watch mode
#--------
single_or_watch_group = p.add_mutually_exclusive_group(required=True)
# Positional argument for single file conversion
single_or_watch_group.add_argument("pdf_filename", nargs="?", help="Scanned pdf file to OCR")
# Watch directory for watch mode
single_or_watch_group.add_argument('-w', '--watch',
dest='watch_dir', help='Watch given directory and run ocr automatically until terminated')
#-----------
# Filing options
#----------
filing_group = p.add_argument_group(title="Filing optinos")
filing_group.add_argument('-f', '--file', action='store_true',
default=False, dest='enable_filing', help='Enable filing of converted PDFs')
#filing_group.add_argument('-c', '--config', type = argparse.FileType('r'),
filing_group.add_argument('-c', '--config', type = lambda x: open_file_with_timeout(p,x),
dest='configfile', help='Configuration file for defaults and PDF filing')
filing_group.add_argument('-e', '--evernote', action='store_true',
default=False, dest='enable_evernote', help='Enable filing to Evernote')
filing_group.add_argument('-n', action='store_true',
default=False, dest='match_using_filename', help='Use filename to match if contents did not match anything, before filing to default folder')
# Add flow option to single mode extract_images,preprocess,ocr,write
args = p.parse_args(argv)
You can use any of those argument to be passed to it's parser, like this :
import pypdfocr
obj = pypdfocr.pypdfocr.pypdfocr()
obj.get_options([]) # this makes it takes default, but you could add CLI option to it. Other option might be [-v] or [-d,-v]
I hope this help you understand in the mean time :)
I am trying to write a program that reads from a file and writes to a new file using the command line. I am using the CommandLine class and using ArgParse to accept different args as well as the file names for input and output. I am trying to redirect stdin and stdout to these files. However, I keep getting an error that I put at the bottom of the code below. Am I inputting the arguments to the command line incorrectly, or is something else going on? All of my files are in the same folder.
class CommandLine() :
'''
Handle the command line, usage and help requests.
CommandLine uses argparse, now standard in 2.7 and beyond.
it implements a standard command line argument parser with various argument options,
a standard usage and help, and an error termination mechanism do-usage_and_die.
attributes:
all arguments received from the commandline using .add_argument will be
avalable within the .args attribute of object instantiated from CommandLine.
For example, if myCommandLine is an object of the class, and requiredbool was
set as an option using add_argument, then myCommandLine.args.requiredbool will
name that option.
'''
def __init__(self, inOpts=None):
'''
CommandLine constructor.
Implements a parser to interpret the command line argv string using argparse.
'''
import argparse
self.parser = argparse.ArgumentParser(description = 'Program prolog - a brief description of what this thing does',
epilog = 'Program epilog - some other stuff you feel compelled to say',
add_help = True, #default is True
prefix_chars = '-',
usage = '%(prog)s [options] -option1[default] <input >output'
)
self.parser.add_argument('inFile', action = 'store', help='input file name')
self.parser.add_argument('outFile', action = 'store', help='output file name')
self.parser.add_argument('-lG', '--longestGene', action = 'store', nargs='?', const=True, default=False, help='longest Gene in an ORF')
self.parser.add_argument('-mG', '--minGene', type=int, choices= (0, 100, 200, 300, 500, 1000), action = 'store', help='minimum Gene length')
self.parser.add_argument('-s', '--start', action = 'append', nargs='?', help='start Codons') #allows multiple list options
self.parser.add_argument('-st', '--stop', action = 'append', nargs='?', help='stop Codons') #allows multiple list options
self.parser.add_argument('-v', '--version', action='version', version='%(prog)s 0.1')
if inOpts is None :
self.args = self.parser.parse_args()
else :
self.args = self.parser.parse_args(inOpts)
C:\Users\Zach\Documents\UCSC\BME 160\Lab 5>python findORFsCmdLine.py -- minGene=3
00 --longestGene --start=ATG --stop=TAG --stop=TGA --stop=TAA <tass2.fa >result.
txt
usage: findORFsCmdLine.py [options] -option1[default] <input >output
findORFsCmdLine.py: error: the following arguments are required: inFile, outFile
The error tells you that you did not provide inFile and outFile. This seems confusing because you did indeed provide <tass2.fa and >result.txt on the command line.
On the command line < and > have special meaning. <tass2.fa is handled by bash (or whichever shell you use) and it opens the file tass2.fa and sends the contents to your program over stdin. It then removes that from the list of command line arguments because bash has already taken care of it.
A similar thing happens with >result.txt where any output from your file that normally goes to stdout such as print calls will be written to that file. Again bash handles this so your program never gets the argument from the command line.
To make sure the filenames go to your program you need to remove the < and > symbols.
In addition to that, the way you are call add_argument for inFile and outFile is not correct. Using action='store' simply stores the value of the argument under the given name. This is the default action so doing that is redundant.
You need to tell add_argument that these are file types. Then you can use these as files and read and write as you desire.
parser.add_argument('inFile', type=argparse.FileType('r'))
parser.add_argument('outFile', type=argparse.FileType('w'))
If you wish to allow these arguments to be optional and use stdin and stdout automatically if they do not supply a filename then you can do this:
parser.add_argument('inFile', type=argparse.FileType('r'),
nargs='?', default=sys.stdin)
parser.add_argument('outFile', type=argparse.FileType('w'),
nargs='?', default=sys.stdout)
You say you are writing a program that reads from a file and writes to a new file using the command line and go on to say that you are trying to redirect these files to stdin and stdout.
If it is really your intention to redirect the input and output files using the shell with the > and < operators, then you do not need to call add_argument with outFile at all. Just let the shell handle it. Remove that line from your program and use print or similar to send your content to stdout and that file will be created. You will still need the call add_argument('inFile', ....
This question is similar to other python from within python script calling but none of those are working for me.
I have a chat script which uses Python XMPP to send a chat message. The syntax is as so:
python chat.py -c "recipient#example.com" -u "sender#example.com" -p "secret" -m "message"
script:
#!/usr/bin/python
import sys
import argparse
import xmpp
def main(argv):
parser = argparse.ArgumentParser()
parser.add_argument('-c', dest='contact', required=True)
parser.add_argument('-u', dest='username', required=True)
parser.add_argument('-p', dest='password', required=True)
parser.add_argument('-m', dest='message', required=False, default='')
args = parser.parse_args(argv)
if (args.message == '') and not sys.stdin.isatty():
for line in sys.stdin:
args.message = args.message + line
jid = xmpp.protocol.JID(args.username)
jabber = xmpp.Client(jid.getDomain(), debug=[])
jabber.connect(server=(jid.getDomain(), 5222) )
jabber.auth(jid.getNode(), args.password)
jabber.send(xmpp.Message(args.contact, args.message.strip()))
if __name__ == "__main__":
main(sys.argv[1:])
as you can see, it takes 4 arguments.
Now I have another python script which is listening to sensors. I am trying to get it to send chat messages when it detects sensor readings so from within listen.py I am doing this:
...
import chat
...
chat.main('-c "chatto#server.com" -u "chatfrom#server.com" -p "password" -m "Yo Yo Yo Wassup"')
....
I have also tried subprocess.call but perhaps have not gotten the syntax correct. both python scripts are in the same directory. So for those of you looking for a specific question, How can I call the chat.py from within listen.py while providing the four required args?
parse_args() is going to expect a list of strings. For simple ones you can just split() on spaces, but in your case you have a complex argument with internal spaces so that won't work.
Don't include quotes in the strings. Those are used to allow an argument to have internal spaces when defined from the shell, but they aren't sent into argv.
chat.main(['-c', 'chatto#server.com', '-u', 'chatfrom#server.com', '-p', 'password', '-m', 'Yo Yo Yo Wassup'])
See more here: https://docs.python.org/2/library/argparse.html#beyond-sys-argv