We Keep Coding

Textract: failed with exit code 127 // windows 10 // pdftotext

When I'm trying to run my (after deploying with pyinstaller) program for reading and converting a PDF file and entering it into a google sheet. I get the error shown in the image below. However I can not seem to figure out what the problem is:
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\utils.py", line 82, in run
pipe = subprocess.Popen(
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\tkinter\__init__.py", line 1883, in __call__
return self.func(*args)
File "EinkaufRGWindows.py", line 40, in InkoopRekeningen
text = textract.process(str(importfolder) + str(i))
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\__init__.py", line 77, in process
return parser.process(filename, encoding, **kwargs)
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\utils.py", line 46, in process
byte_string = self.extract(filename, **kwargs)
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\pdf_parser.py", line 28, in extract
raise ex
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\pdf_parser.py", line 20, in extract
return self.extract_pdftotext(filename, **kwargs)
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\pdf_parser.py", line 43, in extract_pdftotext
stdout, _ = self.run(args)
File "C:\Users\trpfinance\AppData\Local\Programs\Python\Python38-32\lib\site-packages\textract\parsers\utils.py", line 90, in run
raise exceptions.ShellError(
textract.exceptions.ShellError: The command `pdftotext //Mac/Home/Desktop/Wickey Einkauf Test/Rekeningen/Lekkerkerker_ - 20803471.pdf -` failed with exit code 127
------------- stdout -------------
------------- stderr -------------

You're getting a FileNotFoundError it seems. If you look at the error, the command being run is:
pdftotext //Mac/Home/Desktop/Wickey Einkauf Test/Rekeningen/Lekkerkerker_ -
0803471.pdf -
There are a couple of things here I would look at. Firstly, there is an extra slash at the start of your file path, which seems wrong. Secondly, you have spaces in the file path, but there are no quotations enclosing the path. This second part means pdftotext will read this as a few separate command arguments, rather than one. You can fix this by formatting you subprocess call to have the file wrapped in quotation marks, like so:
pdftotext "example file path.pdf" -

You need to install pdftotext using pip.
To install it you need to have Microsoft Visual C++ 14 or greater.

I had the same issue. It seems to be an OS issue. For me, switching to GIT bash worked. https://github.com/deanmalmgren/textract/issues/229
If you are using Pycharm, change default terminal to bash.

What is the correct way to use stdin, stdout, and stderr in Python 2.7?

Does anyone have experience using the subprocess.call command in Python?
I keep getting errors whenever a line like this is in my code:
INFILE1 = open(script_dir+"/Scripts/plot_TSS_profile.R","r")
subprocess.call("Rscript","--slave","--args",filenames["housekeeping_profile"]+" "+filenames["unexpressed_profile"]+" "+filenames["profile_plot"],stdin=INFILE1, stderr=ERR_LOG,stdout=OUT_LOG,shell=True)
INFILE1.close().
If I leave the code as is, I get an error that the program finds multiple values for each th stdin, stderr, and stdout for some reason even though those are the only ones in the code. If I take out these parameters and just put the infile early in the brackets after “--args”, it doesn’t seem to read the file as it says the ‘buffer should be an integer.’
For example, this way gives the buffer error:
INFILE1 = script_dir+"/Scripts/plot_TSS_profile.R"
subprocess.call("Rscript",INFILE1,"--slave","--args",filenames["housekeeping_profile"]+" "+filenames["unexpressed_profile"]+" "+filenames["profile_plot"],stderr=ERR_LOG,shell=True)
INFILE1.close()
Here are my error outputs for more specific information:
The one on buffsize:
Traceback (most recent call last):
File "/mnt/work1/users/pughlab/projects/IEG_MiSEQ/Inferring_DNA_Expression/ExpressionPrediction-master/expression_prediction.py", line 277, in
step5(ERR_LOG,OUT_LOG,args,proj_dir,script_dir,filenames)
File "/mnt/work1/users/pughlab/projects/IEG_MiSEQ/Inferring_DNA_Expression/ExpressionPrediction-master/expression_prediction.py", line 165, in step5
subprocess.call("Rscript",INFILE1,"--slave","--args",filenames["housekeeping_profile"]+" "+filenames["unexpressed_profile"]+" "+filenames["profile_plot"],stderr=ERR_LOG,shell=True)
File "/mnt/work1/software/centos7/python/2.7.15/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
File "/mnt/work1/software/centos7/python/2.7.15/lib/python2.7/subprocess.py", line 343, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
And the other error on there being multiple values:
Traceback (most recent call last):
File "/mnt/work1/users/pughlab/projects/IEG_MiSEQ/Inferring_DNA_Expression/ExpressionPrediction-master/expression_prediction.py", line 272, in <module>
step5(ERR_LOG,OUT_LOG,args,proj_dir,script_dir,filenames)
File "/mnt/work1/users/pughlab/projects/IEG_MiSEQ/Inferring_DNA_Expression/ExpressionPrediction-master/expression_prediction.py", line 165, in step5
subprocess.call("Rscript","--slave","--args",filenames["housekeeping_profile"]+" "+filenames["unexpressed_profile"]+" "+filenames["profile_plot"],stdin=INFILE1,stderr=ERR_LOG,stdout=OUT_LOG,shell=True)
File "/mnt/work1/software/centos7/python/2.7.15/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
TypeError: __init__() got multiple values for keyword argument 'stdin'
Thank you

The main problem is that call takes a list of arguments or a single string to be parsed by the shell. You don't appear to need shell=True here; just create a single list of all the positional arguments you attempting to pass.
with open(script_dir+"/Scripts/plot_TSS_profile.R","r") as INFILE1:
cmd = [
"Rscript",
"--slave",
"--args",
filenames["housekeeping_profile"],
filenames["unexpressed_profile"],
filenames["profile_plot"]
]
subprocess.call(cmd, stdin=INFILE1, stderr=ERR_LOG, stdout=OUT_LOG)
Passing multiple positional arguments means that parameters that were meant to be set via keyword arguments (like stdin, etc) are being assigned values that were meant as part of the command to execute.

subprocess, invoke C-program from within Python

I am trying to invoke a C-program, named “drule.c”, from within my Python-program “drulewrapper.py”. I am trying to use "subprocess" but cannot get it to work.
1) I compile “drule.c” on the Mac’s terminal and all works okay:
$ gcc -o drule drule c
$ ./drule D11
>P>Q>RQ
Fyi, the input -- “D11” -- are axioms in predicate logic; the output -- “>P>Q>RQ” -- is the theorem that is proven and which I then want to process further in my Python program.
2) I write a short Python program (drulewrapper.py) and compile it:
From subprocess import call
def CheckString():
call(“./drule”, “D11”)
3) But when I run CheckString() I get errors:
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
CheckString()
File "/Users/georgeszpiro/Dropbox/metamath/GApl/drulewrapper.py", line 3, in CheckString
call("./drule","D11")
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 267, in call
with Popen(*popenargs, **kwargs) as p:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 609, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integerT
Can anybody help?

subprocess.call() fails on Mac and Linux

I'm running into a weird issue with subprocess.call() function. I am trying to execute Java's 'jar' command using subprocess.call(). Here's the code:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
java_home = os.environ['JAVA_HOME']
jar_path = os.path.join(java_home, 'bin', 'jar')
jar_cmd = jar_path + ' tvf ' + war_file_path
print "command to be executed is : " + jar_cmd
subprocess.call(jar_cmd)
read_war()
I'm using Python v2.7.3 on both Windows and Linux (Oracle Enterprise Linux).
On Windows 7, I see the contents of the war file being displayed. On Linux, however, I see a 'no such file or directory' error.:
$ python example.py
command to be executed is : /usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war
Traceback (most recent call last):
File "example.py", line 24, in <module>
read_war()
File "example.py", line 23, in read_war
subprocess.call(jar_cmd)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
$
I've tried the command '/usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war' from command prompt and it works fine. So, nothing's wrong with the command.
I've tried various combinations of subprocess.call() - passing a string, passing a list etc. None of them worked. Any help at all would be appreciated.

Add shell=True to the call. On windows, the CreateProcess command does string parsing to separate the command and its various arguments. On linux, you only get string processing if you specifically tell subprocess to call the shell. Otherwise, it treats that entire string you handed in as the command and you don't get very far.
subprocess.call(jar_cmd, shell=True)

Use a list (sequence) argument instead of a string as the docs say:
args is required for all calls and should be a string, or a sequence
of program arguments. Providing a sequence of arguments is generally
preferred, as it allows the module to take care of any required
escaping and quoting of arguments (e.g. to permit spaces in file
names). If passing a single string, either shell must be True (see
below) or else the string must simply name the program to be executed
without specifying any arguments.
Example:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
jar_path = os.path.join(os.environ['JAVA_HOME'], 'bin', 'jar')
jar_cmd = [jar_path, 'tvf', war_file_path]
print("command to be executed is: %s" % jar_cmd)
subprocess.check_call(jar_cmd)
read_war()
I've used check_call to raise an exception if the command returns non-zero exit status.

Can't run Popen.communicate() twice. (Python subprocess module)

Can anyone explain why I get this error if I run the communicate function twice?
For instance
from subprocess import *
SVN=Popen('which svn', shell=True, stdout=PIPE)
print SVN.communicate()[0]
returns
"/usr/bin/svn"
but running communicate again...
print SVN.communicate()[0]
returns...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 746, in communicate
stdout = _eintr_retry_call(self.stdout.read)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 478, in _eintr_retry_call
return func(*args)
ValueError: I/O operation on closed file

Because the "file", which is actually the stdout of the program being invoked, has been closed. This means you have already read all the output in the previous communicate(), so calling it again can never produce anything.