I am using subprocess instead of os.system to execute a shell command to convert a document to pdf asynchronously. When being called repeatedly, it fails for approx 50% of files. It doesn't convert the doc file to pdf. How can I fix it.
conversion_process = await asyncio.create_subprocess_shell( 'soffice -env:UserInstallation=file:///tmp/LibreOffice_Conversion_' + author + ' --headless --convert-to pdf --outdir ' + folder + ' ' + input_file, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = await conversion_process.communicate()
if stderr:
app.logger.error(f"stderr: {stderr}")
error message:
stderr: b'javaldx: Could not find a Java Runtime Environment!
Please ensure that a JVM and the package libreoffice-java-common
is installed.
If it is already installed then try removing ~/.config/libreoffice/4/user/config/javasettings_Linux_*.xml
Warning: failed to read path from javaldx
The reason you get the error message is because the path variable used by the subprocess and passed to LibreOffice is not sufficient to find the JRE. I ran into this same issue and changed it to the following which seems to fix it.
subprocess.run(cmd,env={'HOME':'/home/username'})
I can likewise confirm that this error does not happen every time it is called, my speculation as to why this occurs is that JRE is only needed for some document conversions, not all.
Related
I'm running the following code on my windows machine using python's subprocess module.
import subprocess
args = ["abiword --to=pdf '{}'".format('test.docx')]
process = subprocess.run(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, timeout=None)
print(process.stdout.decode())
filename = re.search('-> (.*?) using filter', process.stdout.decode())
print(filename.group(1))
But subprocess.run gives the following error:
b'\'"abiword --to=pdf \'test.docx\'"\' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n'
How to resolve this error and how should I proceed now?
Also, is it correct to use abiword command in my windows machine? I want to convert my docx to pdf without installing any third party software like libreoffice.
You have 2 different problems here.
First one will be simple to solve: you do not give the correct parameters to subprocess. The first parameter (args) can be either a string which contains the full command line or a list containing the command and parameters as separate elements. So you should use either:
args = "abiword --to=pdf '{}'".format('test.docx')
(a simple string and not a list) or:
args = ["abiword", "--to=pdf", '{}'.format('test.docx')]
The second one is that until you can generate the pdf by entering the abiword --to-pdf command in a console CMD.exe window, the same command launched with subprocess.run will not give better results...
Use the full path to abiword: /usr/bin/abiword
I am attempting to use the code from here https://stackoverflow.com/a/56454579 to upload files to a server with WinSCP from Python on Windows 10. The code looks like this:
import subprocess
path = r'C:\mp4s\Sci-Fi Light Flicker.mp4'
process = subprocess.Popen(
['C:\Program Files (x86)\WinSCP\WinSCP.com', '/ini=nul', '/command', 'option batch abort', 'option confirm off', 'open ftp://user:pass#ftp.website.com', f'put "{path}"', 'exit'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
for line in iter(process.stdout.readline, b''):
print(line.decode().rstrip())
I get the output:
batch abort
confirm off
Connecting to ftp.website.com ...
Connected
Starting the session...
Session started.
Active session: [1] user#ftp.website.com
File or folder '\C:\mp4s\Sci-Fi' does not exist.
System Error. Code: 123.
The filename, directory name, or volume label syntax is incorrect
(A)bort, (R)etry, (S)kip, Ski(p) all: Abort
Which tells me that the put command isn't properly handling the spaces in the filename. What do I need to change to get this to work? Removing the double quotes around the path gives me a different error (it thinks I'm trying to upload multiple files).
I do not think you can use an array to provide arguments to WinSCP. The subprocess.Popen escapes double quotes in the arguments using backslash, what conflicts with double double-quotes escaping expected by WinSCP.
You will have to format the WinSCP command-line on your own:
args = "\"C:\Program Files (x86)\WinSCP\WinSCP.com\" /ini=nul " + \
f"/command \"open ftp://user:pass#ftp.website.com\" \"put \"\"{path}\"\"\" exit"
(note that I've omitted your option commands, as they are defaults anyway)
I'd like to execute a simple batch file using Python. But I'm getting some error back from the process saying the file, directory or disc name is not right. I guess the best way to start is to show the code:
import subprocess as sp
from pathlib import Path
file = Path(r'C:\Program Files (x86)\test.bat')
p = sp.Popen(['"' + str(file) + '"'], stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE,
shell=True, universal_newlines=True)
outs, errs = p.communicate('', timeout=5)
print(outs, '\n-----\n', errs)
I extended this with appending to system path and changing the working directory:
import os
import sys
sys.path.append(file.parent)
os.chdir(file.parent)
The batch file contains just a few echo commands for debugging. So I'd expect the code above to print the contents of the echoes. I've verified that I'm able to call it inside a command prompt from any folder. Previously I was getting some file permission error (WinError 5), so that might be related especially as the file is in Program Files. This error was not from the process, but Python itself.
I also tried it with an executable, and a similar error was raised: WinError 2: the system cannot find the file specified. Any idea where I'm stumbling?
Attempts
When the shell=True keyword is removed, the WinError 5 is back
When Popen is called with ['cmd'] and the batch file is run with p.communicate('"' + str(file) + '"\n', timeout=5), no errors are thrown, and the output contains the echoes. However batch files should run without explicitly opening a command prompt, I presume.
Use a command-line string instead of an args list when passing shell=True or when running a batch script with the default shell=False.
On Windows, Popen processes an args list into a command line that's compatible with VC++ argument parsing. But cmd.exe don't use VC++ argument parsing rules, and even for an executable that does use VC++ rules (e.g. python.exe), the extra quotes you're adding (i.e. '"' + str(file) + '"') get backslash-escaped as literal quotation marks in the command line.
I need search a file from network driver (the folder contains twenty thousand or more files)
I tried use glob to search but is very slow. It usually needs 50s~60s to complete the search.
path = r'\\network\driver\'
file = '12AC9K-XXE-849*'
print(glob.glob(path+file))
I also tried use command line tools (all system is in Windows):
os.chdir(path)
print(os.system("dir 12AC9K-XXE-849*"))
I think the issues are in the network so I can not use cmd.exe because I will get the error message:
'\\network\driver\'
CMD.EXE was started with the above path as the current directory.
UNC paths are not supported. Defaulting to Windows directory.
The system cannot find the path specified.
Thanks tdelaney
finally i find answer and get "dir" value.
use subprocess
import subprocess
proc = subprocess.Popen(["dir/b", "\\network\driver\12AC9K-XXE-849*"], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
out = out.splitlines()
print out
get dir value anwser
thanks all help
This is baffling me.
I have a python script that does some work on a Windows platform to generate an XML file, download some images and then call an external console application to generate a video using the xml and images the python script has generated.
The application I call with pOPen is supposed to return a status i.e. [success] or [invalid] or [fail] dependant on how it interprets the data I pass it.
If I use my generation script to generate the information and then call the console application separately in another script it works fine and I get success and a video is generated.
Successful code (please ignore the test prints!):
print ("running console app...")
cmd = '"C:/Program Files/PropertyVideos/propertyvideos.console.exe" -xml data/feed2820712.xml -mpeg -q normal'
print (cmd)
p = subprocess.Popen(cmd , stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = p.communicate()[0]
print ("\n----\n[" + output + "]\n----\n")
if output == "[success]":
print "\nHURRAHHHHH!!!!!!!"
print ("finished...")
But if I include the same code at the end of the script that generates the info to feed the console application then it runs for about 2 seconds and output = []
Same code, just ran at the end of a different script...
EDIT:
Thanks to Dave, Dgrant and ThomasK it seems to be that the generate script is not closing the file as redirecting strerr to stdout shows:
Unhandled Exception: System.IO.IOException: The process cannot access the file '
C:\videos\data\feed2820712.xml' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, I
nt32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions o
ptions, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
However I AM closing the file:
Extract from the generation script:
xmlfileObj.write('</FeedSettings>\n') # write the last line
xmlfileObj.close
# sleep to allow files to close
time.sleep(10)
# NOW GENERATE THE VIDEO
print ("initialising video...")
cmd = '"C:/Program Files/PropertyVideos/propertyvideos.console.exe" -xml data/feed2820712.xml -mpeg -q normal'
print (cmd)
p = subprocess.Popen(cmd , stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = p.communicate()[0]
print ("\n----\n[" + output + "]\n----\n")
if output == "[success]":
print "\nHURRAHHHHH!!!!!!!"
print ("finished...")
Any help would be appreciated.
You're not closing the file. Your code says:
xmlfileObj.close
when it should be
xmlfileObj.close()
Edit: Just to clarify - the code xmlfileObj.close is a valid python expression which returns a reference to the built in close method of a file (or file like) object. Since it is a valid python expression it is perfectly legal code, but it does not have any side effects. Specifically, it does not have the effect of actually calling the close() method. You need to include the open and close brackets to do that.