Running Python code: subprocess.check_output(), getting error: [Winerror 2] - python

I'm running some Python code that is meant to run an Apache Maven program on a file and produce an output:
import os, subprocess
os.chdir("C:/Users/Mohammad/Google Drive/PhD/Spyder workspace/production-consumption/logtool-examples/")
logtoolDir = "C:/Users/Mohammad/Google Drive/PhD/Spyder workspace/production-consumption/logtool-examples/"
processEnv = {'JAVA_HOME': 'C:/Program Files/Java/jdk1.8.0_66/jre',
'mvn': 'C:/Program Files/apache-maven-3.3.3/bin/'}
#processEnv = "C:/Program Files/Java/jdk1.8.0_66"
args = 'org.powertac.logtool.example.ProductionConsumption D:/PowerTAC/Logs/2015/log/powertac-sim-1.state testrunoutput.data'
subprocess.check_output(['mvn', ' exec:exec',
' -Dexec.args=' + args],
env = processEnv,
cwd = logtoolDir)
However, it gives me this error:
File "C:/Users/Mohammad/Google Drive/PhD/Spyder workspace/production-consumption/test.py", line 25, in <module>
cwd = logtoolDir)
File "C:\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\subprocess.py", line 607, in check_output
with Popen(*popenargs, stdout=PIPE, **kwargs) as process:
File "C:\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\subprocess.py", line 859, in __init__
restore_signals, start_new_session)
File "C:\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\subprocess.py", line 1112, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I've investigated some things and I've narrowed it down to what I'm expecting is _winapi.CreateProcess being unable to find the Apache Maven installation (and run the mvn command). The installation is already in my path env variables (the line runs just fine through CMD). It may also be true that I've somehow defined the directories wrongly, but I fail to find a problem there... Can anyone offer a suggestion on how to fix this issue?
Cheers.

You can try appending "shell = True" to the arguments passed to subprocess like so:
subprocess.check_output(['mvn', ' exec:exec',
' -Dexec.args=' + args],
env = processEnv,
cwd = logtoolDir, shell = True)
Though as it says here, this can be a security risk if the contents of the call can be determined by an external source, but I can't tell whether that's the case from the code you've given.

Related

File not found when calling subprocess but it exists, and it is a file

edit below
I am getting the error FileNotFoundError: [WinError 2] The system cannot find the file specified. I am using backslashes (used to escape characters) in the strings but they are not returning a unicode error so I am assuming that is not that problem (I've also tried changing the direction of the slashes). Here is the code with comments:
# I can use cd .\wav to get into the folder from where I am running the file
source_filepath = ".\wav" # I've tried changing to '/' and using the absolute path
target_filepath = ".\spect" # I've tried changing to '/' and using the absolute path
STEP_SIZE = 1.5
for filename in os.listdir(source_filepath):
id, ext = filename.split(".")
print(os.path.join(source_filepath, filename)) # prints ".\wav\UNIT1_20210501_042900.wav" for example
print(os.path.exists(os.path.join(source_filepath, filename))) # prints "True", it exists
print(os.path.isfile(os.path.join(source_filepath, filename))) # prints "True", it is a file
if ext == "wav":
sox_call = subprocess.run(['sox', '-D',
os.path.join(source_filepath, filename)],
stdout=subprocess.PIPE) # fails
duration = int(float(sox_call.stdout))
Here is the traceback:
Traceback (most recent call last):
File "C:\Users\Helana\Documents\projects\plan-b\extract-clips-and-create-spectrograms.py", line 18, in <module>
sox_call = subprocess.run(['sox', '-D',
File "C:\Users\Helana\anaconda3\lib\subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\Helana\anaconda3\lib\subprocess.py", line 951, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\Helana\anaconda3\lib\subprocess.py", line 1420, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
The point of the file is to slice a wav file into pieces and turn each piece into a spectogram and store those spectograms in the "spect" folder. There is more code after this block but it's what I'm getting an error from, do let me know if more code needs to be shown. I am also on Windows 11, not sure if that could cause problems. Sox was installed via pip.
Edit:
Figured out sox was the problem, downloaded the right one from https://sourceforge.net/projects/sox/. Add to my PATH like:
Variable: sox Value: C:\Program Files (x86)\sox-14-4-2\
Based on the question here: How to use sox in windows
Still getting the same error as above when running my file, and running sox in Powershell gives error:
sox : The term 'sox' is not recognized as the name of a cmdlet, function, script file, or operable program.

Python: Can't find file even though file referenced exists

I'm getting this error when trying to run a Python script. Is it saying that it can't find subprocess.py? Because I found it in the location it's listing there, so I doubt that's the issue. What file can't it find?
Traceback (most recent call last):
File "D:\Projects\PythonMathPlots\MandelbrotVideoGenerator.py", line 201, in <module>
run( ['open', 'MandelbrotZoom.mp4'] )
File "C:\Users\Aaron\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\Aaron\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:\Users\Aaron\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
You may need to put the full path in the run(...) command, to the open file, and the path to the .mp4 file as well.
Most likely, open does not exist on your system and you have to use the name of the video player software instead.
Make sure the user you're running the script as has read permission for the file.
You may also try with subprocess.Popen(args, shell=True). The use of shell=True may be useful.
Also, use a path defined as path = os.path.join(filepath, filename) and then before passing the path to Popen, assert if os.path.exists(path)==True.
But note that there are some downsides to using shell=True:
Actual meaning of 'shell=True' in subprocess
https://medium.com/python-pandemonium/a-trap-of-shell-true-in-the-subprocess-module-6db7fc66cdfd

Python subprocess FileNotFoundError

I am trying to follow this blog on how to execute an R script from Python. I have the R script working fine from the command line using Rscript.
Here's my Python code:
import subprocess
import os
command = "C:\Program Files\R\R-3.4.4\bin\Rscript"
path2script = os.getcwd() + "\max.R" # gives me the absolute path to the R script
args = ["11", "3", "9", "42"]
cmd = [command, path2script] + args
x = subprocess.check_output(cmd, universal_newlines = True)
Which gives me this error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I've read a lot of SO posts on this error and in most cases it seems to be a problem with trying to invoke system commands like dir or passing arguments to check_output in the wrong order but in my case I really don't see what should be going wrong.
Following some of the advice I've tried building a string for cmd instead of a list, and then passing it to check_output using the argument shell = True - when I do that I get a CalledProcessError: returned non-zero exit status 1.
I'm assuming this code, which is exactly as it appeared on the blog other than adding the absolute path to the file, is failing now because the behaviour of check_output has changed since 2015...
Can anyone help?
Here's the stack trace:
Traceback (most recent call last):
File "<ipython-input-2-3a0151808726>", line 1, in <module>
runfile('C:/Users/TomWagstaff/Documents/Raising IT/Projects/15 AdWords/Python_R_test/run_max.py', wdir='C:/Users/TomWagstaff/Documents/Raising IT/Projects/15 AdWords/Python_R_test')
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/TomWagstaff/Documents/Raising IT/Projects/15 AdWords/Python_R_test/run_max.py", line 31, in <module>
x = subprocess.check_output(cmd, universal_newlines = True)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\site-packages\spyder\utils\site\sitecustomize.py", line 210, in __init__
super(SubprocessPopen, self).__init__(*args, **kwargs)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
check that you have a right path for command and script
print(os.path.exists(command))
print(os.path.exists(path2script))
note that writing path with backslashes may be dangerous as you can create escape sequence that way which will be interpreted in different way. You can write windows paths with forward slashes and then call os.path.normpath on them, turning them into safe form
(also in command you can use forward slashes only, Python interpret doesn't really care. In path to your R script that would be probably problem though)

Why subprocess.call () is showing error in linux?

Let us consider Linux platform where I need to execute a program called smart.exe which uses input.dat file. Both the files are placed in the same directory with each file having the same file permission 777.
Now if I run the following command in the terminal window smart.exe is fully executed without any error.
$./smart.exe input.dat
On the other hand, if I use the following python script called my_script.py placed in the same directory, then I get an error.
my_script.py has the following code:
#!/usr/bin/python
import os, subprocess
exit_code = subprocess.call("./smart.exe input.dat", shell = False)
The error is as follows:
File "my_script.py", line 4, in <module>
exit_code = subprocess.call("./smart.exe input.dat", shell = False)
File "/usr/lib64/python2.6/subprocess.py", line 478, in call
p = Popen(*popenargs, **kwargs)
File "/usr/lib64/python2.6/subprocess.py", line 642, in __init__
errread, errwrite)
File "/usr/lib64/python2.6/subprocess.py", line 1234, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Can someone please tell me why this is happening. Please note that the smart.exe should take around 10 sec to fully complete. This may be a clue for the problem.
Please also advise if there is any other way to run smart.exe from my_script.py. Your solution is much appreciated!
You should decide if you want shell support or not.
If you want the shell to be used (which is not necessary here), you should use exit_code = subprocess.call("./smart.exe input.dat", shell=True). Then the shell interprets your command line.
If you don't want it (as you don't need it and want to avoid unnecessary complexity), you should do exit_code = subprocess.call(["./smart.exe", "input.dat"], shell=False).
(And there is no point naming your binarys .exe under Linux.)

Biopython error - The system cannot find the file specified

I have encountered an error which I am not able to resolve.
I am trying to perform the easiest set of commands that will perform a tBLASTn algorithm,
looking for a sequence (sequence specified as a "pytanie.fasta" file) in a database (also specified as file -> cucumber.fasta). The result will be saved in the "wynik.txt" file.
The code looks as following:
from Bio.Blast. Applications import NcbitblastnCommandline
database = r"\Biopython\cucumber.fasta"
qr = r"\Biopython\pytanie.fasta"
output = r"\Biopython\wynik.txt"
e = raw_input("Enter e-value: ")
tblastn_cline = NcbitblastnCommandline(cmd='blastn', db=database, query=qr, out=output, evalue=e, outfmt=7)
print tblastn_cline
stdout, stderr = tblastn_cline()
And the error I get:
File "C:\Users\IBM_ADMIN\Desktop\PYTHON\Workspace\Biopython\blast.py", line 20, in <module>
stdout, stderr = tblastn_cline()
File "C:\Python27\lib\site-packages\Bio\Application\__init__.py", line 435, in __call__
shell=(sys.platform!="win32"))
File "C:\Python27\lib\subprocess.py", line 679, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 893, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
I am using:
Eclipse SDK Version: 3.7.1
Python version 2.7
OS: 64 bit Windows 7
I have tried this also on 32-bit Windows XP and it produces the same error.
Biopython package should work fine since it went through the tests suggested by biopython website. I have also tried other formats of the path where the files are located, but it did not work. My friend uses the same code on Ubuntu and it works fine.
Does anybody know how to fix this error?
What are the paths of the files?
The path r"\Biopython\cucumber.fasta", for example, is an absolute path on the current drive (because it starts with a backslash and no drive letter), which I think in your case is r"C:\Biopython\cucumber.fasta". Is that correct?

Categories