I am trying to run some R from python but I keep running into:
'FileNotFoundError: [WinError 2] The system cannot find the file specified'
I have tried searching for this problem but none of the solutions have worked.
The files are in the same directory and I'm running windows.
Here is my attempt the python:
import subprocess
import sys
from os import path
myfile = path.abspath(path.join(path.dirname(__file__), "test.R")
cmd = ['Rscript', myfile]
result = subprocess.check_output(cmd, universal_newlines=True)
print(result)
sys.stdout.flush()
The R code is just a simple Hello World.
Hope that you have added Rscript to path variable. Use below code to change your working directory as current directory
import subprocess
import sys
from os import path
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__)), "."))
os.chdir(os.path.abspath(os.path.dirname(__file__)))
myfile = path.abspath(path.join(path.dirname(__file__), "test.R")
cmd = ['Rscript', myfile]
result = subprocess.check_output(cmd, universal_newlines=True)
print(result)
sys.stdout.flush()
Related
I wanted to run this script that saves images generated by a TDW script but the TDW script is definitely not running.
import glob
import os
import cv2
import subprocess
i = 0
frameSize = (512, 512)
path = 'CRIPP/annotations/icqa_dataset/IID/json'
for i in range(0, 1):
new_path = path + f'/example_{i}.json'
cmd = "recreate.py -json={new_path}"
os.system(cmd)
#subprocess.call(['recreate.py', '-json='+new_path])
I think you forgot to run script using python.
Change cmd line to cmd = "python recreate.py -json={new_path}"
as part of a project I am doing wity Yolov4, I am trying to test multiple images with learned weights and copy the resulting images with added bounding boxes to my google drive.
The following is the code I am running on colab.
It just isn't working, and there is no log messages to debug. Please not if I run the command in the cmd variable directly, it is working fine, but that's just for one image.
My aim is to automate or "batch" the testing process.
Thanks for your help.
#%%capture
import os,sys
import subprocess
directory = '/content/darknet/data/testimages/'
for filename in os.listdir(directory):
#cmd = "./darknet detector test data/obj.data cfg/yolov4-obj.cfg /content/darknet/cfg/yolov4-obj_4000.weights /content/darknet/data/testimages/" + filename
cmd = "!./darknet detector test data/obj.data cfg/yolov4-obj.cfg /content/darknet/cfg/yolov4-obj_4000.weights /content/darknet/data/testimages/" + filename
try:
subprocess.run([sys.executable,cmd])
except:
print("failed")
#print(os.path.join(directory, filename))
cmd = "!cp 'predictions.jpg' '/mydrive/birds/results/" + filename + "_results.jpg' "
#print(cmd)
#cmd = "ls -ltr"
subprocess.run([sys.executable,cmd])
After a few hours of trial and error, a simple documentation search on Colab's page gave me the answer. It was under the heading "passing values to and from the shell". The following modified code does the trick for me.
%%capture
import os,sys
import subprocess
directory = '/content/darknet/data/testimages/'
for filename in os.listdir(directory):
cmd = "./darknet detector test data/obj.data cfg/yolov4-obj.cfg /content/darknet/cfg/yolov4-obj_4000.weights /content/darknet/data/testimages/" + filename
try:
!{cmd}
except:
print("failed")
cmd = "cp 'predictions.jpg' '/mydrive/birds/results/" + filename + "_results.jpg' "
!{cmd}
Try running your subprocess this way!
import os
import subprocess
env = os.environ.copy()
subprocess.run(cmd, shell=True, env=env)
This works for me. I find that this works for all OS. Do not add ! inside cmd.
I have a python script which calls a bunch of other python scripts via subprocess.
One of these scripts has the following line:
items = open("items.csv","w")
I also tried:
path = os.getcwd()
items = open("%s/items.csv","w") %path
But this gives me the following error:
IOError: [Errno 2] No such file or directory: 'items.csv'
If I am using the file in "w" mode, it should get created if it doesn't exist. Then why am I getting this error.
Thanks
EDIT: Also tried using items = open("%s/items.csv" % path,"w"). But getting the same error again
EDIT2: The calling script:
import subprocess
import sys
import shlex
import fileinput
import os
cmd = "python2 convert.py ebay.csv ebay.xml"
arg = shlex.split(cmd)
p = subprocess.Popen(arg)
p.wait()
cmd1 = "python2 convert.py all_products.csv all_products.xml"
arg1 = shlex.split(cmd1)
p1 = subprocess.Popen(arg1)
p1.wait()
ebay = open("ebay.xml")
voylla = open("all_products.xml")
for line in fileinput.input("ebay.xml", inplace=True):
print(line.replace("&", "and"))
for line in fileinput.input("all_products.xml", inplace=True):
print(line.replace("&", "and"))
path = os.getcwd()
print path
cmd2 = "python2 compare.py ebay.xml all_products.xml"
arg2 = shlex.split(cmd2)
print cmd2
p2 = subprocess.Popen(arg2)
p2.wait()
cmd4 = "python2 convert.py items.csv items.xml"
arg4 = shlex.split(cmd4)
p4 = subprocess.Popen(arg4)
p4.wait()
#cmd4 = "python2 ReviseItem.py"
#arg4 = shlex.split(cmd4)
#p4 = subprocess.Popen(arg4)
#p4.wait()
compare.py:
from xml.dom.minidom import parse, parseString
import xml.etree.ElementTree as ET
import sys
import os
#sys.stdout = open('file', 'w')
ebay = sys.argv[1]
voylla = sys.argv[2]
tree = ET.parse(ebay)
root = tree.getroot()
tree1 = ET.parse(voylla)
root1 = tree1.getroot()
path = os.getcwd()
items = open("%s/items.csv" % path,"w")
The main issue here, since you are using subprocess in Python 2.7 to direct other python processes, is that the file will get locked for whatever process first attempts to create and write to the file. From the documentation:
Popen.wait()
Wait for child process to terminate. Set and return returncode attribute.
Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE and the child process
generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to
accept more data. Use communicate() to avoid that.
Another issue to be aware of: you are using the "w" permission with open, which means all previous output will be overwritten. You would do better to open the file in a master process, create and close it, then use the "a" permission with open to append to the file for each process sequentially.
Also note that a data stream from a process is not actually written to the file until the file is closed.
You tried to open the file '%s/items.csv', then apply the % path to the *return value of open().
Move the string formatting to the string:
items = open("%s/items.csv" % path, "w")
However, you should use the os.path library to handle path manipulation:
items = open(os.path.join(path, 'items.csv'), 'w')
or, in this case, use os.path.abspath instead, as it'll use os.getcwd() for you for relative paths:
items = open(os.path.abspath('items.csv'), 'w')
I am struggling to replace commands.getoutput with subprocess.
Please see the codes below:
import subprocess
import commands
dir = subprocess.check_output(['pwd'])
print dir+"/*.py"
dir = commands.getoutput('pwd')
print dir+"/*.py"
Here's the ouput:
/home/akik/py
/*.py
/home/akik/py/*.py
Please help me fixing it.
getoutput() strips the newline printed by pwd command. You have to do it manually with check_output():
from subprocess import check_output
dirname = check_output(['pwd']).rstrip("\n")
print dirname + "/*.py"
So I have Foo.app, which is configured to run Foo.app/Contents/MacOS/foo.py when it is launched. In the app's Info.plist file I also have it set as the application to handle the launching of ".bar" files. When I double-click a .bar file, Foo.app opens as expected.
The problem is that when a file is opened in this way, I can't figure out how to get the path to the opened file in my foo.py script. I assumed it would be in sys.argv[1], but it's not. Instead, sys.argv[1] contains a strange string like "-psn_0_2895682".
Does anyone know how I can get the absolute path to the opened file? I've also checked os.environ, but it's not there either.
You can get your app's process ID then ask the OS for all open files using lsof, looking for your process's ID:
from string import *
from os import getpid
from subprocess import check_output, STDOUT
pid = getpid()
lsof = (check_output(['/usr/sbin/lsof', '-p', str(pid)], stderr=STDOUT)).split("\n")
for line in lsof[1:]:
print line
The regular files will be of type 'REG' in the fifth column, [4] if you're indexing in.
Files open within the running code can be displayed in a similar way:
from string import *
from os import getpid
from subprocess import check_output, STDOUT
import re
pid = getpid()
f = open('./trashme.txt', 'w')
f.write('This is a test\n')
lsof = (check_output(['/usr/sbin/lsof', '-p', str(pid)], stderr=STDOUT)).split("\n")
print lsof[0]
for line in lsof[1:]:
if (re.search('trashme', line)): print line
f.close
Which results in:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python 6995 greg 3w REG 14,2 0 2273252 /Users/greg/Desktop/trashme.txt