import subprocess
sample_file_directory = "..." # directory where file is
SCRIPT_DIR = "..." # directory where script is
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
min_side_len, err = p.communicate()
print len(err)
So i have this script that analyzes .ply files (3d file format) and gives me data about the file. I am trying to find out in my directory which files are corrupted and which aren't. So I am trying to use the subprocess library to run the script to find an arbitrary feature of the .ply files (in this case the minimum edge length) If err has anything in it, it means it couldn't retrieve the arbitrary feature and the file is corrupted. Here, I am only running it on one file. However, I keep getting an error.
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Can anyone help me figure out why I am getting this error? Both my data directory and script directory are valid. Also, sorry if I left anything out that's important. First time posting.
Is your SCRIPT_DIR variable the absolute path or a relative path valid in the location you're running the script? If it's the latter, it may not work, as the subprocess may start in a different environment that has a different working directory.
The problem lies here:
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Notice this part:
Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen']
Now, see, you already know that subprocess takes a list. But the problem is that you are passing the directory as part of the list. The list is formatted like this:
[command, arg1, arg2...]
Now, when you run that command, you are doing this:
[SCRIPT_DIR -> command
sample_file_directory -> arg1
'min' -> arg2
'edgelen' -> arg3]
See the problem? You are passing the script's directory as the command to run and the script's name as an argument. Import the os module and do this:
p = subprocess.Popen([os.path.join(SCRIPT_DIR,sample_file_directory),'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
This does this:
[SCRIPT_DIR and sample_file_directory joined together -> command
'min' -> arg1
'edgelen' -> arg2]
os.path.join automatically adds the path separator.
subprocess.Popen() won't combine path components for you. From what I see, you're only providing a directory. You'll need to add the target executable's name to the path being given to Popen. Try the following:
p = subprocess.Popen([os.path.join(SCRIPT_DIR,SCRIPT_NAME),sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
"One other thing it could potentially be is that you're trying to run a 32-bit binary on a 64-bit system without the 32-bit libraries installed. I actually ran into this issue earlier today. A fix is documented here. I don't know anything about your system, so this may be completely irrelevant advice, but I figured it wouldn't hurt to mention it." Dan Albert. He came up with the answer to the solution! Thanks so much Dan!
Related
Hello I'm trying to run multiple OS command using Python in Mac OS
I want to convert image to tif file by using convert command in Terminal. However I have 100 images to convert, I make Python program to make it easier.
import os
import subprocess
files = os.listdir("/Users/woonie/Downloads/test")
i=1
for file in files:
args=['convert',file,'-resize','100%','-type',"Grayscale","/Users/woonie/Downloads/test/kor.",i,"test.exp0.tif"]
subprocess.Popen(args)
i = i+1
convert filename -...- output_filename.exp0.tif is the form of convert command so I need to change file name and output_filename every time. I have filename list in files. And I want to put number of image after "test" so it becomes kor.test1.exp0.tif, kor.test2.exp0.tif, etc.
Traceback (most recent call last):
File "/Users/woonie/PycharmProjects/image_chage/change.py", line 9, in <module>
subprocess.Popen(args)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 1637, in _execute_child
self.pid = _posixsubprocess.fork_exec(
TypeError: expected str, bytes or os.PathLike object, not int
But this error came out.
So I have changed that code to
args=['convert'+str(file)+'-resize','100%','-type',"Grayscale","/Users/woonie/Downloads/test/kor."+str(i)+"test.exp0.tif"]
But same error came out...
It was work when I put that command directly into Terminal.
Did I use subprocess wrong? or I can't make program what I want to
Use
subprocess.run(args)
instead of
subprocess.Popen(args)
That being said, your args array should probably look like
args=['convert', str(file), '-resize','100%','-type',"Grayscale","/Users/woonie/Downloads/test/kor."+str(i)+"test.exp0.tif"]
because the filename and the resize options are probably separate arguments.
There are quite a few issues with your code.
Rather than carry around long pathnames multi-levels deep in your input and output files, it is often better to just change directory to where the files are and deal with simple filenames - it is less error-prone and less of a maintenance nightmare.
So, consider starting your code with something more like:
import os
# Go to where the images are
os.chdir('/Users/woonie/Downloads/test')
Rather than use use os.listdir(), you get much more flexibility if you use globbing exactly the same as you would in the shell. So, if you want to process just JPEGs rather than TIFFs you produced on the previous run, use:
import glob
# Get list of JPEGs to process
JPEGs = glob.glob('*jpg')
I am unsure why you are using subprocess.Popen() - you would normally do that when you want to capture the output, you might as well use subprocess.run()
I don't know why you use -resize 100%. Where did that come from? The image should be 100% anyway.
When you want to formulate your output filename, you can more simply use f-strings. So, having got rid of all the long paths, your last parameter becomes:
f'kor.{i}test.exp0.tif'
I am trying to replicate another researcher's findings by using the Python file that he added as a supplement to his paper. It is the first time I am diving into Python, so the error might be extremely simple to fix, yet after two days I haven't still. For context, in the Readme file there's the following instruction:
"To run the script, make sure Python2 is installed. Put all files into one folder designated as “cf_dir”.
In the script I get an error at the following lines:
if __name__ == '__main__':
cf_dir, cf_file, cf_phys_file = sys.argv[1:4]
os.chdir(cf_dir)
cf = pd.read_csv(cf_file)
cf_phys = pd.read_csv(cf_phys_file)
ValueError: need more than 0 values to unpack
The "cf_file" and "cf_phys_file" are two major components of all files that are in the one folder named "cf_dir". The "cf_phys_file" relates only to two survey question's (Q22 and Q23), and the "cf_file" includes all other questions 1-21. Now it seems that the code is meant to retrieve those two files from the directory? Only for the "cf_phys_file" the columns 1:4 are needed. The current working directory is already set at the right location.
The path where I located "cf_dir" is as follows:
C:\Users\Marc-Marijn Ossel\Documents\RSM\Thesis\Data\Suitable for ML\Data en Artikelen\Per task Suitability for Machine Learning score readme\cf_dir
Alternative option in readme file,
In the readme file there's this option, but also here I cannot understand how to direct the path to the right location:
"Run the following command in an open terminal (substituting for file names
below): python cfProcessor_AEAPnP.py cf_dir cf_file cf_phys_file task_file jobTaskRatingFile
jobDataFile OESfile
This should generate the data and plots as necessary."
When I run that in "Command Prompt", I get the following error, and I am not sure how to set the working directory correctly.
- python: can't open file 'cfProcessor_AEAPnP.py': [Errno 2] No such file or directory
Thanks for the reading, and I hope there's someone who could help me!
Best regards & stay safe out there during Corona!!
Marc
cf_dir, cf_file, cf_phys_file = sys.argv[1:4]
means, the python file expects few arguments when called.
In order to run
python cfProcessor_AEAPnP.py cf_dir cf_file cf_phys_file task_file jobTaskRatingFile jobDataFile OESfile
the command prompt should be in that folder.
So, open command prompt and type
cd path_to_the_folder_where_ur_python_file_is_located
Now, you would have reached the path of the python file.
Also, make sure you give full path in double quotes for the arguments.
So I am trying to encrypt a directory using python and I'm not sure what the best way to do that is. I am easily able to turn the folder into a zip file, but from there I have tried looking up how to encrypt it with AES, but couldn't get that to work and I have also tried encrypting using 7zip to archive the folder, but also couldn't get that to work, so if anybody has another solution to encrypt a directory or could point me in the right direction on how to use one of the previous methods that would be helpful. (I'm on windows if that has any significance)
I still recommend 7-zip.
let's say you want to name the zip folder as myzip.zip
Import subprocess
zp = subprocess.call(['7z', 'a', 'your password', '-y', 'myzip.zip'] + ['your file'])
An alternative way:
Import pyminzip
level=4 #level of compression
pyminizip.compress("your file", "myzip.zip", "your password", level)
Using 7-Zip through the subprocess module works. Here are some issues I encountered and had to resolve:
You need to specify the path to 7zip separate from the cmd variable in the Popen subprocess, and build the command with variables rather than a solid string:
appPath="C:\Program Files\\7-Zip"
zApp="7z.exe"
zAction='a'
zPass='-pPASSWORD'
zAnswer='-y'
zDir=directoryToZip
progDir=os.path.join(appPath,zApp)
cmd = [zApp, zAction, zipFileName, zPass, zAnswer, zDir]
subprocess.Popen(cmd, executable=progDir, stderr=subprocess.STDOUT, stdout=subprocess.PIPE)
That will create a zip file (in the location with the name in the zipFileName variable), including the contents (directories and files) inside the "directoryToZip" path
progDir has to be specified for separate from the application you are calling as part of the Open command (this is the executable path), and the command string needed to be built out as variables to deal with the windows backslash escaping setup.
This question already has answers here:
os.makedirs doesn't understand "~" in my path
(3 answers)
Closed 6 months ago.
I am trying to read a file in another directory. My current script is in path/to/dir. However, I want to read a file in ~/. I'm not sure how to do this.
I tried f = open("~/.file") but I am getting an error
IOError: [Errno 2] No such file or directory: '~/.file'
use os.path.expanduser:
import os
with open(os.path.expanduser('~/.file')) as f:
print f.read()
This should work
from os.path import expanduser
home = expanduser("~")
os.path.expanduser will help you
The short answer is: use os.path.expanduser, as m.wasowski shows:
f = open(os.path.expanduser("~/.file"))
But why do you have to do this?
Well, ~/.file doesn't actually mean what you think it does. It's just asking for a file named .file in a subdirectory named ~ in the current working directory.
Why does this work on the command line? Bcause shells don't just pass your arguments along as typed, they do all kinds of complicated processing.
This is why ./myscript.py *txt gives you ['./myscript.py', 'a.txt', 'b.txt', 'c.txt', 'd e f.txt'] in your argv—because the shell does glob expansion on *.txt by looking in the directory for everything that matches that pattern and turns it into a.txt b.txt c.txt "d e f.txt".
In the same way, it expands ~/.file into /Users/me/.file by looking at the appropriate environment variables and platform-specific defaults and so on. You can read the gory details for bash; things are similar with dash, tcsh, and most other shells (except Windows cmd and PowerShell).
Python talks to the filesystem directly, it doesn't go through the shell to do it. So, if you want to do the same things shells do, you have to do them explicitly: glob.glob, os.path.expanduser, os.path.expandvars, subprocess.check_output, etc.
The cool thing about doing it all in Python is that it works the same way everywhere. Even on Windows, where the shell doesn't know what ~/.file means, and the right answer may be some hideous thing like 'C:\\Documents and Settings\\me\\Documents' or '\\\\FileServer\\RoamingProfiles\\me', os.path.expanduser will give you the right hideous thing.
I am trying to make a python script that will open a directory, apply a perl script to every file in that directory and get its out put in either multiple text files or just one.
I currently have:
import shlex, subprocess
arg_str = "perl tilt.pl *.pdb > final.txt"
arg = shlex.split(arg_str)
import os
framespdb = os.listdir("prac_frames")
for frames in framespdb:
subprocess.Popen(arg, stdout=True)
I keep getting *.pdb not found. I am very new to all of this so any help trying to complete this script would help.
*.pdb not found means exactly that - there won't be a *.pdb in whatever directory you're running the script... and as I read the code - I don't see anything to imply it's within 'frames' when it runs the perl script.
you probably need os.chdir(path) before the Popen.
How do I "cd" in Python?
...using a python script to run somewhat dubious syscalls to perl may offend some people but everyone's done it.. aside from that I'd point out:
always specify full paths (this becomes a problem if you will later say, want to run your job automatically from cron or an environment that doesn't have your PATH).
i.e. 'which perl' - put that full path in.
./.pdb would be better but not as good as the fullpath/.pdb (which you could use instead of the os.chdir option).
subprocess.Popen(arg, stdout=True)
does not expand filename wildcards. To handle your *.pdb, use shell=True.