I would like to process .wav files in Python. Particularly, I would like to perform following operation
sox input.wav -c 1 -r 16000 output.wav
in every .wav file in my folder. My code is below:
#!/usr/bin/python
# encoding=utf8
# -*- encoding: utf -*-
import glob
import subprocess
segments= []
for filename in glob.glob('*.wav'):
new_filename = "converted_" + filename
subprocess.call("sox" + filename + "-c 1 -r 16000" + new_filename, shell=True)
However, it is not working as expected that it's not calling my command.
When you write
subprocess.call("sox" + filename + "-c 1 -r 16000" + new_filename, shell=True)
what's actually going to be executed for an exemplary TEST.WAV file looks like this:
soxTEST.WAV-c 1 -r 16000converted_TEST.WAV
So you're missing the spaces in between. A nice solution using Python's f-strings (Formatted string literals) would be something like this:
subprocess.call(f"sox {filename} -c 1 -r 16000 {new_filename}", shell=True)
However, I'd recommend switching over to subprocess.run and disregarding the shell=True flag:
subprocess.run(["sox", filename, "-c 1", "-r 16000", new_filename])
More information also at the docs https://docs.python.org/3/library/subprocess.html
Note: Read the Security Considerations section before using shell=True.
Related
In an Anaconda shell environment on Windows with perl, m2-base and maybe some other packages installed:
$ echo "/" > junk
$ more junk
"/"
$ perl -pi.bak -e "s/\"\/\"/\"\\\\\"/" junk
$ more junk junk.bak
::::::::::::::
junk
::::::::::::::
"\"
::::::::::::::
junk.bak
::::::::::::::
"/"
I want to replicate this in Python. My script is this:
import subprocess
cmd = 'perl -pi.bak -e "s/\"\/\"/\"\\\\\"/" junk'
subprocess.call(cmd, shell = True)
which gives the following output:
$python test_perl.py
Substitution replacement not terminated at -e line 1.
I have tried different combinations of backslashes, different quotation styles, and using a different delimiter in perl (i.e. replacing the / with something like #), but can't seem to figure out how to crack this nut.
UPDATE
subprocess.call(['perl', '-pi.bak', '-e', "s!\\\"\/\"!\"\\\\\\\"!", 'junk'], shell = True) works, but I'm confused about why subprocess does not need extra quotes to encapsulate the perl switch statement. Any insights would be appreciated.
--
For more info on what I'm actually doing, I am trying to install a Python module that was designed for Linux/Unix on my Windows platform in an Anaconda environment. For one part, I need to replace "/" with "\" in some of the files of the module. I am aware that I could edit the files directly and use something like os.path.split instead of just split("/") but I am trying to create a file that does all the work, so all one needs to do is clone the git repository and run a setup script.
Following python demo code emulate perl -i.bak ... behavour.
Problem description does not explains why OP resorts to Perl assistance to make simple substitution with preserving .bak file as backup copy.
Python has enough muscle to perform such operation, just a few lines of code.
import os
ext_bak = '.bak'
file_in = 'path_substitute.txt'
file_bak = file_in + ext_bak
# remove backup file if exists
if os.path.exists(file_bak):
os.remove(file_bak)
# rename original file to backup
os.rename(file_in,file_bak);
f = open(file_bak,'r') # read from backup file
o = open(file_in, 'w') # write to a file with original name
for line in f:
o.write(line.replace('/','\\')) # replace / with \ and write
# close files
f.close()
o.close()
Input path_substitute.txt
some path /opt/pkg/dir_1/file_1 word_1
other path /opt/pkg/dir_2/file_2 word_2
one more /opt/pkg/dir_3/file_3 word_3
Output path_substitute.txt
some path \opt\pkg\dir_1\file_1 word_1
other path \opt\pkg\dir_2\file_2 word_2
one more \opt\pkg\dir_3\file_3 word_3
I am using Inkscape to take an input single page pdf file and to output an svg file. The following works from the command line
c:\progra~1\Inkscape\inkscape -z -f "N:\pdf_skunkworks\inflation-report-may-2018-page0.pdf" -l "N:\pdf_skunkworks\inflation-report-may-2018-page0.svg"
where -z is short for --without-gui, -f is short for input file, -l is short for --export-plain-svg. And that works from command line.
I could not get the equivalent to work from Python, either passing the command line as one long string or as separate arguments. stderr and stdout give no error as they both print None
import subprocess #import call,subprocess
#completed = subprocess.run(["c:\Progra~1\Inkscape\Inkscape.exe",r"-z -f \"N:\pdf_skunkworks\inflation-report-may-2018-page0.pdf\" -l \"N:\pdf_skunkworks\inflation-report-may-2018-page0.svg\""])
completed = subprocess.run(["c:\Progra~1\Inkscape\Inkscape.exe","-z", r"-f \"N:\pdf_skunkworks\inflation-report-may-2018-page0.pdf\"" , r"-l \"N:\pdf_skunkworks\inflation-report-may-2018-page0.svg\""])
print ("stderr:" + str(completed.stderr))
print ("stdout:" + str(completed.stdout))
Just to test OS plumbing I wrote some VBA code (my normal language) and this works
Sub TestShellToInkscape()
'* Tools->References->Windows Script Host Object Model (IWshRuntimeLibrary)
Dim sCmd As String
sCmd = "c:\progra~1\Inkscape\inkscape -z -f ""N:\pdf_skunkworks\inflation-report-may-2018-page0.pdf"" -l ""N:\pdf_skunkworks\inflation-report-may-2018-page0.svg"""
Debug.Print sCmd
Dim oWshShell As IWshRuntimeLibrary.WshShell
Set oWshShell = New IWshRuntimeLibrary.WshShell
Dim lProc As Long
lProc = oWshShell.Run(sCmd, 0, True)
End Sub
So I'm obviously doing something silly in the Python code. I'm sure experienced Python programmer could solve easily.
Swap your slashes:
import subprocess #import call,subprocess
completed = subprocess.run(['c:/Progra~1/Inkscape/Inkscape.exe',
'-z',
'-f', r'N:/pdf_skunkworks/inflation-report-may-2018-page0.pdf' ,
'-l', r'N:/pdf_skunkworks/inflation-report-may-2018-page0.svg'])
print ("stderr:" + str(completed.stderr))
print ("stdout:" + str(completed.stdout))
Python knows to swap forward slashes for back slashes on windows OS, and your back slashes are currently acting as escape prefixes.
I have a python script to convert json files to csv. It currently converts all files, but I want it to run only on those json files that have not been converted to csv already. All are in the same directory. How can I modify my code:
#!/bin/bash
# file: foo.sh
for f in *.json; do
python ~/bin/convert.py "$f" "-fcsv"
done
Assuming your script creates basename.csv for an input file named basename.json
for f in *.json; do
test -e "${f%.json}.csv" && continue
python ~/bin/convert.py "$f" "-fcsv"
done
The shell parameter expansion $(variable%pattern} produces the value of variable with any match on the glob pattern removed.
This leverages the find GNU-util and uses python for
the rest. Since you are running a python script, I am assuming that python is
installed on the system.
Full command below:
find . \( -type f -regextype posix-extended -regex '.*json' \) - exec python -c "import sys; import os; filename, file_extension = os.path.splittext(sys.argv[1]); if not os.path.isfile(filename + '.csv') : os.system('python ~/bin/convert ' + filename + file_extension + ' -fcsv')" {}
searches for files with .json extension
find . \( -type f -regextype posix-extended -regex '.*json' \)
takes the output from the find command and enters it into the python namespace
filename, file_extension = os.path.splittext(sys.argv[1]);
checks if there is a filename with extension .csv, if not runs the convert.py program
if not os.path.isfile(filename+'.csv'): os.system('python ~/bin/convert.py ' + filename+file_extension + ' -fcsv')"
You can put the entire script into a bash script and use \ to break it up into multiple lines as shown here: How can I split a shell command over multiple lines when using an IF statement?
I'm working on a script to call an executable for i/o files. I'm using subprocess and trying to shell out the exe and the stdout to a log file. Problem is I would like to output a simple ascii file and I'm getting a hexadecimal file. Just really learning to program python (or any language for that matter) so, I'm assuming there some type of formatting I can do but I just don't get it. I've done a fair bit of searching on this site and others but I haven't anything like what I'm using subprocess for. The "outRadcorr" is what I need help on the most...Any ideas? More code on request.
Import system modules
import os, sys, string, traceback, time, datetime
import params
from subprocess import Popen, PIPE, STDOUT
...some code here.....
Write stdout to log file
rad_log_file = open(dsFolder + '\\radcorr.log', 'w')
# loop through the files in raw file list to run radiometric correction
for rawfiles in rawFolderList:
# Define the file base
rawBase = rawfiles.split(".")[0]
print ('\nProcessing file: %s \n')%( rawBase )
# define variables from raw file to process radcorr
radFile = rawfiles
pixFile = ('%s.pix')%( rawBase )
attFile = ('%s.att')%( rawBase )
# create windose bat file function
def rad_bat_writer( radcorr_bat ):
with open(dsFolder + '\\radcorr.bat', 'a') as rad_bat_file:
rad_bat_file.write(radcorr_bat + '\n')
# grab radcor input/output files and run radcorr.exe
radcorr = ('C:\\itres\\bin\\radcorr.exe -f 1 -j 100 -g 50 -s -1 -n -1 %s %s %s -i '+ rawFolder + '\%s,rb -o ' + radFolder + '\%s -a ' \
+ radFolder + '\%s -c C:\\itres\\rad_cal_files\\%s -I 0 -v 0 -r Y -R Y -^^ 2') %( sum,scatter,shift,radFile,pixFile,attFile,rad_prefix )
# print out radcorr command
print radcorr
# Execute radcorr and write stdout
outRadcorr = Popen("{};".format(radcorr), stdout=PIPE, stderr=STDOUT)
for line in outRadcorr.stdout:
sys.stdout.write(line)
rad_log_file.write(line)
# write output to log
rad_bat_writer( radcorr )
# Close out exe and log files
outRadcorr.wait()
rad_log_file.close()
I was using UltraEdit to view the file, which you can view hex files in. I was not looking in hex mode. I might be confusing the terminology here. File looks normal in NotePad/NotePad++/WordPad etc. As I ran the script in debug mode, I could hit the loop file by file. First two files (1GB/file), the output log looked fine. Once the radcorr.log file was over 10kb, I change from a normal ascii output to this binary file viewed that looked hex. I can't post images yet, but just google ultraedit hexadecimal.
Still not sure why it moved to this format. Finally, output size was 45kb. I change to view/edit hex mode in UltraEdit and it looks fine. Just wanted to get it out there to see if others had any ideas why when I specified the log to be 'w' and not 'wb', for instance.
I do appreciate all you help. #J.F. Sebastian I'll have to test the code you posted, probably help fix potential bugs down the road.
I'm trying to write a python script to copy files from a remote server to a local directory via scp.
Because I'm running this on an OpenELEC distribution (minimal HTPC linux distro, read-only filesystem except for userhome makes it impractical to install python ssh module), I'm doing this ugly and just passing the filename to the scp command via os.system.
SCPCopy = "scp -c blowfish -C user#host:\"" + pipes.quote(file) + "\" /storage/downloads/incoming/"
SCPCopy = SCPCopy.replace('\n','')
os.system(SCPCopy)
This works, except for filenames containing an apostrophe.
Below is an example of what gets passed to os.system in a file with an apostrophe:
scp -c blowfish -C user#host:"'/media/sdi1/home/data/bob'"'"'s file.avi'" /storage/downloads/incoming/
And the error:
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
It looks pipes.quote(x) is escaping the apostrophe (as it should), but obviously the syntax is still incorrect. I've experimented ditching pipes.quote(x) and replacing apostrophes with /' but that isn't getting me anywhere either.
As scp is based on SSH, the filenames you give to it are subject to shell escaping on the remote side as well. Thus you need to escape twice.
A correctly escaped cmdline for the shell:
scp -c blowfish -C user#host:"\"/media/sdi1/home/data/bob's file\"" /storage/.../
To make a python string, we have to add one more level of escaping. To stay sane, we could use triple-quotes:
"""scp -c blowfish -C user#host:"\"/media/sdi1/home/data/bob's file\"" /storage/.../"""
If you do it programmatically (e.g. using the deprecated pipes.quote), then don't touch the filename at all (in your example above, you added apostrophes around the filename).
fp = "/media/sdi1/home/data/bob's file.avi"
fp = "user#host:" + pipes.quote(pipes.quote(fp))
cmdline = "scp -c blowfish -C " + fp + " /storage/downloads/incoming/"
os.system(cmdline)
This is admittedly confusing. For a simple model, the whole point of pipes.quote is to escape the input so that the input will be parsed by the shell as exactly one word, which is equal to the input.
The following is a more generally correct way (and yields the same result):
fp = "/media/sdi1/home/data/bob's file.avi"
# the filepath argument escaped for ssh/scp on the remote side
fp = pipes.quote(fp)
commandargs = ["scp", "-c", "blowfish", "-C", "user#host:"+fp, "/storage/downloads/incoming/"]
# escape all words for the local shell, and then concatenate space-separated
cmdline = " ".join(map(pipes.quote, commandargs))
os.system(cmdline)
It expresses more clearly the intent: Controlling what words exactly the shell will parse.
But why start with a shell in the first place? We don't need one and can save the escaping on the local side. To spawn a process with our args, directly, use commands from the os.exec* family.
fp = pipes.quote("/media/sdi1/home/data/bob's file.avi")
commandargs = ["scp", "-c", "blowfish", "-C", "user#host:"+fp, "/storage/downloads/incoming/"]
if os.fork() == 0:
os.execvp("scp", commandargs)