I wanted to run this script that saves images generated by a TDW script but the TDW script is definitely not running.
import glob
import os
import cv2
import subprocess
i = 0
frameSize = (512, 512)
path = 'CRIPP/annotations/icqa_dataset/IID/json'
for i in range(0, 1):
new_path = path + f'/example_{i}.json'
cmd = "recreate.py -json={new_path}"
os.system(cmd)
#subprocess.call(['recreate.py', '-json='+new_path])
I think you forgot to run script using python.
Change cmd line to cmd = "python recreate.py -json={new_path}"
I am using a Python script to batch convert many images in different folders into single pdfs (with https://pypi.org/project/img2pdf/):
import os
import subprocess
import img2pdf
from shutil import copyfile
def main():
folders = [name for name in os.listdir(".") if os.path.isdir(name)]
for f in folders:
files = [f for f in os.listdir(f)]
p = ""
for ffile in files:
p += f+'\\' + ffile + " "
os.system("py -m img2pdf *.pn* " + p + " --output " + f + "\combined.pdf")
if __name__ == '__main__':
main()
However, despite running the command via Powershell on Windows 10, and despite using very short filenames, when the number of images is very high (eg over 600 or so), Powershell gives me the error "The command line is too long" and it does not create the pdf. I know there is a command-line string limitation (https://learn.microsoft.com/en-us/troubleshoot/windows-client/shell-experience/command-line-string-limitation), but I also know that for powershell this limit is higher (Powershell to avoid cmd 8191 character limit), and I can't figure out how to fix the script. I would like to ask you if you could help me fix the script to avoid violating the character limit. Thank you
PS: I use the script after inserting it in the parent folder that contains the folders with the images; then in each subfolder the output pdf file is created.
Using img2pdf library you can use this script:
import img2pdf
import os
for r, _, f in os.walk("."):
imgs = []
for fname in f:
if fname.endswith(".jpg") or fname.endswith(".png"):
imgs.append(os.path.join(r, fname))
if len(imgs) > 0:
with open(r+"\output.pdf","wb") as f:
f.write(img2pdf.convert(imgs))
I am trying to run a python script from a powershell script inside SQL Server Agent.
I was able to execute most job of a python script (Task1-Task2) except the last portion (Task3) where it runs a third party exe file called SQBConverter (from RedGate) which converts files from SQB format to BAK format.
When I manually run powershell script directly which runs python script, there is no issue.
I modified the "Log On As" from default ("Local System") to my own (JDoh), and it executes the powershell within SQL Server Agent, but it only does the job except where it converts files from SQB to BAK format (Task3).
Without changing to my own (JDoh), it would not even executes the any job of python script.
I don't think there is any issue with powershell script side because it still triggers python script when I changed the "Log On As" to "Local System". It does not show error, but it shows as SQL Server Agent job completed. But, it does not run any tasks within python script at all.
So, I am guessing it might be something to do with SQL Server Agent not able to trigger/run the SQBConverter exe file.
Here is whole python code (ConvertToBAK.py) to give you the whole idea of logic. It does everything until where it converts from SQB to BAK (Task3: last two lines).
import os
from os import path
import datetime
from datetime import timedelta
import glob
import shutil
import re
import time, sys
today = datetime.date.today()
yesterday = today - timedelta(days = 1)
yesterday = str(yesterday)
nonhyphen_yesterday = yesterday.replace('-','')
revised_yesterday = "LOG_us_xxxx_multi_replica_" + nonhyphen_yesterday
src = "Z:\\TestPCC\\FTP"
dst = "Z:\\TestPCC\\Yesterday"
password = "Password"
path = "Z:\\TestPCC\\FTP"
now = time.time()
### Task1: To delete old files (5 days or older)
for f in os.listdir(path):
f = os.path.join(path, f)
if os.stat(f).st_mtime < now - 5 * 86400:
if os.path.isfile(f):
os.remove(os.path.join(path, f))
filelist = glob.glob(os.path.join(dst, "*"))
for f in filelist:
os.remove(f)
### Task2: To move all files from one folder to other folder location
src_files = os.listdir(src)
src_files1 = [g for g in os.listdir(src) if re.match(revised_yesterday, g)]
for file_name in src_files1:
full_file_name = os.path.join(src, file_name)
if os.path.isfile(full_file_name):
shutil.copy(full_file_name, dst)
### Task3: Convert from SQB format to BAK format (running SQBConverter.exe)
for f in glob.glob(r'Z:\\TestPCC\\Yesterday\\*.SQB'):
os.system( f'SQBConverter "{f}" "{f[:-4]}.bak" {password}' )
This is powershell code (Test.ps1):
$path = 'Z:\TestPCC'
$file = 'ConvertToBAK.py'
$cmd = $path+"\\"+$file # This line of code will create the concatenate the path and file
Start-Process $cmd # This line will execute the cmd
This is screenshot of SQL Server Agent's step:
I looked at the properties of SQBConverter exe file itself, and I granted FULL control for all users listed.
I got it working by modifying the last line of my Python code.
From:
os.system( f'SQBConverter "{f}" "{f[:-4]}.bak" {password}' )
To (absolute path):
os.system( f'Z:\\TestPCC\\SQBConverter.exe "{f}" "{f[:-4]}.bak" {password}' )
I'm in the process of writing a python script that takes two arguments that will allow me to output the contents of a folder to a text file for me to use for another process. The snippet of I have is below:
#!/usr/bin/python
import cv2
import numpy as np
import random
import sys
import os
import fileinput
#Variables:
img_path= str(sys.argv[1])
file_path = str(sys.argv[2])
print img_path
print file_path
cmd = 'find ' + img_path + '/*.png | sed -e "s/^/\"/g;s/$/\"/g" >' + file_path + '/desc.txt'
print "command: ", cmd
#Generate desc.txt file:
os.system(cmd)
When I try and run that from my command line, I get the following output, and I have no idea how to fix it.
sh: 1: s/$//g: not found
I tested the command I am using by running the following command in a fresh terminal instance, and it works out fine:
images/*.png | sed -e "s/^/\"/g;s/$/\"/g" > desc.txt
Can anyone see why my snippet isn't working? When I run it, I get an empty file...
Thanks in advance!
its not sending the full text for your regular expression through to bash because of how python processes and escapes string content, so the best quickest solution would be to just manually escape the back slashes in the string, because python thinks they currently are escape codes. so change this line:
cmd = 'find ' + img_path + '/*.png | sed -e "s/^/\"/g;s/$/\"/g" >' + file_path + '/desc.txt'
to this:
cmd = 'find ' + img_path + '/*.png | sed -e "s/^/\\"/g;s/$/\\"/g" >' + file_path + '/desc.txt'
and that should work for you.
although, the comment on your question has a great point, you could totally just do it from python, something like:
import os
import sys
def main():
# variables
img_path= str(sys.argv[1])
file_path = str(sys.argv[2])
with open(file_path,'w') as f:
f.writelines(['{}\n'.format(line) for line in os.listdir(img_path) if line.endswith('*.png')])
if __name__ == "__main__":
main()
I fully agree with Kyle. My recommendation is to do using only python code better than call bash commands from your code. Here it is my recommended code, it is longer and not as optimal than the aforementioned one, but IMHO it is a more easy to understand solution.
#!/usr/bin/python
import glob
import sys
import os
# Taking arguments
img_path = str(sys.argv[1])
file_path = str(sys.argv[2])
# lets put the target filename in a variable (it is better than hardcoding it)
file_name = 'desc.txt'
# folder_separator is used to define how your operating system separates folders (unix / and windows \)
folder_separator = '\\' # Windows folders
# folder_separator = '/' # Unix folders
# better if you make sure that the target folder exists
if not os.path.exists(file_path):
# if it does not exist, you create it
os.makedirs(file_path)
# Create the target file (write mode).
outfile = open(file_path + '/' + file_name, 'w')
# loop over folder contents
for fname in glob.iglob("%s/*" % img_path):
# for every file found you take only the name (assuming that structure is folder/file.ext)
file_name_in_imgPath = fname.split('\\')[1]
# we want to avoid to write 'folders' in the target file
if os.path.isfile(file_name_in_imgPath):
# write filename in the target file
outfile.write(str(file_name_in_imgPath) + '\n')
outfile.close()
I'm trying to create a Python script that would :
Look into the folder "/input"
For each video in that folder, run a mencoder command (to transcode them to something playable on my phone)
Once mencoder has finished his run, delete the original video.
That doesn't seem too hard, but I suck at python :)
Any ideas on what the script should look like ?
Bonus question : Should I use
os.system
or
subprocess.call
?
Subprocess.call seems to allow for a more readable script, since I can write the command like this :
cmdLine = ['mencoder',
sourceVideo,
'-ovc',
'copy',
'-oac',
'copy',
'-ss',
'00:02:54',
'-endpos',
'00:00:54',
'-o',
destinationVideo]
EDIT : Ok, that works :
import os, subprocess
bitrate = '100'
mencoder = 'C:\\Program Files\\_utilitaires\\MPlayer-1.0rc2\\mencoder.exe'
inputdir = 'C:\\Documents and Settings\\Administrator\\Desktop\\input'
outputdir = 'C:\\Documents and Settings\\Administrator\\Desktop\\output'
for fichier in os.listdir(inputdir):
print 'fichier :' + fichier
sourceVideo = inputdir + '\\' + fichier
destinationVideo = outputdir + '\\' + fichier[:-4] + ".mp4"
commande = [mencoder,
'-of',
'lavf',
[...]
'-mc',
'0',
sourceVideo,
'-o',
destinationVideo]
subprocess.call(commande)
os.remove(sourceVideo)
raw_input('Press Enter to exit')
I've removed the mencoder command, for clarity and because I'm still working on it.
Thanks to everyone for your input.
To find all the filenames use os.listdir().
Then you loop over the filenames. Like so:
import os
for filename in os.listdir('dirname'):
callthecommandhere(blablahbla, filename, foo)
If you prefer subprocess, use subprocess. :-)
Use os.walk to iterate recursively over directory content:
import os
root_dir = '.'
for directory, subdirectories, files in os.walk(root_dir):
for file in files:
print os.path.join(directory, file)
No real difference between os.system and subprocess.call here - unless you have to deal with strangely named files (filenames including spaces, quotation marks and so on). If this is the case, subprocess.call is definitely better, because you don't need to do any shell-quoting on file names. os.system is better when you need to accept any valid shell command, e.g. received from user in the configuration file.
The new recommend way in Python3 is to use pathlib:
from pathlib import Path
mydir = Path("path/to/my/dir")
for file in mydir.glob('*.mp4'):
print(file.name)
# do your stuff
Instead of *.mp4 you can use any filter, even a recursive one like **/*.mp4. If you want to use more than one extension, you can simply iterate all with * or **/* (recursive) and check every file's extension with file.name.endswith(('.mp4', '.webp', '.avi', '.wmv', '.mov'))
Python might be overkill for this.
for file in *; do mencoder -some options "$file"; rm -f "$file" ; done
The rm -f "$file" deletes the files.
AVI to MPG (pick your extensions):
files = os.listdir('/input')
for sourceVideo in files:
if sourceVideo[-4:] != ".avi"
continue
destinationVideo = sourceVideo[:-4] + ".mpg"
cmdLine = ['mencoder', sourceVideo, '-ovc', 'copy', '-oac', 'copy', '-ss',
'00:02:54', '-endpos', '00:00:54', '-o', destinationVideo]
output1 = Popen(cmdLine, stdout=PIPE).communicate()[0]
print output1
output2 = Popen(['del', sourceVideo], stdout=PIPE).communicate()[0]
print output2
Or you could use the os.path.walk function, which does more work for you than just os.walk:
A stupid example:
def walk_func(blah_args, dirname,names):
print ' '.join(('In ',dirname,', called with ',blah_args))
for name in names:
print 'Walked on ' + name
if __name__ == '__main__':
import os.path
directory = './'
arguments = '[args go here]'
os.path.walk(directory,walk_func,arguments)
I had a similar problem, with a lot of help from the web and this post I made a small application, my target is VCD and SVCD and I don't delete the source but I reckon it will be fairly easy to adapt to your own needs.
It can convert 1 video and cut it or can convert all videos in a folder, rename them and put them in a subfolder /VCD
I also add a small interface, hope someone else find it useful!
I put the code and file in here btw: http://tequilaphp.wordpress.com/2010/08/27/learning-python-making-a-svcd-gui/