I'm trying to change a file on remote Linux vm, and redirect it's output to another one (on the same machine) using Paramiko. It should be simple, but I'm obviously missing something here. I use tr command to replace every blank space for tab then redirect it to another file:
command= "tr ' ' '\t' <file1 .txt> file2.txt"
This command actually works fine when executed inside shell, but when I send it over SSH:
(stdin, stdout, stderr) = ssh.exec_command(command)
It creates an empty file, and redirects its output to stdout. Can anyone give me an explanation on why is this happening? I tried to look in documentation, but I couldn't find the solution.
You should put quotes around your command:
command='tr " " "\t" < /tmp/file1.txt > /tmp/file2.txt'
Related
I am attempting to use the code from here https://stackoverflow.com/a/56454579 to upload files to a server with WinSCP from Python on Windows 10. The code looks like this:
import subprocess
path = r'C:\mp4s\Sci-Fi Light Flicker.mp4'
process = subprocess.Popen(
['C:\Program Files (x86)\WinSCP\WinSCP.com', '/ini=nul', '/command', 'option batch abort', 'option confirm off', 'open ftp://user:pass#ftp.website.com', f'put "{path}"', 'exit'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
for line in iter(process.stdout.readline, b''):
print(line.decode().rstrip())
I get the output:
batch abort
confirm off
Connecting to ftp.website.com ...
Connected
Starting the session...
Session started.
Active session: [1] user#ftp.website.com
File or folder '\C:\mp4s\Sci-Fi' does not exist.
System Error. Code: 123.
The filename, directory name, or volume label syntax is incorrect
(A)bort, (R)etry, (S)kip, Ski(p) all: Abort
Which tells me that the put command isn't properly handling the spaces in the filename. What do I need to change to get this to work? Removing the double quotes around the path gives me a different error (it thinks I'm trying to upload multiple files).
I do not think you can use an array to provide arguments to WinSCP. The subprocess.Popen escapes double quotes in the arguments using backslash, what conflicts with double double-quotes escaping expected by WinSCP.
You will have to format the WinSCP command-line on your own:
args = "\"C:\Program Files (x86)\WinSCP\WinSCP.com\" /ini=nul " + \
f"/command \"open ftp://user:pass#ftp.website.com\" \"put \"\"{path}\"\"\" exit"
(note that I've omitted your option commands, as they are defaults anyway)
I use libtmux and libtmux uses subprocess.Popen() to invoke Tmux to control session, windows and panes. To set a pane titel Tmux requires to send an escape sequens. This works on command line as expected:
tmux send-keys -t 0 " printf '\033]2;%s\033\\' 'Pane Title'" C-m
When this command is issued by subprocess.Popen() or os.system() - may be others too, I've only tested these two - the escape squence does not make it to the shell. I logg the command sent to a file and the values in the log file are correct but when sent only this survives:
printf '2;%s' 'Pane Title'
I've tested this by executing this command:
echo "printf '\033]2;%s\033\\' 'Pane Title'" > /tmp/setname
The content of /tmp/setname is the above.
The methode finally used is tmux_cmd and I call it via send_keys like this:
p.send_keys("printf '\033]2;%s\033\\' '" + names[i] + "'")
where "p" is the pane object. See my post here.
My question is: How to issue shell commands with escape sequences in Python?
"raw prefixes" was what I've missed! Thanks Jean-François Fabre!
This works:
p.send_keys(r"printf '\033]2;%s\033\\' '" + names[i] + "'")
In windows, I am running a bat script that currently ends with a 'pause' and prompts for the user to 'Press any key to continue...'
I am unable to edit the file in this scenario and I need the script to terminate instead of hang waiting for input that will never come. Is there a way I can run this that will disable or circumvent the prompt?
I have tried piping in input and it does not seem to help. This script is being run from python via subprocess.Popen.
Try to execute cmd.exe /c YourCmdFile < nul
YourCmdFile - full path to your batch script
subprocess.call("mybat.bat", stdin=subprocess.DEVNULL)
Would call mybat.bat and redirect input from nul on windows (which disables pause as shown in other answers)
This one turned out to be a bit of a pain. The redirect of nul from Maximus worked great, thanks!
As for getting that to work in python, it came down to the following. I started with:
BINARY = "C:/Program Files/foo/bar.exe"
subprocess.call([BINARY])
Tried to add the redirection but subprocess.call escapes everything too well and we loose the redirect.
subprocess.call([BINARY + " < nul"])
subprocess.call([BINARY, " < nul"])
subprocess.call([BINARY, "<", "nul"])
Using shell=True didn't work because the space in BINARY made it choke trying to find the executable.
subprocess.call([BINARY + " < nul"], shell=True)
In the end, I had to resort back to os.system and escape myself to get the redirection.
os.system(quote(BINARY) + " < nul")
Not ideal, but it gets the job done.
If anyone knows how to get the last subprocess example to work with the space in the binary, it would be much apprecated!
For me the following code works:
p = Popen("c:\script.bat <nul", cwd="c:\")
This is baffling me.
I have a python script that does some work on a Windows platform to generate an XML file, download some images and then call an external console application to generate a video using the xml and images the python script has generated.
The application I call with pOPen is supposed to return a status i.e. [success] or [invalid] or [fail] dependant on how it interprets the data I pass it.
If I use my generation script to generate the information and then call the console application separately in another script it works fine and I get success and a video is generated.
Successful code (please ignore the test prints!):
print ("running console app...")
cmd = '"C:/Program Files/PropertyVideos/propertyvideos.console.exe" -xml data/feed2820712.xml -mpeg -q normal'
print (cmd)
p = subprocess.Popen(cmd , stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = p.communicate()[0]
print ("\n----\n[" + output + "]\n----\n")
if output == "[success]":
print "\nHURRAHHHHH!!!!!!!"
print ("finished...")
But if I include the same code at the end of the script that generates the info to feed the console application then it runs for about 2 seconds and output = []
Same code, just ran at the end of a different script...
EDIT:
Thanks to Dave, Dgrant and ThomasK it seems to be that the generate script is not closing the file as redirecting strerr to stdout shows:
Unhandled Exception: System.IO.IOException: The process cannot access the file '
C:\videos\data\feed2820712.xml' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, I
nt32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions o
ptions, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
However I AM closing the file:
Extract from the generation script:
xmlfileObj.write('</FeedSettings>\n') # write the last line
xmlfileObj.close
# sleep to allow files to close
time.sleep(10)
# NOW GENERATE THE VIDEO
print ("initialising video...")
cmd = '"C:/Program Files/PropertyVideos/propertyvideos.console.exe" -xml data/feed2820712.xml -mpeg -q normal'
print (cmd)
p = subprocess.Popen(cmd , stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = p.communicate()[0]
print ("\n----\n[" + output + "]\n----\n")
if output == "[success]":
print "\nHURRAHHHHH!!!!!!!"
print ("finished...")
Any help would be appreciated.
You're not closing the file. Your code says:
xmlfileObj.close
when it should be
xmlfileObj.close()
Edit: Just to clarify - the code xmlfileObj.close is a valid python expression which returns a reference to the built in close method of a file (or file like) object. Since it is a valid python expression it is perfectly legal code, but it does not have any side effects. Specifically, it does not have the effect of actually calling the close() method. You need to include the open and close brackets to do that.
Update: When I use the subprocess.call instead of subprocess.Popen, the problem is solved - does anybody know what's the cause? And there came another problem: I can't seem to find a way to control the output... Is there a way to redirect the output from subprocess.call to a string or something like that? Thanks!
I'm trying to use Devenv to build projects, and it runs just fine when i type it in command prompt like devenv A.sln /build "Debug|Win32" - but when I use a python to run it using Popen(cmd,shell=true) where cmd is the same line as above, it shows nothing. If I remove the |, change it to "Debug" only, it works....
Does anybody know why this happens? I've tried putting a \ before |, but still nothing happened..
This is the code I am using:
from subprocess import Popen, PIPE
cmd = ' "C:\\Program Files\\Microsoft Visual Studio 8\\Common7\\IDE\\devenv" solution.sln /build "Debug|Win32" '
sys.stdout.flush()
p = Popen(cmd,shell=True,stdout=PIPE,stderr=PIPE)
lines = []
for line in p.stdout.readlines():
lines.append(line)
out = string.join(lines)
print out
if out.strip():
print out.strip('\n')
sys.stdout.flush()
...which doesn't work, however, if I swap Debug|Win32 with Debug, it works perfectly..
Thanks for every comment here
There is a difference between devenv.exe and devenv.com, both of which are executable and live in the same directory (sigh). The command lines used in the question and some answers don't say which they want so I'm not sure which will get used.
If you want to call from the command line then you need to ensure you use devenv.com, otherwise you're likely to get a GUI popping up. I think this might be the cause of some (but not all) of the confusion.
See section 17.1.5.1. in the python documentation.
On Windows, Python automatically adds the double quotes around the project configuration argument i.e Debug|win32 is passed as "Debug|win32" to devenv. You DON'T need to add the double quotes and you DON'T need to pass shell=True to Popen.
Use ProcMon to view the argument string passed to devenv.
When shell = False is used, it will treat the string as a single command, so you need to pass the command/arugments as a list.. Something like:
from subprocess import Popen, PIPE
cmd = [
r"C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\devenv", # in raw r"blah" string, you don't need to escape backslashes
"solution.sln",
"/build",
"Debug|Win32"
]
p = Popen(cmd, stdout=PIPE, stderr=PIPE)
out = p.stdout.read() # reads full output into string, including line breaks
print out
try double quoting like: 'devenv A.sln /build "Debug|Win32"'
Looks like Windows' shell is taking that | as a pipe (despite the quotes and escapes). Have you tried shell=False instead?