I'm writing a python plugin for vim and it's looking like the only way to call a specific command is with the vim.command function. However, just substituting values into the function seems like a bad idea. How would I escape values so that I can pass untrusted data as an argument into a vim function? As a simple example, let's say I want to echo out untrusted input (I know I could just use print, but this is just an example). I would do something like:
value = get_data_from_untrusted_source()
vim.command("echo %s" % value)
However, if that untrusted data has a | in it, the command is ended and a new one is executed which is bad. Even if I use quotes, we end up with sql injection like attacks where an attacker can just put an apostrophe in their response to end the string. Then if we double quote, it could be possible to put a backslash somewhere to end the quote. For example if we just double quotes we would go from \' to \'' which escapes the first quote.
Basically what I'm asking is if there's a safe way to call vim functions from a python plugin and would appreciate any help.
Related
I would like to add a bash command to my Python script, which linearises a FASTA sequence file while leaving sequence separation intact (hence the specific choice of command). Below is the command, with the example input file of "inputfile.txt":
awk '/^>/ {printf("\n%s\n",$0);next; } { printf("%s",$0);} END {printf("\n");}' < inputfile.txt
The aim is to allow the user to specify the file which is to be modified in the command line, for example:
$ python3 program.py inputfile.txt
I have tried to use string formatting (i.e. %s) in conjunction with sys.argv in order to achieve this. However, I have tried many different locations of " and ', and still cannot get this to work and accept a user input from the command line here.
(The command contains escapes such as \n and so I have tried to counteract this by adding additional backslashes, as well as additional % for the existing %s in the command.)
import sys
import subprocess
path = sys.argv[1]
holder = subprocess.Popen("""awk '/^>/ {printf("\\n%%s\\n",$0);next; } { printf("%%s",$0);} END {printf("\\n");}' < %s""" % path , shell=True, stdout=subprocess.PIPE).stdout.read()
print(holder)
I would very much appreciate any help with identifying the syntax error here, or suggestions for how I could add this user input.
TL;DR: Don't shell out to awk! Just use Python. But let's go step by step...
Your instinct of using triple quotes here is good, then at least you don't need to escape both single and double quotes, that you need in your shell string.
The next useful device you can use is raw strings, using r'...' or r"..." or r"""...""". Raw strings don't expand backslash escapes, so in that case you can leave the \ns intact.
Last is the %s, which you need to escape if you use the % operator, but here I'm going to suggest that instead of using the shell to redirect input, just use Python's subprocess to send stdin from the file! Much simpler and you end up with no substitution.
I'll also recommend that you use subprocess.check_output() instead of Popen(). It's much simpler to use and it's a lot more robust, since it will check that the command exited successfully (with a zero exit status.)
Putting it all together (so far), you get:
with open(path) as inputfile:
holder = subprocess.check_output(
r"""awk '/^>/ {printf("\n%s\n",$0);next; } { printf("%s",$0);} END {printf("\n");}'""",
shell=True,
stdin=inputfile)
But here you can go one step further, since you don't really need a shell anymore, it's only being used to split the command line into two arguments, so just do this split in Python (it's almost always possible and easy to do this and it's a lot more robust since you don't have to deal with the shell's word splitting!)
with open(path) as inputfile:
holder = subprocess.check_output(
['awk', r'/^>/ {printf("\n%s\n",$0);next; } { printf("%s",$0);} END {printf("\n");}'],
stdin=inputfile)
The second string in the list is still a raw string, since you want to preserve the bacsklash escapes.
I could go into how you can do this without using printf() in awk, using print instead, which should get rid of both \ns and %s, but instead I'll tell you that it's much easier to do what you're doing in Python directly!
In fact, everything that awk (or sed, tr, cut, etc.) can do, Python can do better (or, at least, in a more readable and maintainable way.)
In the case of your particular code:
with open(path) as inputfile:
for line in inputfile:
if line.startswith('>'):
# Insert a blank line before this one.
print()
print(line)
if line.startswith('>'):
# Also insert a blank line after this.
print()
# And a blank line at the end.
print()
Isn't this better?
And you can put this into a function, into a module, and reuse it anywhere you'd like. It's easy to store the result in a string, save it into a variable if you like, much more flexible...
Anyways, if you still want to stick to shelling out, see my previous code, I think that's the best you can do while still shelling out, without significantly changing the external command.
I want to open a file in python 3.5 in its default application, specifically 'screen.txt' in Notepad.
I have searched the internet, and found os.startfile(path) on most of the answers. I tried that with the file's path os.startfile(C:\[directories n stuff]\screen.txt) but it returned an error saying 'unexpected character after line continuation character'. I tried it without the file's path, just the file's name but it still didn't work.
What does this error mean? I have never seen it before.
Please provide a solution for opening a .txt file that works.
EDIT: I am on Windows 7 on a restricted (school) computer.
It's hard to be certain from your question as it stands, but I bet your problem is backslashes.
[EDITED to add:] Or actually maybe it's something simpler. Did you put quotes around your pathname at all? If not, that will certainly not work -- but once you do, you will find that then you need the rest of what I've written below.
In a Windows filesystem, the backslash \ is the standard way to separate directories.
In a Python string literal, the backslash \ is used for putting things into the string that would otherwise be difficult to enter. For instance, if you are writing a single-quoted string and you want a single quote in it, you can do this: 'don\'t'. Or if you want a newline character, you can do this: 'First line.\nSecond line.'
So if you take a Windows pathname and plug it into Python like this:
os.startfile('C:\foo\bar\baz')
then the string actually passed to os.startfile will not contain those backslashes; it will contain a form-feed character (from the \f) and two backspace characters (from the \bs), which is not what you want at all.
You can deal with this in three ways.
You can use forward slashes instead of backslashes. Although Windows prefers backslashes in its user interface, forward slashes work too, and they don't have special meaning in Python string literals.
You can "escape" the backslashes: two backslashes in a row mean an actual backslash. os.startfile('C:\\foo\\bar\\baz')
You can use a "raw string literal". Put an r before the opening single or double quotes. This will make backslashes not get interpreted specially. os.startfile(r'C:\foo\bar\baz')
The last is maybe the nicest, except for one annoying quirk: backslash-quote is still special in a raw string literal so that you can still say 'don\'t', which means you can't end a raw string literal with a backslash.
The recommended way to open a file with the default program is os.startfile. You can do something a bit more manual using os.system or subprocess though:
os.system(r'start ' + path_to_file')
or
subprocess.Popen('{start} {path}'.format(
start='start', path=path_to_file), shell=True)
Of course, this won't work cross-platform, but it might be enough for your use case.
For example I created file "test file.txt" on my drive D: so file path is 'D:/test file.txt'
Now I can open it with associated program with that script:
import os
os.startfile('d:/test file.txt')
I need a python script to call a bash script on windows.
So basically I must make a subprocess call form python, that will call cygwin with the -c option that will call the script I need,
The problem is that this script takes a few arguments and that these arguments are full os spaces and quotes and slashes.
I'm using code like the following
arq_saida_unix = arq_saida.replace("\\","/")
subprocess.call("C:\\cygwin64\\bin\\bash \".\\retirarVirgula.sh\\ \""+arq_saida+"\"")
Or I'm directly escaping, which sometimes takes me to as much as 8 backslashes in a row, for a backslash to get to my script must be escaped i) in bash ii) in cmd.exe iii) in python
all of this is error prone and takes quite some time every time to get it right.
Is there a better way of doing it? Ideally I wouldn't have any escaping backslashes, but anything that avoids the triple-slash double quote above would be nice.
I tried to use re.escape, but could figure out how exactly to use it , except as a replacement to .replace("\","/") and similar.
Don't pass a single string to call; instead, pass a list consisting of the command name and one argument per element. This saves you from needing to protect special characters from shell interpretation.
subprocess.call(["retirarVirgula.sh", arq_saida], executable=r"C:\cygwin64\bin\bash")
Note: I'm assuming arq_saida contains the single argument to pass to the script; if the script takes multiple arguments, then arc_saida should probably be built as a list as well:
arq_saida = ["arg", "arg two", "arg three"]
subprocess.call(["retirarVirgula.sh"] + arq_saida, executable=r"C:\cygwin64\bin\bash")
I noticed this code:
os.system("'{0}'".format(path))
and saw that some one had change it to this:
os.system("\"{0}\"".format(path))
I was wondering by changing it from single to double quotes what advantages does it give you?
Here is the original commit I pulled it from: https://github.com/mattn/legit/commit/84bd1b1796b749a7fb40e0b734d2de29ddc9d3d9
Not much really but rule of thumb use single quotes for literal assignments and prints. That way you will avoid printing things that shouldn't be there in the first place.
Single quotes are often useful because they are literal, and contain exactly the characters you type e.g. 'Hi there/' will actually print Hi there/
However, if you need something like 'Hi there /n', if you put it in single quotes it will give you literally 'Hi there /n' whereas double quotes will give you the result you need "Hi there" and then break line.
On windows, command line arguments are parsed by the program it-self, not shell or cmd.exe. And most of windows programs parse quoted strings with double quote in generally. python.exe is the same. On unix OSs, command line arguments are parsed by shell. And most of shells parse single/double quote both. of course, double quote expand $ or something which the shell can treat. However, path will not contains $.
This change is workaround to be possible to work legit on many OSs.
In Vim, it's a quick 3-character command to change what's inside the current quoted string (e.g., ci"), but is there a simple way to change what type of quotes are currently surrounding the cursor?
Sometimes I need to go from "blah" to """blah""" or "blah" to 'blah' (in Python source code) and I'd ideally like to do it quickly using default key bindings.
Try the surround.vim plugin. I find it an essential addition to any vim installation.
Surround.vim is great, but I don't think it'll handle your triple-quoted needs directly.
The way I've done stuff along these lines (when surround wasn't appropriate) was to use %, make the change, then double-backtick to go back to the starting point. E.g. if the cursor is somewhere in a single-quoted string, do f'%, make the change, then double-backtick and ..