In bash (as started by Python), I want to print this string \033[31m so that I can use a pipe | operator after it, followed by a command to copy that string to the clipboard. This means that in practice, I'm trying to run something like:
os.system('echo \\033[31m | xsel -ib')
...but the xsel -ib part is working fine, so this question is focused specifically on the behavior of echo.
Most of my attempts have been similar to:
echo -e \\033[31m
I have tried it with single quotes, double quotes, no quotes, removing the -e flag, etc. The closest I got was:
echo -n "\\ 033[31m"
which prints this string \ 033[31m
I don't want that space between \ and 0
-n flag is used to not append a new line after the printed string
I use Ubuntu 20.04, and xsel is a selection and clipboard manipulation tool for the X11 Window System (which Ubuntu 20.04 uses).
echo is the wrong tool for the job. It's a shell builtin, and one for which the POSIX sh standard explicitly does not guarantee portable behavior for when escape sequences (such as \033) are present. system() starts /bin/sh instead of bash, so POSIX behavior -- not that of your regular interactive shell -- is expected.
Use subprocess.run() instead of os.system(), and you don't need echo in the first place.
If you want to put an escape sequence into the clipboard (so not \033 but instead the ESC key that this gets converted to by an echo with XSI extensions to POSIX):
# to store \033 as a single escape character, use a regular Python bytestring
subprocess.run(['xsel', '-ib'], input=b'\033[31m')
If you want to put the literal text without being interpreted (so there's an actual backslash and an actual zero), use a raw bytestring instead:
# to store \033 as four separate characters, use a raw string
subprocess.run(['xsel', '-ib'], input=rb'\033[31m')
For a more detailed description of why echo causes problems in this context, see the excellent answer by Stephane to the Unix & Linux Stack Exchange question Why is printf better than echo?.
If you for some reason do want to keep using a shell pipeline, switch to printf instead:
# to store \033 as four separate characters, use %s
subprocess.run(r''' printf '%s\n' '\033[31m' | xsel -ib ''', shell=True)
# to store \033 as a single escape character, use %b
subprocess.run(r''' printf '%b\n' '\033[31m' | xsel -ib ''', shell=True)
I'm having an issue reading output from a python subprocess command.
The bash command from whose output I want to read:
pacmd list-sink-inputs | tr '\n' '\r' | perl -pe 's/ *index: ([0-9]+).+?application\.process\.id = "([^\r]+)"\r.+?(?=index:|$)/\2:\1\r/g' | tr '\r' '\n'
When I run this via bash I get the intended output:
4 sink input(s) available.
6249:72
20341:84
20344:86
20350:87
When I try to get it's output via python's subprocess running either one :
subprocess.Popen(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()[0].decode('UTF-8')
check_output(cmnd,shell=True).decode('UTF-8')
subprocess.run(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.decode('utf-8')
where cmnd = """pacmd list-sink-inputs | tr '\n' '\r' | perl -pe 's/ *index: ([0-9]+).+?application\.process\.id = "([^\r]+)"\r.+?(?=index:|$)/\2:\1\r/g' | tr '\r' '\n'"""
It gives the following output:
'4 sink input(s) available.\n\x02:\x01\n\x02:\x01\n\x02:\x01\n\x02:\x01\n'
Which is unintended as it doesn't have the 6249:72 ,etc. numbers I want. Even stderr is blank and returncode is 0 as intended.
The only workaround, I could find was to redirect the bash output to a text file and then read the text file via python which I don't want to use because that's unnecessary file IO.
I've already gone through Missing output from subprocess command, Python Subprocess Grep, Python subprocess run() is giving abnormal output [duplicate] and many others but can't wrap my head around what's going wrong.
You have a quoting issue. """\1""" means chr(0o1). To produce the string \1, you could use """\\1""". The other instances of \ should be \\ as well.
Since all instances of \ need to be escaped, you could also use r"""\1""".
Other issues:
\1 and \2 outside of a regular expression is wrong anyways. You should be using $1 and $2.
There's no use for a mutliline literal here. "..." or r"..." would suffice.
The whole tr business can be avoided by using -0777 to cause perl to treat the entire file as one line.
This gives us:
cmnd = "pacmd list-sink-inputs | perl -0777pe's/ *index: (\\d+).+?application\\.process\\.id = "([^\\n]+)"\\n.+?(?=index:|$)/$2:$1\\n/sag'"
or
cmnd = r"pacmd list-sink-inputs | perl -0777pe's/ *index: (\d+).+?application\.process\.id = "([^\n]+)"\n.+?(?=index:|$)/$2:$1\n/sag'"
But why is Perl being used at all here? You could easily do the same thing in Python!
Normally I work with Python but I have a project in Perl. So: What is the process for directing the results of an snmpwalk to a string? I would like to search the string to see if it contains a smaller string.
Here is what I have so far:
foreach (#list){
chomp($_);
system("snmpwalk -v 2c -c community-string $_ oid-hidden");
if (index($string, $substring) != -1) {
print "'$string' contains '$substring'\n";
}
}
system function doesn't return the function output, use qx// or backticks, so your snmpwalk call line will look like this:
my $output = qx/snmpwalk -v 2c -c community-string $_ oid-hidden/;
And then you do with the output variable what you need, for more info I'd refer you to http://perldoc.perl.org/perlop.html#Quote-Like-Operators
However in more general terms I'd follow the advice in #ThisSuitIsBlackNot's comment...
I have a python script that essentially parses an xml file, uses the package re and prints text as follows:
string = str(search_compiled.groups(0)[0].encode('utf-8')) + "%" + str(text.encode('utf-8'))
print string
I receive the text in the shell script as follows:
string="$($file.py $arg1 $arg2 $arg3)"
varA="$(echo "$string" | cut -d'%' -f1)"
varB="$(echo "$string" | cut -d'%' -f2)"
echo "$string"
So, in summary, I need the passed string to be cut into two by the delimiter '%' and store the results in varA and varB.
The splitting does not happen.
string shows the entire thingy: part A plus the part B. Here's the catch, the '%' I added in the python script does not get printed though.
Could anyone please help me in understanding what is going wrong?
You can use the pipe and cut commands as you have in the question but without the quotes on the delimiter character use -d% instead of -d'%'
varA=$(echo $string | cut -f1 -d%)
varB=$(echo $string | cut -f2 -d%)
[root#test /tmp]$ eval `echo "aaa%bbb%ccc" | awk -F '%' '{print "a="$1" b="$2}'`
[root#test /tmp]$ echo $a
aaa
[root#test /tmp]$ echo $b
bbb
Explanation
Use awk -F '%' '{print "a="$1" b="$2}' get like this a=aaa b=bbb
eval a=aaa b=bbb Equivalent to the input terminal
$ a=aaa
$ b=bbb
I re-read this for a 3rd time, and I think this is the basic problem (from your description):
string shows the entire thingy: part A plus the part B. Here's the catch, the '%' I added in the python script does not get printed though.
The conversion of data to utf-8 then back to string seems suspect to me. Can you change the string creation line in your python program to this:
string = u'{}%{}'.format(search_compiled.groups(0)[0].encode('utf-8'), text.encode('utf-8'))
You might be double encoding, so this could be what you need:
string = u'{}%{}'.format(search_compiled.groups(0)[0], text)
Add this in the shell script before it calls the python script:
export PYTHONIOENCODING=UTF-8
I have a file which contains multiple like this:
s10123-yyy.bkp.abc01.zone,Windows File =
System,N/A,defaultBackupSet,default,272188(* )(S =
),Completed,INCR,Mixed,02/28/2015 19:00:27,02/28/2015 =
19:03:06,02/28/2015 20:32:11,02/28/2015 =
20:32:09,12.08,53.93%,0.18,98.52%,0%,0.12,1:28:23,N/A,8.203,N/A,67303,0,8=
3,"Disk_Library2, Disk_Library6,",N/A,N/A,=0A=
Which I need to make it in one line like this:
s10123-yyy.bkp.abc01.zone,Windows File System,N/A,defaultBackupSet,default,272188(* )(S ),Completed,INCR,Mixed,02/28/2015 19:00:27,02/28/2015 19:03:06,02/28/2015 20:32:11,02/28/2015 20:32:09,12.08,53.93%,0.18,98.52%,0%,0.12,1:28:23,N/A,8.203,N/A,67303,0,83,"Disk_Library2, Disk_Library6,",N/A,N/A
If I do it manually, I highlight the "=" and press "delete" button twice to connect and get the desired result.
The last 5 character ",=0A=" needs to be deleted too.
Awk, Sed, Bash, Perl or Python script would be preferred.
Appreciate you help.
Thanks!
This is most simple with awk1:
awk -v RS=',=0A=\n' -F '=\n' -v OFS= '{ $1 = $1 } 1' filename
The trick is to
use ,=0A=\n as record separator RS
=\n as field separator
have an empty output field separator OFS, so that the fields are printed directly one after the other, and
force the rebuilding of the output record with $1 = $1 before printing it.
Addendum: Obligatory crazy sed solution:
sed -n '/,=0A=$/ { s///; H; s/.*//; x; s/\n//g; p; d; }; /=$/ { s///; H; }' filename
I don't recommend that you use that; I just like writing things in sed that shouldn't be written in sed. It's fun!
1 Tested with GNU awk and mawk, which are the most common ones. Multi-character RS is not strictly required by POSIX, though, so more esoteric awks may reject this. Thanks to #TomFenech for pointing this out.
Through Perl.
perl -0777pe 's/=\n|,=[^,]*$//sg' file
In Python, Create a list and then use the extend method to add the lines to the list, versus append.
This is a Perl solution:
perl -l -0777 -pwe"s/,?=(?:0A=)?\n//g" file
-0777 disables input record separator, making the file into one single line.
-p reads input from file and prints it back to standard output.
-l (before -0) adds newline to your print statements.
The regex s/,?=(?:0A=)?\n//g finds an optional comma, followed by =, followed by optional 0A= string, and ending with newline.
I don't know if all your files are just one of these long lines. If it is multiple such lines, you should set the input record separator to =0A=\n, most likely, chomp the lines and delete =\n.
sed
sed '
:a
/,=0A=$/ {s///; s/\n//g} # "end of line", remove the chars and newlines
/ \?=$/ {s///; N; ba} # line continuation: remove the chars, append
# the next line, goto a
' file