I'm trying to run a Python script in PowerShell. While using sys module I stumble upon a problem of getting the return value of a PowerShell function. The function in question is Date -f 'dd\/MM\/yyyy' that returns 14/03/2019 and I'd like that value to be used as an argument in the Python script. I've been able so far to get the literal arguments from the command line, e.g. text.
This Python script (see sys.argv docs):
import sys
print(sys.argv)
called like this in Powershell:
python .\test.py -date $(Date -f "dd\/MM\/yyyy")
outputs:
['.\\test.py', '-date', '14/03/2019']
On a general note, I recommend using better date formats than dd/MM/yyyy - ISO 8061 yyyy-MM-dd would be a good one.
If I understand correctly, you want to have the output of the command Date -f 'dd\/MM\/yyyy' to be fed into your python script (script.py).
What you are looking for are so called "pipes" or output redirects. Looking at the officical documentation (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/write-output?view=powershell-6) the following might work:
$DATE=Date -f 'dd/\MM\/yyyy'
Write-Output $DATE | python script.py
Related
I am trying to trigger the python script or shell script whenever a desktop notification has arrived using dbus-monitor
I am using the command in this way
dbus-monitor "interface='org.freedesktop.Notifications'" | grep --line-buffered "string" | xargs -I '{}' python3 ./test.py {}
after that, I am trying to send the desktop notification from another terminal using
-> notify-send "hello" "world"
the output for the above custom notification is
string "notify-send"
string ""
string "hello"
string "world "
string "urgency"
string "notify-send"
string ""
string "hello"
string "world "
string "urgency"
but if my output of this command is 10 lines, then the python script is getting called for every line.
but my expectation is to call the python script once for every notification and then get all the output in a single line as a param for the python script.
It is wise to take advantage of systemd integration with dbus.
Using systemd integration the programmer has better controls/sensors over the dbus integration. Also can take advantage on systemd loging/monitors mechanisms.
There is a good article here about systemd dbus with python..
Also there is very related answer to your question in this answer. as well.
I'm passing the result of the execution of a command to python as input, like so:
$ python parse_ips.py "$(kubectl get configmap ...)"
This works fine when executing from the command line, however I'm now trying to edit the file using PyCharm. Therefore I need the escaped version of the result of this command which I can paste into PyCharm's debug configuration, as I can't execute the command in real-time like I can do on the command line.
However, I am struggling to find a way to replicate the escaping bash does behind the scenes, so I can use the result as an argument within the PyCharm configuration. Running the above kubectl command results in a multi-line string which includes spaces and quotes. When I paste this into PyCharm it just interprets it as multiple arguments. I'm looking for the escaped result, which I could paste directly into the command line, or into PyCharm's debug configuration, to achieve the same result with a fixed parameter for testing.
Any help would be greatly appreciated!
Edit: To clarify, I mean on the command line the result of the $(kubectl ...) command is passed into the python program as a single command line argument when it is surrounded by quotes ("$(kubectl ...)"). So in the python program, you can access sys.argv[1] and it will contain the entire execution output of $(kubectl get configmap ...). However, if I execute that command myself on the command line, the result is a multi-line string.
If I then copy the result of that into PyCharm (or even on the command line again), it is interpreted as many command line arguments. E.g. it would look something like this:
$ python parse_ips.py apiVersion: v1
data:
item1: ifconfig-push 127.0.0.0 255.255.0.0
item2: ifconfig-push 127.0.0.1 255.255.0.0
item3: ifconfig-push 127.0.0.2 255.255.0.0
...
And so on. This obviously doesn't work in the same way as it did before. So I am unable to test my program without making the kubectl call from the command line each time. I was looking to replicate what "$(kubectl ...)" gets converted into so it is able to pass the entire output as a single command line entry.
I am struggling to find a way to replicate the escaping bash does behind the scenes
Typically use printf "%q" to escape stuff.
printf "%q" "$(kubectl get configmap ....)"
This is printf as the bash builtin command. It differs from coreutils printf, and newest ones also support %q with different quoting style:
/usr/bin/printf "%q" "$(kubectl get configmap ....)"
Modern bash also has quoting expansion:
var="$(kubectl get configmap ....)"
echo "${var#Q}"
And there is also the quoting style outputted by set -x.
I would suggest to use a file:
kubectl get configmap ... > /tmp/tempfile
python parse_ips.py "$(cat /tmp/tempfile)"
With xclip you can copy command output straight to the X server clipboard, which is handy:
printf "%q" "$(kubectl get configmap ...)" | xclip -selection clipboard
# then in another window:
python parse_ips.py <right mouse click><select paste>
How do I approach converting this:
echo -e 'FROM busybox\nRUN echo "hello world"' | docker build -
into sh (https://github.com/amoffat/sh) syntax.
The issue I have is that the args are tokenized and quoted. So docker isn't able to pass stdin onto the end of the command since - has single quotes placed around it. Digging in the docs I'm not seeing a way to support this.
Thanks for any help you can provide (alternatively if you have good examples of doing this in subprocess, that would work.)!
The issue I have is that the args are tokenized and quoted. So docker isn't able to pass stdin onto the end of the command since - has single quotes placed around it.
Like I commented,
- isn’t special shell syntax. It should work fine as a normal argument.
Still, here’s how it looks like you do it, going by the docs. Using _in,
dockerfile = r"""
FROM busybox
RUN echo "hello world"
"""
docker.build("-", _in=dockerfile)
I have to write the time it takes for several C programs to run on several files using:
time ./program filename
to a spread sheet and am using subprocess.check_output to get the stdout as a string. I should get something along the lines of:
real 0m0.001s
user 0m0.001s
sys 0m0.000s
but I get:
b'0.00user 0.00system 0:00.00elapsed ?%CPU (0avgtext+0avgdata
1388maxresident)k\n0inputs+0outputs (0major+60minor)pagefaults
0swaps\n'
I see the user and system time but they get cut off after two decimal places. Is there a way to make sure the output reads all 3 decimal places?
Here is my code:
import xlwt
import subprocess
files = ('100KB.txt', '1MB.txt', '10MB.txt', '100MB.txt')
programs = ('./10kBuffer', './step2', './step3', './step4')
command = ['time', programs[0], files[0]]
out = subprocess.check_output(command, stderr=subprocess.STDOUT)
print(out)
that's because GNU time uses the default format string, more detailed, but you need -p option.
Quoting the manual:
The default format string is:
%Uuser %Ssystem %Eelapsed %PCPU (%Xtext+%Ddata %Mmax)k
%Iinputs+%Ooutputs (%Fmajor+%Rminor)pagefaults %Wswaps
When the -p option is given, the (portable) output format is used:
real %e
user %U
sys %S
You also need to decode the output or you'll get bytes instead of str, and newlines won't be interpreted. ex:
>>> print(b'hello\nworld\n')
b'hello\nworld\n'
>>> print('hello\nworld\n')
hello
world
So I would fix your code as is:
command = ['time', '-p', programs[0], files[0]]
out = subprocess.check_output(command, stderr=subprocess.STDOUT)
print(out.decode())
EDIT: the other answer seems to help fixing the missing decimals by using shell built-in. You can mix both answers to get the output you need as string, with enough decimals.
Note that it doesn't seem you can do much better, unless you want to use a profiler for your command (see How do I get time of a Python program's execution?)
It looks like you are running into confusion between GNU time getting used by your python script and the time shell built-in being used on command line.
This comes from the man page for GNU time:
Note: some shells (e.g., bash(1)) have a built-in time command that provides less functionality than the command described here. To access the
real command, you may need to specify its pathname (something like /usr/bin/time).
Based on the output you were expecting, it looks like you are wanting the bash built-in, which gives 3 decimal places:
$ bash -c time time
real 0m0.000s
user 0m0.000s
sys 0m0.000s
$ sh -c time time
user 0m0.00s
sys 0m0.00s
$ ksh -c time time
user 0m0.00s
sys 0m0.00s
$ tcsh -c time time
0.016u 0.011s 0:00.02 100.0% 0+0k 0+0io 0pf+0w
So in order to specify the bash built-in rather than GNU time, you can change your command to:
command = ['bash', '-c', 'time', programs[0], files[0]]
and you should get the output you were expecting.
Suppose I have a python file main.py, and it has some optional parameters, --learning-rate, --batch-size, and etc.
If I want to run this file, I can input the following into the terminal(Ubuntu Linux for example).
python3 main.py --learning-rate 0.1 --batch-size 100
And now, I want to write some code in main.py, in order that after I enter the command above, I can get this command in a string by executing those code. The following is the string I want to get:
"python3 main.py --learning-rate 0.1 --batch-size 100"
The reason I want to do this is that I want to write this string into my recording file so that I can know better what command I have run.
Could anyone tell me what package should I import and what code should I write to get that command information during running the python file?
Thanks!
You cannot always get precisely what you typed, because the shell will have first done substitutions and expanded filenames before starting your script. For example, if you type python "foo.py" *.txt, your script won't see *.txt, it will see the list of files, and it won't see the quotes around foo.py.
With that caveat out of the way, the sys module has a variable named argv that contains all of the arguments. argv[0] is the name of the script.
To get the name of the python executable you can use sys.executable.
To tie it all together, you can do something like this:
print(sys.executable + " " + " ".join(sys.argv))
Why not just remake the command using the arguments you parsed? It won't be exactly what you typed, but that might be nice as it will be in a common format.
Ex (assuming the learning rate and batch size are stored in similarly named variables):
command = "python3 main.py --learning-rate {} --batch-size {}".format(learning_rate, batch_size)
Of course it will be a little more complicated if some commands are optional, but I assume in that case there would be a default value for these parameters, since your network would need those parameters every time.