Background: I have a python program that imports and uses the readline module to build a homemade command line interface. I have a second python program (built around bottle, a web micro-framework) that acts as a front-end for that CLI. The second python program opens a pipe-like interface to the first, essentially passing user input and CLI output back and forth between the two.
Problem: In the outer wrapper program (the web interface), whenever the end-user presses the TAB key (or any other key that I bind the readline completer function), that key is inserted into the CLI's stdin without firing the readline completer function. I need this to trigger readline's command completion function instead, as normally occurs during an interactive CLI session.
Possible Solution #1: Is there some way to send the TAB key to a subprocess' stdin, so that a batch usage works the same as an interactive usage?
Possible Solution #2: Or, if there was some way to trigger the entire completion process manually (including matches generation and display), I could insert and scan for a special text sequence, like "<TAB_KEY_HERE>", firing the possible completion matches display function manually. (I wrote the completer function, which generates the possible matches, so all I really need is access to readline's function to display the possible matches.)
Possible Solution #3: I guess, if I cannot access readline's matches-display function, the last option is to rewrite readline's built-in display-completion function, so I can call it directly. :(
Is there a better solution? Any suggestions on following the paths presented by any of the above solutions? I am stuck on #1 and #2, and I'm trying to avoid #3.
Thanks!
Solution #1 proved to be a workable approach. The key was to not connect the web socket directly to the CLI app. Apparently, readline was falling back into some simpler mode, which filtered out all TAB's, since it was not connected to a real PTY/TTY. (I may not be remembering this exactly right. Many cobwebs have formed.) Instead, a PTY/TTY pair needed to be opened and inserted in between the CLI app and web-sockets app, which tricked the CLI app into thinking it was connected to a real keyboard-based terminal, like so:
import pty
masterPTY, slaveTTY = pty.openpty()
appHandle = subprocess.Popen(
['/bin/python', 'myapp.py'],
shell=False,
stdin=slaveTTY,
stdout=slaveTTY,
stderr=slaveTTY,
)
...
while True
# read output from CLI app
output = os.read(masterPTY, 1024)
...
# write output to CLI app
while input_data:
chars_written = os.write(masterPTY, input_data)
input_data = input_data[chars_written:]
...
appHandle.terminate()
os.close(masterPTY)
os.close(slaveTTY)
HTH someone else. :)
See this answer to a related question for more background:
https://stackoverflow.com/a/14565848/538418
Related
I am trying to use Paramiko to access the input and output of a program running inside of a screen session. Let's assume there is a screen screenName with a single window running the program I wish to access the I/O of. When I try something like client.exec_command('screen -r screenName') for example, I get the message "Must be connected to a terminal" in the stdout.
Searching around, I learned that for some programs, one needs to request a "pseudo-terminal" by adding the get_pty=True parameter to exec_command. When I try this, my code just hangs, and my terminal becomes unresponsive.
What else can I try to make this work?
To provide a little more context to my problem, I am creating a web app that allows users to perform basic operations that I have previously done only in a PuTTy terminal. A feature I wish to have is a kind of web terminal designed to interact with the input and output of this particular program. Effectively all I want to do is continuously pipe output from the program to my backend, and pipe input to the program.
I'm working on a script that should check on certain system events (like opening of a file, or changing of a registry key) and start further actions depending on that. But I haven't found a clean way to get the information into my script.
I'm looking for a way to get the output of Sysinternals Process Monitor into another program. This should happen without user interaction in close to real time; so saving into a CSV/XML and than using this doesn't work.
I've checked on using the backing file, but this is in the Process Monitor PML format, which i haven't found to be documented anywhere.
Does anybody know a way how I can get the output of Process Monitor into my script?
Or an other (not too messy) way to get a real time list of opened files, registry keys etc into a python program?
Thanks!
If you want to parse stdout or a file, and your ok with a 32 bit only solution, try Dr Strace or ntstrace.
YOu could also look into ospy or another ProcMon alternative. ospy is open source, so at the very least you could look at the source code for capturing events.
Here is a list of alternates to ProcMon.
I want to create a build pipeline, and developers need to set up a few things into a properties file which gets populated using a front end GUI.
I tried running sample CLI interactive script using python that just asked for a name and prints it out afterwards, but Jenkins just waited for ages then hanged. I see that it asked for the input, but there was no way for the user to input the data.
EDIT: Currently running Jenkins as a service..Or is there a good plugin anyone recommends or is it the way I created the python script?
Preference:
I would prefer to use Python because it is a little lightweight, but if people had success with other languages I can comprise.
Using a GUI menu to populate the data, would be cool because I can use option boxes, drop down menus and make it fancy but it isn't a necessity, a CLI is considerably better than our current deployment.
BTW, running all this on Windows 7 laptop running Python 2.7 and Java 1.7
Sorry for the essay! Hopefully people can help me!
Sorry, but Jenkins is not an interactive application. It is designed for automated execution.
The only viable way to get input to a Jenkins job (and everything that is executed from that job) is with the job parameters that are populated before the job is started. Granted, Jenkins GUI for parameter entry is not the greatest, but it does the job. Once the Jenkins job collected the job parameters at the start of the job, it can pass those parameters to anything it executes (Python, shell, whatever) at any time during the job. Two things have to be true for that to happen:
You need to collect all the input data before the job starts
Whatever your job calls (Python, shell, etc) need to be able to receive their input not interactively, but through command line.
How to get input into program
A well designed script should be able to simply accept parameters on the command line:
./goodscript.sh MyName will be the simplest way of doing it, where value MyName will be stored in $1 first parameter of the script. Subsequent command line parameters will be available in variables $2, $3 and so on.
./goodscript.sh -name MyName -age 30 will be a better way of doing it, where the script can take multiple parameters regardless of their order by specifying a parameter name before parameter value. You can read about using getopt for this method of parameter passing
Both examples above assume that the goodscript.sh is written well enough to be able to process those command line parameters. If the script does not explicitly process command line parameters, doing the above will be useless.
You can "pipe" some output to an interactive script that is not designed to handle command line parameters explicitly:
echo MyName | ./interactivescript.sh will pass value MyName to the first interactive prompt that interactivescript.sh provides to the user. Problem with this is that you can only pass a value to the first interactive prompt.
Jenkins job parameters GUI
Like I said above, you can use Jenkins GUI to gather all sorts of job parameters (dropdown lists, checkboxes, text entry). I assume you know how to setup Jenkins job with parameters. If not, in the job configuration click "This build is parameterized" checkbox. If you can't figure out how to set this up, that's a different question and will need to be explained separately.
However, once your Jenkins job collected all the parameters up front, you can reference them in your "execute shell" step. If you are using Windows, you will reference them as %PARAM_NAME%, and for Linux as $PARAM_NAME.
Explain what you need help with: getting your script to accept command line parameters, or passing those command line parameters from jenkins job GUI, and I will expand this answer further
Prior info: I'm on a Mac.
Q: How can I get terminal-like text output from the program execution, if I compile it with py2app for redistribution?
My case is a program that copies a lot of big files and takes a while to process so I would like to at least have an output notification everytime each file is copied.
This is easy if I run it on the command line, I can just print a new line.
But when I make a self-sufficient package, it simply opens on the bottom dock, with no window, and closes upon completion.
A simple text window would be fine.
Thanks in advance.
If you want to create a simple text window, you need to pick a GUI framework to do that with. For something this simple, there's no reason not to use Tkinter (which comes with any Python) or PyObjC (which is pre-installed with Apple's Python 2.7), unless you happen to be more familiar with wx, gobject, Qt, etc.
At any rate, however you do it, you'll need to write a function that takes a message and appends it to the text window (maybe creating it lazily, if necessary), and call that function wherever you would normally print. You may also want to write and install a logging handler that does the same thing, so you can just log.info stuff. (You could instead create a file-like object that does this and redirect stdout and/or stderr, but unless you have no control over the printing code, that's going to be a lot more work.)
The only real problem here is that a GUI needs an event loop, and you probably just wrote your code as a sequential script.
One way around that is to turn your whole current script into a background thread. If you're using a GUI library that allows you to access the widgets from background threads, everything is easy; your printfunc just does textwidget.append(msg). If not, it may at least have a call_on_main_thread type function, so your printfunc does call_on_main_thread(textwidget.append, msg). If worst comes to worst (and I believe with Tkinter, it does), you have to create an explicit queue to push messages through, and write a queue handler in the event loop. This recipe should give you an idea. Replace the body of workerThread with your code, and end it with self.endApplication(). (There are probably better examples out there; this was just what I found first in a quick search.)
The other way around that is to have your code cooperatively operate with the event loop. Some libraries, like wx, have functions like SafeYield that make things work if you just call it after every chunk of processing. Others don't have that, but have a way to explicitly drive the event loop from your code. Others have neither—but every event loop framework has to have a way to schedule new events, so you can break your code up into a sequence of functions that each finish quickly and then do something like root.after_idle(nextfunc).
However… are you sure you need to do this?
First, any app, including one created by py2app, will send its stdout to the terminal if you run it with Foo.app/Contents/MacOS/Foo. And you can even set things up so that open Foo.app works that way, if you want. Obviously this doesn't help for people who just double-click the app in Finder (because then there is no terminal), but sometimes it's sufficient to just have to output available when people need it and know how to follow instructions.
And you can take this farther: Create a Foo.command file that just does something like $(dirname $0)/Foo.app/Contents/MacOS/Foo, and when you double-click that file, it launches Terminal.app and runs your script.
Or you can get even simpler: Just use logging to syslog the output, and if you want to see when each file is done, just watch the log messages go by in Console.app.
Finally, do you even need py2app in the first place? If you don't have any external dependencies, just rename you script to Foo.command, and double-clicking it will run it in Terminal.app. If you do have external dependencies, you might still be able to get away with bundling it all together as a folder with a .command in it instead of as a .app.
Obviously none of these ideas are exactly a professional or newbie-friendly way to build an interface, so if that matters, you will have to create a GUI.
I would like to daemonize a python process, and now want to ask if it is good practice to have a daemon running, like a parent process and call another class which opens 10-30 threads.
I'm planning on writing a monitoring script for group of servers and would like to check every server every 5 mins, that each server is checked exactly 5minutes.
I would like to have it this way ( sort of speak, ps auxf style output ):
|monitor-daemon.py
\-check-server.py
\-check-server.py
....
Thank you!
Maybe you should use http://pypi.python.org/pypi/python-daemon
You can use supervisord for this. You can configure tasks to respond to events. The events can be manually created or automatically by monitoring processes or based on regular intervals.
It is fully customizable and written in Python.
Example:
[program:your_daemon_name]
command=your_daemon_process
# Add extra options here according to the manual...
[eventlistener:your_monitor_name]
command=your_monitor_process
events=PROCESS_STATE_RUNNING # Will be triggered after a program changes from starting to running
# Add extra options here according to the manual...
Or if you want the eventlistener to respond to the process output use the event PROCESS_COMMUNICATION_STDOUT or TICK_60 for a check every minute. The logs can be redirected to files and such so you can always view the state.
There's really not much to creating your own daemonize function: The source for Advanced Programming in the Unix Environment (2nd edition) is freely available: http://www.apuebook.com/src.tar.gz -- you're looking for the apue.2e/daemons/init.c file.
There is a small helper program that does all the work of creating a proper daemon, it can be used to wrap arbitrary programs; this might save some hassle.