We Keep Coding

How to notify a daemon given a pid

So I have been getting my feet wet with python, attempting to build a reminder system that ties into the gnome notification ui. The basic idea is you type a command into your shell like remind me to check on dinner in 20 min and then in 20 min you get a desktop notification saying "check on dinner". The way I am doing this is by having a script parse the message and write the time the notification should be sent and the message that should be sent to a log file.
The notifications are getting triggered by a python daemon. I am using this daemon design I found online. The issue I am seeing is when this daemon is running it is taking 100% of my cpu! I stripped down all the code the daemon was doing and it I still have this problem when all the daemon is doing is
while True:
last_modified = os.path.getmtime(self.logfile)
I presume that this is a bad approach and I should instead be notifying the daemon when there is a new reminder and then most of the time the reminder daemon should be sleeping. Now this is just an idea but I am having a hard time finding resources on 'how to notify a process' when all I know is the daemons pid. So if I have suspend the daemon with something like time.sleep(time_to_next_notification) would there be a way for me to send a signal to to the daemon letting it know that there was a new reminder?

Though I believe you're better off using a server - client type solution that listens on a port, what you are asking is 100% possible using the signal and os libraries. This approach will not work well with multi threaded programs however as signals are only handled by the parent thread in python. Additionally windows doesn't implement signals in the same way so the options are more limited.
Signals
The "client" process can send arbitrary signals using os.kill(pid, signal). You will have to go through the available signals and determine which one you want to use (signal.NSIG may be a good option because it shouldn't stomp on any other default behavior).
The "daemon" process on startup must register a handler for what to do when it receives your chosen signal. The handler is a function you must define that receives the signal itself that was received as well as the current stack frame of execuiton (def handler(signum, frame):). If you're only doing one thing with this handler, and it doesn't need to know what was happening when it was called, you can probably ignore both these parameters. Then you must register the handler with signal.signal ex: signal.signal(signal.NSIG, handler).
From there you will want to find some appropriate way to wait until the next signal without consuming too many resources. This could be as simple as looping on a os.sleep
command, or you could try to get fancy. I'm not sure 100% how execution resumes on returning from a signal handler, so you may need to concern yourself with recursion depth (ie, make sure you don't recurse every time a signal is handled or you'll only ever be able to handle a limited number of signals before needing to re-start).
Server
Having a process listen on a port (generally referred to as a server, but functionally the same as your 'daemon' description) instead of listen for operating system signals has several main advantages.
Ports are able to send data where signals are only able to trigger events
Ports are more similar cross-platform
Ports play nice[r] with multi-threading
Ports make it easy to send messages across a network (ie: create reminder from phone and execute on PC)
Waiting for multiple things at once
In order to address the need to wait for multiple processes at once (listening for input as well as waiting to deliver next notification) you have quite a few options:
Signals actually may be a good use case as signal.SIGALRM can be used as a conveniently re-settable alarm clock (if you're using UNIX). You would set up the handler in the same way as before, and simply set an alarm for the next notification. After setting the alarm, you could simply resume listening on the port for new tasks. If a new task comes in, setting the alarm again will override the existing one, so the handler would need to retrieve the next queued notification and re-set the alarm once done with the first task.
Threads could either be used to poll a queue of notification tasks, or an individual thread could be created to wait for each task. This is not a particularly elegant solution, however it would be effective and easy to implement.
The most elegant solution would likely be to use asyncio co-routines, however I am not as well versed in asyncio, and will admit they're a bit more confusing than threads.

processing HTTPS requests and ZeroMQ messages in parallel inside Twisted

I am trying to do two things in parallel:
intercept HTTPS POST request and make Twisted hold the connection open indefinitely (return NOT_DONE_YET)
listen to ZeroMQ queue (and close one of the pending connections as a result of a ZeroMQ message)
I know how to make each of these two things separately, but how can I do them in parallel? To be specific, how can I receive messages from ZeroMQ inside the Twisted framework, when I am already listening to TCP sockets?

You can get support for ZMQ here, in txZMQ.
It sounds like you already know how to deal with the HTTP request.
To do these things in parallel, just create your ZmqSubConnection and your twisted.web.server.Site, with references to each other as necessary for your application. No special set-up is required.
Just make sure you only create and run one reactor. Other people with similar questions sometimes don't understand that reactor.run() means "run the event loop for the whole process", and not "run this one thing I set up".

Python: Interrupting sender with incoming messages

I'm working with socket, asynchronous event-driven programming. I would like to send a message, once I receive a response, send another message. But I may be doing something besides listening. That is, I want to get interrupted when socket.recv() actually receives a message.
Question 1: How can I let layer 3 interrupt layer 4? i.e. How can I handle the event of a non-null returning socket.recv() without actually dedicating "program time" to actually wait for a specific time to listen to incoming messages?

In asynchronous programming you don't interrupt an operation triggered by a message. All operations should be done in a short and fast fashion so you can process lots of messages per second. This way every operation is atomic and you don't suffer any race conditions so easily.
If you are in need to do more complex processing in parallel you could hand those problems over to a helper thread. Libraries like twisted are prepared for such use cases.

Do you need to use sockets directly? I would otherwise recommend looking into the excellent Twisted library for python. It handles all the async work for you, so you can focus on writing handlers and other logic your code needs. Twisted is relatively easy to get started with. Take a look at some examples at http://twistedmatrix.com/documents/current/core/howto/index.html.

Twisted irc python bot - buffering messages

"Why does it take a long time for data I send with transport.write to arrive at the other side of the connection?"
Twisted can only send data after you give up control of execution to the reactor. For example, if you have an infinite loop writing data to a transport, the data will never actually be sent since control will never leave your code and return to the reactor."
I found this in the twisted FAQ, it is exactly as my problem. Is there a way to fix this so I can send messages instantly and not wait for the loop to finish?

No. You cannot send messages instantly. Control must return to the event loop. Fortunately, the problem you're really trying to solve (inferred from comments on the question), rate limiting the messages you send, doesn't require being able to do this. Instead, stop using time.sleep in a loop and start using reactor.callLater to delay the execution of code. Or use features of the IRC API which do these things for you (which you seem to have discovered already, lineRate).

what's the tornado ioloop, and tornado's workflow?

i want to know tornado's internal workflow, and have seen this article, it's great, but something i just can't figure out
within the ioloop.py, there is such a function
def add_handler(self, fd, handler, events):
"""Registers the given handler to receive the given events for fd."""
self._handlers[fd] = handler
self._impl.register(fd, events | self.ERROR)
so what's this mean? every request will trigger add_handler or it's just triggered once when init?
every socket connect will generate a file descriptor , or it's just generated once?
what's the relationship between ioloop and iostream ?
how does httpserver work with ioloop and iostream ?
is there any workflow chart, so i can see it clearly ?
sorry for these questiones, i just confused
any link, suggestion, tip helps. many thanks :)

I'll see if I can answer your questions in order:
Here _impl is whichever socket polling mechanism is available, epoll on Linux, select on Windows. So self._impl.register(fd, events | self.ERROR) passes the "wait for some event" request to the underlying operating system, also specifically including error events.
When run, the HTTPServer will register sockets to accept connections on, using IOLoop.add_handler(). When connections are accepted, they will generate more communication sockets, which will likely also add event handlers via an IOStream, which may also call add_handler(). So new handlers will be added both at the beginning, and as connections are recieved.
Yes, every new socket connection will have a unique file descriptor. The original socket the HTTPServer is listening on should keep its file descriptor though. File descriptors are provided by the operating system.
The IOLoop handles events to do with the sockets, for example whether they have data available to be read, whether they may be written to, and whether an error has occured. By using operating system services such as epoll or select, it can do this very efficiently.
An IOStream handles streaming data over a single connection, and uses the IOLoop to do this asynchronously. For example an IOStream can read as much data as is available, then use IOLoop.add_handler() to wait until more data is available.
On listen(), the HTTPServer creates a socket which it uses to listen for connections using the IOLoop. When a connection is obtained, it uses socket.accept() to create a new socket which is then used for communicating with the client using a new HTTPConnection.
The HTTPConnection uses an IOStream to transfer data to or from the client. This IOStream uses the IOLoop to do this in an asynchronous and non-blocking way. Many IOStream and HTTPConnection objects can be active at once, all using the same IOLoop.
I hope this answers some of your questions. I don't know of a good structural chart, but the overall idea should be fairly similar for other webservers too, so there might be some good info around. That in-depth article you linked to did look pretty useful, so if you understand enough I'd recommend giving it another go :).