We Keep Coding

Using ThreadPoolExecutor to run ongoing task as well as another separate service

Is it possible to use ThreadPoolExecutor to run tasks in the background?
I want to map an ongoing listener to a task and then run another service after this.
I can achieve that with threading.Thread, but not ThreadPoolExecutor
i.e. this works:
# set a custom thread name prefix for all threads in the pool
import threading
from concurrent.futures import ThreadPoolExecutor
import time
# a mock task that does nothing
def task(name):
i = 0
while 1:
print(i)
i = i + 1
time.sleep(5)
threading.Thread(target=lambda: task("test")).start()
print("to here")
and gives
0to here
1
2
Where as if I use this:
with ThreadPoolExecutor(thread_name_prefix='TaskPool') as executor:
executor.map(task("test"))
print("to here")
We never reach "to here"
How can I configure ThreadPoolExecutor appropriately?

How to schedule python script to exit at given time

I need to schedule a python script which can exit and kill it self at a given time. For scheduling, I am using python schedule and below is the code:
import schedule
from threading import Thread
import time
import sys
def exit_data():
print("Exiting")
sys.exit()
def exit_data_thread():
schedule.every().day.at('13:20').do(exit_data)
while True:
schedule.run_pending()
time.sleep(1)
def main():
Thread(target=exit_data_thread).start()
while True:
time.sleep(1)
main()
Function exit_data() runs at given time and it prints Exiting but do not exit. It only prints Exiting and then it keeps running. I have also used quit instead of sys.exit(). Please help. Thanks

Try to send signal to yourself :p
import schedule
from threading import Thread
import time
import sys
import os
import signal
def exit_data():
print("Exiting")
# sys.exit()
os.kill(os.getpid(), signal.SIGTERM)
def exit_data_thread():
schedule.every(3).seconds.do(exit_data)
while True:
schedule.run_pending()
time.sleep(1)
def main():
Thread(target=exit_data_thread).start()
while True:
time.sleep(1)
main()

To close the entire program within a thread, you can use os._exit(). Calling sys.exit() will only exit the thread, not the entire program.

Python Event loop w/ gevent

import gevent
from gevent.event import AsyncResult
import time
class Job(object):
def __init__(self, name):
self.name = name
def setter(job):
print 'starting'
gevent.sleep(3)
job.result.set('%s done' % job.name)
def waiter(job):
print job.result.get()
# event loop
running = []
for i in range(5):
print 'creating'
j = Job(i)
j.result = AsyncResult()
running.append(gevent.spawn(setter, j))
running.append(gevent.spawn(waiter, j))
print 'started greenlets, event loop go do something else'
time.sleep(5)
gevent.joinall(running)
gevent doesnt actually start until joinall is called
Is there something that would start/spawn gevent asynchronously (why does it not start right away as soon as spawn is called)?
Is there a select/epoll on running greenlets to see which one needs to be joined instead of joinall()?

No, it does not start straight away. It will start as soon as your main greenlet yields to the hub (releases control by calling sleep or join for example)
Clearly your intention is that it starts when you call time. It does not, because you have not monkey patched it.
Add these lines to the very top of your file:
from gevent import monkey
monkey.patch_all()
This will then have the behaviour that you want (because under the hood, time will be modified to yield to the hub).
Alternatively, you can call gevent.sleep.

Since you did not monkey patch, time.sleep() is causing your app to pause. Use gevent.sleep(5) instead.

The very first step should be monkey patching
from gevent import monkey;
monkey.patch_all()
This will spawn the greenlets asynchronously.

How to properly initialize a python queue?

I'm new to python and having trouble with the python queue, I'm initializing queue in my init constructor by when I run my python app it crashes, I've included a snippet of my code is there a better way to do it?
import os, sys
import time
if(sys.hexversion < 0x03000000):
import Queue
else:
import queue as Queue
class Appwindow():
def __init__(self):
self.myQueue = Queue.Queue()
def displayMeth(self, stuff):
if self.displayed:
self.myQueue.put(stuff)

try:
from queue import Queue
except ImportError:
from Queue import Queue
# shiny fancy Atomic Message Queue for concurrency
q = Queue()

Strange problems when using requests and multiprocessing

Please check this python code:
#!/usr/bin/env python
import requests
import multiprocessing
from time import sleep, time
from requests import async
def do_req():
r = requests.get("http://w3c.org/")
def do_sth():
while True:
sleep(10)
if __name__ == '__main__':
do_req()
multiprocessing.Process( target=do_sth, args=() ).start()
When I press Ctrl-C (wait 2sec after run - let Process run), it doesn't stop. When I change the import order to:
from requests import async
from time import sleep, time
it stops after Ctrl-C. Why it doesn't stop/kill in first example?
It's a bug or a feature?
Notes:
Yes I know, that I didn't use async in this code, this is just stripped down code. In real code I use it. I did it to simplify my question.
After pressing Ctrl-C there is a new (child) process running. Why?
multiprocessing.__version__ == 0.70a1, requests.__version__ == 0.11.2, gevent.__version__ == 0.13.7

Requests async module uses gevent. If you look at the source code of gevent you will see that it monkey patches many of Python's standard library functions, including sleep:
request.async module during import executes:
from gevent import monkey as curious_george
# Monkey-patch.
curious_george.patch_all(thread=False, select=False)
Looking at the monkey.py module of gevent you can see:
https://bitbucket.org/denis/gevent/src/f838056c793d/gevent/monkey.py#cl-128
def patch_time():
"""Replace :func:`time.sleep` with :func:`gevent.sleep`."""
from gevent.hub import sleep
import time
patch_item(time, 'sleep', sleep)
Take a look at the code from the gevent's repository for details.