I have the following program here
`
import datetime
import asyncio
import time
import math
async def count1():
s = 0
print('start time count 1: ' +str(datetime.datetime.now()))
for i in range(100000000):
s += math.cos(i)
print('end time count 1: ' +str(datetime.datetime.now()))
return s
async def count2():
s = 0
print('start time count 2: ' +str(datetime.datetime.now()))
for i in range(1000000):
s += math.cos(i)
print('end time count 2: ' +str(datetime.datetime.now()))
return s
async def main():
start_time = time.time()
task = asyncio.gather(count1(), count2())
results = await task
end_time = time.time()
print(f"Result 1: {results[0]}")
print(f"Result 2: {results[1]}")
print(f"Total time taken: {end_time - start_time:.2f} seconds")
asyncio.run(main())
The output is
start time count 1: 2023-02-16 12:26:19.322523
end time count 1: 2023-02-16 12:26:40.866866
start time count 2: 2023-02-16 12:26:40.868166
end time count 2: 2023-02-16 12:26:41.055005
Result 1: 1.534369444774577
Result 2: -0.28870546796843
Total time taken: 21.73 seconds
I am trying to get count1() and count2() to start working at the same time, as it seen it the output it's not happening. count2() only starts after count1() has ended, and I am not sure why.
I also tried replacing the lines in main() with:
task1 = asyncio.create_task(count1())
task2 = asyncio.create_task(count2())
result1 = await task1
result2 = await task2
not also does not result in count1() and count2() starting at the same time.
async is essentially cooperative multitasking. Neither function awaits, so they hog the entire process for as long as they run and don't yield to other functions.
You could add a strategic
if i % 10000 == 0:
await asyncio.sleep(0)
in the loop so they "give way" to other async coroutines.
Async doesn’t mean “running at the same time”. Async means that when one of the functions waits (typically for some I/O), the control can be passed to another function.
In practice async can be used e.g. to have one process wait for several responses from web at the same time. It won’t allow you to e.g. run the same computation on several CPU cores (this stuff is generally hard to achieve in Python due to the infamous GIL problem).
Related
I've got a program that is pulling UPD data being transmitted by my home weather station. One of those values that is a "rain_accum" which is the total rain accumulation in the past minute. When that "rain_accum" is > 0, I want the light to blink. When the rain_accum == 0, I want it to not blink.
I'm fairly new to Python and very new to uasyncio (note: I'm using Micropython, so I may not have all the capablities of asyncio) and I'm just getting lost. I've been messing with this for 3 days and have actually gotten pretty close to getting it working, but the timing was way off the blink would take several seconds and UDP messages were getting missed. I've cut my code back to the bare basics of what I've been trying to do hoping I could get some direction as to where to go with this.
import uasyncio
async def blink():
while True:
print("Here 1")
await uasyncio.sleep(1)
print("Here 2")
await uasyncio.sleep(1)
async def getData():
while True:
print("Getting data")
if True:
await blink()
print("Next loop")
await uasyncio.sleep(5)
try:
uasyncio.run(getData())
except KeyboardInterrupt:
print("Quitting")
In the example in your question, once you call await blink() you never return: your current coroutine has entered an infinite loop.
Take a look at the example from the documentation, which shows how to start two coroutines concurrently. If we were to take that examples and fit it to the example from your question, we might get something like:
import random
import time
import uasyncio
blink_active = False
async def blink():
while True:
if blink_active:
print("Here 1")
await uasyncio.sleep(1)
print("Here 2")
await uasyncio.sleep(1)
else:
await uasyncio.sleep(1)
async def getData():
global blink_active
# Start a concurrent async task
uasyncio.create_task(blink())
last_check = 0
while True:
now = time.time()
if now - last_check > 5:
data = random.randint(0, 10)
print("Got data", data)
if data > 7:
print("Start blinking")
blink_active = True
else:
print("Stop blinking")
blink_active = False
last_check = now
print("Next loop")
await uasyncio.sleep(1)
try:
uasyncio.run(getData())
except KeyboardInterrupt:
print("Quitting")
Here, getData() starts a blink task that will run forever. Whether it "blinks" (or prints out text) is controlled by a global variable, blink_active.
In getData, our loop runs once/second, but we only gather new data every five seconds. Depending on the value we retrieve from our "sensor", we start or stop the blinker by setting blink_active appropriately.
I have a function which constantly yields some objects, say 1 per second and a handler which works 2 seconds and handles this objects. For example:
from time import sleep
import asyncio
from datetime import datetime
def generator():
i = 0
while True:
yield i
i += 1
sleep(1)
def handler(number):
sleep(2)
if number % 2 == 0:
print(str(number) + ' is even')
else:
print(str(number) + ' is odd')
for number in generator():
handler(number)
So, for example '2 is even' is printed 6 seconds after the program starts. How do I reduce this time to 4 seconds ( 2 seconds for generator + 2 seconds for handler) using asyncio? I need to set up asynchronous handling of the numbers.
You need couple of changes here:
your generator currently is a "generator", change it to "asynchronous generator" so that you can use async for. This way it can give the control back to eventloop.
Use async version of sleep in asyncio library: asyncio.sleep. time.sleep doesn't cooperate with other tasks.
Change your handler sync function to a "coroutine".
import asyncio
async def generator():
i = 0
while True:
yield i
i += 1
await asyncio.sleep(1)
async def handler(number):
await asyncio.sleep(2)
if number % 2 == 0:
print(str(number) + " is even")
else:
print(str(number) + " is odd")
async def main():
async for number in generator():
asyncio.create_task(handler(number))
asyncio.run(main())
Now, your first task is main, asyncio.run() automatically creates it as Task. Then when this task is running, it iterates asynchronously through the generator(). The values are received then for each number, you create a new Task out of handler coroutine.
This way the sleep times are overlapped. When it waits for new number for 1 second, it also actually waits for handler() one second. Then when the number is received, one second of handler() task is already passed, it only needs 1 second.
You can see the number of tasks if you want:
async def main():
async for number in generator():
print(f"Number of all tasks: {len(asyncio.all_tasks())}")
asyncio.create_task(handler(number))
Because each handler sleeps 2 seconds, and your number generator sleeps 1 seconds, You see that in every iteration 2 Tasks are exist in event loop. Change await asyncio.sleep(1) to await asyncio.sleep(0.5) in generator coroutine, you will see that 4 tasks are in event loop in every iteration.
Answer to the comment:
Can I do the same thing if I use API which doesn't let me create
asynchronous generator, but just a normal one? Can I still
asynchronously handle yielded objects?
Yes you can. Just note that if you don't have asynchronous generator, you can't use async for, which means your iteration is synchronous. But, you have to do a little trick for it to work. When your main() task is being executed, it constantly get a value from generator generator and creates a Task for it, but it doesn't give a chance to other tasks to run. You need await asyncio.sleep(0):
import asyncio
import time
def generator():
i = 0
while True:
yield i
i += 1
time.sleep(1)
async def handler(number):
await asyncio.sleep(2)
if number % 2 == 0:
print(str(number) + " is even")
else:
print(str(number) + " is odd")
async def main():
for number in generator():
print(f"Number of all tasks: {len(asyncio.all_tasks())}")
asyncio.create_task(handler(number))
await asyncio.sleep(0)
asyncio.run(main())
I have a script that will scrape Facebook post in a while loop. But to prevent getting banned by Facebook I prefer the scraping function run(self) only run at a certain time and stop after 1 hour. How can I achieve that? I saw someone post about starting a function at a certain time but not stopping it after a specific hour.
Code to start a function at a certain time:
import schedule
import time
def job(t):
print ("I'm working...", t)
return
schedule.every().day.at("18:44").do(job,'It is 01:00')
while True:
schedule.run_pending()
time.sleep(60) # wait one minute
My script code
def run(self):
while True:
try:
wanted = "Pecahan setiap negeri (Kumulatif):" # wanted post
for post in get_posts("myhealthkkm", pages=5):
if post.get("post_text") is not None and wanted in post.get("post_text"):
# print("Found", t)
listposts.append(post.get("post_text"))
# append until 3 page finish then go here
time.sleep(1)
print(listposts)
global listView
if listposts != 0:
listView = listposts.copy()
print(listView)
listposts.clear()
except exceptions.TemporarilyBanned:
print("Temporarily banned, sleeping for 10m")
time.sleep(600)
I want the run() function only execute at 3pm and stop after 1 hour which is 4pm everyday. How can I achieve this goal ? Thanks
Thanks to the respond from #larsks, I tried to put a time check inside the function and exit the loop if it reached 3600second which is equal to 1 hour.
Here is the sample code:
import schedule
import time
def job():
program_starts = time.time()
while True:
now = time.time()
timeframe = now - program_starts
print("It has been {0} seconds since the loop started".format(timeframe))
if timeframe > 3600:
break
schedule.every().day.at("19:20:30").do(job)
while True:
schedule.run_pending()
#time.sleep(60) # wait one minute
I am learning asyncio library to do some tasks I want to achieve. I wrote the following code to teach myself on how to be able to switch to so another task while the original one is being executed. As you can see below, the summation() should be performed until a condition is met where it should jump to the secondaryTask(). After secondaryTask() is finished, it should return back to summation() where hopefully it gets finished. The potential results should be sum=1225 and mul=24.
import asyncio, time
async def summation():
print('Running summation from 0 to 50:')
sum = 0
for i in range(25):
sum = sum + i
if i != 25:
time.sleep(0.1)
else:
await asyncio.sleep(0) # pretend to be non-blocking work (Jump to the next task)
print('This message is shown because summation() is completed! sum= %d' % sum)
async def secondaryTask():
print('Do some secondaryTask here while summation() is on progress')
mul = 1
for i in range(1, 5):
mul = mul * i
time.sleep(0.1)
await asyncio.sleep(0)
print('This message is shown because secondaryTask() is completed! Mul= %d' % mul)
t0 = time.time()
ioloop = asyncio.get_event_loop()
tasks = [ioloop.create_task(summation()), ioloop.create_task(secondaryTask())]
wait_tasks = asyncio.wait(tasks)
ioloop.run_until_complete(wait_tasks)
ioloop.close()
t1 = time.time()
print('Total time= %.3f' % (t1-t0))
This code does not perform as expected because sum=300 as oppose to be sum=1225. Clearly that summation() does not continue while secondaryTask() is being processed. How can I modify summation() to be able to do the summation of the remaining 25 values on the background?
Thank you
It's just your careless. You want to run summation from 0 to 50, your summation function should be for i in range(50)
The following code snippet has two coroutines each for server and client. The client coroutine has a logic to break the while loop after 10 seconds and server should stop after 15 seconds.
When I run the script this doesn't stop, ideally, it should stop after 15 seconds but this is not happening.
import asyncio
import time
import zmq
import zmq.asyncio
zmq.asyncio.install()
ctx = zmq.asyncio.Context()
server_socket = ctx.socket(zmq.REP)
client_socket = ctx.socket(zmq.REQ)
server_socket.bind("tcp://127.0.0.1:8899")
client_socket.connect("tcp://127.0.0.1:8899")
t0 = time.time()
#asyncio.coroutine
def server_coroutine():
while True:
msg = yield from server_socket.recv_string()
print(msg)
msg = "Server:: {}".format(msg)
yield from server_socket.send_string(msg)
t1 = time.time()
elapsed_time = t1 - t0
# print('elapsed time is {}'.format(elapsed_time))
if elapsed_time > 15:
print("Breaking Server loop")
break
#asyncio.coroutine
def client_coroutine():
counter = 0
while True:
yield from asyncio.sleep(2)
msg = 'Message: {}'.format(counter)
yield from client_socket.send_string(msg)
res = yield from client_socket.recv_string()
print(res)
t1 = time.time()
elapsed_time = t1 - t0
print('elapsed time is {}'.format(elapsed_time))
if elapsed_time > 10:
print("Breaking Client loop")
break
counter += 1
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(
asyncio.ensure_future(server_coroutine()),
asyncio.ensure_future(client_coroutine())
))
If you run code you will see something like this:
Server:: Message: 4
elapsed time is 10.022311687469482
Breaking Client loop
ok, client_coroutine finished successfully, but what state of server_coroutine at this moment? It stuck at this line msg = yield from server_socket.recv_string() waiting for possibility to recive string from server_socket, but it won't happen since there's already no client to send it! And since your event loop runs until both coroutines done it would run forever.
Here's the simplest fix:
#asyncio.coroutine
def server_coroutine():
while True:
msg = yield from server_socket.recv_string()
if msg == 'CLOSE': # LOOK HERE 1
break
print(msg)
msg = "Server:: {}".format(msg)
yield from server_socket.send_string(msg)
t1 = time.time()
elapsed_time = t1 - t0
# print('elapsed time is {}'.format(elapsed_time))
if elapsed_time > 15:
print("Breaking Server loop")
break
#asyncio.coroutine
def client_coroutine():
counter = 0
while True:
yield from asyncio.sleep(2)
msg = 'Message: {}'.format(counter)
yield from client_socket.send_string(msg)
res = yield from client_socket.recv_string()
print(res)
t1 = time.time()
elapsed_time = t1 - t0
print('elapsed time is {}'.format(elapsed_time))
if elapsed_time > 10:
print("Breaking Client loop")
yield from client_socket.send_string('CLOSE') # LOOK HERE 2
break
counter += 1
Note, this fix is only to demonstrate issue and one possible way to solve it.
In real life I think you would want to do something different: probably, set timeouts to you coroutines to guarantee they won't stuck forever if client/server stops responding.