I'm currently reading physics in the university, and im learning python as a little hobby.
To practise both at the same time, i figured I'll write a little "physics engine" that calculates the movement of an object based on x,y and z coordinates. Im only gonna return the movement in text (at least for now!) but i want the position updates to be real-time.
To do that i need to update the position of an object, lets say a hundred times a second, and print it back to the screen. So every 10 ms the program prints the current position.
So if the execution of the calculations take 2 ms, then the loop must wait 8ms before it prints and recalculate for the next position.
Whats the best way of constructing a loop like that, and is 100 times a second a fair frequency or would you go slower, like 25 times/sec?
The basic way to wait in python is to import time and use time.sleep. Then the question is, how long to sleep? This depends on how you want to handle cases where your loop misses the desired timing. The following implementation tries to catch up to the target interval if it misses.
import time
import random
def doTimeConsumingStep(N):
"""
This represents the computational part of your simulation.
For the sake of illustration, I've set it up so that it takes a random
amount of time which is occasionally longer than the interval you want.
"""
r = random.random()
computationTime = N * (r + 0.2)
print("...computing for %f seconds..."%(computationTime,))
time.sleep(computationTime)
def timerTest(N=1):
repsCompleted = 0
beginningOfTime = time.clock()
start = time.clock()
goAgainAt = start + N
while 1:
print("Loop #%d at time %f"%(repsCompleted, time.clock() - beginningOfTime))
repsCompleted += 1
doTimeConsumingStep(N)
#If we missed our interval, iterate immediately and increment the target time
if time.clock() > goAgainAt:
print("Oops, missed an iteration")
goAgainAt += N
continue
#Otherwise, wait for next interval
timeToSleep = goAgainAt - time.clock()
goAgainAt += N
time.sleep(timeToSleep)
if __name__ == "__main__":
timerTest()
Note that you will miss your desired timing on a normal OS, so things like this are necessary. Note that even with asynchronous frameworks like tulip and twisted you can't guarantee timing on a normal operating system.
Since you cannot know in advance how long each iteration will take, you need some sort of event-driven loop. A possible solution would be using the twisted module, which is based on the reactor pattern.
from twisted.internet import task
from twisted.internet import reactor
delay = 0.1
def work():
print "called"
l = task.LoopingCall(work)
l.start(delay)
reactor.run()
However, as has been noted, don't expect a true real-time responsiveness.
A piece of warning. You may not expect a real time on a non-realtime system. The sleep family of calls guarantees at least a given delay, but may well delay you for more.
Therefore, once you returned from sleep, query current time, and make the calculations into the "future" (accounting for the calculation time).
Related
I'm an amateur coder. I'm working on a small little game for a project in biology, but I have come across an issue in my code. I have a loop that adds +1 to the variable sunlight every two seconds. However, all code below the loop is non-functional now that I have made the loop. I'm guessing it's because it's waiting for the loop to finish. Any way to have the loop always run but allow the code to run through it's sequence at the same time?
print("Game started!")
sunlight = 0
while True:
time.sleep(2)
sunlight += 1
commands = input("Type stats to see which molecules you have, type carbon to get carbon\ndioxide, and type water to get water: ")
if commands == ("stats"):
print("Sunlight: ",sunlight,"")
As you are beginner, i would not recommend to use multithreading or asyncio. Instead just start the time and when user enter "stats", elapsed time//2 will be equal to sunlight.
import time
start_time = time.time()
while True:
commands = input("Type stats to see which molecules you have, type carbon to get carbon\ndioxide, and type water to get water: ")
if commands == ("stats"):
sunlight = (time.time()-start_time)//2 # elapsed time // 2
print("Sunlight: ", sunlight, "")
Your sunlight variable basically functions as a clock; it counts half of the number of seconds since the program begins. Rather than implement your own clock using time.sleep(), it's better to just use an existing clock from the time library.
The function time.monotonic returns a number of seconds, so you can use this to get the current sunlight by saving the start time, then each time you want to know the value of sunlight, take the difference between the current time and the start time, divided by 2.
start_time = time.monotonic()
def get_sunlight():
current_time = time.monotonic()
return int(current_time - start_time) // 2
It is better to use the monotonic() function than the clock() function for this purpose, since the clock() function is deprecated as of Python 3.3:
The time.clock() function is deprecated because it is not portable: it behaves differently depending on the operating system.
It's also better than the time() function for this purpose, because changes to the system clock (such as going forwards or back due to daylight savings time) will affect the result of time():
While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.
You should look into the multithreading library. That's probably a good resource. You can fire off a thread running your sunlight incrementer that updates a global variable (not a good idea but you seem to have just 1 writer, so you can get by till you have time to pick up more advanced parallel processing concepts).
Reference: https://www.geeksforgeeks.org/multithreading-python-set-1/
I have this program that checks the amount of CPU being used by the current Python process.
import os
import psutil
p = psutil.Process(os.getpid())
counter = 0
while True:
if counter % 1000 == 0:
print(p.cpu_percent())
counter += 1
The output is:
98.79987719751766
98.79981257674615
98.79975442677997
98.80031017770553
98.80022615662917
98.80020675841527
98.80027781367056
98.80038116157328
98.80055555555509
98.80054906013777
98.8006523704943
98.80072337402265
98.80081374321833
98.80092993219198
98.80030995738038
98.79962549234794
98.79963842975158
98.79916715088079
98.79930277598402
98.7993480085206
98.79921895171654
98.799154456851
As seen by the output, this program is taking up 100% of my CPU and I'm having a tough time understanding why. I noticed that putting a time.sleep(0.25) causes CPU usage to go down to zero.
In my actual application, I have a similar while loop and can't afford to have a sleep in the while loop since it is reading a video from OpenCV and needs to stay realtime.
import cv2
cap = cv2.VideoCapture("video.mp4")
while True:
success, frame = cap.retrieve()
This program takes the same amount of CPU as the first program I wrote, but this one decodes video!
If someone could explain a bit more that'd be awesome.
Your original loop is doing something as fast as it can. Any program that is doing purely CPU bound work, with no significant blocking operations involved, will happily consume the whole CPU, it just does whatever it's doing faster if the CPU is faster.
Your particular something is mostly just incrementing a value and dividing it by 1000 over and over, which is inefficient, but making it more efficient, e.g. by changing the loop to:
import os
import psutil
p = psutil.Process(os.getpid())
while True:
for i in range(1000): pass
print(p.cpu_percent())
removing all the division work and having a more efficient addition (range does the work at the C layer), would just mean you do the 1000 loops faster and print the cpu_percent more often (it might slightly reduce the CPU usage, but only because you might be printing enough that the output buffer is filled faster than it can be drained for display, and your program might end up blocking on I/O occasionally).
Point is, if you tell the computer to do something forever as fast as it can, it will. So don't. Use a sleep; even a small one (time.sleep(0.001)) would make a huge difference.
For your video capture scenario, seems like that's what you want in the first place. If the video source is producing enough output to occupy your whole CPU, so be it; if it isn't, and your code blocks when it gets ahead, that's great, but you don't want to slow processing just for the sake of lower CPU usage if it means falling behind/dropping frames.
I hope you are doing great!
When you do:
while True:
if counter % 1000 == 0:
print(p.cpu_percent())
counter += 1
You actually ask your computer to process constantly.
It is going to increment counter as fast as possible and will display the cpu_percent every time counter is modulo of 1000.
It means that your program will feed the CPU constantly with the instructions of incrementing that counter.
When you use sleep, you basically say to the OS (operating system) that your code shouldn't execute anything new before sleep time. Your code will then not flood the CPU with instructions.
Sleep suspends execution for the given number of seconds.
Currently it is better to use a sleep than the counter.
import os
import psutil
import time
p = psutil.Process(os.getpid())
counter = 0
while True:
time.sleep(1)
print(p.cpu_percent())
I hope it is helping.
Have a lovely day,
G
You might want to add in a time.sleep(numberOfSeconds) to your loop if you don't want it to be using 100% CPU all the time, if it's only checking for a certain condition over and over.
I am trying to sample a signal at 10Khz in Python. There is no problem when try to run this code(at 1KHz):
import sched, time
i = 0
def f(): # sampling function
s.enter(0.001, 1, f, ())
global i
i += 1
if i == 1000:
i = 0
print "one second"
s = sched.scheduler(time.time, time.sleep)
s.enter(0.001, 1, f, ())
s.run()
When I try to make the time less, it starts to exceed one second(in my computer, 1.66s at 10e-6).
It it possible to run a sampling function at a specific frequency in Python?
You didn't account for the code's overhead. Each iteration, this error adds up and skews the "clock".
I'd suggest to use a loop with time.sleep() instead (see comments to https://stackoverflow.com/a/14813874/648265) and count the time to sleep from the next reference moment so the inevitable error doesn't add up:
period=0.001
t=time.time()
while True:
t+=period
<...>
time.sleep(max(0,t-time.time())) #max is needed in Windows due to
#sleep's behaviour with negative argument
Note that the OS scheduling will not allow you to reach precisions beyond a certain level since other processes have to preempt yours from time to time. In this case, you'll need to use some OS-specific facilities for multimedia applications or work out a solution that doesn't need this level of accuracy (e.g. sample the signal with a specialized app and work with its saved output).
How can I best create a high frequency, 100 times a second, loop in python on Linux? It does not need to be highly accurate, just good enough so on average does actually loop about 100 times a second.
I tried:
import time
count = 0
start = time.time()
while count <= 300:
time.sleep(0.01)
count = count + 1
end = time.time()
print('avg. {0}'.format((end - start) / count))
And it works pretty well! Getting
avg. 0.0103...
output.
But is there a better way to do such tight loops?
(This is for a server that needs to frequently read incoming network packets and process them, and perform periodic processing whether there are new packets or not).
Since you are on UNIX, you can also use the signal module.
def handler(signum, frame):
print 'do it'
# This will fire an ITIMER_REAL signal every 0.01 seconds
signal.setitimer(signal.ITIMER_REAL, 0, 0.01)
# Tell the signal module to execute handler when upon signal
# ITIMER_REAL
signal.signal(signal.ITIMER_REAL, handler)
For more info see: https://docs.python.org/2/library/signal.html. I have used signals in my own code. I found it highly robust and accurate but somewhat limited in scope (you can only set a limited number of signals). A big advantage is that you do not need threads for concurrency. Be aware, that when you combine multi-threaded code with signals, you must take good care that you set the signal handler (signal.signal) in the main thread.
Here's a simple ad-hock solution without drift:
import time
INTERVAL = 0.01
last = time.time()
while True:
next = last + INTERVAL
time.sleep(next - time.time()) # it's ok to sleep negative time
last = next
do_your_thing_here()
This is for Pygame only but at the end of the loop you could do...
mainClock = pygame.time.Clock()
mainClock.tick(100)
100 or 1000 or whatever really.
I want to be able to easily set a benchmarking program's file write rate. It's a python program that I'm using to test another system. I'd like to be able to control the rate of file creation. One method I have thought of is to have a function with an argument for the number of files to create. This could be called in a loop which keeps track of the clock and only calls the function every second. This would fulfill the requirement of creating a certain number of files every second. The problem with this is that there could be a chunk of dead time (milliseconds, but still). I'd like a continuous load.
You'll have to somehow keep track of the time it takes to actually perform the file I/O calls, and adjust the sleep times between the operations. Adjustment needs to be continuous, as the sleeps and IO calls might take different amount of time depending on system load.
If you'd like to do N operations per second on average, you could run loops of few seconds (or longer), and after every round see if you're running too fast or slow, and adjust the sleep() time done between each operation upwards or downwards based on that. If you're running much too fast, increment the sleep time more, if you're only a little bit too fast, increment less.
import time
# target rate: 100 ops / 1 second
target = 100.0
round_time = 1.0
# at first, assume the writes are immediate
sleepTime = round_time/target
ops = 0
t_start = time.time()
while True:
#doYourIOoperationHere()
ops += 1
time.sleep(sleepTime)
# adjust sleep time periodically
if ops == target:
t_end = time.time()
elapsed = t_end - t_start
difference = round_time - elapsed
# print out the vars here to debug adjustment
print "%d ops done, elapsed %.3f, difference %.3f" % (ops, elapsed, difference)
# increase or decrease the sleep time, approach the target time slowly
sleepTime += difference/target/2
t_start = time.time()
ops = 0
Or something along those lines (simplistic code untested). This might not work well for very high IO rates or system loads, you might have to start doing multiple write operations per single sleep call. Also, a longer averaging than 1 second is likely to be necessary.