I am measuring the response time on a function using the time module. The time module is supposed to output seconds as a float, so I am saving a start time value (time.clock()) and taking another reading at the end, and using the difference as a runtime. While watching the results, we noted the runtimes seemed high -- something that seemed to take less than 2 seconds, was printing as 3-and-change, for instance. Based on the perceived issue, I decided to double-check the results using the datetime module. Printing the two side-by-side shows the time module values are almost double the datetime values.
Anyone know why that might be?
Here is my code:
for datum in data:
start = datetime.datetime.now()
startts = time.clock()
check = test_func(datum)
runtime = datetime.datetime.now() - start
runts = time.clock() - startts
print(check, "Time required:", runtime, "or", runts)
Some of my results:
XYZ Time required: 0:00:01.985303 or 3.7836029999999994
XYZ Time required: 0:00:01.476289 or 3.3465039999999817
XYZ Time required: 0:00:01.454407 or 3.7140109999999993
XYZ Time required: 0:00:01.550416 or 3.860824000000008
I am assuming this sort of issue would have been noticed before, and I am just missing something basic in my implementation. Can someone clue me in?
Looks like time.clock() has been Deprecated since version 3.3
Maybe this will help ?
time.clock()
On Unix, return the current processor time as a floating point number
expressed in seconds. The precision, and in fact the very definition
of the meaning of “processor time”, depends on that of the C function
of the same name.
On Windows, this function returns wall-clock seconds elapsed since the
first call to this function, as a floating point number, based on the
Win32 function QueryPerformanceCounter(). The resolution is typically
better than one microsecond.
Deprecated since version 3.3: The behaviour of this function depends
on the platform: use perf_counter() or process_time() instead,
depending on your requirements, to have a well defined behaviour.
We found the issue. The test_func I am testing is using a multi-threaded process. I both did not know that, and did not know it was an issue.
The time module uses processor time (https://docs.python.org/3.6/library/time.html), while the datetime module uses wall clock time (https://docs.python.org/3.6/library/datetime.html). Using the difference in the datetime timestamps told me how much actual time had elapsed, and for our purposes was the relevant information.
I hope this helps someone else in the future!
Related
I am developing a Tkinter python game - long story short it needs to be able to run at different FPS values. However, I am having trouble maintaining a consistent second length.
I have tried to make it detect lag and take it away from the .after() function:
def UpdatesEveryFrame():
s = time.perf_counter()
# Code here
s = int((time.perf_counter() - s)*1000)
LabelProductionTime3.after(int(1000 / fps) - s, UpdatesEveryFrame)
However, this is unsuccessful. It seems to create an accurate value in milliseconds (usually around 15) but this does not make an accurate second delay. I have tried replacing perf_counter() with time() but this has the same effect.
Because of what the game is based upon, it is essential that there is an accurate second delay. Can you help? Thanks.
If the goal here is precision, then perhaps you should try the time.perf_counter_ns method of the time module, it specifically is made to be more precise than time.perf_counter, and gives time in nanoseconds, further if the time has to be converted back into seconds, it can be done using unit conversion.
Further, the documentation of time.perf_counter method mentions this as well -:
Use perf_counter_ns() to avoid the precision loss caused by the float
type.
def UpdatesEveryFrame():
s = time.perf_counter_ns()/(10 ** 9) # used perf_counter_ns, divided by (10 ** 9) to convert to seconds.
# Code here
s = int((time.perf_counter_ns()/(10 ** 9) - s)*1000) # used perf_counter_ns, divided by (10 ** 9) to convert to seconds.
LabelProductionTime3.after(int(1000 / fps) - s, UpdatesEveryFrame)
EDIT:
There also exists time.monotonic method, designed specifically to measure the time elapsed between two consecutive calls, it returns time in fractional seconds similar to time.perf_counter, so no changes have to be made in the current code except the name of the function itself.
def UpdatesEveryFrame():
s = time.monotonic() # Changed method name.
# Code here
s = int((time.monotonic() - s)*1000)
LabelProductionTime3.after(int(1000 / fps) - s, UpdatesEveryFrame) # Changed method name.
Further, similar to the method time.perf_counter_ns available as a more precise version of time.perf_counter, there also exists a more precise version of the time.monotonic method that returns time in nanoseconds and functions similar to time.monotonic, namely time.monotonic_ns.
I have a very simple code that is supposed to print out '2' every time the microsecond of the current time is 2, or once per second. However, when I run this code, it returns nothing. How can I fix this?
import datetime
while True:
if datetime.datetime.now().microsecond == 2:
print (datetime.datetime.now().microsecond)
You can't reliably build a loop that will only print exactly every second at the point where the number of microseconds in the current time is equal to 2.
Computer clocks don't necessarily tick every single microsecond, nor is Python always fast enough to produce a datetime instance for a given microsecond. On many systems, calling datetime.now() will produce the exact same value for several microseconds at a stretch, all the way up to only producing a new, distinct time once a second. Your code also creates multiple datetime objects, so after you test for the microsecond attribute, the next one created when you call print() may easily have a higher microsecond value.
See the time module which documents available clocks in a little more detail. datetime uses the same clock as time.time:
The precision of the various real-time functions may be less than suggested by the units in which their value or argument is expressed. E.g. on most Unix systems, the clock “ticks” only 50 or 100 times a second.
and
Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.
It depends both on your OS and your computer hardware what the precision of your system clock is; it is not the resolution of the clock that matters (how much information can be given in the clock value) but rather how often that value is refreshed.
If you want to limit printing to once a second, just track the last time you printed and make sure the new time is at least 1 second newer:
import datetime
last = datetime.datetime.now()
while True:
now = datetime.datetime.now()
if (now - last).total_seconds() >= 1:
print(now)
I'm trying to create a simple game using pygame and everything was ok so far. The last couple of days though I realized that a problem occurred with the time.clock(). I read the documentation and the function should count the time of the game since it starts. I wanted to spawn an alien group every 8 seconds and it worked(I'm working on debian os) but as I mentioned the last 2 days it doesn't count properly. The system needs about 20 seconds in real time in order for time.clock to print 8.0 and the aliens to spawn and at first I thought that I messed up with the counters but how can this be, It worked fine in the beginning so I tried to run the same code on the windows partition and it was also fine. So is this a problem with the system clock or anything else? I replaced the time.clock on debian with time.time and also worked fine. Did anyone in the past run into the same problem? Can you help me check if something else is the problem(both operating systems run python 3.6)? If you don't understand something or need anything more just ask me.
Thank you for your time
here is a portion of the time.clock use in the game:
sergeant_spawn_time_limit = 8.0
sergeant_spawn_time = 0.0
if game_stage == 2 or game_stage == 3 or score >= 400:
if time.clock() - sergeant_spawn_time > sergeant_spawn_time_limit:
for spawn_sergeant in range(5):
sergeant = AlienSergeant(display_width, display_height, 50, 88)
all_sprites_list.add(sergeant)
alien_sergeant_list.add(sergeant)
sergeant_spawn_time = time.clock()
The behaviour of time.clock() is platform dependend:
time.clock()
On Unix, return the current processor time as a floating point number expressed in seconds. The precision, and in fact
the very definition of the meaning of “processor time”, depends on
that of the C function of the same name.
On Windows, this function returns wall-clock seconds elapsed since the
first call to this function, as a floating point number, based on the
Win32 function QueryPerformanceCounter(). The resolution is typically
better than one microsecond.
Deprecated since version 3.3: The behaviour of this function depends
on the platform: use perf_counter() or process_time() instead,
depending on your requirements, to have a well defined behaviour.
So it's really the wrong tool to use here. Either use time.time() or pygame's clock or it's build-in event system. You'll find a lot of examples, like here or here.
When I run PyCharm profiler (a quick intro video is here -- https://www.youtube.com/watch?v=QSueV8MYtlw ) I get thousands of lines like hasattr or npyio.py (where did it come from, I do not even use numpy) which do not help me to understand what's going on at all.
How can I make make PyCharm profiler to show only timings of my source code, not any libraries or system calls?
In other words, can the time spent in system calls and libraries be assigned to my functions which call them?
In other words (version two), all I want is number of milliseconds next to each line of my python code, not anything else.
I created a code to provide an example and hopefully provide an acceptable answer:
import datetime as dt
class something:
def something_else(self):
other_list = range(100000)
for num in other_list:
datetimeobj = dt.datetime.fromtimestamp(num)
print(num)
print(datetimeobj)
def something_different(self):
other_list = range(100000)
for num in other_list:
datetimeobj = dt.datetime.fromtimestamp(num)
print(num)
print(datetimeobj)
st = something()
st.something_else()
st.something_different()
The code resulted in the below picture, which I have sorted based on name. (This is in my case possible because all of the builtin methods are prefixed by "<". After doing this I can now see that main took 100 % of the total time (Column: Time (ms)). something_else took 50.8 % of the time and something_different took 49.2 % of the time (Totaling to 100 % as well) (Column: Time (ms)) The time spent inside each of the two home-grown methods was 2.0 % for each (Column: Own Time (ms)) -> This means that underlying calls from something_else accounted for 48.8 % and for something_different 47.2 % and the parts that I wrote accounted for 4.0 % of the total time. The remaining 96.0 % of the code happens from the built-in methods, that I call.
Your questions were:
How can I make make PyCharm profiler to show only timings of my source code, not any libraries or system calls? -> That's what you see in the column: "Own Time (ms)" -> 2.0 % (Time spent inside the specific method.)
In other words, can the time spent in system calls and libraries be assigned to my functions which call them? -> That's what you see in the column: "Time (ms)" (Time spent including underlying methods.)
Subtract the two columns and you get time spent only in underlying methods.
I have unfortunately been unable to find a method for filtering in the profiler, but it is possible to export the list by copying it and this way you could create something else to do the filtering on e.g. "<built_in" to clean up the data.
I'm posing this question mostly out of curiosity. I've written some code that is doing some very time intensive work. So, before executing my workhorse function, I wrapped it up in a couple of calls to time.clock(). It looks something like this:
t1 = time.clock()
print this_function_takes_forever(how_long_parameter = 20)
t2 = time.clock()
print t2 - t1
This worked fine. My function returned correctly and t2 - t1 gave me a result of 972.29, or about 16 minutes.
However, when I changed my code to this
t1 = time.clock()
print this_function_takes_forever(how_long_parameter = 80)
t2 = time.clock()
print t2 - t1
My function still returned fine, but the result of t2 - t1 was:
None
-1741
I'm curious as to what implementation detail causes this. Both the None, and the negative number are perplexing to me. Does it have something to do with a signed type? How does this explain the None?
The Python docs say:
On Unix, return the current processor time as a floating point number expressed in seconds. The precision, and in fact the very definition of the meaning of “processor time”, depends on that of the C function of the same name
The manpage of the referenced C function then explains the issue:
Note that the time can wrap around. On a 32-bit system where CLOCKS_PER_SEC equals 1000000 this function will
return the same value approximately every 72 minutes.
A quick guess... Looks like an overflow. The default data type is probably a signed data type (putting the first bit to 1 on a signed integer gives a negative number).
Try putting the result of the substraction in a variable (double), and then printing that.
If it still prints like that, you can try converting it from double to string, and then using 'print' function on the string.
The None has a very simple answer, your function does not return a value. Actually I gather that is does under normal circumstances, but not when how_long_parameter = 80. Because your function seems to be returning early (probably because execution reaches the end of the function where there is an implicit return None in Python) the negative time might be because your function takes almost no time to complete in this case? So look for the bug in your function and correct it.
The actual answer as to why you get a negative time depends on the operating system you are using, because clock() is implemented differently on different platforms. On Windows it uses QueryPerformanceCounter(), on *nix it uses the C function clock().