I got a question regarding calculating time code delta.
I read metadata from a movie file containing a timecode formated HH:MM:SS:FF
(FF = frame, 00->23 for example. So its like 00 to framerate-1)
So i get some data like 15:41:08:02 and from another refrence file i get 15:41:07:00
Now I have to calculate the timeoffset (like timedelta but just with frames).
How would i go around doing this?
framerate = 24
def timecode_to_frames(timecode):
return sum(f * int(t) for f,t in zip((3600*framerate, 60*framerate, framerate, 1), timecode.split(':')))
print timecode_to_frames('15:41:08:02') - timecode_to_frames('15:41:07:00')
# returns 26
def frames_to_timecode(frames):
return '{0:02d}:{1:02d}:{2:02d}:{3:02d}'.format(frames / (3600*framerate),
frames / (60*framerate) % 60,
frames / framerate % 60,
frames % framerate)
print frames_to_timecode(26)
# returns "00:00:01:02"
I'd just use gobal frame numbers for all computations, converting back to timecodes only for display
def tc_to_frame(hh, mm, ss, ff):
return ff + (ss + mm*60 + hh*3600) * frame_rate
def frame_to_tc(fn):
ff = fn % frame_rate
s = fn // frame_rate
return (s // 3600, s // 60 % 60, s % 60, ff)
for negative frame numbers I'd prepend a minus to the representation of the absolute value
If the timecode is SMPTE timecode, you may need to take into account drop frames. Drop-frame timecodes drop frame numbers 0 and 1 of the first second of every minute, except when the number of minutes is divisible by 10.
This page provides some history background with formulas to convert between timecodes and frame numbers.
Using the timecode module this is quite easy:
import timecode as tc
tc1 = tc.Timecode(24, "15:41:08:02")
tc2 = tc.Timecode(24, "15:41:07:00")
delta = tc1 - tc2
print(delta.frames)
gives you 26
Related
I have a CSV file having online Teams meeting data. It has two columns, one is "names" and the other with their duration of attendance during the meeting. I want to convert this information into a graph to quickly identify who remained for a long time in the meeting and who for less time. The duration column data is in this format, 3h 54m. This means it has the characters h and m in the column. See the picture below too.
Now how can I convert this data into decimal values like 3.54 or 234?
3.54 will mean hours and 234 will mean minutes in total. I am happy with any solution like either hour i.e. 3.54 or minutes 234.
Numpy or Pandas are both welcome.
The conversion of the timestring can be achieved with the datetime module:
from datetime import datetime
time_string = '3h 54m'
datetime_obj = datetime.strptime(time_string, '%Hh %Mm')
total_mins = (datetime_obj.hour * 60) + datetime_obj.minute
time_in_hours = total_mins / 60
# Outputs: `3.9 234`
print(time_in_hours, total_mins)
Here '%H' means the number of hours and '%M' is the number of minutes (both zero-padded). If this is encapsulated in a function, it can applied on the data from your spreadsheet that has been read-in via numpy or pandas.
REFERENCE:
https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime
3h 54m is not 3.54. It's 3 + 54/60 = 3.9. Also since you have some items with seconds, it may be best to do all the conversions to seconds so you don't lose significant digits due to rounding if you need to add any of the items. For example, if you have 37 minutes, that's 0.61666667 hours. So using seconds, you get more precise results if you have to combine things.
The function below can handle h m and s and any combo of them. I provide examples at the bottom. I'm sure there are hundreds of ways to do this conversion. There are tons of examples on StackOverflow of reading and using CSV files so I didn't provide info. I like playing with the math stuff. :)
def hms_to_seconds(time_string):
timeparts = []
# split string. Assumes blank space(s) between each time componet
timeparts = time_string.split()
h = 0
m = 0
s = 0
# loop through the componets and get values
for part in timeparts:
if (part[-1] == "h"):
h = int( part.partition("h")[0] )
if (part[-1] == "m"):
m = int( part.partition("m")[0] )
if (part[-1] == "s"):
s = int( part.partition("s")[0] )
return (h*3600 + m*60 + s)
print(hms_to_seconds("3h 45m")) # 13500 sec
print(hms_to_seconds("1m 1s")) # 61 sec
print(hms_to_seconds("1h 1s")) # 3601 sec
print(hms_to_seconds("2h")) # 7200 sec
print(hms_to_seconds("10m")) # 600 sec
print(hms_to_seconds("33s")) # 33 sec
1. Read .csv
Reading the csv file into memory can be done using pd.read_csv()
2. Parse time string
Parser
Ideally you want to write a parser that can parse the time string for you. This can be done using ANTLR or by writing one yourself. See e.g. this blog post. This would be the most interesting solution and also the most challenging one. There might be a parser already out there which can handle this as the format is quite common, so you might wanna search for one.
RegEx
A quick and dirty solution however can be implemented using RegEx. The idea is to use this RegEx [0-9]*(?=h) for hours, [0-9]*(?=m) for minutes, [0-9]*(?=s) for seconds to extract the actual values before those identifiers and then calculate the total duration in seconds.
Please note: this is not an ideal solution and only works for "h", "m", "s" and there are edge cases that are not handled in this implementation but it does work in principle.
import re
duration_calcs = {
"h": 60 * 60, # convert hours to seconds => hours * 60 * 60
"m": 60, # convert minutes to seconds => minutes * 60
"s": 1, # convert seconds to seconds => seconds * 1
}
tests_cases = [
"3h 54m",
"4h 9m",
"12s",
"41m 1s"
]
tests_results = [
int(14040), # 3 * 60 * 60 + 54 * 60
14940, # 4 * 60 * 60 + 9 * 60
12, # 12 * 1
2461 # 41 * 60 + 1 * 1
]
def get_duration_component(identifier, text):
"""
Gets one component of the duration from the string and returns the duration in seconds.
:param identifier: either "h", "m" or "s"
:type identifier: str
:param text: text to extract the information from
:type text: str
:return duration in seconds
"""
# RegEx using positive lookahead to either "h", "m" or "s" to extract number before that identifier
regex = re.compile(f"[0-9]*(?={identifier})")
match = regex.search(text)
if not match:
return 0
return int(match.group()) * duration_calcs[identifier]
def get_duration(text):
"""
Get duration from text which contains duration like "4h 43m 12s".
Only "h", "m" and "s" supported.
:param text: text which contains a duration
:type text: str
:return: duration in seconds
"""
idents = ["h", "m", "s"]
total_duration = 0
for ident in idents:
total_duration += get_duration_component(ident, text)
return total_duration
def run_tests(tests, test_results):
"""
Run tests to verify that this quick and dirty implementation works at least for the given test cases.
:param tests: test cases to run
:type tests: list[str]
:param test_results: expected test results
:type test_results: list[int]
:return:
"""
for i, (test_inp, expected_output) in enumerate(zip(tests, test_results)):
result = get_duration(test_inp)
print(f"Test {i}: Input={test_inp}\tExpected Output={expected_output}\tResult: {result}", end="")
if result == expected_output:
print(f"\t[SUCCESS]")
else:
print(f"\t[FAILURE]")
run_tests(tests_cases, tests_results)
Expected result
Test 0: Input=3h 54m Expected Output=14040 Result: 14040 [SUCCESS]
Test 1: Input=4h 9m Expected Output=14940 Result: 14940 [SUCCESS]
Test 2: Input=12s Expected Output=12 Result: 12 [SUCCESS]
Test 3: Input=41m 1s Expected Output=2461 Result: 2461 [SUCCESS]
split()
Another (even simpler) solution could be to split() at space and use some if statements to determine whether a part is "h", "m" or "s" and then parse the preceding string to int and convert is as shown above. The idea is similar, so I did not write a program for this.
I bought a cheap ublox7 GPS dongle and stuck it on my raspberry pi 3. When I looked at the output and tried to stick it into a map program I got weird results. Here is some sample output from the device after parsing with a library called "pynmea2".
$GPGLL,3745.81303,N,12214.62049,W,175033.00,A,D*7C
I did some research about how to convert this output to something useful and I found a formula that involved splitting the number up and dividing it by .6.
Doing GPS Conversion – Degrees to Latitude Longitude and vice versa
so I wrote a python program to try to capture and convert all of this, and the output is off by like a mile. I am trying to figure out what I am doing wrong, how could I be this close yet still off by about one mile?
from time import sleep
import pynmea2
import serial
import re
degree_sign = u"\N{DEGREE SIGN}"
ser = serial.Serial('/dev/ttyACM0', 9600, timeout=1.0)
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser))
while True:
line = sio.readline()
msg = pynmea2.parse(line)
msg = str(msg)
if re.search("GPGLL", msg):
raw_nums = re.findall(r'\b\d*\.\d*', msg)
lat_whole = (raw_nums[0])
lat_part1 = lat_whole[0:2]
lat_part2 = lat_whole[2:4]
lat_part2 = int(lat_part2)
lat_part2 = lat_part2 / .6
lat_part2 = int(lat_part2)
lat_part2 = str(lat_part2)
lat_part3 = lat_whole[5:9]
lat_part3 = float(lat_part3)
lat_part3 = lat_part3 / .6
lat_part3 = round(lat_part3, 0)
lat_part3 = int(lat_part3)
lat_part3 = str(lat_part3)
lon_whole = raw_nums[1]
lon_part1 = lon_whole[0:3]
lon_part1 = int(lon_part1)
lon_part1 = -lon_part1
lon_part1 = str(lon_part1)
lon_part2 = lon_whole[3:5]
lon_part2 = int(lon_part2)
lon_part2 = lon_part2 / .6
lon_part2 = str(lon_part2)
lon_part2 = lon_part2[0:2]
lon_part3 = lon_whole[6:10]
lon_part3 = float(lon_part3)
lon_part3 = lon_part3 / .6
lon_part3 = round(lon_part3, 0)
lon_part3 = int(lon_part3)
lon_part3 = str(lon_part3)
print(lat_part1 + "." + lat_part2 + lat_part3 +"," , lon_part1 + "." + lon_part2 + lon_part3)
print(lat_part1+degree_sign+lat_part2+"'"+lat_part3+"\"" + "N", lon_part1 + degree_sign+ lon_part2 + "'" + lon_part3+"\"" + "W")
sleep(1)
Here is the list that regex generated using the pynmea2 output:
['3745.81246', '12214.61512', '224329.00'] assigned to raw_nums.
Output from the script:
37.7513540, -122.2310268
37°75'13540"N -122°23'10268"W
Entering the first bit of output into google maps brings up a place near me but about a mile away, the second number doesn't work on Google maps for some reason - but it works on apple maps.
My questions:
I know there must be at least 100 better ways to write this code, do you have suggestions for getting there quicker?
Does the formula make sense? Am I applying it correctly?
Do you see a reason why this should return a result that is close but no cigar?
Do you know why the second line of output would not work as input into google maps?
What accuracy should I expect from a ublox 7 GPS dongle I got from Amazon for $12?
Thanks in advance, I really appreciate it.
Update: I looked up my address on gps coordinates conversion
and the latitude they show for my address is 3745.50084 while my gps is reporting 3745.81246. So it just seems like I am starting with bad data...
If your parsed string from the GPS device is always of the form you specified, you can simply split the string on the commas like split_msg = msg.split(","). Then your lat will be split_msg[1] and your long split_msg[3]. With indexes 2 and 4 being the heading direction.
The lat is provided as DDmm.mm and long is provided as DDDmm.mm, which you seem to have captured above. So 3745.81246 would be 37 degrees and 45.81246 minutes. You can take the decimal portion of the minutes (i.e. 0.81246) and multiply times 60 to get seconds. So you would get 37 degrees, 45 minutes, and 48.75 seconds. As a sanity check, minutes and seconds should always be less than 60 as either of them being 60 would increment the next value (e.g. 60 minutes in a degree, 60 seconds in a minute).
To convert the minutes to a decimal degree number, simply divide the minutes number by 60 (45.81246/60=0.763541) then add that to your degrees. So 3745.81246 would become 37.763541.
So within the if statement:
split_msg = msg.split(",")
lat, lat_dir, long, long_dir = split_msg[1:5]
lat_d, lat_m = float(lat[:2]), float(lat[2:])
long_d, long_m = float(long[:3]), float(long[3:])
lat_dec = lat_d + lat_m/60
long_dec = long_d + long_m/60
lat_min = math.floor(lat_m)
lat_sec = 60*(lat_m - lat_min)
long_min = math.floor(long_m)
long_sec = 60*(long_m - long_min)
print(f"{lat_dec} {lat_dir}, {long_dec} {long_dir}")
print(f"{lat_d}{degree_sign} {lat_min}' {lat_sec}\" {lat_dir}, {long_d}{degree_sign} {long_min}' {long_sec}\" {long_dir}")
I have not tested the above code, but this is the general way I would approach this problem.
Since you're using the pynmea2 library you can make use of the object properties to ease the subsequent steps.
import pynmea2
line = "$GPGLL,3745.81303,N,12214.62049,W,175033.00,A,D*7C"
nmeaobj = pynmea2.parse(line)
coord = f'{nmeaobj.latitude} {nmeaobj.longitude}'
print(coord)
# 37.763551 -122.243675
The decimal degree difference between the script output and the library is around 0.012 which is similar to the precision length (1.1132 km) cited in the Degree precision versus length table. This would explain why you are seen a discrepancy of about a mile.
abs(-122.243675 - -122.2310268)
0.012648200000000998
abs(37.7635505 - 37.7513540)
0.01219650000000172
You could use the formula cited in the previous link to convert from decimal degree to DMS components, and this would yield a valid location. But notice that the directions of the coordinate (NS/WE) were left out of the final string formation.
def dd_to_dms(coord):
d = int(coord)
abs_d = abs(coord-d)
m = int(60 * abs_d)
s = 3600 * abs_d - 60 * m
return d,m,s
lat = '''%02d°%02d'%07.4f"''' % dd_to_dms(nmeaobj.latitude)
lon = '''%02d°%02d'%07.4f"''' % dd_to_dms(nmeaobj.longitude)
print(f'{lat} {lon}')
# 37°45'48.7818" -122°14'37.2294"
The second line of output would not work as input into google maps because, as already mentioned, minutes and seconds should always be less than 60. Moreover, adding the directions of the coordinate to a negative degree could make you (depending on the algorithm used to parse the string) "walk" in the opposite direction of the desired location as a consequence of having the degree sign (or the coordinate direction NS/WE) ignored, or, in Google maps case, simply not understanding the coordinate.
37°45'48.7818" -122°14'37.2294" # works
37°45'48.7818"N -122°14'37.2294"W # don't work
37°45'48.7818"N 122°14'37.2294"W # works
I should not use any date or time libraries! Actually I was able to sum them as normal integers but I couldn't find how to sum them as hours and minutes so as an example "03:50" + "04:20" should be "08:10" but actually I got "7:70" because I still couldn't find valid approach. Any hint or approach is much appreciated. Thank you.
import re
def add_time(time, duration):
elems1 = re.split(r"\s", time)
elems2 = re.split(r":", elems1[0])
elems2.append(elems1[1])
elems3 = re.split(r":", duration)
resH = eval(elems2[0]) + eval(elems3[0])
resM = eval(elems2[1]) + eval(elems3[1])
return f'{str(resH)}:{str(resM)} {elems2[2]}'
print(add_time("3:50 AM", "4:20"))
Actual output
7:70 AM
Desired output
08:10 AM
You can implement it by yourself by using % to keep track of hours and minutes format, but keep in mind that the remainder of the minutes, if came across 60, should passed to the hours.
take a look in this implementation:
import re
import numpy as np
def add_time(time, duration):
elems1 = re.split(r"\s", time)
elems2 = re.split(r":", elems1[0])
elems2.append(elems1[1])
elems3 = re.split(r":", duration)
remainder_from_minutes = int(np.floor((int(elems2[1]) + int(elems3[1]))/60))
resH = (int(elems2[0]) + int(elems3[0]))%24 + remainder_from_minutes
resM = (int(elems2[1]) + int(elems3[1]))%60
am_or_pm = elems2[2] if int(resH/12)%2==0 else chr(145-ord(elems2[2][0])) + "M"
return str(resH).rjust(2,"0") + ":" + str(resM).rjust(2,"0") + " " + str(am_or_pm)
print(add_time("3:50 AM", "8:50"))
print(add_time("3:20 PM", "8:50"))
output:
12:40 PM
12:10 AM
fixed AM/PM notation as well, with some ascii table tricks, in such way that also 12:40 AM and 03:50 PM will be acceptable
i think you should add just condition for handling when minutes is greater than 59 and when hour is greater than 12.
import re
def add_time(time, duration):
elems1 = re.split(r"\s", time)
elems2 = re.split(r":", elems1[0])
elems2.append(elems1[1])
elems3 = re.split(r":", duration)
cf = 0
resM = eval(elems2[1]) + eval(elems3[1])
if (resM > 60):
cf = 1
resM = resM - 60
resH = eval(elems2[0]) + eval(elems3[0]) + cf
if(resH > 12):
resH = resH - 12
return f'{str(resH)}:{str(resM)} {elems2[2]}'
I think a good way to do this is to convert everything to minutes, sum them and convert them to hours + minutes back. This is good because this way you will be able to scale your system in future, for example, make it possible to add more than just two time values (which is not possible in one of the other answers). Because you are using Python which has built-in long arithmetic I don't see any disadvantages of this solution
Here is my code:
def send_file(self,source,destination):
self.client.open_sftp().put(source,destination,self.upload_status)
def upload_status(self,sent,size):
sent_mb=round(sent/1000000,1)
remaining_mb=round((size-sent)/1000000,1)
size=round(size/1000000,1)
sys.stdout.write("Total size:{0} MB|Sent:{1} MB|Remaining:{2} MB".
format(size,sent_mb,remaining_mb))
sys.stdout.write('\r')
I get the following output:
Total size:30.6|Sent:30.5 MB|Remaining:0.1 MB
My expected output is:
Total size:30.6|Sent:30.5 MB|Remaining:0.1 MB|Time remaining:00:00:01
Is there any module in Paramiko that can give me time stamp? If not, how can I achieve this?
Remember time, when the transfer started;
On each update, calculate how long the transfer is already taking place;
Based on that, calculate a transfer speed;
Based on that, calculate, how long will it take to transfer the rest.
import datetime
# ...
start = datetime.datetime.now()
def upload_status(self, sent, size):
sent_mb = round(float(sent) / 1000000, 1)
remaining_mb = round(float(size - sent) / 1000000, 1)
size_mb = round(size / 1000000, 1)
time = datetime.datetime.now()
elapsed = time - start
if sent > 0:
remaining_seconds = elapsed.total_seconds() * (float(size - sent) / sent)
else:
remaining_seconds = 0
remaining_hours, remaining_remainder = divmod(remaining_seconds, 3600)
remaining_minutes, remaining_seconds = divmod(remaining_remainder, 60)
print(
("Total size:{0} MB|Sent:{1} MB|Remaining:{2} MB|" +
"Time remaining:{3:02}:{4:02}:{5:02}").
format(
size_mb, sent_mb, remaining_mb,
int(remaining_hours), int(remaining_minutes), int(remaining_seconds)))
Also note that your MB calculation works in Python 3 only. In Python 2, you would be striping all digits. I've fixed that by a cast to float.
I want to take the time that some operations need; so I wrote:
def get_formatted_delta(start_time):
seconds = ( datetime.datetime.now() - start_time ).total_seconds()
m, s = divmod(seconds, 60)
min = '{:02.0f}'.format(m)
sec = '{:02.4f}'.format(s)
return '{} minute(s) {} seconds'.format(min, sec)
but when I run that; I get printouts like:
00 minute(s) 1.0010 seconds
Meaning: as expected, 0 minutes are showing up as "00".
But 1 seconds shows up as 1.xxxx - instead of 01.xxxx
So, what is wrong with my format specification?
The field width applies to the whole field including the decimals and the decimal point. For 4 decimals plus the point, plus 2 places for the integer portion, you need 7 characters:
>>> format(1.001, '07.4f')
'01.0010'
You don't need to format those floats separately, you can do all formatting and interpolation in one step:
def get_formatted_delta(start_time):
seconds = ( datetime.datetime.now() - start_time ).total_seconds()
m, s = divmod(seconds, 60)
return '{:02.0f} minute(s) {:07.4f} seconds'.format(m, s)