We Keep Coding

matplotlib automatically sorting time values

I am trying to plot data from temperature sensor with time steps. I have time steps in format "hh:mm:ss" after conversion from string to datetime format. First value in the list is "21:47:22" and the last one is "06:12:22" the next day.I have been trying to plot these values with order of indexes in the list however Python automaticaly sorting it from "00:00:00" to "24:00:00" on the x axis. Here is the image.
Could you please advice how to solve this issue? Below my code:
import matplotlib.pyplot as plt
import datetime
data = []
sensor1 = []
sensor2 = []
time = []
with open("output.txt","r") as f:
data = f.readlines()
first_sensor_len = len(data[0])
for var in data:
if var[2:7] == "First" and len(var) == first_sensor_len:
sensor1.append(var[28:33])
sensor2.append(var[75:80])
time.append(datetime.datetime.strptime(var[36:44], "%H:%M:%S"))
elif var[2:8] == "Second" and len(var) == first_sensor_len:
sensor2.append(var[29:34])
sensor1.append(var[75:80])
time.append(datetime.datetime.strptime(var[83:91], "%H:%M:%S"))
plt.plot(time, sensor1)
plt.show()

Supposed time looks like
timestr = ["21:47:22", "22:12:22", "23:12:22", "00:12:22", "01:12:22", "03:12:22", "06:12:22"]
time = [datetime.datetime.strptime(ts, "%H:%M:%S") for ts in timestr]
time
[datetime.datetime(1900, 1, 1, 21, 47, 22),
datetime.datetime(1900, 1, 1, 22, 12, 22),
datetime.datetime(1900, 1, 1, 23, 12, 22),
datetime.datetime(1900, 1, 1, 0, 12, 22),
datetime.datetime(1900, 1, 1, 1, 12, 22),
datetime.datetime(1900, 1, 1, 3, 12, 22),
datetime.datetime(1900, 1, 1, 6, 12, 22)]
You can use np.diff from numpy to mark every first time of a new day. If the difference of two consecutive time values is negative, there was midnight in between.
(This boolean array is appended to one initial False, which states that the first time value has always no day offset; the result of np.diff is generally one entry shorter than its input.)
import numpy as np
newday_marker = np.append(False, np.diff(time) < datetime.timedelta(0))
newday_marker
array([False, False, False, True, False, False, False], dtype=bool)
With np.cumsum this array can be transformed into the array of dayoffsets for each time value.
day_offset = np.cumsum(newday_marker)
day_offset
array([0, 0, 0, 1, 1, 1, 1], dtype=int32)
In the end this has to be converted to timedeltas and then can be added to the original list of time values:
date_offset = [datetime.timedelta(int(dt)) for dt in day_offset]
dtime = [t + dos for t, dos in zip(time, date_offset)]
dtime
[datetime.datetime(1900, 1, 1, 21, 47, 22),
datetime.datetime(1900, 1, 1, 22, 12, 22),
datetime.datetime(1900, 1, 1, 23, 12, 22),
datetime.datetime(1900, 1, 2, 0, 12, 22),
datetime.datetime(1900, 1, 2, 1, 12, 22),
datetime.datetime(1900, 1, 2, 3, 12, 22),
datetime.datetime(1900, 1, 2, 6, 12, 22)]

List comprehension of 2+ variables in R

What's the best R equivalent of the Python 2-variable list comprehension
[datetime(y,m,15) for y in xrange(2000,2020) for m in [3,6,9,12]]
The result
[datetime.datetime(2000, 3, 15, 0, 0),
datetime.datetime(2000, 6, 15, 0, 0),
datetime.datetime(2000, 9, 15, 0, 0),
datetime.datetime(2000, 12, 15, 0, 0),
datetime.datetime(2001, 3, 15, 0, 0) ... ]

This will produce equivalent results in R
with(expand.grid(m=c(3,6,9,12), y=2000:2020), ISOdate(y,m,15))
We use expand.grid to get all combinations of year and month, and then we just use the vectorized ISOdate function to get the values.

Another way to express the desired result using the listcompr package:
library(listcompr)
gen.list(ISOdate(y, m, 15), y = 2000:2019, m = c(3, 6, 9, 12))
Note that range(a,b) (or xrange(a, b) in Python2) is equivalent to a:(b-1) (stop element b excluded) in R (and not a:b, which includes the stop element).

accessing two list elements to get results

I have two lists. One list name 'date' has dates in it which are related to persons birth date.
data = [ datetime.datetime(1958, 3, 15, 0, 0), datetime.datetime(1958, 9, 15, 0, 0), datetime.datetime(1930, 10, 23, 0, 0), datetime.datetime(1928, 9, 15, 0, 0), datetime.datetime(1928, 1, 23, 0, 0), datetime.datetime(1925, 11, 15, 0, 0), datetime.datetime(1962, 7, 20, 0, 0),datetime.datetime(1960, 12, 14, 0, 0), datetime.datetime(1960, 5, 10, 0, 0),datetime.datetime(1963, 9, 7, 0, 0), datetime.datetime(1956, 3, 10, 0, 0), datetime.datetime(1955, 2, 15, 0, 0),datetime.datetime(1958, 11, 14, 0, 0),datetime.datetime(1956, 8, 24, 0, 0),datetime.datetime(1990, 4, 30, 0, 0)]
Now next list contains marriage dates.
marriage = [ datetime.datetime(1985, 5, 14, 0, 0),datetime.datetime(1945, 6, 15, 0, 0), datetime.datetime(1938, 6, 11, 0, 0), datetime.datetime(1995, 4, 5, 0, 0), datetime.datetime(1987, 2, 26, 0, 0), datetime.datetime(1983, 12, 13, 0, 0), datetime.datetime(1980, 9, 16, 0, 0), datetime.datetime(2011, 6, 19, 0, 0)]
each date from the 'marriage' list is related to 2 dates from 'date' list. Now, I want to compare one date from marriage list to two dates from date list so that i can print"birth date is less than marriage.
How can accomplish this task using loop? confused with this one.
Please note that I used import datetime, import re to accomplish date comparison.

for i in range(len(data)):
if data[i] < marriage[i]:
print "birthdate is lt marriage date"
else:
print "birthdate is gt or eq to marriage date"
I'm not sure what you are trying to accomplish here... Also you don't need re for date comparison, you can use normal < > == <= >= operators.
This also sounds like a job for a hash(dictionary)...
marriage = {
'marriage1' : {
'1' : <birthday>,
'2' : <birthday>,
'marriage-date' : <marriage-date>
},
'marriage2' : {
'1' : <birthday>,
'2' : <birthday>,
'marriage-date' : <marriage-date>
}
}
A hash(dictionary) will make comparisons much easier with lists that don't contain the same number of values.

This assumes that the marriage and birth dates are in the same order (i.e., the first two birth dates correspond to the first marriage date and the next 2 birth dates correspond to the second marriage date)
for i in range(len(marriage)):
if marriage[i] > data[i*2] and marriage[i] > data[(i*2)+1]:
print "Both birthdates less than marriage data"
I believe my assumption is correct because there are twice as many entries in the data list as there are in the marriage list.

adding elements in nested lists

I have got a list containing nested lists like this :
[ [datetime.datetime(2000, 12, 10, 0, 0), 0.0011] , [datetime.datetime(2000, 12, 11, 0, 0), 0.0013 , [datetime.datetime(2000, 12, 12, 0, 0), 0.0014]]
etc..
How do I go about adding sub elements 2 by 2 like this :
sum(0.0011,0.0013) + 0.0014
then taking the result of this sum and adding it to the next sub element ?
I`m basically trying to compound the values .
thanks!

The easiest way to do this is with the sum() builtin and a generator expression:
>>>items = [[datetime.datetime(2000, 12, 10, 0, 0), 0.0011], [datetime.datetime(2000, 12, 11, 0, 0), 0.0013 ], [datetime.datetime(2000, 12, 12, 0, 0), 0.0014]]
>>>sum(item[1] for item in items)
0.0038000000000000004
Edit:
If you want to print out the result of each stage of the summation, you want to use functools.reduce() (which, in 2.x is the reduce builtin).
from functools import reduce
import datetime
items = [[datetime.datetime(2000, 12, 10, 0, 0), 0.0011], [datetime.datetime(2000, 12, 11, 0, 0), 0.0013 ], [datetime.datetime(2000, 12, 12, 0, 0), 0.0014]]
def add_printing_result(a, b):
total = a+b
print(total)
return total
reduce(add_printing_result, (item[1] for item in items))
Which gives us:
0.0024000000000000002
0.0038000000000000004

sum = 0, myarr = [ [datetime.datetime(2000, 12, 10, 0, 0), 0.0011] , [datetime.datetime(2000, 12, 11, 0, 0), 0.0013] , [datetime.datetime(2000, 12, 12, 0, 0), 0.0014]]
for(i in myarr):
sum+=i[1]
I'm sure there are better ways to do this (I'm no Python expert) but this should sum your values properly such that sum is the sub elements' sum.

How can I specify date and time in Python?

What is the object used in Python to specify date (and time) in Python?
For instance, to create an object that holds a given date and time, (let's say '05/10/09 18:00').
As per S.Lott's request, so far I have:
class Some:
date =
I stop there. After the "=" sign for, I realize I didn't knew what the right object was ;)

Simple example:
>>> import datetime
# 05/10/09 18:00
>>> d = datetime.datetime(2009, 10, 5, 18, 00)
>>> print d.year, d.month, d.day, d.hour, d.second
2009 10 5 18 0
>>> print d.isoformat(' ')
2009-10-05 18:00:00
>>>

Nick D has the official way of handling your problem. If you want to pass in a string like you did in your question, the dateutil module (http://labix.org/python-dateutil) has excellent support for that kind of thing.
For examples, I'm going to copy and paste from another answer I gave a while back now:
Simple example:
>>> parse("Thu Sep 25 2003")
datetime.datetime(2003, 9, 25, 0, 0)
>>> parse("Sep 25 2003")
datetime.datetime(2003, 9, 25, 0, 0)
>>> parse("Sep 2003", default=DEFAULT)
datetime.datetime(2003, 9, 25, 0, 0)
>>> parse("Sep", default=DEFAULT)
datetime.datetime(2003, 9, 25, 0, 0)
>>> parse("2003", default=DEFAULT)
datetime.datetime(2003, 9, 25, 0, 0)
To ambigous:
>>> parse("10-09-2003")
datetime.datetime(2003, 10, 9, 0, 0)
>>> parse("10-09-2003", dayfirst=True)
datetime.datetime(2003, 9, 10, 0, 0)
>>> parse("10-09-03")
datetime.datetime(2003, 10, 9, 0, 0)
>>> parse("10-09-03", yearfirst=True)
datetime.datetime(2010, 9, 3, 0, 0)
To all over the board:
>>> parse("Wed, July 10, '96")
datetime.datetime(1996, 7, 10, 0, 0)
>>> parse("1996.07.10 AD at 15:08:56 PDT", ignoretz=True)
datetime.datetime(1996, 7, 10, 15, 8, 56)
>>> parse("Tuesday, April 12, 1952 AD 3:30:42pm PST", ignoretz=True)
datetime.datetime(1952, 4, 12, 15, 30, 42)
>>> parse("November 5, 1994, 8:15:30 am EST", ignoretz=True)
datetime.datetime(1994, 11, 5, 8, 15, 30)
>>> parse("3rd of May 2001")
datetime.datetime(2001, 5, 3, 0, 0)
>>> parse("5:50 A.M. on June 13, 1990")
datetime.datetime(1990, 6, 13, 5, 50)
Take a look at the documentation for it here:
http://labix.org/python-dateutil#head-c0e81a473b647dfa787dc11e8c69557ec2c3ecd2

Look at the datetime module; there are datetime, date and timedelta class definitions.

>>> import datetime
>>> datetime.datetime.strptime('05/10/09 18:00', '%d/%m/%y %H:%M')
datetime.datetime(2009, 10, 5, 18, 0)
>>> datetime.datetime.today()
datetime.datetime(2009, 10, 5, 21, 3, 55, 827787)
So, you can either use format string to convert to datetime.datetime object or if you're particularly looking at today's date could use today() function.