The following code throws an exception, which most probably pertains to the TZID replacements I needed to do to fix some other bugs. If I remove the the "UNTIL" statement from the ical string, the code works just fine.
from icalendar.cal import Calendar
import datetime
from dateutil import rrule
from dateutil.tz import gettz
cal_str = "BEGIN:VEVENT\nDTSTART;TZID=America/Los_Angeles:20171019T010000\nDTEND;TZID=America/Los_Angeles:20171019T230000\nRRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;UNTIL=20180423T191500\nX-OP-ENTRY-STATE:unlocked\nEND:VEVENT"
ical = Calendar.from_ical(cal_str)
start_time_dt = ical.get("DTSTART").dt
end_time_dt = ical.get("DTEND").dt
tzinfo = gettz(str(start_time_dt.tzinfo))
start_time_dt = start_time_dt.replace(tzinfo=tzinfo)
recurring_rule = ical.get('RRULE').to_ical().decode('utf-8')
rules = rrule.rruleset()
first_rule = rrule.rrulestr(recurring_rule, dtstart=start_time_dt)
rules.rrule(first_rule)
event_delta = end_time_dt -start_time_dt
now = datetime.datetime.now(datetime.timezone.utc)
for s in rules.between(now - event_delta, now + datetime.timedelta(minutes=1)):
print(s)
Here is the exception:
Traceback (most recent call last):
File "ical_test.py", line 27, in <module>
for s in rules.between(now - event_delta, now + datetime.timedelta(minutes=1)):
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 290, in between
for i in gen:
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1362, in _iter
self._genitem(rlist, gen)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1292, in __init__
self.dt = advance_iterator(gen)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 861, in _iter
if until and res > until:
TypeError: can't compare offset-naive and offset-aware datetimes
Anyone help for finding out the root cause of this error and a way to fix this?
First of all they fixed the exception to be more explicit in dateutil>2.7.1 to this:
Traceback (most recent call last):
File "ical_test.py", line 23, in <module>
first_rule = rrule.rrulestr(recurring_rule, dtstart=start_time_dt)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1664, in __call__
return self._parse_rfc(s, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1547, in _parse_rfc
tzinfos=tzinfos)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1506, in _parse_rfc_rrule
return rrule(dtstart=dtstart, cache=cache, **rrkwargs)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 461, in __init__
'RRULE UNTIL values must be specified in UTC when DTSTART '
ValueError: RRULE UNTIL values must be specified in UTC when DTSTART is timezone-aware
The solution is to calculate the UNTIL time in UTC and add Z to the end of the time string as described in the RFC:
https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html
the correct RRULE string should look like this:
cal_str = "BEGIN:VEVENT\nDTSTART;TZID=America/Los_Angeles:20171019T010000\nDTEND;TZID=America/Los_Angeles:20171019T230000\nRRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;UNTIL=20180423T001500Z\nX-OP-ENTRY-STATE:unlocked\nEND:VEVENT"
Related
I need to make my scheduler fire every X days at the specific time (e.g. every 7 days at 11:30)
my code:
def make_interval(record_date: str, record_time: str, record_title: str):
hours, minutes = _get_hours_minutes(record_time)
trigger = AndTrigger([IntervalTrigger(days=int(record_date)),
CronTrigger(hour=hours, minute=minutes)])
scheduler.add_job(_send_notification, trigger=trigger,
kwargs={...},
id=record_title,
timezone=user_timezone)
but I got error: [Errno 22] Invalid argument on the third line.
What's wrong? Can't get why it doesnt work
_get_hours_minutes (just returns separately the value of hours and minutes from "HH:MM")
def _get_hours_minutes(user_time: str) -> (str, str):
return user_time[:2], user_time[3:5]
tracebacks:
Traceback (most recent call last):
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\scheduler\main.py", line 92, in make_interval
scheduler.add_job(_send_notification, trigger=trigger,
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\venv\Lib\site-packages\apscheduler\schedulers\base.py", line 447, in add_job
self._real_add_job(job, jobstore, replace_existing)
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\venv\Lib\site-packages\apscheduler\schedulers\base.py", line 863, in _real_add_job
replacements['next_run_time'] = job.trigger.get_next_fire_time(None, now)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\venv\Lib\site-packages\apscheduler\triggers\combining.py", line 55, in get_next_fire_time
fire_times = [trigger.get_next_fire_time(previous_fire_time, now)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\venv\Lib\site-packages\apscheduler\triggers\combining.py", line 55, in <listcomp>
fire_times = [trigger.get_next_fire_time(previous_fire_time, now)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\venv\Lib\site-packages\apscheduler\triggers\interval.py", line 68, in get_next_fire_time
return normalize(next_fire_time)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\pizhlo21\Desktop\Folder\python\tg_bot_reminder\venv\Lib\site-packages\apscheduler\util.py", line 431, in normalize
return datetime.fromtimestamp(dt.timestamp(), dt.tzinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 22] Invalid argument
My OS platform: Windows 10
I would take a closer look at "record_time" and the format that you pass to "_get_hours_minutes". It has to be in the format of "HH:MM" (assuming that the Errno 22 error is for that line. There are other ways to use the datetime library so that you don't have to slice strings.
I'm using a market data API and am using Pandas dataframes to filter and restructure before storing in a database. Before it goes into InfluxDB, I need to restructure a date column which currently looks like:
Earnings
May 15/b
Apr 09/a
What I have so far is below, but I'm getting the error -
TypeError: Unrecognized value type: <class 'str'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/OMT/Demo/scratchpad.py", line 41, in <module>
filtered['NewEarnings'] = pd.to_datetime(filtered['NewEarnings'], format='%b%d')
File "/usr/local/lib/python3.8/dist-packages/pandas/core/tools/datetimes.py", line 801, in to_datetime
cache_array = _maybe_cache(arg, format, cache, convert_listlike)
File "/usr/local/lib/python3.8/dist-packages/pandas/core/tools/datetimes.py", line 178, in _maybe_cache
cache_dates = convert_listlike(unique_dates, format)
File "/usr/local/lib/python3.8/dist-packages/pandas/core/tools/datetimes.py", line 460, in _convert_listlike_datetimes
raise e
File "/usr/local/lib/python3.8/dist-packages/pandas/core/tools/datetimes.py", line 423, in _convert_listlike_datetimes
result, timezones = array_strptime(
File "pandas/_libs/tslibs/strptime.pyx", line 144, in pandas._libs.tslibs.strptime.array_strptime
ValueError: time data '-' does not match format '%b%d' (match)
The code in question -
filtered = df
earningsColumn = filtered['Earnings'].squeeze()
stripped = earningsColumn.str.rstrip('. /a/b')
filtered['NewEarnings'] = stripped
filtered['NewEarnings'] = pd.to_datetime(filtered['NewEarnings'], format='%b%d')
This works fine:
{{ strptime(states('input_datetime.music_alarm'), "%H:%M:%S") - strptime("10", "%M") }}
But this throws an error:
{{ strptime(states('input_datetime.music_alarm'), "%H:%M:%S") + strptime("10", "%M") }}
states('input_datetime.music_alarm') equals a time, like 08:00:00
I'm using jinja2 for homeassistant. Here is the error.
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
result = coro.send(None)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/automation/__init__.py", line 336, in async_trigger
yield from self._async_action(self.entity_id, variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/automation/__init__.py", line 425, in action
yield from script_obj.async_run(variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/script.py", line 158, in async_run
await self._async_call_service(action, variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/script.py", line 187, in _async_call_service
self.hass, action, True, variables, validate_config=False)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/service.py", line 72, in async_call_from_config
config[CONF_SERVICE_DATA_TEMPLATE], variables))
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 56, in render_complex
for key, item in value.items()}
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 56, in <dictcomp>
for key, item in value.items()}
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 57, in render_complex
return value.async_render(variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 132, in async_render
return self._compiled.render(kwargs).strip()
File "/srv/homeassistant/lib/python3.5/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/srv/homeassistant/lib/python3.5/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/srv/homeassistant/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 1, in top-level template code
TypeError: bad operand type for unary -: 'datetime.datetime'
You MUST use timedelta and you have multiple issue in your code.
1/ Wrong use of datetime.datetime.strptime()
>>> import datetime
>>> print(datetime.datetime.strptime("10", "%M"))
1900-01-01 00:10:00
>>> print(datetime.datetime.strptime('08:00:00', "%H:%M:%S"))
1900-01-01 08:00:00
You have to parse full datetime to have correct behavior.
2/ You can't sum two datetime
Basically + is forbidden to avoid mistake, you need only - because you just have to reverse variables in your expression to have your negative timedelta.
>>> print(type(datetime.datetime.strptime("10", "%M") - datetime.datetime.strptime('08:00:00', "%H:%M:%S")))
<class 'datetime.timedelta'>
>>> print(datetime.datetime.strptime("10", "%M") - datetime.datetime.strptime('08:00:00', "%H:%M:%S"))
-1 day, 16:10:00
>>> print(datetime.datetime.strptime('08:00:00', "%H:%M:%S") - datetime.datetime.strptime("10", "%M"))
7:50:00
You can see in the reverse order this will give you incorrectly -1 day, 16:10:00 because this can't be handled without error.
3/ You can register timedelta to your template
strptime() is not available by default in Jinja2, so do like it with timedelta()...
Something like that :
import datetime
from jinja2 import Template
jinga = Template('{{ strptime(states("input_datetime.music_alarm"), "%H:%M:%S") - timedelta(minutes=10) }} ')
jinga.globals['timedelta'] = datetime.timedelta
print(jinga.render())
I am getting an error when creating a series in pandas.
Whenever I try to print the series I have created, I get an error.
The code I am running:
import pandas as pd
data2 = [1,2,3,4]
index = ['a','b','c','d']
s = pd.Series(data2, index)
print(s.shape)
s
The error:
Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
s
File "C:\Python34\lib\idlelib\rpc.py", line 611, in displayhook
text = repr(value)
File "C:\Python34\lib\site-packages\pandas\core\base.py", line 80, in __repr__
return str(self)
File "C:\Python34\lib\site-packages\pandas\core\base.py", line 59, in __str__
return self.__unicode__()
File "C:\Python34\lib\site-packages\pandas\core\series.py", line 1060, in __unicode__
width, height = get_terminal_size()
File "C:\Python34\lib\site-packages\pandas\io\formats\terminal.py", line 33, in get_terminal_size
return shutil.get_terminal_size()
File "C:\Python34\lib\shutil.py", line 1071, in get_terminal_size
size = os.get_terminal_size(sys.__stdout__.fileno())
AttributeError: 'NoneType' object has no attribute 'fileno'
Your error is related to pyshell, not to pandas.
Try to run it through python directly or jupyter console, because the code you provided is correct.
I've just started using pycassa, so if this is a stupid question, I apologize upfront.
I have a column family with the following schema:
create column family MyColumnFamilyTest
with column_type = 'Standard'
and comparator = 'CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.TimeUUIDType)'
and default_validation_class = 'BytesType'
and key_validation_class = 'UTF8Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and populate_io_cache_on_flush = false
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'};
When I try to do a get() with a valid key (works fine in cassandra-cli) I get:
Traceback (most recent call last):
File "<pyshell#19>", line 1, in <module>
cf.get('mykey',column_count=3)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 664, in get
return self._cosc_to_dict(list_col_or_super, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 368, in _cosc_to_dict
ret[self._unpack_name(col.name)] = self._col_to_dict(col, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 444, in _unpack_name
return self._name_unpacker(b)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 140, in unpack_composite
components.append(unpacker(bytestr[2:2 + length]))
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 374, in <lambda>
return lambda v: uuid.UUID(bytes=v)
File "/usr/lib/python2.7/uuid.py", line 144, in __init__
raise ValueError('bytes is not a 16-char string')
ValueError: bytes is not a 16-char string
Here's some more information I've discovered:
When using cassandra-cli I can see the data as:
% cassandra-cli -h 10.249.238.131
Connected to: "LocalDB" on 10.249.238.131/9160
Welcome to Cassandra CLI version 1.2.10-SNAPSHOT
Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.
[default#unknown] use Keyspace;
[default#Keyspace] list ColumnFamily;
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: urn:keyspace:ColumnFamily:a36e8ab1-7032-4e4c-a53d-e3317f63a640:
=> (name=autoZoning:::, value=01, timestamp=1391298393966000)
=> (name=creationTime:::, value=00000143efd8b76e, timestamp=1391298393966000)
=> (name=inactive:::14fe78e0-8b9b-11e3-b171-005056b700bb, value=00, timestamp=1391298393966000)
=> (name=label:::14fe78e0-8b9b-11e3-b171-005056b700bb, value=726a6d2d766e782d76613031, timestamp=1391298393966000)
1 Row Returned.
Elapsed time: 16 msec(s).
Since it was unclear what was causing the exception, I decided to add a print prior to the 'return self._name_unpacker(b)' line in columnfamily.py and I see:
>>> cf.get(dict(cf.get_range(column_count=0,filter_empty=False)).keys()[0])
Attempting to unpack: <00>\rautoZoning<00><00><00><00><00><00><00><00><00><00>
Traceback (most recent call last):
File "<pyshell#172>", line 1, in <module>
cf.get(dict(cf.get_range(column_count=0,filter_empty=False)).keys()[0])
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 665, in get
return self._cosc_to_dict(list_col_or_super, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 368, in _cosc_to_dict
ret[self._unpack_name(col.name)] = self._col_to_dict(col, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 445, in _unpack_name
return self._name_unpacker(b)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 140, in unpack_composite
components.append(unpacker(bytestr[2:2 + length]))
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 374, in <lambda>
return lambda v: uuid.UUID(bytes=v)
File "/usr/lib/python2.7/uuid.py", line 144, in __init__
raise ValueError('bytes is not a 16-char string')
ValueError: bytes is not a 16-char string
I have no idea where the extra characters are coming from around the column name. But that got me curious so I added another print in _cosc_to_dict in columnfamily.py and I see:
>>> cf.get(dict(cf.get_range(column_count=0,filter_empty=False)).keys()[0])
list_col_or_super is: []
list_col_or_super is: [ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\rautoZoning\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', value='\x01', ttl=None),
counter_super_column=None, super_column=None, counter_column=None),
ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\x0ccreationTime\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
value='\x00\x00\x01C\xef\xd8\xb7n', ttl=None), counter_super_column=None, super_column=None,
counter_column=None), ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\x08inactive\x00\x00\x00\x00\x00\x00\x00\x00\x10\x14\xfex\xe0\x8b\x9b\x11\xe3\xb1q\x00PV\xb7\x00\xbb\x00', value='\x00', ttl=None), counter_super_column=None, super_column=None,
counter_column=None), ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\x05label\x00\x00\x00\x00\x00\x00\x00\x00\x10\x14\xfex\xe0\x8b\x9b\x11\xe3\xb1q\x00PV\xb7\x00\xbb\x00', value='thisIsATest', ttl=None), counter_super_column=None, super_column=None, counter_column=None)]
autoZoning unpack:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/columnfamily.py", line 666, in get
return self._cosc_to_dict(list_col_or_super, include_timestamp, include_ttl)
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/columnfamily.py", line 369, in _cosc_to_dict
ret[self._unpack_name(col.name)] = self._col_to_dict(col, include_timestamp, include_ttl)
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/columnfamily.py", line 446, in _unpack_name
return self._name_unpacker(b)
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/marshal.py", line 140, in unpack_composite
components.append(unpacker(bytestr[2:2 + length]))
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/marshal.py", line 374, in <lambda>
return lambda v: uuid.UUID(bytes=v)
File "/usr/lib64/python2.6/uuid.py", line 144, in __init__
raise ValueError('bytes is not a 16-char string')
ValueError: bytes is not a 16-char string
Am I correct in assuming that the extra characters around the column names are what is responsible for the 'ValueError: bytes is not a 16-char string' exception?
Also if I try to use the column name and select it I get:
>>> cf.get(u'urn:keyspace:ColumnFamily:a36e8ab1-7032-4e4c-a53d-e3317f63a640:',columns=['autoZoning:::'])
Traceback (most recent call last):
File "<pyshell#184>", line 1, in <module>
cf.get(u'urn:keyspace:ColumnFamily:a36e8ab1-7032-4e4c-a53d-e3317f63a640:',columns=['autoZoning:::'])
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 651, in get
cp = self._column_path(super_column, column)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 383, in _column_path
self._pack_name(column, False))
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 426, in _pack_name
return self._name_packer(value, slice_start)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 115, in pack_composite
packed = packer(item)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 298, in pack_uuid
randomize=True)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/util.py", line 75, in convert_time_to_uuid
'neither a UUID, a datetime, or a number')
ValueError: Argument for a v1 UUID column name or value was neither a UUID, a datetime, or a number
Any further thoughts?
Thanks,
Rob
Turns out that the problem wasn't with the key, it was being caused, in part, by a bug in pycassa that wasn't handling an empty (null) string in the column UUID. A short-term fix is in the answer in google groups:
https://groups.google.com/d/msg/pycassa-discuss/Vf_bSgDIi9M/KTA1kbE9IXAJ
The other part of the answer was to get at the columns by using tuples (with the UUID as a UUID and not a str) instead of a string with ':' separators because that's, as I found out, a cassandra-cli thing.