This works fine:
{{ strptime(states('input_datetime.music_alarm'), "%H:%M:%S") - strptime("10", "%M") }}
But this throws an error:
{{ strptime(states('input_datetime.music_alarm'), "%H:%M:%S") + strptime("10", "%M") }}
states('input_datetime.music_alarm') equals a time, like 08:00:00
I'm using jinja2 for homeassistant. Here is the error.
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
result = coro.send(None)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/automation/__init__.py", line 336, in async_trigger
yield from self._async_action(self.entity_id, variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/automation/__init__.py", line 425, in action
yield from script_obj.async_run(variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/script.py", line 158, in async_run
await self._async_call_service(action, variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/script.py", line 187, in _async_call_service
self.hass, action, True, variables, validate_config=False)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/service.py", line 72, in async_call_from_config
config[CONF_SERVICE_DATA_TEMPLATE], variables))
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 56, in render_complex
for key, item in value.items()}
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 56, in <dictcomp>
for key, item in value.items()}
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 57, in render_complex
return value.async_render(variables)
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/template.py", line 132, in async_render
return self._compiled.render(kwargs).strip()
File "/srv/homeassistant/lib/python3.5/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/srv/homeassistant/lib/python3.5/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/srv/homeassistant/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 1, in top-level template code
TypeError: bad operand type for unary -: 'datetime.datetime'
You MUST use timedelta and you have multiple issue in your code.
1/ Wrong use of datetime.datetime.strptime()
>>> import datetime
>>> print(datetime.datetime.strptime("10", "%M"))
1900-01-01 00:10:00
>>> print(datetime.datetime.strptime('08:00:00', "%H:%M:%S"))
1900-01-01 08:00:00
You have to parse full datetime to have correct behavior.
2/ You can't sum two datetime
Basically + is forbidden to avoid mistake, you need only - because you just have to reverse variables in your expression to have your negative timedelta.
>>> print(type(datetime.datetime.strptime("10", "%M") - datetime.datetime.strptime('08:00:00', "%H:%M:%S")))
<class 'datetime.timedelta'>
>>> print(datetime.datetime.strptime("10", "%M") - datetime.datetime.strptime('08:00:00', "%H:%M:%S"))
-1 day, 16:10:00
>>> print(datetime.datetime.strptime('08:00:00', "%H:%M:%S") - datetime.datetime.strptime("10", "%M"))
7:50:00
You can see in the reverse order this will give you incorrectly -1 day, 16:10:00 because this can't be handled without error.
3/ You can register timedelta to your template
strptime() is not available by default in Jinja2, so do like it with timedelta()...
Something like that :
import datetime
from jinja2 import Template
jinga = Template('{{ strptime(states("input_datetime.music_alarm"), "%H:%M:%S") - timedelta(minutes=10) }} ')
jinga.globals['timedelta'] = datetime.timedelta
print(jinga.render())
Related
The pandas-market-calendars module documentation gives a quickstart example of how to create a schedule here: https://pypi.org/project/pandas-market-calendars/
I copy-pasted their quickstart example:
import pandas_market_calendars as mcal
# Create a calendar
nyse = mcal.get_calendar('NYSE')
# Show available calendars
print(mcal.get_calendar_names())
early = nyse.schedule(start_date='2012-07-01', end_date='2012-07-10')
print(early)
In their example, we should get a nice pretty schedule output:
market_open market_close
=========== ========================= =========================
2012-07-02 2012-07-02 13:30:00+00:00 2012-07-02 20:00:00+00:00
2012-07-03 2012-07-03 13:30:00+00:00 2012-07-03 17:00:00+00:00
2012-07-05 2012-07-05 13:30:00+00:00 2012-07-05 20:00:00+00:00
2012-07-06 2012-07-06 13:30:00+00:00 2012-07-06 20:00:00+00:00
2012-07-09 2012-07-09 13:30:00+00:00 2012-07-09 20:00:00+00:00
2012-07-10 2012-07-10 13:30:00+00:00 2012-07-10 20:00:00+00:00
instead, I get an error:
ValueError: The dtype of 'values' is incorrect. Must be 'datetime64[ns]'. Got object instead.
What am I doing wrong? I thought I input the dates exactly like the example showed. Why is it requiring a different format? Some people have ran the above code without error. My traceback is:
Traceback (most recent call last):
File "D:/Strawberry/Strawberry_Omega/tests/pandas_mkt_clndr_test.py", line 9, in <module>
early = nyse.schedule(start_date='2012-07-01', end_date='2012-07-10')
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas_market_calendars\market_calendar.py", line 632, in schedule
adjusted = schedule.loc[_close_adj].apply(
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\frame.py", line 7547, in apply
return op.get_result()
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\apply.py", line 180, in get_result
return self.apply_standard()
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\apply.py", line 255, in apply_standard
results, res_index = self.apply_series_generator()
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\apply.py", line 284, in apply_series_generator
results[i] = self.f(v)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas_market_calendars\market_calendar.py", line 633, in <lambda>
lambda x: x.where(x.le(x["market_close"]), x["market_close"]), axis= 1)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\generic.py", line 9001, in where
return self._where(
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\generic.py", line 8741, in _where
cond, _ = cond.align(self, join="right", broadcast_axis=1)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\series.py", line 4274, in align
return super().align(
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\generic.py", line 8556, in align
return self._align_series(
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\generic.py", line 8664, in _align_series
right = other._reindex_indexer(join_index, ridx, copy)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\series.py", line 4241, in _reindex_indexer
return self.copy()
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\generic.py", line 5660, in copy
data = self._mgr.copy(deep=deep)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\internals\managers.py", line 802, in copy
res = self.apply("copy", deep=deep)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\internals\managers.py", line 406, in apply
applied = getattr(b, f)(**kwargs)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\internals\blocks.py", line 679, in copy
return self.make_block_same_class(values, ndim=self.ndim)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\internals\blocks.py", line 261, in make_block_same_class
return type(self)(values, placement=placement, ndim=ndim)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\internals\blocks.py", line 1544, in __init__
values = self._maybe_coerce_values(values)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\internals\blocks.py", line 2170, in _maybe_coerce_values
values = self._holder(values)
File "C:\Users\Wes\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\arrays\datetimes.py", line 254, in __init__
raise ValueError(
ValueError: The dtype of 'values' is incorrect. Must be 'datetime64[ns]'. Got object instead.
I'm pretty new to Python and I've been struggling to figure out how to solve the following error I started seeing in certain instances
for idKey, idKey_dataFrame in dataFrame[~pd.isna(dataFrame.idKey)].groupby("idKey", sort=False):
idKey_dataFrame = idKey_dataFrame[idKey_dataFrame.matched_customerProps.astype(str).str.contains("entitlement") & idKey_dataFrame.matched_customerProps.astype(str).str.contains("entitlement address")
]
for customer_type, idKey_sub_dataFrame in idKey_dataFrame.groupby("customer_type"):
if customer_type != “repeat” and customer_type != “mvp”:
continue
self.log.info(f”Link made to {idKey}")
idKey_csv_export = self.make_csv(idKey_sub_dataFrame, inventory=False)
I noticed that whenever I have more than 1 value, I get
"ERROR - unhashable type: 'list'" error.
I'm not sure how to make it so I execute a groupby when there's multiple values and also perform a comparison against the object type. Any help would be greatly appreciated.
These are my columns
customer_name customer_id customer_type
xyz 003 repeat
zzz 389 repeat, mvp, intl
yyy 002 repeat
yay 005 repeat
kdi 083 mvp, repeat
If I provide single valued customer_type, I get no error. Whenever I test with customers that have more than 1 value under customer_type, I get an error. The offending line is the first for statement for idKey, idKey_dataFrame in dataFrame[~pd.isna(dataFrame.idKey)
Here is the complete error:
ERROR - unhashable type: 'list' in line 258, in execute for customer_type, idKey_sub_dataFrame in idKey_dataFrame.groupby("customer_type"):
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/ops.py", line 162, in get_iterator
splitter = self._get_splitter(data, axis=axis)
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/ops.py", line 168, in _get_splitter
comp_ids, _, ngroups = self.group_info
File "pandas/_libs/properties.pyx", line 34, in pandas._libs.properties.CachedProperty.__get__
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/ops.py", line 296, in group_info
comp_ids, obs_group_ids = self._get_compressed_labels()
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/ops.py", line 312, in _get_compressed_labels
all_labels = [ping.labels for ping in self.groupings]
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/ops.py", line 312, in <listcomp>
all_labels = [ping.labels for ping in self.groupings]
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/grouper.py", line 397, in labels
self._make_labels()
File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/grouper.py", line 421, in _make_labels
labels, uniques = algorithms.factorize(self.grouper, sort=self.sort)
File "/usr/local/lib/python3.7/site-packages/pandas/util/_decorators.py", line 208, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pandas/core/algorithms.py", line 672, in factorize
values, na_sentinel=na_sentinel, size_hint=size_hint, na_value=na_value
File "/usr/local/lib/python3.7/site-packages/pandas/core/algorithms.py", line 508, in _factorize_array
values, na_sentinel=na_sentinel, na_value=na_value
File "pandas/_libs/hashtable_class_helper.pxi", line 1798, in pandas._libs.hashtable.PyObjectHashTable.factorize
File "pandas/_libs/hashtable_class_helper.pxi", line 1718, in pandas._libs.hashtable.PyObjectHashTable._unique
This code is throwing a type error, yet I cannot seem to pinpoint exactly what the issue is.
I'm assuming my frame has some bad data, however there are 10,000 rows, so tough to understand whats happening there.
for i in cat_cols:
data_frame[i] = lbl.fit_transform(data_frame[i].fillna('0'))
Error:
> Traceback (most recent call last): File "test_makeprediction.py",
> line 187, in test_maximum_condition
> test_response = MakePrediction.predict(json_input) File "C:\dev\anaconda3\lib\site-packages\predict-1.0.0-py3.7.egg\prediction\makeprediction.py",
> line 51, in predict
> data_frame[i] = lbl.fit_transform(data_frame[i].fillna('0')) File
> "C:\dev\anaconda3\lib\site-packages\sklearn\preprocessing\label.py",
> line 236, in fit_transform
> self.classes_, y = _encode(y, encode=True) File "C:\dev\anaconda3\lib\site-packages\sklearn\preprocessing\label.py",
> line 107, in _encode
> raise TypeError("argument must be a string or number") TypeError: argument must be a string or number
I added some additional code, but it generates a new error.
Transform to Numerical Values for Comparison
for i in cat_cols.describe(include=[np.object, np.numeric]).columns:
I'm very confident this has something to do with the way the JSON is coming from my API, however everything I can see looks like it checks out.
New Error:
Data Frame Row Count: 10000
Category Count: 74
Numerical Count: 19
[2019-10-02 13:45:31,993] ERROR in app: Exception on /predict [POST]
Traceback (most recent call last):
File "C:\dev\anaconda3\lib\site-packages\flask\app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "C:\dev\anaconda3\lib\site-packages\flask\app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\dev\anaconda3\lib\site-packages\flask\app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\dev\anaconda3\lib\site-packages\flask\_compat.py", line 39, in reraise
raise value
File "C:\dev\anaconda3\lib\site-packages\flask\app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "C:\dev\anaconda3\lib\site-packages\flask\app.py", line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "predictservice.py", line 63, in predict
return_response = MakePrediction.predict(json_obj)
File "C:\dev\anaconda3\lib\site-packages\predict-1.0.0-py3.7.egg\prediction\makeprediction.py", line 50, in predict
for i in cat_cols.describe(include=[np.object, np.numeric]).columns:
AttributeError: 'Index' object has no attribute 'describe'
It looks like my issue was resolved by this bit of code, which i found in another stack post.
data_frame[i] = lbl.fit_transform(
data_frame[i].astype(str).fillna('0')
)
You can try this:
for i in data_frame.describe(include=[object, bool]).columns:
data_frame[i] = lbl.fit_transform(
data_frame[i].fillna('0')
)
with .describe(include=[object, bool]) I make sure I only take categoric or Boolean columns.
Other alternative:
for i in cat_cols:
data_frame[i] = lbl.fit_transform(
data_frame[i].fillna('0').astype(str)
)
The following code throws an exception, which most probably pertains to the TZID replacements I needed to do to fix some other bugs. If I remove the the "UNTIL" statement from the ical string, the code works just fine.
from icalendar.cal import Calendar
import datetime
from dateutil import rrule
from dateutil.tz import gettz
cal_str = "BEGIN:VEVENT\nDTSTART;TZID=America/Los_Angeles:20171019T010000\nDTEND;TZID=America/Los_Angeles:20171019T230000\nRRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;UNTIL=20180423T191500\nX-OP-ENTRY-STATE:unlocked\nEND:VEVENT"
ical = Calendar.from_ical(cal_str)
start_time_dt = ical.get("DTSTART").dt
end_time_dt = ical.get("DTEND").dt
tzinfo = gettz(str(start_time_dt.tzinfo))
start_time_dt = start_time_dt.replace(tzinfo=tzinfo)
recurring_rule = ical.get('RRULE').to_ical().decode('utf-8')
rules = rrule.rruleset()
first_rule = rrule.rrulestr(recurring_rule, dtstart=start_time_dt)
rules.rrule(first_rule)
event_delta = end_time_dt -start_time_dt
now = datetime.datetime.now(datetime.timezone.utc)
for s in rules.between(now - event_delta, now + datetime.timedelta(minutes=1)):
print(s)
Here is the exception:
Traceback (most recent call last):
File "ical_test.py", line 27, in <module>
for s in rules.between(now - event_delta, now + datetime.timedelta(minutes=1)):
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 290, in between
for i in gen:
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1362, in _iter
self._genitem(rlist, gen)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1292, in __init__
self.dt = advance_iterator(gen)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 861, in _iter
if until and res > until:
TypeError: can't compare offset-naive and offset-aware datetimes
Anyone help for finding out the root cause of this error and a way to fix this?
First of all they fixed the exception to be more explicit in dateutil>2.7.1 to this:
Traceback (most recent call last):
File "ical_test.py", line 23, in <module>
first_rule = rrule.rrulestr(recurring_rule, dtstart=start_time_dt)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1664, in __call__
return self._parse_rfc(s, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1547, in _parse_rfc
tzinfos=tzinfos)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 1506, in _parse_rfc_rrule
return rrule(dtstart=dtstart, cache=cache, **rrkwargs)
File "/usr/local/lib/python3.5/dist-packages/dateutil/rrule.py", line 461, in __init__
'RRULE UNTIL values must be specified in UTC when DTSTART '
ValueError: RRULE UNTIL values must be specified in UTC when DTSTART is timezone-aware
The solution is to calculate the UNTIL time in UTC and add Z to the end of the time string as described in the RFC:
https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html
the correct RRULE string should look like this:
cal_str = "BEGIN:VEVENT\nDTSTART;TZID=America/Los_Angeles:20171019T010000\nDTEND;TZID=America/Los_Angeles:20171019T230000\nRRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;UNTIL=20180423T001500Z\nX-OP-ENTRY-STATE:unlocked\nEND:VEVENT"
I've just started using pycassa, so if this is a stupid question, I apologize upfront.
I have a column family with the following schema:
create column family MyColumnFamilyTest
with column_type = 'Standard'
and comparator = 'CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.TimeUUIDType)'
and default_validation_class = 'BytesType'
and key_validation_class = 'UTF8Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and populate_io_cache_on_flush = false
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'};
When I try to do a get() with a valid key (works fine in cassandra-cli) I get:
Traceback (most recent call last):
File "<pyshell#19>", line 1, in <module>
cf.get('mykey',column_count=3)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 664, in get
return self._cosc_to_dict(list_col_or_super, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 368, in _cosc_to_dict
ret[self._unpack_name(col.name)] = self._col_to_dict(col, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 444, in _unpack_name
return self._name_unpacker(b)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 140, in unpack_composite
components.append(unpacker(bytestr[2:2 + length]))
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 374, in <lambda>
return lambda v: uuid.UUID(bytes=v)
File "/usr/lib/python2.7/uuid.py", line 144, in __init__
raise ValueError('bytes is not a 16-char string')
ValueError: bytes is not a 16-char string
Here's some more information I've discovered:
When using cassandra-cli I can see the data as:
% cassandra-cli -h 10.249.238.131
Connected to: "LocalDB" on 10.249.238.131/9160
Welcome to Cassandra CLI version 1.2.10-SNAPSHOT
Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.
[default#unknown] use Keyspace;
[default#Keyspace] list ColumnFamily;
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: urn:keyspace:ColumnFamily:a36e8ab1-7032-4e4c-a53d-e3317f63a640:
=> (name=autoZoning:::, value=01, timestamp=1391298393966000)
=> (name=creationTime:::, value=00000143efd8b76e, timestamp=1391298393966000)
=> (name=inactive:::14fe78e0-8b9b-11e3-b171-005056b700bb, value=00, timestamp=1391298393966000)
=> (name=label:::14fe78e0-8b9b-11e3-b171-005056b700bb, value=726a6d2d766e782d76613031, timestamp=1391298393966000)
1 Row Returned.
Elapsed time: 16 msec(s).
Since it was unclear what was causing the exception, I decided to add a print prior to the 'return self._name_unpacker(b)' line in columnfamily.py and I see:
>>> cf.get(dict(cf.get_range(column_count=0,filter_empty=False)).keys()[0])
Attempting to unpack: <00>\rautoZoning<00><00><00><00><00><00><00><00><00><00>
Traceback (most recent call last):
File "<pyshell#172>", line 1, in <module>
cf.get(dict(cf.get_range(column_count=0,filter_empty=False)).keys()[0])
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 665, in get
return self._cosc_to_dict(list_col_or_super, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 368, in _cosc_to_dict
ret[self._unpack_name(col.name)] = self._col_to_dict(col, include_timestamp, include_ttl)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 445, in _unpack_name
return self._name_unpacker(b)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 140, in unpack_composite
components.append(unpacker(bytestr[2:2 + length]))
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 374, in <lambda>
return lambda v: uuid.UUID(bytes=v)
File "/usr/lib/python2.7/uuid.py", line 144, in __init__
raise ValueError('bytes is not a 16-char string')
ValueError: bytes is not a 16-char string
I have no idea where the extra characters are coming from around the column name. But that got me curious so I added another print in _cosc_to_dict in columnfamily.py and I see:
>>> cf.get(dict(cf.get_range(column_count=0,filter_empty=False)).keys()[0])
list_col_or_super is: []
list_col_or_super is: [ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\rautoZoning\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', value='\x01', ttl=None),
counter_super_column=None, super_column=None, counter_column=None),
ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\x0ccreationTime\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
value='\x00\x00\x01C\xef\xd8\xb7n', ttl=None), counter_super_column=None, super_column=None,
counter_column=None), ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\x08inactive\x00\x00\x00\x00\x00\x00\x00\x00\x10\x14\xfex\xe0\x8b\x9b\x11\xe3\xb1q\x00PV\xb7\x00\xbb\x00', value='\x00', ttl=None), counter_super_column=None, super_column=None,
counter_column=None), ColumnOrSuperColumn(column=Column(timestamp=1391298393966000,
name='\x00\x05label\x00\x00\x00\x00\x00\x00\x00\x00\x10\x14\xfex\xe0\x8b\x9b\x11\xe3\xb1q\x00PV\xb7\x00\xbb\x00', value='thisIsATest', ttl=None), counter_super_column=None, super_column=None, counter_column=None)]
autoZoning unpack:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/columnfamily.py", line 666, in get
return self._cosc_to_dict(list_col_or_super, include_timestamp, include_ttl)
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/columnfamily.py", line 369, in _cosc_to_dict
ret[self._unpack_name(col.name)] = self._col_to_dict(col, include_timestamp, include_ttl)
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/columnfamily.py", line 446, in _unpack_name
return self._name_unpacker(b)
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/marshal.py", line 140, in unpack_composite
components.append(unpacker(bytestr[2:2 + length]))
File "/usr/local/lib64/python2.6/site-packages/pycassa-1.11.0-py2.6.egg/pycassa/marshal.py", line 374, in <lambda>
return lambda v: uuid.UUID(bytes=v)
File "/usr/lib64/python2.6/uuid.py", line 144, in __init__
raise ValueError('bytes is not a 16-char string')
ValueError: bytes is not a 16-char string
Am I correct in assuming that the extra characters around the column names are what is responsible for the 'ValueError: bytes is not a 16-char string' exception?
Also if I try to use the column name and select it I get:
>>> cf.get(u'urn:keyspace:ColumnFamily:a36e8ab1-7032-4e4c-a53d-e3317f63a640:',columns=['autoZoning:::'])
Traceback (most recent call last):
File "<pyshell#184>", line 1, in <module>
cf.get(u'urn:keyspace:ColumnFamily:a36e8ab1-7032-4e4c-a53d-e3317f63a640:',columns=['autoZoning:::'])
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 651, in get
cp = self._column_path(super_column, column)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 383, in _column_path
self._pack_name(column, False))
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/columnfamily.py", line 426, in _pack_name
return self._name_packer(value, slice_start)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 115, in pack_composite
packed = packer(item)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/marshal.py", line 298, in pack_uuid
randomize=True)
File "/usr/local/lib/python2.7/dist-packages/pycassa-1.11.0-py2.7.egg/pycassa/util.py", line 75, in convert_time_to_uuid
'neither a UUID, a datetime, or a number')
ValueError: Argument for a v1 UUID column name or value was neither a UUID, a datetime, or a number
Any further thoughts?
Thanks,
Rob
Turns out that the problem wasn't with the key, it was being caused, in part, by a bug in pycassa that wasn't handling an empty (null) string in the column UUID. A short-term fix is in the answer in google groups:
https://groups.google.com/d/msg/pycassa-discuss/Vf_bSgDIi9M/KTA1kbE9IXAJ
The other part of the answer was to get at the columns by using tuples (with the UUID as a UUID and not a str) instead of a string with ':' separators because that's, as I found out, a cassandra-cli thing.