We Keep Coding

Missing data for required field in Marshmallow 3.10.0

I am new to Marshmallow (3.10.0) and I am lost and need help trying to figure out what causes the following error:
AssertionError: ["Input Error - exten: ['Missing data for required field.']"]
The traceback of the error is the following:
Traceback (most recent call last):
File "/root/wazo_virtualenv_python37/lib/python3.7/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/root/wazo-confd/integration_tests/suite/helpers/wrappers.py", line 81, in decorated
result = func(*new_args, **kwargs)
File "/root/wazo-confd/integration_tests/suite/helpers/wrappers.py", line 81, in decorated
result = func(*new_args, **kwargs)
File "/root/wazo-confd/integration_tests/suite/base/test_call_filter_surrogate_user.py", line 216, in test_get_surrogates_callfilter_exten_when_disabled
confd.extensions.features(feature['id']).put({'enabled': False}).assert_updated()
File "/root/wazo-confd/integration_tests/suite/helpers/client.py", line 272, in assert_updated
self.assert_status(204)
File "/root/wazo-confd/integration_tests/suite/helpers/client.py", line 242, in assert_status
assert_that(self.response.status_code, is_in(statuses), self.response.text)
So it seems that the test function test_get_surrogates_callfilter_exten_when_disabled is failing:
def test_get_surrogates_callfilter_exten_when_disabled(call_filter, user):
response = confd.extensions.features.get(search="bsfilter")
feature = response.items[0]
---> (line 216 in traceback): confd.extensions.features(feature['id']).put({'enabled': False}).assert_updated()
with a.call_filter_surrogate_user(call_filter, user):
response = confd.callfilters(call_filter['id']).get()
assert_that(
response.item,
has_entries(
surrogates=has_entries(
users=contains(has_entries(exten=None, uuid=user['uuid']))
)
),
)
confd.extensions.features(feature['id']).put(
{'enabled': feature['enabled']}
).assert_updated()
the feature_extension schema is defined as the following:
class ExtensionFeatureSchema(BaseSchema):
id = fields.Integer(dump_only=True)
exten = fields.String(validate=Regexp(EXTEN_REGEX), required=True)
context = fields.String(dump_only=True)
feature = fields.String(attribute='typeval', dump_only=True)
enabled = fields.Boolean()
links = ListLink(Link('extensions_features'))
and the put function:
def put(self):
form = self.schema().load(request.get_json())
variables = [self.model(**option) for option in form]
self.service.edit(self.section_name, variables)
return '', 204
I have tried many solutions that I found online; but they did not fix the issue for me:
1 + pass partial=True to the load function:
form = self.schema().load(request.get_json(), partial=True)
2 + remove required=True from the field definition; this made the above error go away but failed many other tests that I have.
I am currently out of ideas, so if anyone has an idea how to fix the issue.

Discord.py Random errors from running bot?

This question is mostly code because I have no clue what is happening and how to fix it...
I will also provide the main file. Thanks!
Code (looks something like this; Microsoft Lens is good, but not great) :
import discord
from discord.ext.commands import CommandNotFound
from discord_slash import SlashCommand
def mixedCase(*args) :
total = []
for string in args:
a = map( ''. join,
itertools.product(*((c.upper(), c.lower( ))
for c in string) ) )
for x in list(a):
total.append(x)
return list(total)
client = commands.Bot(case_insensitive=True, command_prefix=mixedCase( "mc" ) )
slash = SlashCommand ( client, sync_commands=True, sync_on_cog_reload=True)
startTime = int(time.time( ))
client.main_startTime = startTime
class MyHelpCommand (commands.MinimalHelpCommand):
async def send_pages(self):
destination = self.get_destination( )
e = discord.Embed(color=discord.Color.green( ), description='')
for page in self.paginator. pages:
e.description += page
e.set_footer (text="Need help? Do 'mcsupport to join the support server!")
await destination.send(embed=e)
client.help_command = MyHelpCommand( )
client.run(TOKEN)
Error:
Traceback (most recent call last):
File "Bot.py", line 127, in <module> client. run( "TOKEN" )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 723, in run return future. result( )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 702, in runner await self. start( *args,** kwargs )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 666, in start await self. connect (reconnect=reconnect)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/client.py", line 566, in connect await self.ws. poll_event( )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/gateway.py", line 559, in poll_event await self. received_message(msg. data)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/gateway.py", line 509, in received_message func (data)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/state.py", line 490, in parse_message_crea te message = Message(channel=channel, data=data, state=self )
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/message. py", line 584, in _init ref. resolved = self.__class__(channel=chan, data=resol ved, state=state)
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/message.py", line 562, in _init self. stickers = [Sticker(data=data, state=state) for d ata in data. get( 'stickers', [])]
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/message.py", line 562, in <listcomp> self. stickers = [Sticker(data=data, state=state) for d ata in data. get( 'stickers', [])]
File "/home/quartz_i_warrior/.local/lib/python3.7/site-p ackages/discord/sticker.py", line 77, in _init_ self. pack_id = int(data['pack_id'])
KeyError: 'pack_id'
Extracted using Microsoft Lens, I'll try to correct missed words.

Looking at the output and traceback I can see that the error comes from within the discord module. When sticker.py is used, and the class within it is initialized, the pack_id attribute is instantiated with int(data['pack_id']) which should return the value from the dictionary data with key 'pack_id'. It seems that it cannot find such an item. The keyerror suggests that the key 'pack_id' cannot be found in the dictionary data. Usually, data.get('pack_id', 'default value') is better used in order to avoid problems (if value doesn't exist, just replaced with a default value. I am not sure where the issue comes from here, but I don't believe its due to your code.

virtual wx.ListCtrl raises wxAssertionError on SetItemCount

I want to create a custom ListCtrl "MyListCtrlCrafting" which pulls the data from a different class called "DBInterface" (actually it is not a real database, but a sophisticated python dictionary). There are tutorials on how to do that, I followed "Python in Action". The main points are: inherit from ListCtrl an set the style parameter to wx.LC_REPORT | wx.LC_SINGLE_SEL | wx.LC_VIRTUAL, call SetItemCount() during init() and override some methods.
To test how that works I made a small App which consists only of one (main-)frame, the virtual ListCtrl and a rudimentary version of DBInterface. Everything works out fine in this setting. But when I connect the classes of the real App, I get a Traceback:
Traceback (most recent call last):
File "DesktopCrafter.py", line 198, in <module>
controller = AppCtrl()
File "DesktopCrafter.py", line 186, in __init__
self.frame = GUI.MainWindow(back_end=self.back_end, crafting_controller=self.crafting_controller, parent=None, title='Desktop Crafter')
...
self.view = MyListCtrlCrafting(name='crafting-hand', back_end=self.db, crafting_controller=self.cc, parent=self, id=wx.ID_ANY)
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 107, in __init__
self.bindData()
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 121, in bindData
self.SetItemCount(count)
wx._core.wxAssertionError: C++ assertion "m_count == ListView_GetItemCount(GetHwnd())" failed at ..\..\src\msw\listctrl.cpp(3120) in wxListCtrl::SetItemCount(): m_count should match ListView_GetItemCount
In contrast to the simple App, the virtualListCtrl is now deeply nested. Can this error just be produced by wrong connections inside this nesting or between DBInterface and the ListCtrl? Or do I have to understand how m_count is calculated, to solve this error? If so, how can I read the _core file? I already read about ListCtrl in the core.py file, but it doesn't contain the relevant parts.
My problem with this traceback is that I don't understand why it is raised during SetItemCount(). This method should be something like a definition and since it is dealing with rows of a list, it should accept positive integers, and maybe 0, and maybe also -1 for standard. I plug in 5, so this cannot be the real problem going on here(?)
Any help or hint is much appreciated!
The complete Traceback:
Traceback (most recent call last):
File "DesktopCrafter.py", line 198, in <module>
controller = AppCtrl()
File "DesktopCrafter.py", line 186, in __init__
self.frame = GUI.MainWindow(back_end=self.back_end, crafting_controller=self.crafting_controller, parent=None, title='Desktop Crafter')
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 285, in __init__
self.InitUI()
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 299, in InitUI
self.panel = MainPanel(back_end=self.db, crafting_controller=self.cc)
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 251, in __init__
self.splitter = MySplitter(back_end=back_end, crafting_controller=crafting_controller)
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 229, in __init__
self.l_frame = LPanel(back_end=back_end, crafting_controller=crafting_controller)
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 188, in __init__
self.panel = CraftingPanel(back_end=back_end, crafting_controller=crafting_controller, *args, **kwargs)
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 154, in __init__
self.view = MyListCtrlCrafting(name='crafting-hand', back_end=self.db, crafting_controller=self.cc, parent=self, id=wx.ID_ANY)
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 107, in __init__
self.bindData()
File "d:\workspace-fun\DesktopCrafter\dc\util\DCUI.py", line 121, in bindData
self.SetItemCount(count)
wx._core.wxAssertionError: C++ assertion "m_count == ListView_GetItemCount(GetHwnd())" failed at ..\..\src\msw\listctrl.cpp(3120) in wxListCtrl::SetItemCount(): m_count should match ListView_GetItemCount
The virtual ListCtrl (both print's give me the expected results):
class MyListCtrlCrafting(wx.ListCtrl):
def __init__(self, name, back_end, crafting_controller, *args, **kwargs):
wx.ListCtrl.__init__(self, style=wx.LC_REPORT | wx.LC_SINGLE_SEL | wx.LC_VIRTUAL, *args, **kwargs)
self.name = name
self.db = back_end
self.cc = crafting_controller
#print(self.db.retrieveValue(['player', self.name]))
self.Bind(wx.EVT_LIST_CACHE_HINT, self.DoCacheItems)
self.bindData()
self.InsertColumn(0, "Slot")
self.InsertColumn(1, "Item")
self.InsertColumn(2, "Amount")
print("initialized MyListCtrl")
def bindData(self):
data = self.db.retrieveValue(['player', self.name])
count = len(data)
#print(count)
self.itemDataMap = {i: ('slot'+str(i+1), data[str(i)]['item'], data[str(i)]['amount']) for i in range(count)}
self.SetItemCount(count)
def DoCacheItems(self, e):
self.db.updateCache(e.GetCacheFrom(), e.GetCacheTo())
def OnGetItemText(self, row_no, col_no):
data = self.db.retrieveValue(['player', self.name, str(row_no)])
return data[col_no]
def OnGetItemAttr(self, item): return None
def OnGetItemImage(self, item): return -1
def OnBackEndUpdated(self):
self.bindData()
self.RefreshItems(0, self.GetItemCount())
The DBInterface:
class DBInterface(dict):
def __init__(self, path, *args, **kwargs):
super(DBInterface, self).__init__(*args, **kwargs)
self.path = path
def load(self):
with open(self.path, 'r') as f:
try:
self.update(json.loads(f.read()))
except Exception as e:
print(e); print(self.path)
def save(self):
self.path.ensure()
with open(self.path, 'w') as f:
f.write(json.dumps(self))
def retrieveValue(self, path):
a = self
for i in range(len(path)):
a = a[path[i]]
return a

That assert is basically a check to ensure that the operation (setting the count) was successful, by asking the native control how many items it has (which should have been set by the prior API call. It seems like something that really shouldn't be able to fail, although obviously it is. Perhaps something is resetting or changing the native control somehow? Are there any threads involved?
I don't have a solid answer for you, but as advice I would suggest taking the small sample that works and build it up step by step until it matches the window hierarchy and environment and basic features of that part of the full application where it is broken. Eventually it should start failing the same way and then you'll know what changed that triggered it and can look more closely at that.

How to extend `psycopg2.extras.DictCursor`?

I want to use a python3-psycopg2 (2.5.4) cursor that can:
1) access query result in a dict-like manner
2) log every sql executed by this cursor with logging module automatically
I tried the following code, it didn't work, what did I do wrong? I can extend psycopg2.extensions.cursor, but that would not allow me to do 1).
class LoggingCursor(psycopg2.extras.DictCursor):
def __init__(self, *args, **kwargs): # it didn't work with or
super().__init__(*args, **kwargs) # without these 2 lines
def execute(self, sql, args=None):
from beautiful import app
# logger = blah blah
logger.debug(self.mogrify(sql, args).decode())
psycopg2.extensions.cursor.execute(self, sql, args)
cursor = db_conn.cursor(cursor_factory=LoggingCursor)
When I cursor.execute(some_sql), it gives me:
File "some_file.py", line 123, in some_function
some_var = cursor.fetchone()
File "/usr/local/lib/python3.4/dist-packages/psycopg2/extras.py", line 63, in fetchone
res = super(DictCursorBase, self).fetchone()
File "/usr/local/lib/python3.4/dist-packages/psycopg2/extras.py", line 139, in __init__
self._index = cursor.index
AttributeError: 'LoggingCursor' object has no attribute 'index'

How about changing
psycopg2.extensions.cursor.execute(self, sql, args)
to
super().execute(sql, args)
so that the base class execute() method is called?

Django 1.2 + South 0.7 + django-annoying's AutoOneToOneField leads to TypeError: 'LegacyConnection' object is not iterable

I'm using Django 1.2 trunk with South 0.7 and an AutoOneToOneField copied from django-annoying. South complained that the field does not have rules defined and the new version of South no longer has an automatic field type parser. So I read the South documentation and wrote the following definition (basically an exact copy of the OneToOneField rules):
rules = [
(
(AutoOneToOneField),
[],
{
"to": ["rel.to", {}],
"to_field": ["rel.field_name", {"default_attr": "rel.to._meta.pk.name"}],
"related_name": ["rel.related_name", {"default": None}],
"db_index": ["db_index", {"default": True}],
},
)
]
from south.modelsinspector import add_introspection_rules
add_introspection_rules(rules, ["^myapp"])
Now South raises the following error when I do a schemamigration.
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "django/core/management/base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "django/core/management/base.py", line 223, in execute
output = self.handle(*args, **options)
File "South-0.7-py2.6.egg/south/management/commands/schemamigration.py", line 92, in handle
(k, v) for k, v in freezer.freeze_apps([migrations.app_label()]).items()
File "South-0.7-py2.6.egg/south/creator/freezer.py", line 33, in freeze_apps
model_defs[model_key(model)] = prep_for_freeze(model)
File "South-0.7-py2.6.egg/south/creator/freezer.py", line 65, in prep_for_freeze
fields = modelsinspector.get_model_fields(model, m2m=True)
File "South-0.7-py2.6.egg/south/modelsinspector.py", line 322, in get_model_fields
args, kwargs = introspector(field)
File "South-0.7-py2.6.egg/south/modelsinspector.py", line 271, in introspector
arg_defs, kwarg_defs = matching_details(field)
File "South-0.7-py2.6.egg/south/modelsinspector.py", line 187, in matching_details
if any([isinstance(field, x) for x in classes]):
TypeError: 'LegacyConnection' object is not iterable
Is this related to a recent change in Django 1.2 trunk? How do I fix this?
I use this field as follows:
class Bar(models.Model):
foo = AutoOneToOneField("foo.Foo", primary_key=True, related_name="bar")
For reference the field code from django-tagging:
class AutoSingleRelatedObjectDescriptor(SingleRelatedObjectDescriptor):
def __get__(self, instance, instance_type=None):
try:
return super(AutoSingleRelatedObjectDescriptor, self).__get__(instance, instance_type)
except self.related.model.DoesNotExist:
obj = self.related.model(**{self.related.field.name: instance})
obj.save()
return obj
class AutoOneToOneField(OneToOneField):
def contribute_to_related_class(self, cls, related):
setattr(cls, related.get_accessor_name(), AutoSingleRelatedObjectDescriptor(related))

Try to change this line
(AutoOneToOneField),
To this:
(AutoOneToOneField,),
A tuple declared like you did, is not iterable.

Solved the problem by removing the rules and adding the following method to AutoOneToOneField:
def south_field_triple(self):
"Returns a suitable description of this field for South."
from south.modelsinspector import introspector
field_class = OneToOneField.__module__ + "." + OneToOneField.__name__
args, kwargs = introspector(self)
return (field_class, args, kwargs)

Your rule have simple python related problem.. In tuple, you must add comma if only single item inside.
So change (AutoOneToOneField), to (AutoOneToOneField,),
But to be honest, i didn't know that I can use method inside field instead of rules. I will apply your patch and submit to django-annoying repository.