We Keep Coding

Python Get Account using tda-api returns ValueError

I have the following api call to tda-api
orders = client.get_account(config.account_id,fields=['positions'])
Gives the Error:
File "/opt/anaconda3/lib/python3.7/site-packages/tda/client/base.py", line 361, in get_account
fields = self.convert_enum_iterable(fields, self.Account.Fields)
File "/opt/anaconda3/lib/python3.7/site-packages/tda/utils.py", line 66, in convert_enum_iterable
self.type_error(value, required_enum_type)
File "/opt/anaconda3/lib/python3.7/site-packages/tda/utils.py", line 41, in type_error
possible_members_message))
ValueError: expected type "Fields", got type "str". (initialize with enforce_enums=False to disable this checking)
the documentation follows:
Client.get_account(account_id, *, fields=None)
and if i replace with:
client.get_account(config.account_id,fields=positions)
'positions' is not defined
And if i look into the api the code for the get_account() function looks like:
class Fields(Enum):
'''Account fields passed to :meth:`get_account` and
:meth:`get_accounts`'''
POSITIONS = 'positions'
ORDERS = 'orders'
def get_account(self, account_id, *, fields=None):
fields = self.convert_enum_iterable(fields, self.Account.Fields)
params = {}
if fields:
params['fields'] = ','.join(fields)

Most likely not the correct way to go about this but I found a fix for now.
I commented out the following that checks for the type in the utils.py file for the tda package.
def convert_enum_iterable(self, iterable, required_enum_type):
if iterable is None:
return None
if isinstance(iterable, required_enum_type):
return [iterable.value]
values = []
for value in iterable:
if isinstance(value, required_enum_type):
values.append(value.value)
# elif self.enforce_enums:
# self.type_error(value, required_enum_type)
else:
values.append(value)
return values
I believe if you create your own client you can set this property to false as well.
And then I am able to use the following to get my current positions:
data = client.get_account(config.account_id,fields=['positions'])

At one point, I had these working but haven't tried in a while. I do recall it being difficult to decipher what a field type was. Depending on your import statements, you might need the entire tda.client.Client.Account.Fields.POSITIONS, for example.
r_acct_orders = client.get_account(config.ACCOUNT_ID, fields=tda.client.Client.Account.Fields.ORDERS).json() # field options None, ORDERS, POSITIONS
r = client.get_account(config.ACCOUNT_ID, fields=tda.client.Client.Account.Fields.POSITIONS).json() # field options None, ORDERS, POSITIONS
r = client.get_accounts(fields=tda.client.Client.Account.Fields.ORDERS).json() # field options None, ORDERS, POSITIONS
r = client.get_accounts(fields=None).json() # field options None, ORDERS, POSITIONS
Also, I use a config.py for account_id but you can just use your syntax as needed.

Creating namedtuple valid for differents parameters

I'm trying to figure it out a way to create a namedtuple with variable fields depending on the data you receive, in my case, I'm using the data from StatCounter and not on all the periods are the same browsers. I tried this way but it is a bit ugly and I'm sure there is a better way to achieve it.
def namedtuple_fixed(name: str, fields: List[str]) -> namedtuple:
"""Check the fields of the namedtuple and changes the invalid ones."""
fields_fixed: List[str] = []
for field in fields:
field = field.replace(" ", "_")
if field[0].isdigit():
field = f"n{field}"
fields_fixed.append(field)
return namedtuple(name, fields_fixed)
Records: namedtuple = namedtuple("empty_namedtuple", "")
def read_file(file: str) -> List["Records"]:
"""
Read the file with info about the percentage of use of various browsers
"""
global Records
with open(file, encoding="UTF-8") as browsers_file:
reader: Iterator[List[str]] = csv.reader(browsers_file)
field_names: List[str] = next(reader)
Records = namedtuple_fixed("Record", field_names)
result: List[Records] = [
Records(
*[
dt.datetime.strptime(n, "%Y-%m").date()
if record.index(n) == 0
else float(n)
for n in record
]
)
for record in reader
]
return result
The "namedtuple_fixed" function is to fix the names that have invalid identifiers.
Basically, I want to create a named tuple that receives a variable number of parameters, depending on the file you want to analyze. And if it's with type checking incorporated (I mean using NamedTuple from the typing module), much better.
Thanks in advance.

This solves my problem, but just partially
class Record(SimpleNamespace):
def __repr__(self):
items = [f"{key}={value!r}" for key, value in self.__dict__.items()]
return f"Record({', '.join(items)})"
Using the types.SimpleSpace documentation
And it can cause problems, like for example if you initiallize a Record like the following:
foo = Record(**{"a": 1, "3a": 2})
print(foo.a) # Ok
print(foo.3a) # Syntax Error

Django ORM, how to use values() and still work with choicefield?

I am using django v1.10.2
I am trying to create dynamic reports whereby I store fields and conditions and the main ORM model information into database.
My code for the generation of the dynamic report is
class_object = class_for_name("app.models", main_model_name)
results = (class_object.objects.filter(**conditions_dict)
.values(*display_columns)
.order_by(*sort_columns)
[:50])
So main_model_name can be anything.
This works great except that sometimes associated models of the main_model have choicefield.
So for one of the reports main_model is Pallet.
Pallet has many PalletMovement.
My display columns are :serial_number, created_at, pallet_movement__location
The first two columns are fields that belong to Pallet model.
The last one is from PalletMovement
What happens is that PalletMovement model looks like this:
class PalletMovement(models.Model):
pallet = models.ForeignKey(Pallet, related_name='pallet_movements',
verbose_name=_('Pallet'))
WAREHOUSE_CHOICES = (
('AB', 'AB-Delaware'),
('CD', 'CD-Delaware'),
)
location = models.CharField(choices=WAREHOUSE_CHOICES,
max_length=2,
default='AB',
verbose_name=_('Warehouse Location'))
Since the queryset will return me the raw values, how can I make use of the choicefield in PalletMovement model to ensure that the pallet_movement__location gives me the display of AB-Delaware or CD-Delaware?
Bear in mind that the main_model can be anything depending on what I store in the database.
Presumably, I can store more information in the database to help me do the filtering and presentation of data even better.

The values() method returns a dictionary of key-value pairs representing your field name and a corresponding value.
For example:
Model:
class MyModel(models.Model):
name = models.CharField()
surname = models.CharField()
age = models.IntegerField()
...
Query:
result = MyModel.objects.filter(surname='moutafis').values('name', 'surname')
Result:
< Queryset [{'name': 'moutafis', 'surname': 'john'}] >
You can now manipulate this result as you would a normal dictionary:
if main_model_name is 'PalletMovement':
# Make life easier
choices = dict(PalletMovement.WAREHOUSE_CHOICES)
for item in result:
item.update({
pallet_movement__location: verbal_choice.get(
pallet_movement__location, pallet_movement__location)
})
You can even make this into a function for better re-usability:
def verbalize_choices(choices_dict, queryset, search_key):
result = queryset
for item in result:
item.update({ search_key: choices_dict.get(search_key, search_key) })
return result
verbal_result = verbalize_choices(
dict(PalletMovement.WAREHOUSE_CHOICES),
result,
'pallet_movement__location'
)
I suggest the use of the update() and get() methods because they will save you from potential errors, like:
The search_key does not exist in the choice_dict then get() will return the value of the search_key
update() will try to update the given key-value pair if exists, else it will add it to the dictionary.
If the usage of the above will be in the template representation of your data, you can create a custom template filter instead:
#register.filter(name='verbalize_choice')
def choice_to_verbal(choice):
return dict(PalletMovement.WAREHOUSE_CHOICES)[choice]
Have an extra look here: Django: How to access the display value of a ChoiceField in template given the actual value and the choices?

You would use get_foo_display
In your template:
{{ obj.get_location_display }}
or
{{ obj.pallet_movement.get_location_display }}
[Edit:] As pointed out in the comments this will not work when calling values()

an alternative to create a templatetag is :
{{form.choicefield.1}}
This shows the value of the initial data of the foreign key field instead the id.

The universal solution for any main_model_name is by Django Model _meta API introspection: class_object._meta.get_field(field_name).choices
That is:
choice_dicts = {}
for field_name in display_columns:
choice_dicts[field_name] = {
k: v for k, v in class_object._meta.get_field(field_name).choices
}
out = []
for row in results:
out.append({name: choice_dicts[name].get(value, value)
for name, value in row.items()
})
The rest is a trivial example, mostly copied code from the question
>>> pallet = app.models.Pallet.objects.create()
>>> palletm = app.models.PalletMovement.objects.create(pallet=pallet, location='AB')
>>>
>>> main_model_name = 'PalletMovement'
>>> conditions_dict = {}
>>> display_columns = ['pallet_id', 'location']
>>> sort_columns = []
>>>
>>> class_object = class_for_name("app.models", main_model_name)
>>> results = (class_object.objects.filter(**conditions_dict)
... .values(*display_columns)
... .order_by(*sort_columns)
... )[:50]
>>>
>>> # *** INSERT HERE ALL CODE THAT WAS ABOVE ***
>>>
>>> print(out)
[{'location': 'AB-Delaware', 'pallet_id': 1}]
It works equally with 'pallet_id' or with 'pallet' in display_columns. Even that "_meta" starts with underscore, it is a documented API.

is it possible to add a button with variable in templatetags

templatetags.py
from django import template
from django.utils.safestring import mark_safe
register = template.Library()
#register.filter("as_span")
def as_span(ZergitForm):
ZergitForm_as_span = ZergitForm.as_ul().replace("<ul", "<span").replace("</ul", "</span")
ZergitForm_as_span = ZergitForm_as_span.replace("<li", "<span").replace("</li", "</span")
return mark_safe(ZergitForm_as_span)
I am using MultipleChoiceField.After using this templatetag it is printing the form data in a span instead of <li> tag.I want to do an delete operation for each data inside the span.Now it is printing each <li> value in individual <span> tag.I need to insert a input button for each span.
Is it possible to do using templatetag concept.
Thanks

You could do something similar when rendering the field instead of the whole form.
If you build a templatetag to render a field you can use its id which you need to adress an ajax-call to delete it.

You are better off replacing those fields' widget. It seems like you are using a CheckboxSelectMultiple, which uses <ul> and <li>. Create a class that descends from CheckboxSelectMultiple and replace its render() method:
(from django/forms/widgets.py)
class CheckboxSelectMultiple(SelectMultiple):
def render(self, name, value, attrs=None, choices=()):
if value is None: value = []
has_id = attrs and 'id' in attrs
final_attrs = self.build_attrs(attrs, name=name)
output = [u'<ul>']
# Normalize to strings
str_values = set([force_unicode(v) for v in value])
for i, (option_value, option_label) in enumerate(chain(self.choices, choices)):
# If an ID attribute was given, add a numeric index as a suffix,
# so that the checkboxes don't all have the same ID attribute.
if has_id:
final_attrs = dict(final_attrs, id='%s_%s' % (attrs['id'], i))
label_for = u' for="%s"' % final_attrs['id']
else:
label_for = ''
cb = CheckboxInput(final_attrs, check_test=lambda value: value in str_values)
option_value = force_unicode(option_value)
rendered_cb = cb.render(name, option_value)
option_label = conditional_escape(force_unicode(option_label))
output.append(u'<li><label%s>%s %s</label></li>' % (label_for, rendered_cb, option_label))
output.append(u'</ul>')
return mark_safe(u'\n'.join(output))

ndb.Key filter for MapReduce input_reader

Playing with new Google App Engine MapReduce library filters for input_reader I would like to know how can I filter by ndb.Key.
I read this post and I've played with datetime, string, int, float, in filters tuples, but How I can filter by ndb.Key?
When I try to filter by a ndb.Key I get this error:
BadReaderParamsError: Expected Key, got u"Key('Clients', 406)"
Or this error:
TypeError: Key('Clients', 406) is not JSON serializable
I tried to pass a ndb.Key object and string representation of the ndb.Key.
Here are my two filters tuples:
Sample 1:
input_reader': {
'input_reader': 'mapreduce.input_readers.DatastoreInputReader',
'entity_kind': 'model.Sales',
'filters': [("client","=", ndb.Key('Clients', 406))]
}
Sample 2:
input_reader': {
'input_reader': 'mapreduce.input_readers.DatastoreInputReader',
'entity_kind': 'model.Sales',
'filters': [("client","=", "%s" % ndb.Key('Clients', 406))]
}

This is a bit tricky.
If you look at the code on Google Code you can see that mapreduce.model defines a JSON_DEFAULTS dict which determines the classes that get special-case handling in JSON serialization/deserialization: by default, just datetime. So, you can monkey-patch the ndb.Key class into there, and provide it with functions to do that serialization/deserialization - something like:
from mapreduce import model
def _JsonEncodeKey(o):
"""Json encode an ndb.Key object."""
return {'key_string': o.urlsafe()}
def _JsonDecodeKey(d):
"""Json decode a ndb.Key object."""
return ndb.Key(urlsafe=d['key_string'])
model.JSON_DEFAULTS[ndb.Key] = (_JsonEncodeKey, _JsonDecodeKey)
model._TYPE_IDS['Key'] = ndb.Key
You may also need to repeat those last two lines to patch mapreduce.lib.pipeline.util as well.
Also note if you do this, you'll need to ensure that this gets run on any instance that runs any part of a mapreduce: the easiest way to do this is to write a wrapper script that imports the above registration code, as well as mapreduce.main.APP, and override the mapreduce URL in your app.yaml to point to your wrapper.

Make your own input reader based on DatastoreInputReader, which knows how to decode key-based filters:
class DatastoreKeyInputReader(input_readers.DatastoreKeyInputReader):
"""Augment the base input reader to accommodate ReferenceProperty filters"""
def __init__(self, *args, **kwargs):
try:
filters = kwargs['filters']
decoded = []
for f in filters:
value = f[2]
if isinstance(value, list):
value = db.Key.from_path(*value)
decoded.append((f[0], f[1], value))
kwargs['filters'] = decoded
except KeyError:
pass
super(DatastoreKeyInputReader, self).__init__(*args, **kwargs)
Run this function on your filters before passing them in as options:
def encode_filters(filters):
if filters is not None:
encoded = []
for f in filters:
value = f[2]
if isinstance(value, db.Model):
value = value.key()
if isinstance(value, db.Key):
value = value.to_path()
entry = (f[0], f[1], value)
encoded.append(entry)
filters = encoded
return filters

Are you aware of the to_old_key() and from_old_key() methods?

I had the same problem and came up with a workaround with computed properties.
You can add to your Sales model a new ndb.ComputedProperty with the Key id. Ids are just strings, so you wont have any JSON problems.
client_id = ndb.ComputedProperty(lambda self: self.client.id())
And then add that condition to your mapreduce query filters
input_reader': {
'input_reader': 'mapreduce.input_readers.DatastoreInputReader',
'entity_kind': 'model.Sales',
'filters': [("client_id","=", '406']
}
The only drawback is that Computed properties are not indexed and stored until you call the put() parameter, so you will have to traverse all the Sales entities and save them:
for sale in Sales.query().fetch():
sale.put()