How does one return a dict like object through protoRPC ?
I tried using the FieldList to no avail. I only see the following field definitions:
'IntegerField',
'FloatField',
'BooleanField',
'BytesField',
'StringField',
'MessageField',
'EnumField',
There are two scenarios:
1) Your dict has a well-defined schema: This is the best use case for ProtoRPC and if possible you should try to fit it into a schema. In this case, you would use a MessageField with some Message class that matches the schema in your dictionary.
For example, instead of
{'amount': 31, 'type': 'fish', mine: False}
you could define
from protorpc import messages
class MyCatch(messages.Message):
amount = messages.IntegerField(1)
type = messages.StringField(2)
mine = messages.BooleanField(3)
and then use this message definition in a field via
messages.MessageField(MyCatch, index, ...)
2) Your dict does not have a well-defined schema: In this case you can use json to dump your dictionary to a string and request ensure_ascii=True to make sure the return type is a bytes (str) object. Then you can just use a BytesField.
For example:
import json
class MyMessage(messages.Message):
some_dict = messages.BytesField(1)
my_dict = {'amount': 31, 'type': 'fish', mine: False}
message = MyMessage(some_dict=json.dumps(my_dict, ensure_ascii=True))
The use of ensure_ascii is optional as True is the default, but this may change depending on your environment.
Instead you could use pickle to serialize your dictionary. The method pickle.dumps always outputs ASCII/binary, so by swapping out json.dumps for pickle.dumps and dropping the ensure_ascii=True, you'd have the same outcome.
It's possible to create a custom JsonField like this :
In [1]: class JsonField(messages.StringField):
type = dict
You can then use it as any other field :
In [2]: class MyMessage(messages.Message):
data = JsonField(1)
In [3]: m = MyMessage(data={"foo": "bar"})
In [4]: m.data
Out [4]: {'foo': 'bar'}
For the first option in the approved answer, we can add a parameter repeated=True, so we'll have a list of json as the answer. I checked about it at https://developers.google.com/appengine/docs/python/tools/protorpc/overview?hl=en#Defining_the_Response_Message
A bit involved, but I have a recipe for something quite close to a dict implementation for protorpc: https://gist.github.com/linuxluser/32d4a9c36ca0b8715ad4
It is restricted to using string-only keys and simple (not nested) values. But if your data fits in that category, this solution should work well.
The idea has 2 parts:
Create a new field type MultiField that can hold an arbitrary value type.
Create a dict-like type MapField that stores key-value pairs in a list of MultiField types.
You use it like so:
import messages
import mapfield
class MyMessage(messages.Message):
some_dict = mapfield.MapField(1)
my_message = MyMessage(some_dict={"foo": 7, "bar": False, "baz": 9.2, "qux": "nog"})
It's only a start. Probably could be better. Improvements are welcomed. :)
Related
If I have a dictionary that is nested, and I pass in a string like "key1.key2.key3" which would translate to:
myDict["key1"]["key2"]["key3"]
What would be an elegant way to be able to have a method where I could pass on that string and it would translate to that key assignment? Something like
myDict.set_nested('key1.key2.key3', someValue)
Using only builtin stuff:
def set(my_dict, key_string, value):
"""Given `foo`, 'key1.key2.key3', 'something', set foo['key1']['key2']['key3'] = 'something'"""
# Start off pointing at the original dictionary that was passed in.
here = my_dict
# Turn the string of key names into a list of strings.
keys = key_string.split(".")
# For every key *before* the last one, we concentrate on navigating through the dictionary.
for key in keys[:-1]:
# Try to find here[key]. If it doesn't exist, create it with an empty dictionary. Then,
# update our `here` pointer to refer to the thing we just found (or created).
here = here.setdefault(key, {})
# Finally, set the final key to the given value
here[keys[-1]] = value
myDict = {}
set(myDict, "key1.key2.key3", "some_value")
assert myDict == {"key1": {"key2": {"key3": "some_value"}}}
This traverses myDict one key at a time, ensuring that each sub-key refers to a nested dictionary.
You could also solve this recursively, but then you risk RecursionError exceptions without any real benefit.
There are a number of existing modules that will already do this, or something very much like it. For example, the jmespath module will resolve jmespath expressions, so given:
>>> mydict={'key1': {'key2': {'key3': 'value'}}}
You can run:
>>> import jmespath
>>> jmespath.search('key1.key2.key3', mydict)
'value'
The jsonpointer module does something similar, although it likes / for a separator instead of ..
Given the number of pre-existing modules I would avoid trying to write your own code to do this.
EDIT: OP's clarification makes it clear that this answer isn't what he's looking for. I'm leaving it up here for people who find it by title.
I implemented a class that did this a while back... it should serve your purposes.
I achieved this by overriding the default getattr/setattr functions for an object.
Check it out! AndroxxTraxxon/cfgutils
This lets you do some code like the following...
from cfgutils import obj
a = obj({
"b": 123,
"c": "apple",
"d": {
"e": "nested dictionary value"
}
})
print(a.d.e)
>>> nested dictionary value
I am currently writing a script that extracts data from an xml and writes it into an html file for easy viewing on a webpage.
Each piece of data has 2 pieces of "sub data": Owner and Type.
In order for the html to work properly I need the "owner" string and the "type" string to be written in the correct place. If it was just a single piece of data then I would use dictionaries and just use the data name as the key and then write the value to html, however there are 2 pieces of data.
My question is, can a dictionary have 2 values (in my case owner and type) assigned to a single key?
Any object can be a value in a dictionary, so you can use any collection to hold more than one value against the same key. To expand my comments into some code samples, in order of increasing complexity (and, in my opinion, readability):
Tuple
The simplest option is a two-tuple of strings, which you can access by index:
>>> d1 = {'key': ('owner', 'type')}
>>> d1['key'][0]
'owner'
>>> d1['key'][1]
'type'
Dictionary
Next up is a sub-dictionary, which allows you to access the values by key name:
>>> d2 = {'key': {'owner': 'owner', 'type': 'type'}}
>>> d2['key']['owner']
'owner'
>>> d2['key']['type']
'type'
Named tuple
Finally the collections module provides namedtuple, which requires a little setup but then allows you to access the values by attribute name:
>>> from collections import namedtuple
>>> MyTuple = namedtuple('MyTuple', ('owner', 'type'))
>>> d3 = {'key': MyTuple('owner', 'type')}
>>> d3['key'].owner
'owner'
>>> d3['key'].type
'type'
Using named keys/attributes makes your subsequent access to the values clearer (d3['key'].owner and d2['key']['owner'] are less ambiguous than d1['key'][0]).
As long as keys are hash-able you can have keys of any format. Note, tuples are hash-able so that would be a possible solution to your problem
Make a tuple of case-owner and type and use it as a key to your dictionary.
Note, generally all objects that are hashable should also be immutable, but not vice-versa. So
I have a list of python dictionaries called checkout_items, created by a simple function (further simplified here, for ease of reading):
def checkout_items(request):
items = get_cart_items(request)
co_items = [] #a list of dictionaries to be used by the cart
#iterate through the items and homogenize them into a standardized checkout_item format
for i in items:
co_item = {'number': num,
'name': i.name,
'sku': i.sku,
'quantity': i.quantity,
'price': i.price}
I reference this list elsewhere in a view (again, simplified):
checkout_items = cart.checkout_items(request)
attributes = {}
for i in checkout_items:
attributes['item_name_'+ str(i['number'])] = str(i['name'])
attributes['item_number_'+ str(i['number'])] = str(i['sku'])
attributes['quantity_'+ str(i['number'])] = str(i['quantity'])
But the name variable is being set to this: <bound method CartItem.name of <CartItem: CartItem object>
Yet the sku (alphanumeric string, just like "name") comes through just fine, it seems. Both are coming directly from MySQL. Any ideas what's going on?
<bound method CartItem.name of <CartItem: CartItem object>
is an error usually given when you are calling to see if a method is there, not actually calling the method itself. I would try adding () so i.name()
as you say, the data you give to the function checkout_items() is from get_cart_items().
So items is not a stupid namespace-like object, but an object that has methods that you wrote, and all of them are CartItems.
Then when you build co_item, you give i.name to the dictionary, that is later printed out as bound method CartItem.name of <CartItem: CartItem object>.
It looks like your CartItem object has a method name! You should try i.name(). Or use whatever property that really has the name if name() is something else.
How does one pre-populate a Formish form? The obvious method as per the documentation doesn't seem right. Using one of the provided examples:
import formish, schemaish
structure = schemaish.Structure()
structure.add( 'a', schemaish.String() )
structure.add( 'b', schemaish.Integer() )
schema = schemaish.Structure()
schema.add( 'myStruct', structure )
form = formish.Form(schema, 'form')
If we pass this a valid request object:
form.validate(request)
The output is a structure like this:
{'myStruct': {'a': 'value', 'b': 0 }}
However, pre-populating the form using defaults requires this:
form.defaults = {'myStruct.a': 'value', 'myStruct.b': 0}
The dottedish package has a DottedDict object that can convert a nested dict to a dotted dict, but this asymmetry doesn't seem right. Is there a better way to do this?
No, don't require to use dotted dict, you can easily use the post-validate style dict to pre-populate the form:
form.defaults={'myStruct': {'a': None, 'b': 'default_value'}}
maybe have old version of formish, try update the libs.
Imagine you got an entity in the Google App Engine datastore, storing links for anonymous users.
You would like to perform the following SQL query, which is not supported:
SELECT DISTINCT user_hash FROM links
Instead you could use:
user = db.GqlQuery("SELECT user_hash FROM links")
How to use Python most efficiently to filter the result, so it returns a DISTINCT result set?
How to count the DISTINCT result set?
Reviving this question for completion:
The DISTINCT keyword has been introduced in release 1.7.4.
You can find the updated GQL reference (for example for Python) here.
A set is good way to deal with that:
>>> a = ['google.com', 'livejournal.com', 'livejournal.com', 'google.com', 'stackoverflow.com']
>>> b = set(a)
>>> b
set(['livejournal.com', 'google.com', 'stackoverflow.com'])
>>>
One suggestion w/r/t the first answer, is that sets and dicts are better at retrieving unique results quickly, membership in lists is O(n) versus O(1) for the other types, so if you want to store additional data, or do something like create the mentioned unique_results list, it may be better to do something like:
unique_results = {}
>>> for item in a:
unique_results[item] = ''
>>> unique_results
{'livejournal.com': '', 'google.com': '', 'stackoverflow.com': ''}
One option would be to put the results into a set object:
http://www.python.org/doc/2.6/library/sets.html#sets.Set
The resulting set will consist only of the distinct values passed into it.
Failing that, building up a new list containing only the unique objects would work. Something like:
unique_results = []
for obj in user:
if obj not in unique_results:
unique_results.append(obj)
That for loop can be condensed into a list comprehension as well.
Sorry to dig this question up but in GAE I cannot compare objects like that, I must use .key() for comparison like that:
Beware, this is very inefficient :
def unique_result(array):
urk={} #unique results with key
for c in array:
if c.key() not in urwk:
urk[str(c.key())]=c
return urk.values()
If anyone has a better solution, please share.