I'm using nobonobo's python bindings to unqlite and am running into an issue when attempting to work with a JSON document collection.
In the README, there is this JX9 script:
sample = (
"db_create('users'); /* Create the collection users */"
"db_store('users',{ 'name' : 'dean' , 'age' : 32 });"
"db_store('users',{ 'name' : 'chems' , 'age' : 27 });"
"print db_fetch_all('users')..'\n';"
"while( ($rec = db_fetch('users')) != NULL ){"
" print $rec; print '\n';"
"}"
)
This correctly prints each record:
[{"name":"dean","age":32,"__id":0},{"name":"chems","age":27,"__id":1}]
{"name":"dean","age":32,"__id":0}
{"name":"chems","age":27,"__id":1}
However when I try to read the collection in Python using a callback, I get garbage back:
#unqlitepy.OutputCallback
def f(output, outlen, udata):
output = (c_char*outlen).from_address(output).raw
print locals()
return unqlitepy.UNQLITE_OK
db.fetch_cb('users', f)
This is the output:
{'udata': None, 'output': 'a\x1e\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x02D\xa7\x83\x0b', 'outlen': 22L}
Similarly if I grab a cursor and print the first user in the users collection, I get this:
'users_0' '\x01\x08\x00\x00\x00\x04name\x05\x08\x00\x00\x00\x04dean\x06\x08\x00\x00\x00\x03age\x05\n\x00\x00\x00\x00\x00\x00\x00 \x06\x08\x00\x00\x00\x04__id\x05\n\x00\x00\x00\x00\x00\x00\x00\x00\x06\x02'
Does anybody know what might be happening? Is there some way to decode the data returned to python?
I wrote some new bindings which make all of this way easier: https://github.com/coleifer/unqlite-python
>>> users.store([
... {'name': 'Charlie', 'color': 'green'},
... {'name': 'Huey', 'color': 'white'},
... {'name': 'Mickey', 'color': 'black'}])
True
>>> users.store({'name': 'Leslie', 'color': 'also green'})
True
>>> users.fetch(0) # Fetch the first record.
{'__id': 0, 'color': 'green', 'name': 'Charlie'}
>>> users.delete(0) # Delete the first record.
True
>>> users.delete(users.last_record_id()) # Delete the last record.
True
>>> users.all()
[{'__id': 1, 'color': 'white', 'name': 'Huey'},
{'__id': 2, 'color': 'black', 'name': 'Mickey'}]
>>> users.filter(lambda obj: obj['name'].startswith('H'))
[{'__id': 1, 'color': 'white', 'name': 'Huey'}]
Related
I am calling the list operation to retrieve the metadata values of a blob storage.
My code looks like:
blob_service_list = storage_client.blob_services.list('rg-exercise1', 'sa36730')
for items in blob_service_list:
print((items.as_dict()))
What's happening in this case is that the returned output only contains the items which had a corresponding Azure object:
{'id': '/subscriptions/0601ba03-2e68-461a-a239-98cxxxxxx/resourceGroups/rg-exercise1/providers/Microsoft.Storage/storageAccounts/sa36730/blobServices/default', 'name': 'default', 'type': 'Microsoft.Storage/storageAccounts/blobServices', 'sku': {'name': 'Standard_LRS', 'tier': 'Standard'}, 'cors': {'cors_rules': [{'allowed_origins': ['www.xyz.com'], 'allowed_methods': ['GET'], 'max_age_in_seconds': 0, 'exposed_headers': [''], 'allowed_headers': ['']}]}, 'delete_retention_policy': {'enabled': False}}
Where-as, If I do a simple print of items, the output is much larger:
{'additional_properties': {}, 'id': '/subscriptions/0601ba03-2e68-461a-a239-98c1xxxxx/resourceGroups/rg-exercise1/providers/Microsoft.Storage/storageAccounts/sa36730/blobServices/default', 'name': 'default', 'type': 'Microsoft.Storage/storageAccounts/blobServices', 'sku': <azure.mgmt.storage.v2021_06_01.models._models_py3.Sku object at 0x7ff2f8f1a520>, 'cors': <azure.mgmt.storage.v2021_06_01.models._models_py3.CorsRules object at 0x7ff2f8f1a640>, 'default_service_version': None, 'delete_retention_policy': <azure.mgmt.storage.v2021_06_01.models._models_py3.DeleteRetentionPolicy object at 0x7ff2f8f1a6d0>, 'is_versioning_enabled': None, 'automatic_snapshot_policy_enabled': None, 'change_feed': None, 'restore_policy': None, 'container_delete_retention_policy': None, 'last_access_time_tracking_policy': None}
Any value which is None has been removed from my example code. How can I extend my example code to include the None fields and have the final output as a list?
I tried in my environment and got below results:
If you need to include the None values in the dictionary you can follow the below code:
Code:
from azure.mgmt.storage import StorageManagementClient
from azure.identity import DefaultAzureCredential
storage_client=StorageManagementClient(credential=DefaultAzureCredential(),subscription_id="<your sub id>")
blob_service_list = storage_client.blob_services.list('v-venkat-rg', 'venkat123')
for items in blob_service_list:
items_dict = items.as_dict()
for key, value in items.__dict__.items():
if value is None:
items_dict[key] = value
print(items_dict)
Console:
The above code executed with None value successfully.
I currently aggregate a query to get a unique list of strings used in my endpoint. As of now, the fetched data looks like this:
{
"data": [
"Beige",
"Grey",
...
]
}
However, I'm attempting to do something like this:
{
"data": [
{
name: "Beige",
count: 7
},
{
name: "Grey",
count: 3
},
...
]
}
Where count is the amount of times the value occurs in the datbase.
Currently, my viewset is structured like this:
class ProductFiltersByCategory(APIView):
"""
This viewset takes the category parameter from the url and returns related product filters
"""
def get(self, request, *args, **kwargs):
"""
Gets parameter in urls and aggregated filtered result
"""
category = self.kwargs['category']
aggregated_result = Product.objects.filter(
category__parent__name__iexact=category,
status='available'
).distinct().aggregate(data=ArrayAgg('colors__name', distinct=True))
return Response(aggregated_result)
How do i go about making the key-value pair structure i want, as well as do the count for each "color"?
You can annotate the Count of each category and group by that category using .values(). Since your relations involve a many-to-many field, you probably just want to do the query on the Color model rather than the Product model.
Color.objects.filter(
product__category__parent__name__iexact=category,
product__status='available'
).values('name', 'product').annotate(
colors_count=Count('product', distinct=True)
).values_list('name', 'colors_count')
You can do this with collections.Counter:
from random import choices
from collections import Counter
from pprint import pprint
colours = ['White', 'Yellow', 'Blue', 'Red', 'Green', 'Black', 'Brown', 'Azure', 'Ivory', 'Teal', 'Silver', 'Purple', 'Navy blue', 'Pea green', 'Gray',
'Orange', 'Maroon', 'Charcoal', 'Aquamarine', 'Coral', 'Fuchsia', 'Wheat', 'Lime', 'Crimson', 'Khaki', 'Hot pink', 'Magenta', 'Olden', 'Plum', 'Olive', 'Cyan']
data = {"data": choices(colours, k=10)}
pprint(data)
newData = {"data": [{"name": k, "count": v} for k,v in Counter(data["data"]).items()]}
pprint(newData)
Sample Output:
{'data': ['Coral',
'Ivory',
'Red',
'Crimson',
'Azure',
'Ivory',
'Red',
'Red',
'Khaki',
'Hot pink']}
{'data': [{'count': 1, 'name': 'Coral'},
{'count': 2, 'name': 'Ivory'},
{'count': 3, 'name': 'Red'},
{'count': 1, 'name': 'Crimson'},
{'count': 1, 'name': 'Azure'},
{'count': 1, 'name': 'Khaki'},
{'count': 1, 'name': 'Hot pink'}]}
You can use Counter form the collections library to count the values.
You can try something like this:
from collections import Counter
data = get_data_from_endpoint(request)[data]
c = Counter(data)
result = {'data': [{'name': element, 'count': c[element]} for element in c]}
I have two dictionaries - basic dictionary and additional, and I want to use additional dictionary data to interact with function and append data to the 1st one.
dict_report = {'Id': p_id, 'First name': person_name, 'Age': p_age} #basic dict
iter_params = {'Shape': 0, 'Margin': 1} #additional
I have a function which accepts 3 arguments and I want to use 2nd dict value as 1st argument.
Let's say, in basic language:
def function():
shape = concatvals(iter_params_mammo(['Shape'],14,19)) #goes to cell value directly
margin = concatvals(iter_params_mammo(['Margin'], 14, 19))
dict_report.add(shape,margin)
return dict_report
Should return
dict_report = {'Id': p_id, 'First name': person_name, 'Age': p_age, 'Shape': "specific shape", 'Margin': "specific margin"}
Can I somehow not specify shape and margin, but just append new values to dict automatically, depending on additional dictionary size and values, but use their values to create new data?
Not sure if this is what you wanted but here is a rough copy.
toda will pull a value from existing dict based on the key passed in.
arba reinserts the key.
shimi will update become the existing dict then update to add new key value pair
dict_report = {'Id': 'p_id', 'First name': 'person_name', 'Age': 'p_age'}
kv = iter_params = {'Shape': 0, 'Margin': 1}
toda = iter_params.get('Shape')
arba = {'Shape': toda}
print(arba)
def dict_merge(x):
shimi = dict_report
shimi.update(x)
print(shimi)
return shimi
dict_merge(arba)
I am not sure what function you want to create or parameters, but as far as I can tell you just want to merge a dictionary and return it otherwise, so, here's that:
dict_report = {'Id': "p_id", 'First name': "person_name", 'Age': "p_age"} #basic dict
iter_params = {'Shape': 0, 'Margin': 1}
def func(d1,d2,arg3):
for k,v in d2.items():
#perform function on v
#e.g. v += 1 or whatever you want
#add to dict_report
d1[k] = v
return d1
dict_report = func(dict_report,iter_params,"3rd parameter?")
print (dict_report)
prints:
{'Id': 'p_id', 'Shape': 0, 'First name': 'person_name', 'Margin': 1, 'Age': 'p_age'}
I have below json as input from the client
[
{'id': 0, 'name': 'Housing', 'value': 3},
{'id': 1, 'name': 'Bank', 'value': 8},
{'id': 2, 'name': 'Entertainment', 'value': 3}
]
It is being assigned to inputV_wc object in my view like below
View:
def savemore(request):
if request.method == "POST":
data=json.loads(request.body.decode())
inputV_wc = data['wc']
else:
response_data = 'You have not saved any data!'
return HttpResponse(response_data, content_type="text/plain")
try:
if not inputV_wc:
test=''
else:
# WC - Insert wc again on each save rather update - time consuming
if js_wex.objects.filter(pid = request.session.get('pid')).exists():
js_wex.objects.filter(pid=request.session.get('pid')).delete()
wc = js_wex(pid=request.session.get('pid'), wcname=inputV_wc[0]['name'],rating=inputV_wc[0]['value'],ordernum=inputV_wc[0]['id'])
wc.save()
wc = js_wex(pid=request.session.get('pid'), wcname=inputV_wc[1]['name'],rating=inputV_wc[1]['value'],ordernum=inputV_wc[1]['id'])
wc.save()
wc = js_wex(pid=request.session.get('pid'), wcname=inputV_wc[2]['name'],rating=inputV_wc[2]['value'],ordernum=inputV_wc[2]['id'])
wc.save()
except Exception as e:
response_data = 'Ouch! Something went wrong!'+str(e)
return HttpResponse(response_data, content_type="text/plain")
Currently if my input json has 5 rows, certainly the above view fails with Index out of range..
and if input json has 2 rows it again fails with missing entry - model cannot be saved.
How can I write my view such that if json has varied number of objects like
input from one user -
[
{'id': 0, 'name': 'Housing', 'value': 14},
{'id': 1, 'name': 'Bank', 'value': 18}
]
input from other user -
[
{'id': 0, 'name': 'Housing', 'value': 3},
{'id': 1, 'name': 'Bank', 'value': 18},
{'id': 2, 'name': 'Housing1', 'value': 14},
{'id': 3, 'name': 'Bank1', 'value': 12}
]
can be handled ?
Json rows could be from 1 to 15 maximum for each input.
I read about using **kwargs, for handling similar scenario.. but I couldn't figure out how to apply for saving my model for varied json input.
If I understand you correctly, I think you just want to use a for statement:
for row in inputV_wc:
wc = js_wex(pid=request.session.get('pid'), wcname=row['name'], rating=row['value'], ordernum=row['id'])
wc.save()
If you want to insert all the objects at once, based on what you have shown you could probably use bulk_create:
rows = []
for row in inputV_wc:
wc = js_wex(pid=request.session.get('pid'), wcname=row['name'], rating=row['value'], ordernum=row['id'])
rows.append(wc)
js_wex.objects.bulk_create(rows)
I'm feeling stumped and looking for help. I'm trying to access data that lives inside of a dictionary that's inside of an array that is inside of a dictionary. See below:
{
'files': [
{
'type': 'diskDescriptor',
'name': '[VM] VM1/VM1.vmdk',
'key': 4,
'size': 0
},
{
'type': 'diskExtent',
'name': '[VM] VM1/VM1-flat.vmdk',
'key': 5,
'size': 32457621504
}
],
'capacity': 32505856,
'label': 'Hard disk 1',
'descriptor': '[VM] VM1/VM1.vmdk',
'committed': 31696896,
'device': {
'summary': '32,505,856 KB',
'_obj': <pysphere.vi_property.VIProperty object at 0x17442910>,
'unitNumber': 0,
'key': 2000,
'label': 'Hard disk 1',
'type': 'VirtualDisk',
'capacityInKB': 32505856
}
}
If I want to access, let's say the descriptor key value how would I go about this with Python? For some reason all of the combinations I've tried do not work.
Any help and guidance would be appreciated and if more information is needed I can provide. Thanks.
Lets call your main dictionary bob, because I like bob:
bob['files'] #get you the list with second dictionary
bob['files'][0] #get you the first item in the list, which is the nested 2nd dictionary
bob['files'][0]['type'] == 'diskDescriptor'