How is a list of values passed into a feature_list? The documentation suggests that this is valid but it is not clear how this is accomplished given that passing a list result in an error.
>>> tf.train.SequenceExample().feature_lists.feature_list["multiple"].feature.add().int64_list.value.append([1,2,3,4])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal /containers.py", line 251, in append
self._values.append(self._type_checker.CheckValue(value))
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal /type_checkers.py", line 132, in CheckValue
raise TypeError(message)
TypeError: [1, 2, 3, 4] has type <type 'list'>, but expected one of: (<type 'int'>, <type 'long'>)
This is an example given in the example proto file.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/example/example.proto
// Conditionally conformant FeatureLists, the parser configuration determines
// if the feature sizes must match:
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0, 6.0 ] } } }
// } }
Is it necessary to use something other than append when adding a list?
An example of a sequence would be...
[[1,2,3],[4,5],[6,7],[8,9,10]]
...where there are four steps in this sequence, and at each step there is a set of values. The desired result would look something like the example below.
feature_lists: { feature_list: {
key: "movie_ratings"
value: { feature: { float_list: { value: [ 1, 2, 3 ] } }
feature: { float_list: { value: [ 4, 5 ] } }
feature: { float_list: { value: [ 6, 7 ] } }
feature: { float_list: { value: [ 8, 9, 10 ] } } }
} }
Use extend instead of append.
tf.train.SequenceExample().feature_lists.feature_list["multiple"].feature.add().int64_list.value.extend([1,2,3,4])
Related
I want to limit text scores using the $gt operator.
Using the find function, I can sort the text scores according to the text similarity status from largest to smallest. I can get the cursor with the highest score by putting a limit of 1 on the rank.
deneme = user.find(
{ '$text': { '$search': "dogan can" } },
{ 'score': { '$meta': "textScore" }})
deneme_sort = deneme.sort([('score', {'$meta': 'textScore'})]).limit(1)
But I don't want the ones whose text score is below the value I gave, to be listed.
For example, I don't want text scores below 1.5 to appear in the list. I'm trying to use the '$gt' operator for this but I'm getting an error.
deneme = user.find(
{ '$text': { '$search': "dogan can" } },
{ 'score': { '$meta': "textScore"}}, {'score': { '$gt': 1.5 } })
TypeError: skip must be an instance of int
it gives this error because the find function can only take two values.
I'm trying to query using the '$and' operator. This time it does not recognize the '$meta' operator. Or the '$gt' operator must take two values.
deneme = user.find({ '$text': { '$search': "dogan can" }} ,
{'$and':[{ 'score': { '$meta': "textScore" }},{'score': { '$gt': 1.5 }}]})
doc = []
for doc in deneme:
print(doc)
Expression $gt takes exactly 2 arguments. 1 were passed in., full error: {'ok': 0.0, 'errmsg': 'Expression $gt takes exactly 2 arguments. 1 were passed in.', 'code': 16020, 'codeName': 'Location16020'}
I just started learning mongodb. Can you help me?
I think what you're requesting is documented here. In short - you will need to use the aggregation framework so that you can have 3 stages to accomplish each of the following:
Perform the initial $text searching in a $match stage
Persist the text score as part of the document via an $addFields stage
Use an additional $match stage to perform the $gt filtering against that new score field.
Given a collection with the following documents:
test> db.foo.find()
[
{ _id: 1, key: 'dogan can' },
{ _id: 2, key: 'dogan' },
{ _id: 3, key: 'can' },
{ _id: 4, key: 'abc' }
]
A text search against dogan can will return the first three documents:
test> db.foo.aggregate([ { $match: { $text: { $search: "dogan can" } } },{$addFields:{score:{$meta:'textScore'}}}])
[
{ _id: 3, key: 'can', score: 1.1 },
{ _id: 1, key: 'dogan can', score: 1.5 },
{ _id: 2, key: 'dogan', score: 1.1 }
]
Appending the final $match (using a filter of 1.2), only one of the documents is returned:
test> db.foo.aggregate([ { $match: { $text: { $search: "dogan can" } } },{$addFields:{score:{$meta:'textScore'}}},{$match:{score:{$gt:1.2}}}])
[
{ _id: 1, key: 'dogan can', score: 1.5 }
]
If desired, you can of course include a $sort stage on the score as well.
I want to get the data from a json. I have the idea of a loop to access all levels.
I have only been able to pull data from a single block.
print(output['body']['data'][0]['list'][0]['outUcastPkts'])
How do I get the other data?
import json,urllib.request
data = urllib.request.urlopen("http://172.0.0.0/statistic").read()
output = json.loads(data)
for elt in output['body']['data']:
print(output['body']['data'][0]['inUcastPktsAll'])
for elt in output['list']:
print(output['body']['data'][0]['list'][0]['outUcastPkts'])
{
"body": {
"data": [
{
"inUcastPktsAll": 3100617019,
"inMcastPktsAll": 7567,
"inBcastPktsAll": 8872,
"outPktsAll": 8585575441,
"outUcastPktsAll": 8220240108,
"outMcastPktsAll": 286184143,
"outBcastPktsAll": 79151190,
"list": [
{
"outUcastPkts": 117427359,
"outMcastPkts": 1990586,
"outBcastPkts": 246120
},
{
"outUcastPkts": 0,
"outMcastPkts": 0,
"outBcastPkts": 0
}
]
},
{
"inUcastPktsAll": 8269483865,
"inMcastPktsAll": 2405765,
"inBcastPktsAll": 124466,
"outPktsAll": 3101194852,
"outUcastPktsAll": 3101012296,
"outMcastPktsAll": 173409,
"outBcastPktsAll": 9147,
"list": [
{
"outUcastPkts": 3101012296,
"outMcastPkts": 90488,
"outBcastPkts": 9147
},
{
"outUcastPkts": 0,
"outMcastPkts": 0,
"outBcastPkts": 0
}
]
}
],
"msgs": [ "successful" ]
},
"header": {
"opCode": "1",
"token": "",
"state": "",
"version": 1
}
}
output = json.loads(data) #Type of output is a dictionary.
#Try to use ".get()" method.
print(output.get('body')) #Get values of key 'body'
print(output.get('body').get('data')) #Get a list of key 'data'
If a key doesn't exist, the '.get()' method will return None.
https://docs.python.org/3/library/stdtypes.html#dict.get
In python you can easily iterate over the objects of a list like so:
>>> l = [1, 2, 3, 7]
>>> for elem in l:
... print(elem)
...
1
2
3
7
This works regarding what can of object do you have in the list (integers, tuples, dictionaries). Having that in mind, your solution was not far off, you only to do the following changes:
for entry in output['body']['data']:
print(entry['inUcastPktsAll'])
for list_element in entry['list']:
print(list_element['outUcastPkts'])
This will give you the following for the json object you have provided:
3100617019
117427359
0
8269483865
3101012296
0
How to append list of dictionaries in python?
I'm trying to create JSON data using python where data is retrieved from MongoDB.
I need to get the JSON data in the below format.
Note: I retrieved all the required data from db.I'm not sure to append that data in the desired JSON format.
JSON Data:
"System_Details":
{
"System_id":"001",
"Details":[
{
"name":"Job-Info"
"job":[
{
"category":"1",
{
"eid":"01",
"role":"associate-1"
},
{
"eid":"02",
"role":"associate-2"
},
{
"eid";"03",
"role":"associate-3"
}
},
{
"category":"2",
{
"eid":"04",
"role":"associate-4"
},
{
"eid":"05",
"role":"associate-5"
},
{
"eid";"06",
"role":"associate-6"
}
},
]
}
]}
My script:
System_Details = {}
System_Details['Details'] = []
job = []
job_dict = {}
System_Details["System_id"]= <SYSTEMID ## which is retrieved from db>
job.append(job_dict) #job_dict is having above json values which is mentioned inside job list("job":[])
Now job = [ ] Contains
"job":[
{
"category":"1",
{
"eid":"01",
"role":"associate-1"
},
{
"eid':"02",
"role":"associate-2"
},
{
"eid";"03",
"role":"associate-3"
}
},
{
"category":"2",
{
"eid":"04",
"role":"associate-4"
},
{
"eid':"05",
"role":"associate-5"
},
{
"eid";"06",
"role":"associate-6"
}
},
]
How can I append this list to System_Details['Details']?
Please note the curly braces inside System_Details ['Details'] while appending.
In short
"System_Details":
{
"Details":[
{
...
...
"job":[
{
...
},
{
...
}
]
}
]}
How to append "job":[] to "Details":[] ?
Thanks in advance.
Since the question is not clear, answering this based on the given understanding.
"System_Details":
{
"Details":[
{
...
...
"job":[
{
...
},
{
...
}
]
}
]
}
So, there are two lists Details and job. You want to append all the items present in the job to appended on Details list.
You can append a list to another list by using extend() in python.
>>> l = [1, 2]
>>> l.extend([3, 4])
>>> l
[1, 2, 3, 4]
>>> l.extend('foo')
>>> l
[1, 2, 3, 4, 'f', 'o', 'o']
>>> l.extend((5, 6))
>>> l
[1, 2, 3, 4, 'f', 'o', 'o', 5, 6]
>>> l.extend({'x': 100, 'y': 200})
>>> l
[1, 2, 3, 4, 'f', 'o', 'o', 5, 6, 'y', 'x']
Please update your question or comment on the answer, if it answers your question.
I am trying to deserialise a json which is a valid format w.r.to the yang model defined. This example is given in the pyangbind documentation. But the json format is a little different from what I pasted here.
https://github.com/robshakir/pyangbind/tree/master/docs/example/simple-serialise
JSON:
{
"a-container": {
"a-value": 8
},
"a-list": [
{
"the-key": "entry-one"
},
{
"the-key": "entry-two"
}
]
}
Yang:
module simple_serialise {
yang-version "1";
namespace "http://rob.sh/yang/examples/ss";
prefix "ss";
container a-container {
leaf a-value {
type int8;
}
}
list a-list {
key 'the-key';
leaf the-key {
type string;
}
}
}
I tried to deserialise with:
from pyangbind.lib import pybindJSON
from lib import simple_serialise
s = '''{
"a-container": {
"a-value": 8
},
"a-list": [
{
"the-key": "entry-one"
},
{
"the-key": "entry-two"
}
]
}'''
sip = pybindJSON.loads(s, simple_serialise, 'simple_serialise')
I get the following error when I try to deserialise.
Traceback (most recent call last):
File "/Users/joshisk/PycharmProjects/tapi-pyang/src/main.py", line 38, in <module>
sip = pybindJSON.loads(di1, simple_serialise, 'simple_serialise') #type: simple_serialise.simple_serialise
File "/anaconda3/lib/python3.6/site-packages/pyangbind/lib/pybindJSON.py", line 58, in loads
path_helper=path_helper, extmethods=extmethods, overwrite=overwrite)
File "/anaconda3/lib/python3.6/site-packages/pyangbind/lib/serialise.py", line 302, in load_json
key_order = d[key].keys()
AttributeError: 'list' object has no attribute 'keys'
Pyangbind expects a dictionary with the key as the 'key' value you gave in the model.
list a-list {
key 'the-key';
In your case, the value for 'the-key'.
from pyangbind.lib import pybindJSON
from lib import simple_serialise
s = '''{
"a-container": {
"a-value": 8
},
"a-list": {
"entry-one": {
"the-key": "entry-one"
},
"entry-two": {
"the-key": "entry-two"
}
}
}'''
sip = pybindJSON.loads(s, simple_serialise, 'simple_serialise')
I am pretty new to Python and I am currently working on large string formatting that I need for a library I am using.
The problem occurs as I do not understand where exactly the error is happening within the large string format. More precisely I get an error of the form
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2731, in run_code
exec code_obj in self.user_global_ns, self.user_ns
File "<ipython-input-3-f6a3bb7fe2f9>", line 13, in <module>
trainCV = trainCV % (train_params)
ValueError: unsupported format character ',' (0x2c) at index 2726
Is there an way to precisely detect the line get where the error occurs ?
My complete code looks like this:
trainCV = open('Conv_Constructor.yaml','r').read()
train_params = {'batch_size': 100,
'output_channels_h2': 64,
'conv_kernel_size': 8,
'pool_size': 2,
'stride_size': 1,
'output_channels_h3': 64,
'num_classes': 6,
'valid_stop': 4200,
'test_start': 4200,
'test_stop': 4400,
'max_epochs': 5}
trainCV = trainCV % (train_params)
print trainCV
And the Conv_Constructor.yaml file I am trying to format as a string is the following
# ---------- INPUTS ---------
#
# batch_size
# output_channels_h2
# conv_kernel_size
# pool_size
# stride_size
# output_channels_h3
# num_classes
# valid_stop
# test_start
# test_stop
# max_epochs
##################################################################
!obj:pylearn2.train.Train {
dataset: !obj:pylearn2.official_train_data.load_data {
start: 0,
stop: 4000
# one_hot: 1,
},
model: !obj:pylearn2.models.mlp.MLP {
batch_size: %(batch_size)i,
input_space: !obj:pylearn2.space.Conv2DSpace {
shape: [32, 32],
num_channels: 1,
axes = ('b',0,1,'c')
},
layers: [ !obj:pylearn2.models.mlp.ConvRectifiedLinear {
layer_name: 'h2',
output_channels: %(output_channels_h2)i,
#params : !pkl: 'dae_layer_1_weights.plk',
irange: .05,
kernel_shape: [%(conv_kernel_size)i, %(conv_kernel_size)i],
pool_shape: [%(pool_size)i, %(pool_size)i],
pool_stride: [%(stride_size)i, %(stride_size)i],
max_kernel_norm: 1.9365
}, !obj:pylearn2.models.mlp.ConvRectifiedLinear {
layer_name: 'h3',
output_channels: %(output_channels_h3)i,
#params : !pkl: 'dae_layer_1_weights.plk',
irange: .05,
kernel_shape: %(conv_kernel_size)i, %(conv_kernel_size)i],
pool_shape:[%(pool_size)i, %(pool_size)i],
pool_stride: [%(stride_size)i, %(stride_size)i],
max_kernel_norm: 1.9365
}, !obj:pylearn2.models.mlp.Softmax {
max_col_norm: 1.9365,
layer_name: 'y',
n_classes: %(num_classes)i,
istdev: .05
}
],
},
algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
batch_size: %(batch_size)i,
learning_rate: .01,
init_momentum: .5,
monitoring_dataset:
{
'valid' : !obj:pylearn2.official_train_data.load_data {
start: 4000,
stop: %(valid_stop)i
#one_hot: 1,
},
'test' : !obj:pylearn2.official_train_data.load_data {
start: %(test_start),
stop: %(test_stop)
#one_hot: 1,
}
},
cost: !obj:pylearn2.costs.cost.SumOfCosts { costs: [
!obj:pylearn2.costs.cost.MethodCost {
method: 'cost_from_X'
}, !obj:pylearn2.costs.mlp.WeightDecay {
coeffs: [ .00005, .00005, .00005 ]
}
]
},
termination_criterion: !obj:pylearn2.termination_criteria.And {
criteria: [
!obj:pylearn2.termination_criteria.MonitorBased {
channel_name: "valid_y_misclass",
prop_decrease: 0.50,
N: 50
},
!obj:pylearn2.termination_criteria.EpochCounter {
max_epochs: %(max_epochs)i
},
]
},
},
extensions:
[ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
channel_name: 'valid_y_misclass',
save_path: "%(save_path)s/convolutional_network_best.pkl"
}, !obj:pylearn2.training_algorithms.sgd.MomentumAdjustor {
start: 1,
saturate: 10,
final_momentum: .99
}
]
}
You can locate the error more easily by processing each line separately instead of your whole string.
Replace
trainCV = trainCV % (train_params)
with
trainCV = trainCV.split('\n')
t1=[]
try:
for i, t in enumerate(trainCV):
t1.append(t % (train_params))
except :
print 'Error in line {}:'.format(i)
print t[i]
raise
and you will get the following output:
78
start: %(test_start),
meaning your string formating didn't quite work (in this case I think there is the i after the bracket missing). Debug your large string in this way and you should have a working code.
After that is done you can print it by joining the list:
print '\n'.join(t1)