How to set a protobuf Timestamp field in python? - python

I am exploring the use of protocol buffers and would like to use the new Timestamp data type which is in protobuf3. Here is my .proto file:
syntax = "proto3";
package shoppingbasket;
import "google/protobuf/timestamp.proto";
message TransactionItem {
optional string product = 1;
optional int32 quantity = 2;
optional double price = 3;
optional double discount = 4;
}
message Basket {
optional string basket = 1;
optional google.protobuf.Timestamp tstamp = 2;
optional string customer = 3;
optional string store = 4;
optional string channel = 5;
repeated TransactionItem transactionItems = 6;
}
message Baskets {
repeated Basket baskets = 1;
}
After generating python classes from this .proto file I'm attempting to create some objects using the generated classes. Here's the code:
import shoppingbasket_pb2
from google.protobuf.timestamp_pb2 import Timestamp
baskets = shoppingbasket_pb2.Baskets()
basket1 = baskets.baskets.add()
basket1.basket = "001"
basket1.tstamp = Timestamp().GetCurrentTime()
which fails with error:
AttributeError: Assignment not allowed to composite field "tstamp" in protocol message object.
Can anyone explain to me why this isn't working as I am nonplussed.

See Timestamp.
I think you want:
basket1.tstamp.GetCurrentTime()

You could also parse it:
Timestamp().FromJsonString("2022-03-26T22:23:34Z")

I found this highly confusing, as it differs from how other values was assigned in my demo, so I'd like to add this method using .FromDatetime():
.proto:
message User {
int64 id = 1;
string first_name = 2;
...
string phone_number = 7;
google.protobuf.Timestamp created_on = 8; # <-- NB
}
The response, UserMsgs.User(), is here a class generated from the above protofile, and is aware of what type each field has.
def GetUser(self, request, context):
response = UserMsgs.User()
if request.id is not None and request.id > 0:
usr = User.get_by_id(request.id)
response.id = usr.id
response.first_name = usr.first_name
...
response.phone_number = str(usr.phone_number)
response.created_on.FromDatetime(usr.created_on) # <-- NB
return response
So instead of assigning response.created_on with = as the others, we can use the built in function .FromDatetime as mentioned here.
NB: Notice the lowercase t in Datetime
usr.created_on in my example is a python datetime, and is assigned to a google.protobuf.Timestamp field.

Related

How to declare variables in a GraphqQL query using the gql client?

I'm new with GraphQL schemas and I would like to do a mutation using the gql client. The query below works like a charme in the graphql web interface after replacing the 5 variables with the corresponding strings and integers.
But when I put a $ before every variables in the query, as mentionned in the documentation, it throws an error saying Variable '$w' is not defined by operation 'createMutation'.
What am'I missing ?
transport = AIOHTTPTransport(url="http://x.x.x.x:8000/graphql")
client = Client(transport=transport, fetch_schema_from_transport=True)
query = gql(
"""
mutation createMutation {
createTarget(targetData: {weight: $w, dt: $dt,
exchangeId: $exchangeId,
strategyId: $strategyId,
marketId:$marketId
}) {
target {
dt,
weight,
market,
exchange,
strategy
}
}
}
"""
)
params = {"w": self.weight,
"dt": self.dt,
"exchangeId": self.exchange.pk,
"strategyId": self.strategy.pk,
"marketId": self.market.pk
}
result = client.execute(query, variable_values=params)
When I remove the $ it says Float cannot represent non numeric value: w.
And this is how the graphene code looks like at the server side :
class TargetInput(graphene.InputObjectType):
weight = graphene.Float()
dt = graphene.DateTime()
strategy_id = graphene.Int()
exchange_id = graphene.Int()
market_id = graphene.Int()
class CreateTarget(graphene.Mutation):
class Arguments:
target_data = TargetInput(required=True)
target = graphene.Field(CustomObject)
#staticmethod
def mutate(root, info, target_data):
target = Target.objects.create(**target_data)
return CreateTarget(target=target)
class Mutation(graphene.ObjectType):
create_target = CreateTarget.Field()
schema = graphene.Schema(query=Query, mutation=Mutation)
There is also another question related to gql variables but it doesn't solve my problem.
I have found the answer to my own question. When using variables it's necessary to declare each of them between ( and ) at the beggining of the query, as stipulated here.
So in my case the correct query was:
query = gql(
"""
mutation createMutation ($w: Float, $dt: DateTime, $exchangeId: Int, $strategyId: Int, $marketId: Int){
createTarget(targetData: {weight: $w, dt: $dt,
exchangeId: $exchangeId,
strategyId: $strategyId,
marketId: $marketId
}) {
target {
dt
}
}
}
"""
)

Django Rest Framework + Angular 2 - uploading multiple files

I'm using Django Rest Framework as my backend and Angular 2 for my frontend. I've got this page in Angular, where I create a form:
createResourcesForm() {
this.resourcesForm = this.formBuilder.group({
resources: this.formBuilder.array([
this.formBuilder.group({
title: ['', Validators.compose([Validators.required])],
file: ['', Validators.compose([])],
})
])
})
}
As you can see, the form consists of FormArray, where every element has two inputs: title and file - a text input and a file input respectively. On submitting the form I'm trying to send the data to Django but I get an error Missing filename. Request should include a Content-Disposition header with a filename parameter.. I could set it easily but I'm expecting to receive a list of {title, file}, so how to set multiple file names? Any other idea how I could do this?
The error in Django Rest Framework comes from parse method in FileUploadParser.
I'm not pasting any Python code here because it's a standard ListCreateAPIView, nothing special about it. Here is my serializer:
class ResourceCreateSerializer2(serializers.Serializer):
author_pk = serializers.IntegerField(required=False)
author_first_name = serializers.CharField(max_length=50, required=False)
author_last_name = serializers.CharField(max_length=50, required=False)
resources = ResourceWithoutAuthorSerializer(many=True)
class ResourceWithoutAuthorSerializer(serializers.ModelSerializer):
instruments = InstrumentSerializer(many=True)
class Meta:
model = MusicResource
fields = ['title', 'file', 'instruments']
Don't mind the other fields, they are being sent just fine (as the file does). One more thing to say - I'm adding a content-type header in Angular just before sending the data.
UPDATE 1
Here is my method for uploading files (Angular 2):
get value() {
let formData = new FormData();
for (let i = 0; i < this.resources.length; i++) {
let resource = this.resources.value[i];
let fileName = resource.file;
let fileInputs = $('input[type="file"]').filter(function () {
return this.value == fileName;
});
if (fileInputs.length == 0) {
return null;
}
let fileInput = <HTMLInputElement>fileInputs[0];
formData.append('resources-' + i + '-title', resource.title);
formData.append('resources-' + i + '-file', fileInput.files[0], fileInput.files[0].name);
for (let j = 0; j < this.instrumentsSelect.value.length; j++) {
formData.append('resources-' + i + '-instruments', this.instrumentsSelect.value[j]);
}
}
return formData;
}
then
this.musicResourcesService.addMusicResource(toSend).subscribe(
data => console.log('successfuly added resources'),
err => console.log('MusicResourcesAddComponent', 'onMusicResourceFormSubmit', err)
);
addMusicResource(data) {
let headers = new Headers({});
headers.append('Content-Type', 'multipart/form-data');
headers.append('Accept', 'application/json');
let options = new RequestOptions({headers});
return this.api.post('resources/resources/list_create/', data, true, options);
}
public post(url: any, payload: any, noToken?, options?: any): Observable<any> {
const provider = noToken ? this.http : this.authHttp;
const fulLUrl = this.conf.getAPIUrl() + url;
return provider.post(fulLUrl, payload, options)
.delay(100)
.map(this.extractData)
.catch(this.handleError).share();
}
I did not like #Robert's answer and did not receive any other idea so after hours of reseraching it turns out that I was missing two things:
The parser should have been set to MultiPartParser, not FileUploadParser
There is no need to set the Content-Type header manually, it will get filled automatically along with the boundary which I was missing
Also, to make sure Django receives all the data and understands it, I had to change
formData.append('resources-' + i + '-title', resource.title);
and similar lines to
formData.append('resources[' + i + ']title', resource.title);

How to get the type of a variable defined in a protobuf message?

I'm trying to do some 'translation' from protobuf files to Objective-C classes using Python. For example, given the protobuf message:
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
}
I want to translate it into an objc class:
#interface Person : NSObject
#property (nonatomic, copy) NSString *name;
#property (nonatomic, assign) int ID;
#property (nonatomic, copy) NSString *email;
#end
The key point is to acquire every property's name and type. For example, 'optional string email' in the protobuf message, its name is 'email', type is 'string', so it should be NSString *email in objective-c. I followed the official tutorial, wrote an addressbook.proto just the same as the one in the tutorial and compiled it. Then I wrote my python code:
import addressbook_pb2 as addressbook
p = addressbook.Person()
all_fields = p.DESCRIPTOR.fields_by_name
# print "all fields: %s" %all_fields
field_keys = all_fields.keys()
# print "all keys: %s" %field_keys
for key in field_keys:
one_field = all_fields[key]
print one_field.label
This just gave me:
1
2
3
2
So I guess label is not what I need, while field_keys is just the list of names that I expect. I tried some other words, and did some search on the web, but didn't find the right answer.
If there's no way to acquire the type, I have another thought, which is to read and analyze every line of the protobuf source file in a pure 'Pythonic' way, but I really don't want to do this if its not necessary.
Can anybody help me?
The FieldDescriptor class has a message_type member which, if a composite field, is a descriptor of the message type contained in this field. Otherwise, this is None.
Combine this with iterating through a dictionary of DESCRIPTORS means you can get the name and type of composite and non-composite (raw) fields.
import addressbook_pb2 as addressbook
DESCRIPTORS = addressbook.Person.DESCRIPTOR.fields_by_name
for (field_name, field_descriptor) in DESCRIPTORS.items():
if field_descriptor.message_type:
# Composite field
print(field_name, field_descriptor.message_type.name)
else:
# Raw type
print(field_name, field_descriptor.type)
# TYPE_DOUBLE
# TYPE_FLOAT
# TYPE_INT64
# TYPE_UINT64
# TYPE_INT32
# TYPE_FIXED64
# TYPE_FIXED32
# TYPE_BOOL
# TYPE_STRING
# TYPE_GROUP
# TYPE_MESSAGE
# TYPE_BYTES
# TYPE_UINT32
# TYPE_ENUM
# TYPE_SFIXED32
# TYPE_SFIXED64
# TYPE_SINT32
# TYPE_SINT64
# MAX_TYPE
The raw types are class attributes; https://github.com/protocolbuffers/protobuf/blob/master/python/google/protobuf/descriptor.py
Thanks to Marc's answer, I figured out some solution. This is just a thought, but it's a huge step for me.
Python code:
import addressbook_pb2 as addressbook
typeDict = {"1":"CGFloat", "2":"CGFloat", "3":"NSInteger", "4":"NSUinteger", "5":"NSInteger", "8":"BOOL", "9":"NSString", "13":"NSUinteger", "17":"NSInteger", "18":"NSInteger"}
attrDict = {"CGFloat":"assign", "NSInteger":"assign", "NSUinteger":"assign", "BOOL":"assign", "NSString":"copy"}
p = addressbook.Person()
all_fields = p.DESCRIPTOR.fields_by_name
field_keys = all_fields.keys()
for key in field_keys:
one_field = all_fields[key]
typeNumStr = str(one_field.type)
className = typeDict.get(typeNumStr, "NSObject")
attrStr = attrDict.get(className, "retain")
propertyStr = "#property (nonatomic, %s) %s *%s" %(attrStr, className, key)
print propertyStr
For the addressbook example, it prints:
#property (nonatomic, copy) NSString *email
#property (nonatomic, copy) NSString *name
#property (nonatomic, retain) NSObject *phone
#property (nonatomic, assign) NSInteger *id
Not the final solution, but it means a lot. Thank you, Marc!

Looping over Protocol Buffers attributes in Python

I would like help with recursively looping over all attributes/sub objects contained in a protocol buffers message, assuming that we do not know the names of them, or how many there are.
As an example, take the following .proto file from the tutorial on the google website:
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
and to use it...:
person = tutorial.Person()
person.id = 1234
person.name = "John Doe"
person.email = "jdoe#example.com"
phone = person.phone.add()
phone.number = "555-4321"
phone.type = tutorial.Person.HOME
Given Person, How do I then access both the name of the attribute and its value for each element: person.id, person.name, person.email, person.phone.number, person.phone.type?
I have tried the following, however it doesn't seem to recurs into person.phone.number or person.phone.type.
object_of_interest = Person
while( hasattr(object_of_interest, "_fields") ):
for obj in object_of_interest._fields:
# Do_something_with_object(obj) # eg print obj.name
object_of_interest = obj
I have tried using obj.DESCRIPTOR.fields_by_name.keys to access the sub elements, but these are the string representations of the sub objects, not the objects themselves.
obj.name gives me the attribute of the name, but im not sure how to actually get the value of that attribute, eg obj.name may give me 'name', but how do i get 'john doe' out of it?
I'm not super familiar with protobufs, so there may well be an easier way or api for this kind of thing. However, below shows an example of how you could iterate/introspect and objects fields and print them out. Hopefully enough to get you going in the right direction at least...
import addressbook_pb2 as addressbook
person = addressbook.Person(id=1234, name="John Doe", email="foo#example.com")
person.phone.add(number="1234567890")
def dump_object(obj):
for descriptor in obj.DESCRIPTOR.fields:
value = getattr(obj, descriptor.name)
if descriptor.type == descriptor.TYPE_MESSAGE:
if descriptor.label == descriptor.LABEL_REPEATED:
map(dump_object, value)
else:
dump_object(value)
elif descriptor.type == descriptor.TYPE_ENUM:
enum_name = descriptor.enum_type.values[value].name
print "%s: %s" % (descriptor.full_name, enum_name)
else:
print "%s: %s" % (descriptor.full_name, value)
dump_object(person)
which outputs
tutorial.Person.name: John Doe
tutorial.Person.id: 1234
tutorial.Person.email: foo#example.com
tutorial.Person.PhoneNumber.number: 1234567890
tutorial.Person.PhoneNumber.type: HOME

in protobuf python api how to add element to nested message repeated property?

message items {
message require {
optional bool require_sex = 1; //
repeated int32 fate = 2 [packed = true];
}
optional int32 sub_type = 1;
repeated int32 levels = 2 [packed = true];
}
I tried
raw.require.require_sex = 1
raw.require.fate.append(1)
raw.require.fate.extend([2,3])
got an error
AttributeError: 'property' object has no attribute 'append'
but the first level repeated field works fine:
raw = down_pb2.items()
raw.levels.append(4)
is this kind of definition not supported?
You need to create a field using that require type and then access that field in the code.
message items {
message require {
optional bool require_sex = 1; //
repeated int32 fate = 2 [packed = true];
}
optional int32 sub_type = 1;
repeated int32 levels = 2 [packed = true];
required require sub = 3;
}
then
raw.sub.fate.append(1)

Categories