I am looking for some resources in understanding the envelope pattern and google isn't being too helpful. Can someone illustrate this with an example? For more info, I am working with Pubsub in Python and I'm looking to tag messages with some additional info (apart from plainly adding that info in the message body) for complex applications.
The envelope pattern is a way to wrap a message with metadata to help determine what to do with that message without having to understand the contents of the message itself. The metadata can be used to determine how to route a message to the appropriate receiver to deal with the message contents or it could indicate what to do with the message itself.
Typically, the way this pattern works is that different layers add a wrapper on top of a message that is passed to it. There can be multiple, nested envelopes. When a message is received, there is a corresponding layer that removes one of the envelopes for every layer that added one.
The side that creates messages would look like this:
The side that consume messages would look like this:
A Google Cloud Pub/Sub message has an attributes map that can be used for precisely this purpose. You can put data in the attributes that can indicate what to do with the message without having to decode the bytes stored in the data portion of the message.
For example, imagine you have a topic that contains messages that correspond to different user actions on an ecommerce website. Some of the messages are for profile updates while others are transactions. You want to perform different actions on these types of messages. Instead of looking into the data field in the message to determine which type it is and what to do with it, you could add key/value pairs in the attributes, e.g. action: UPDATE and action: TRANSACTION. Then, your subscriber would look at the value for the action attribute and determine what to do with the message. It could execute a different piece of code or maybe even send it to a different service, acting as a router for messages. The key point is that the determination of what to do could be performed without having to decode the message data itself.
Messages sent to push subscription endpoints have an additional envelope added that indicates the subscription the message came from. This is necessary because it is possible for multiple push subscriptions to point to he same endpoints.
The PubSub envelope pattern is simple
The message content (absolutely what you want, base64 encoded)
Attributes on the message (that you can add to add meta data on the content). You can use attribute for taking decision on the message without processing it. For exemple, add a message version. If your code can only process v1 and you have a v2 message, skip it instead of having unmarshalling error.
Id of publication
Timestamp
When you consume the envelop, it is presented in 2 differents manner:
Cloud Function PubSub trigger event
Push Subscription
You will find the same information but not structured with the exact same JSON schema.
Related
I am looking at this sample: https://github.com/microsoft/BotBuilder-Samples/blob/main/samples/python/57.teams-conversation-bot/bots/teams_conversation_bot.py
_message_all_members iterates thorough a list of all members and sends each one a message.
I need a way to send one message to one user. I know their user id. Can you help me write a function that takes a text message and a user id and sends it. Whoever wrote that sample is a pervert, and I can't unwrap all the layers. tc1, tc2. wtf?
I think what you're looking for is "proactive notification", a method to send an activity to a user without the bot first receiving a message from them?
The thing that makes this awkward is that to use the SDK, you need to have a TurnContext that represents the conversation between the bot and the user... this is typically done using the adapter.continue_conversation method which requires you to pass the conversation reference. The conversation reference itself can either be cached from a previous bot message from that user or obtained by creating a new conversation with that user (this would be using the adapter.create_conversation method).
The tc1, tc2 stuff are essentially just references to turn contexts in nested callback methods as this is how create_conversation and continue_conversation work.
There is a specific proactive notification sample that might work better for you here: https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/python/16.proactive-messages
I have a bot which polls data regularly from some input and if a specific logic applies, sends a message to a telegram channel.
The messages are only relevant for readers if the reader in the channel more or less directly read the message, i.e. after let's say 30 min, the message is irrelevant. I do not want to delete the message for historic purposes, BUT I'd like to set it to read for everyone.
Is there a way to set the "read"-status for the receivers? Is it possible to do so, i.e. change, for already sent, i.e. previous messages?
To avoid having to keep track of sent messages with their timings, is it even possible to specify that directly when sending the message?
No, this is not possible. The read status can only be changed by the user itself. Anything not mentioned in the official API docs at https://core.telegram.org/bots/api can not be done with Telegram Bots.
As a side note: AFAIK the read status is not binary for each user. This can be observed by noting that viewing a channel message from different devices/clients with the same account will up the view count by more than 1.
I am using MQTT to send and receive messages from/to Google IoT Core (telemetry messages).
Messages are then fowarded to different Pub/Sub Topic.
I'd like to add custom attributes to my messages.
The goal is to use differents subscriptions on my topic. Then, filter incoming messages by my custom attributes, and finally get my messages by 'theme' on my dataflow pipeline.
I know we can do it when we use the Pub/Sub client but I can't manage to do this from the mqtt client.
I am using the python client.
So as it was requested I add some details. This is a very classic situation.
I am using a MQTT client as in the google example here: https://cloud.google.com/iot/docs/how-tos/mqtt-bridge#configuring_mqtt_clients. Then using the publish code from the same documentation: https://cloud.google.com/iot/docs/how-tos/mqtt-bridge#publishing_telemetry_events (I juste replaced
for i in range(0, 60):
time.sleep(1)
client.loop()
by time.sleep(1) because I do not want to wait a minute between each message.
I publish my messages calling the previous code like this:
publisher.publish(topic, payload)
where topic is my PubSub topic and payload is my data.
The documentation says:
The forwarded message data field contains a copy of the message published by the device, and the following message attributes are added to each message in the Cloud Pub/Sub topic
(link if you want to see the attributes: https://cloud.google.com/iot/docs/how-tos/mqtt-bridge#publishing_telemetry_events)
What I want to do is add custom attributes to this list.
If I call the Pub/Sub client directly I can do this (from documentation):
# Add two attributes, origin and username, to the message
future = publisher.publish(
topic_path, data, origin="python-sample", username="gcp"
)
where origin and username are custom attributes, Is it possible to do this using the MQTT client ?
Thanks :)
So it is NOT possible.
What google advice is to use topic sub-directories to 'group' messages by theme.
You can associate topics to your registry, then set subfolder for your topic. You can have as much subdirectory as you want as long that it follows this convention:
topic: topic-name
subtopic: topic_name/theme1, topic_name/theme2....
I have a use-case wherein I want to read the messages in Pubsub without acknowledging the messages. I would need help in on how to rule out the possibility of "duplicate messages" which will remain in Pubsub store when I don't ACK the delivered message.
Solutions that I have thought of:
Store the pulled messages in Datastore and see if they are same.
Store the pulled messages at runtime and check if my message is duplicate O(n) time complexity and space complexity O(n).
Store the pulled messages in a file and compare the new incoming messages from the messages in the file.
Use Dataflow and rule out the possibility (least expected)
I see that there is no offset like feature in Pubsub which is similar to Kafka, I think.
Which is the best approach that you would suggest in this matter/ or any other alternative approach that I can use?
I am using python google-cloud-pubsub_v1 to create a python client and pulling messages from Pubsub.
I am sharing the code which is the logic to pull the data
subscription_path = subscriber.subscription_path(
project_id, subscription_name)
NUM_MESSAGES = 3
# The subscriber pulls a specific number of messages.
response = subscriber.pull(subscription_path, max_messages=NUM_MESSAGES)
for received_message in response.received_messages:
print(received_message.message.data)
It sounds like Pub/Sub is probably not the right tool for the job. It seems as if you are trying to use Pub/Sub as a persistent data store, which is not the intended use case. Acking is a fundamental part of the lifecycle of a Cloud Pub/Sub message. Pub/Sub messages are deleted if they are unacked after the provided message retention period, which cannot be longer than 7 days.
I would suggest instead that you consider using an SQL database like Cloud Spanner. Then, you could generate a uuid for each message, use this as the primary key for deduplication, and transactionally update the database to ensure there are no duplicates.
I might be able to provide a better answer for you if you provide more information about what you are planning on doing with the deduplicated messages.
I'm having a problem forwarding messages with multiple media files (in my particular case, photos).
I'm using forwardMessage method to forward the message that user sends to bot (which, in some cases, might contain more than one photo):
bot.forwardMessage(admin_id, from_chat_id, message_id)
But, the thing is, according to Telegram API v3.5, the message which contain media group breaks up into array of messages, and hence the bot breaks up the message from user into multiple messages. E.g. if user have sent a message with 3 photos, then the bot forwards 3 messages, each of which contains a photo.
What I want is to forward a message with multiple media files as it is, as one message, not as multiple messages.
I know that probably I can take each photo from each message that the bot receives, put them one-by-one into array, and then send it from the bot using sendMediaGroup method, but I'd like to know if I can get it working with forwardMessage method.
Thanks in advance.
You can't forward original content via forwardMessage, for instance, you can't forward with an inline keyboard.
I think this may be a mistake, you can suggest that to #BotSupport.
Telegram API had updated a method call sendMediaGroup for sending images as album.
https://core.telegram.org/bots/api#sendmediagroup
Have a look :)