When I run my gRPC client and it attempts to stream a request to the server I get this error: "TypeError: has type list_iterator, but expected one of: bytes, unicode"
Do I need to encode the text I'm sending in some way? Error message makes some sense, as I am definitely passing in an iterator. I assumed from the gRPC documentation that this is what was needed. (https://grpc.io/docs/tutorials/basic/python.html#request-streaming-rpc)Anyway, sending a list or string yields a similar error.
At the moment I am sending a small test list of strings to the server in the request, but I plan to stream requests with very large amounts of text in the future.
Here's some of my client code.
def gen_tweet_space(text):
for tweet in text:
yield tweet
def run():
channel = grpc.insecure_channel('localhost:50050')
stub = ProseAndBabel_pb2_grpc.ProseAndBabelStub(channel)
while True:
iterator = iter(block_of_text)
response = stub.UserMarkov(ProseAndBabel_pb2.UserTweets(tweets=iterator))
Here's relevant server code:
def UserMarkov(self, request_iterator, context):
return ProseAndBabel_pb2.Babel(prose=markov.get_sentence(request_iterator.tweets))
Here's the proto where the rpc and messages are defined:
service ProseAndBabel {
rpc GetHaiku (BabelRequest) returns (Babel) {}
rpc GetBabel (BabelRequest) returns (Babel) {}
rpc UserMarkov (stream UserTweets) returns (UserBabel) {}
}
message BabelRequest{
string ask = 1;
}
message Babel{
string prose = 1;
}
message UserTweets{
string tweets = 1;
}
message UserBabel{
string prose = 1;
}
I've been successful getting the non-streaming rpc to work, but having trouble finding walkthroughs for request side streaming for python applications so I'm sure I'm missing something here. Any guidance/direction appreciated!
You need to pass the iterator of requests to the gRPC client stub, not to the protobuf constructor. The current code tries to instantiate a UserTweets protobuf with an iterator rather than an individual string, resulting in the type error.
response = stub.UserMarkov(ProseAndBabel_pb2.UserTweets(tweets=iterator))
You'll instead need to have your iterator to return instances of ProseAndBabel_pb2.UserTweets, each of which wraps one of the request strings you would like to send, and pass the iterator itself to the stub. Something like:
iterator = iter([ProseAndBabel_pb2.UserTweets(tweets=x) for x in block_of_text])
response = stub.UserMarkov(iterator)
Related
I am have a python client listening to SSE events from a server with node.js API
The flow is I sent an event to the node.js API through call_notification.py and run seevents.py in loop using run.sh(see below)
However I don't see that python client is receiving this SSE event? any guidance on why is that?
call_notification.py
import requests
input_json = {'BATS':'678910','root_version':'12A12'}
url = 'http://company.com/api/root_event_notification?params=%s'%input_json
response = requests.get(url)
print response.text
node.js API
app.get("/api/root_event_notification", (req, res, next) => {
console.log(req.query.params)
var events = require('events');
var eventEmitter = new events.EventEmitter();
//Create an event handler:
var myEventHandler = function () {
console.log('new_root_announced!');
res.status(200).json({
message: "New root build released!",
posts: req.query.params
});
}
seevents.py (python client listening to SSE events)
import json
import pprint
import sseclient
def with_urllib3(url):
"""Get a streaming response for the given event feed using urllib3."""
import urllib3
http = urllib3.PoolManager()
return http.request('GET', url, preload_content=False)
def with_requests(url):
"""Get a streaming response for the given event feed using requests."""
import requests
return requests.get(url, stream=True)
url = 'http://company.com/api/root_event_notification'
response = with_urllib3(url) # or with_requests(url)
client = sseclient.SSEClient(response)
#print client.events()
for event in client.events():
print "inside"
pprint.pprint(json.loads(event.data))
run.sh
#!/bin/sh
while [ /usr/bin/true ]
do
echo "Running sseevents.py"
python sseevents.py 2>&1 | tee -a sseevents.log.txt
echo "sleeping for 30 sec"
sleep 30
done
OUTPUT:-
Run call_notification.py on Terminal
node.js API OUTPUT
new_root_announced!
{'root_version': 'ABCD', 'BATS': '143'}
./run.sh --> DON'T SEE ABOVE EVENT below
Running sseevents.py
sleeping for 30 sec
Running sseevents.py
sleeping for 30 sec
Running sseevents.py
sleeping for 30 sec
Very short answer to you question:
The server code is not sending a SSE message back to the client.
Why? Because you need to follow the SSE format.
According to JASON BUTZ in Server-Sent Events With Node
You should send a Connection: keep-alive header to ensure the client keeps the connection open as well. A Cache-Control header should be sent with the value no-cache to discourage the data being cached. Finally, the Content-Type needs to be set to text/event-stream.
With all of that done a newline (\n) should be sent to the client and then the events can be sent. Events must be sent as strings, but what is in that string doesn’t matter. JSON strings are perfectly fine.
Event data must be sent in the format "data: <DATA TO SEND HERE>\n".
It’s important to note that at the end of each line should be a newline character. To signify the end of an event an extra newline character needs to be added as well.
Multiple data lines are perfectly fine.
Long answer to your question:
According to Eric Bidelman in html5rocks.com:
When communicating using SSEs, a server can push data to your app whenever it wants, without the need to make an initial request. In other words, updates can be streamed from server to client as they happen.
But, in order for this to happen, the client has to "start" by asking for it AND prepare to receive a stream of messages (when they happen).
The "start" is done by calling a SSE API endpoint (in your case, calling the Node.js API code).
The preparation is done by preparing to handle a stream of asynchronous messages.
SSEs open a single unidirectional channel between server and client.*
* The emphasis is mine
This means that the server has a "direct" channel to the client. It is not intended to be "started" (opened) by some other process/code that is not "the client" code.
Assuming from OP comments...
Expected behavior (verbose)
A client Alice calls the API endpoint with params {name: "Alice"}, nothing (visible) happens.
...then a client Bob calls the API endpoint with params {name: "Bob"}, client Alice receives a SSE with payload {name: "Bob", says: "Hi"}.
...then a client Carol calls the API endpoint with params {name: "Carol"}, clients Alice AND Bob each one receives a SSE with payload {name: "Carol", says: "Hi"}.
...and so on. Every time a new client calls the API endpoint with params, every other client who has a channel "open" will receive a SSE with the new "Hi" payload.
...and then client Bob "disconnects" from the server, client Alice, client Carol and all the clients that have a channel "open" will receive a SSE with payload {name: "Bob", says: "Bye"}.
...and so on. Every time an old client "disconnects" from the server, every other client who has a channel "open" will receive a SSE with the new "Bye" payload.
Abstracted behavior
Each new client that asks to "open" a channel sending some params or an old client "disconnects" from the server, they cause and event in the server.
Every time such an event happens in the server, the server sends a SSE message with the params and a message as payload to all the "open" channels.
Note on blocking Each client with an "open" channel will be "stuck" in an infinite waiting loop for events to happen. It is client design responsibility to use "threading" code techniques to avoid blocking.
Code
Your Python client should "ask" to start the single unidirectional channel AND keep waiting UNTIL the channel is closed. Should not end and start all over again with a different channel. It should keep the same channel open.
From the network perspective, it will be like a "long" response that does not end (until the SSE messaging is over). The response just "keeps coming and coming".
Your Python client code does that. I noted it is the exact sample code used from sseclient-py library.
Client code for Python 3.4
To include the parameters you want to send to the server, use some code from the Requests library docs/#passing-parameters-in-urls.
So, mixing those samples we end up with the following code as your Python 3.4 client:
import json
import pprint
import requests
import sseclient # sseclient-py
# change the name for each client
input_json = {'name':'Alice'}
#input_json = {'name':'Bob'}
#input_json = {'name':'Carol'}
url = 'http://company.com/api/root_event_notification'
stream_response = requests.get(url, params=input_json, stream=True)
client = sseclient.SSEClient(stream_response)
# Loop forever (while connection "open")
for event in client.events():
print ("got a new event from server")
pprint.pprint(event.data)
Client code for Python 2.7
To include the parameters you want to send to the server, encode them in the URL as query parameters using urllib.urlencode() library.
Make the http request with urllib3.PoolManager().request() so you will end up with a stream response.
Note that the sseclient library returns event data as unicode string. To convert back the JSON object to python object (with python strings) use byteify, a recursive custom function ( thanks to Mark Amery ).
Use the following code as your Python 2.7 client:
import json
import pprint
import urllib
import urllib3
import sseclient # sseclient-py
# Function that returns byte strings instead of unicode strings
# Thanks to:
# [Mark Amery](https://stackoverflow.com/users/1709587/mark-amery)
def byteify(input):
if isinstance(input, dict):
return {byteify(key): byteify(value)
for key, value in input.iteritems()}
elif isinstance(input, list):
return [byteify(element) for element in input]
elif isinstance(input, unicode):
return input.encode('utf-8')
else:
return input
# change the name for each client
input_json = {'name':'Alice'}
#input_json = {'name':'Bob'}
#input_json = {'name':'Carol'}
base_url = 'http://localhost:3000/api/root_event_notification'
url = base_url + '?' + urllib.urlencode(input_json)
http = urllib3.PoolManager()
stream_response = http.request('GET', url, preload_content=False)
client = sseclient.SSEClient(stream_response)
# Loop forever (while connection "open")
for event in client.events():
print ("got a new event from server")
pprint.pprint(byteify(json.loads(event.data)))
Now, the server code should:
emit an inside-server 'hello' event so other clients listen to the event
"open" the channel
Register to listen for all possible inside-server events to happen (this means, keeping the channel "open" and not sending anything between messages, just keeping the channel "open").
This includes to emit an inside-server 'goodbye' event so other clients listen to the event WHEN channel is closed by the client/network (and finally "wrap up").
Use the following Node.js API code:
var EventEmitter = require('events').EventEmitter;
var myEmitter = new EventEmitter;
function registerEventHandlers(req, res) {
// Save received parameters
const myParams = req.query;
// Define function that adds "Hi" and send a SSE formated message
const sayHi = function(params) {
params['says'] = "Hi";
let payloadString = JSON.stringify(params);
res.write(`data: ${payloadString}\n\n`);
}
// Define function that adds "Bye" and send a SSE formated message
const sayBye = function(params) {
params['says'] = "Bye";
let payloadString = JSON.stringify(params);
res.write(`data: ${payloadString}\n\n`);
}
// Register what to do when inside-server 'hello' event happens
myEmitter.on('hello', sayHi);
// Register what to do when inside-server 'goodbye' event happens
myEmitter.on('goodbye', sayBye);
// Register what to do when this channel closes
req.on('close', () => {
// Emit a server 'goodbye' event with "saved" params
myEmitter.emit('goodbye', myParams);
// Unregister this particular client listener functions
myEmitter.off('hello', sayHi);
myEmitter.off('goodbye', sayBye);
console.log("<- close ", req.query);
});
}
app.get("/api/root_event_notification", (req, res, next) => {
console.log("open -> ", req.query);
// Emit a inside-server 'hello' event with the received params
myEmitter.emit('hello', req.query);
// SSE Setup
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
});
res.write('\n');
// Register what to do when possible inside-server events happen
registerEventHandlers(req, res);
// Code execution ends here but channel stays open
// Event handlers will use the open channel when inside-server events happen
})
...continue quoting Eric Bidelman in html5rocks.com:
Sending an event stream from the source is a matter of constructing a plaintext response, served with a text/event-stream Content-Type, that follows the SSE format. In its basic form, the response should contain a "data:" line, followed by your message, followed by two "\n" characters to end the stream
In the client code, the sseclient-py library takes care of interpreting the SSE format so every time the two "\n" characters arrive, the library "iterates" a new "iterable" object (a new event) that has the data property with the message sent from the server.
This is how I tested the code
Started server with Node.js API code
Run a client with only the "Alice" line uncommented (Nothing is seen on this client console yet).
Run a second client with only "Bob" line uncommented. The console of the first client "Alice" shows: Bob saying "Hi" (Nothing is seen on Bob's client console yet).
Run a third client with only "Carol" line uncommented. Alice's and Bob's consoles show: Carol saying "Hi" (Nothing is seen on Carol's client console yet).
Stop/kill Bob's client. Alice's and Carol's consoles show: Bob saying "Bye".
So, code works OK :)
I am trying to create a grpc python server that can keep track of all clients connected.
I am referencing a talk/demo that Ray Tsang did where he kept a collection of StreamObservers and just iterated through them to send to all the clients. Here is a video of that for reference.
Now my question is how do you get a StreamObserver in python? I only see self, request and context as being available to me in the definition.
This is my first python project so there might be something obvious I am missing here.
Here is my proto, its basically the sample proto
syntax = "proto3";
package hellostreamingworld;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
// Sends another greeting
rpc SayHelloAgain (HelloRequest) returns (stream HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
If I have understand you need to create a class ClientInterceptor that extends grpc.UnaryUnaryClientInterceptor(https://grpc.io/grpc/python/grpc.html?highlight=unaryunaryclientinterceptor#grpc.UnaryUnaryClientInterceptor)
and then assign it with the intercept_channel method in this way
self.channel = grpc.insecure_channel(address, options)
self.channel = grpc.intercept_channel(
self.channel,
ClientInterceptor()
)
You can use the intercept_unary_unary method for receive infos about the various clients.
If you want to have infos server side extends the ServerInterceptor(https://grpc.io/grpc/python/grpc.html?highlight=serverinterceptor#grpc.ServerInterceptor) and assign it on the server init
self.server = grpc.server(futures.ThreadPoolExecutor(max_workers=1000),
options=(('grpc.max_send_message_length', 1000 * 1024 * 1024,),
('grpc.max_receive_message_length', 1000 * 1024 * 1024,),
),
interceptors=(MyServerInterceptor())
)
And the use the intercept_service method for receive infos.
What you probably need is a bidirectional stream service. That is both the client and server are continuously sending data over. In this case gRPC + python will keep track of your client!
Check out this example code and corresponding explanation.
I'm trying to adapt this Java codeLabs to Python / SocketIO (Flask Socket IO on server side).
My socket is working, I'm able to pass both the sample rate (and to retrieve on server side), and the audio data.
The problem is that the format of my audio data is not correct. I send it this way:
// Create a node that sends raw bytes across the websocket
var scriptNode = context.createScriptProcessor(4096, 1, 1);
// Need the maximum value for 16-bit signed samples, to convert from float.
const MAX_INT = Math.pow(2, 16 - 1) - 1;
scriptNode.addEventListener('audioprocess', function(e) {
var floatSamples = e.inputBuffer.getChannelData(0);
// The samples are floats in range [-1, 1]. Convert to 16-bit signed
// integer.
socket.emit('audiodata', Int16Array.from(floatSamples.map(function(n) {
return n * MAX_INT;
})));
This is the code from the codeLabs but I use socket.emit instead of socket.send.
The python code to handle and send it to Speech API is this one:
credentials = service_account.Credentials.from_service_account_file(CONFIG[u"service_account"])
client = speech.SpeechClient(credentials=credentials)
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16,
sample_rate_hertz=44100,
language_code=u"fr"
)
streaming_config = types.StreamingRecognitionConfig(
config=config,
interim_results=True,
single_utterance=False)
#socketio.on(u'audiodata')
def handle_audio_data_msg(audio_data_msg):
print u'Received raw audio.'
print audio_data_msg
for response in client.streaming_recognize(streaming_config, [audio_data_msg]):
print response
emit(u'audiodatareceived', {u"AudioData": u"Acquired"})
I got TypeError: descriptor 'SerializeToString' requires a 'google.protobuf.pyext._message.CMessage' object but received a 'dict'
so I tried, on front side, to pass Int16Array.from(...).buffer but I get TypeError: descriptor 'SerializeToString' requires a 'google.protobuf.pyext._message.CMessage' object but received a 'str'
So I'm not sure how I should pass the data... Any help appreciated!
[edit]
I re-wrote my handler cause I think I was not using the right type, it is now like that:
#socketio.on(u'audiodata')
def handle_audio_data_msg(audio_data_msg):
print u'Received raw audio.'
request = types.StreamingRecognizeRequest(audio_content=audio_data_msg)
response = client.streaming_recognize(streaming_config, [request])
for item in response:
print u"error", item.get(u"error")
print u"results", item.get(u"results")
print u"resultIndex", item.get(u"resultIndex")
print u"endpointerType", item.get(u"endpointerType")
emit(u'audiodatareceived', {u"AudioData": u"Acquired"})
I don't get any error but I get no response (empty list) and so nothing printed...
[edit2]: I realized that the type of my input was never binary data but str. Could the problem come from here ? I try to instantiate my Flask Socket IO app as follow, but still the type is str.
socketio = SocketIO(app, binary=True)
I'm following this Route_Guide sample.
The sample in question fires off and reads messages without replying to a specific message. The latter is what i'm trying to achieve.
Here's what i have so far:
import grpc
...
channel = grpc.insecure_channel(conn_str)
try:
grpc.channel_ready_future(channel).result(timeout=5)
except grpc.FutureTimeoutError:
sys.exit('Error connecting to server')
else:
stub = MyService_pb2_grpc.MyServiceStub(channel)
print('Connected to gRPC server.')
this_is_just_read_maybe(stub)
def this_is_just_read_maybe(stub):
responses = stub.MyEventStream(stream())
for response in responses:
print(f'Received message: {response}')
if response.something:
# okay, now what? how do i send a message here?
def stream():
yield my_start_stream_msg
# this is fine, i receive this server-side
# but i can't check for incoming messages here
I don't seem to have a read() or write() on the stub, everything seems to be implemented with iterators.
How do i send a message from this_is_just_read_maybe(stub)?
Is that even the right approach?
My Proto is a bidirectional stream:
service MyService {
rpc MyEventStream (stream StreamingMessage) returns (stream StreamingMessage) {}
}
What you're trying to do is perfectly possible and will probably involve writing your own request iterator object that can be given responses as they arrive rather than using a simple generator as your request iterator. Perhaps something like
class MySmarterRequestIterator(object):
def __init__(self):
self._lock = threading.Lock()
self._responses_so_far = []
def __iter__(self):
return self
def _next(self):
# some logic that depends upon what responses have been seen
# before returning the next request message
return <your message value>
def __next__(self): # Python 3
return self._next()
def next(self): # Python 2
return self._next()
def add_response(self, response):
with self._lock:
self._responses.append(response)
that you then use like
my_smarter_request_iterator = MySmarterRequestIterator()
responses = stub.MyEventStream(my_smarter_request_iterator)
for response in responses:
my_smarter_request_iterator.add_response(response)
. There will probably be locking and blocking in your _next implementation to handle the situation of gRPC Python asking your object for the next request that it wants to send and your responding (in effect) "wait, hold on, I don't know what request I want to send until after I've seen how the next response turned out".
Instead of writing a custom iterator, you can also use a blocking queue to implement send and receive like behaviour for client stub:
import queue
...
send_queue = queue.SimpleQueue() # or Queue if using Python before 3.7
my_event_stream = stub.MyEventStream(iter(send_queue.get, None))
# send
send_queue.push(StreamingMessage())
# receive
response = next(my_event_stream) # type: StreamingMessage
This makes use of the sentinel form of iter, which converts a regular function into an iterator that stops when it reaches a sentinel value (in this case None).
I'm writing an http server in python3.3, just to learn how to do this sort of thing. In my function that parses a request, I want to use fcntl.ioctl to get the number of bytes that I can read in the socket, and I only do this when I see a kevent in the result of checking a kqueue that says there is stuff to read on the socket. But whenever I try to call fcntl.ioctl, I get OSError: [Errno 14] Bad address. What am I doing wrong? Also, this seems to be happening on the first call. Here is the relevant code:
def client_thread(kq, client_socket, methods):
while True:
events = kq.control([], 2, POLLTIME) #we pass an empty list of changes, because we don't have any changes to make to the events we are interested in.
#we want a list that is at most two long. We listen for POLLTIME seconds.
for event in events:
if event != KILL_KEV: #there are only two events in our kqueue
handle_client(client_socket, methods)
else: #KILL_SOCK has a connection
break
client_socket.close()
client_socket.shutdown()
def handle_client(client_socket, methods):
request = parse_request(client_socket) #parse the request data in the client socket
handlers = methods[request["request"]["method"]] #retrieve the appropriate list of handlers from the methods dict
for path_match_pred, handler_func in handlers:
if path_match_pred(path): #if the path matches whatever path predicate you've created...
break
response = handler_func(request) #... then call the appropriate handler function to handle the request
send_response(client_socket, response) #and finally, send the response.
def parse_request(client_socket):
"""Returns the request data, parsed into a dictionary like this:
{
"request": {
"method": method,
"path": path,
"version": HTTP version
},
"headers": header dictionary,
"body": body data as a string
}
This should only be called if the client socket is ready for reading!
"""
client_fd = client_socket.fileno() #get the file descriptor for the socket
bytes_in_socket = 0
fcntl.ioctl(client_fd, termios.FIONREAD, bytes_in_socket) #count the bytes in it
#^^^^^^^^^THIS IS WHERE IT BREAKS
print(bytes_in_socket, "bytes in socket")
msg = bytearray() #make empty byte array
while bytes_in_socket:
msg.extend(client_socket.recv(bytes_in_socket)) #read the bytes we counted earlier
fcntl.ioctl(client_fd, termios.FIONREAD, bytes_in_socket) #check for more bytes
print(bytes_in_socket, "bytes left to read")
Note that in fcntl.ioctl's documentation, they mention the acceptable types for arg (the argument to the ioctl operation). In some cases, (like this), you need to pass a buffer, or an object that supports its interface. In particular, when you want to receive a value back. You're just passing an integer.