After trying a few things, I got to a point where I can run client-server gRPC connection that is UDS based on the same host. Since I didn't find any detailed documentation for this setup, my main concern is that I don't know if I use gRPC API properly.
Any reference/resource/explanation would be warmly welcomed
sources I looked at:
gRPC python API
gRPC python docs
gRPC server in Python with Unix domain socket (SO)
gRPC connection:
server
import grpc
import my_pb2_grpc
from concurrent import futures
server = None
#...
class Listener(my_pb2_grpc.MyServiceServicer):
#...
def create_connection():
if server is None:
server = grpc.server(futures.ThreadPoolExecutor(max_workers=1))
my_pb2_grpc.add_MyServiceServicer_to_server(Listener(), server)
server.add_insecure_port("unix:///path/to/uds.sock")
server.start()
def stop_connection():
if server:
server.stop(0)
server = None
client
import grpc
import my_pb2_grpc
channel = None
stub = None
#...
def create_connection():
if channel is None:
channel = grpc.insecure_channel("unix:///path/to/uds.sock")
stub = my_pb2_grpc.MyServiceStub(channel)
def stop_connection():
if channel:
channel.close()
channel = None
stub = None
Related
my goal is to implement a gRPC client that is unfamiliar of the proto file (only knows server/service port number) - the client find the service on run-time (reflection)
my client should be able to:
connect to the server
learn which server capabilities exists (including messages)
send rpc requests to the enable-reflection server
i already see an example in Go & Java and failed to find one in python that work for me.
i successfully used "evans" cli client with my server but i want to implement cli client on my own.
https://github.com/grpc/grpc/blob/master/doc/server_reflection_tutorial.md
i look on this example of how to implementing the server & i can use some help to better understand how to implement the client.
https://github.com/grpc/grpc/blob/master/examples/python/helloworld/greeter_server_with_reflection.py
my questions are:
how to find the current service from the client only from port number?
how to communicate with the server without importing the generated proto files?
server side:
from __future__ import print_function
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
def run():
# NOTE(gRPC Python Team): .close() is possible on a channel and should be
# used in circumstances in which the with statement does not fit the needs
# of the code.
#
# For more channel options, please see https://grpc.io/grpc/core/group__grpc__arg__keys.html
with grpc.insecure_channel(target='localhost:50051',
options=[('grpc.lb_policy_name', 'pick_first'),
('grpc.enable_retries', 0),
('grpc.keepalive_timeout_ms', 10000)
]) as channel:
stub = helloworld_pb2_grpc.GreeterStub(channel)
# Timeout in seconds.
# Please refer gRPC Python documents for more detail. https://grpc.io/grpc/python/grpc.html
response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'),
timeout=10)
print("Greeter client received: " + response.message)
if __name__ == '__main__':
logging.basicConfig()
run()
client side:
from __future__ import print_function
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
def run():
# NOTE(gRPC Python Team): .close() is possible on a channel and should be
# used in circumstances in which the with statement does not fit the needs
# of the code.
with grpc.insecure_channel('localhost:50051') as channel:
stub = helloworld_pb2_grpc.GreeterStub(channel)
response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
print("Greeter client received: " + response.message)
if __name__ == '__main__':
logging.basicConfig()
run()
I'm writing a program with websockets in python. I've got an example server and client code running and they work well if only one client is connected. If there are multiple clients, data from the server will go randomly to one of the clients.
I would like for:
Server to keep track of the various clients connected
Server to be able to direct messages to a specific client out of multiple(For eg. 5) clients
websockets is the library I'm using.
Python version 3.7.2
Server Code:
import asyncio
import websockets
uri='localhost'
async def response(websocket, path):
msg = input("What do you want to send : ")
print("message:",msg)
await websocket.send(msg)
start_server = websockets.serve(response, uri, 5000)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Client Code:
import asyncio
import websockets
uri="ws://localhost:5000"
async def message():
async with websockets.connect(uri) as socket:
print(await socket.recv())
while True:
asyncio.get_event_loop().run_until_complete(message())
If I create 2 files with the client code as client1.py and client2.py, and send message from the server side, I get the sent data going to either on of the clients.
I would like to:
Server keeps track of the various clients connected
Server is able to direct messages to a specific client out of multiple clients
As I am just starting out with websockets, all input is appreciated.
In this output given, I intended to send all my messages to client 1, yet they got split up between client 1 and 2
"websocket" targets the current connection and if you say "websocket.send(msg)" you're sending a message to the client that has just connected and websocket is an object that is reusable while the client is connected. You can assign the websocket as a variable then send a message some other time as long as the connection is still opened.
NOT RECOMMENED
Requiring user's input from the server is not a good idea because now you're awaiting the server until it receives user inputs. Nothing really happens to the server while it's waiting for user's input and this may crush your server.
RECOMMENED
A client has to tell the server which connection / client to send the message to. I would recommend using a JSON format when sending messages within client's and the server and then convert the String to a python-dict since websocket requires only strings.
Click here to check out my GitHub repository. A websockets server made only Python.
SERVER EXAMPLE
Example on how you can send a packet to a specific client
I'm not familiar with asyncio, so I will try to get to the point with functions/threads;
Usually, my server side listens to one connection and once it accepts it, I have a function 'handler' that is threaded to each connection that gets accepted.
part of my server and handler:
def handler(conn, addr):
global data1
while True:
data = conn.recv(2048)
data1 = json.loads(data.decode())
# treat it as you need
while True:
s.listen(1)
conn, addr = s.accept()
print('Conectado com', addr[0],':', str(addr[1]))
thr = threading.Thread(target = handler, args = (conn, addr)).start()
Now, for the control of the clients and such, I always use a dictionary. The key should be the username or any other particular info. The value of the key is the 'conn' from that user. This way you can get the connection of user 'x' by its specific key.
Something like:
import socket
import time
import datetime as dt
import base64
import os
import json
import threading
HOST = ''
PORT = 12999
global s
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST,PORT))
whoto = {}
def autenticacao():
global data1
global conn
global nomeuser
user1 = data1[1]
passwd = data1[2]
auth = c.execute('SELECT usuario FROM fullinfo WHERE usuario= ? AND password = ?', (user1,passwd)).fetchone()
if auth is not None:
word = 'autenticado'
conn.sendall(word.encode())
nomeuser = auth[0]
whoto[nomeuser] = conn
I'm sorry i'm leaving it unreproducible, but my point is to show the 'algorithm'. This dictionary is what I use to keep record of who is online, the 'adress' (conn) of each client to send messages to single clients and such.
On the example above, I add the user (key) and conn (value) once it's authenticated inside my server.
Hope this helps. Good luck!
I am currently using rpyc for constructing a server and multiple clients that will connect to it. I have data in the clients that I would like to push to the server for further processing, and I would like to do it with the server calling a client method whenever the client connects to the server. From their tutorial, it says that clients can expose their service to the server, but I am getting errors.
Code for my server:
import rpyc
class TestService(rpyc.Service):
def on_connect(self, conn):
conn.root.example()
if __name__ == "__main__":
from rpyc.utils.server import ThreadedServer
t = ThreadedServer(TestService, port=18861, auto_register=True)
t.start()
Code for my client:
import rpyc
class ClientService(rpyc.Service):
def exposed_example(self):
print "example"
if __name__ == "__main__":
try:
rpyc.discover("test")
c = rpyc.connect_by_service("test", service=ClientService)
c.close()
except:
print "could not find server"
The client is able to connect to the server, but there will be an exception in thread and an error: raise EOFError("stream has been closed"). It is complaining about the line conn.root.example() and I don't know what the correct syntax would be, and the tutorial did not specify at all.
Any help will be much appreciated!
I deployed my python gRPC server to Heroku and was wondering how I could test it with a local Python client.
server.py
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
icp_pb2_grpc.add_MyServicer_to_server(MyServicer(), server)
server_port = os.environ.get('PORT', 50051)
server.add_insecure_port('[::]:'+ str(server_port))
server.start()
print("==== SERVER RUNNING =====")
try:
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
server.stop(0)
if __name__ == '__main__':
serve()
client.py
def run():
# Is the channel url correct?
channel = grpc.insecure_channel('https://www.HEROKUURL.com:50051')
stub = my_grpc.MyStub(channel)
file = _get_file_content()
response = stub.Predict(icp_pb2.MyRequest(file_content=file))
print("received: " + response.results)
I am using the client from my computer and am not receiving any response from the server. I am able to successfully communicate with the server if it is launched locally. What am I doing wrong?
Heroku does not support HTTP 2. On the other hand, GRPC uses a http/2 based transport. I think that's why you can connect it locally but not from Heroku.
I'm writing python client using thrift, but I can't find any available option to set rpc time out.
My client code goes below:
socket = TSocket.TSocket(address, port)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
server = Client.Client(protocol)
transport.open()
You can use socket.setTimeout() method.
from thrift.transport.THttpClient import THttpClient
socket = THttpClient(server_url)
socket.setTimeout(SERVICE_TIMEOUT_IN_mS)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocol(transport)