We Keep Coding

How to receive data in websocket with Webdis?

I used webdis in docker, with a python file in the backend:
import redis
r = redis.Redis(host='localhost', port=6379, db=0)
value = "tests"
r.publish('mychannel', value)
And a javascript frontend that uses a websocket
var time1 = null
var time2 = null
var socket = new WebSocket('ws://localhost:8765/ws');
socket.onmessage = function(e) {
var server_message = e.data;
if (server_message == "start") {
time1 = new Date().valueOf()
}
if (server_message == "end") {
time2 = new Date().valueOf()
console.log("100000 messages took ", (time2 - time1) / 1000, "seconds")
window.location.reload()
}
document.getElementById("test").innerText = server_message
}
The python script runs fine and the javascript can connect to the websocket but it's not doing anything

.py file executed by C# process not waiting to finish

I want to run .py file from my C# project, and get the result. The python script is making an API request, and returns an auth_key token, which I want to use in my C# code. The only problem is that, for some reason the C# code doesn't wait for the process to finish, and thus that not every account has auth_key. Here is my C# code.
private static void GenerateTokens()
{
var url = ConfigurationManager.AppSetting[GeSettingsNode() + ":ip"];
for (int i = 0; i < accounts.Count; i++)
{
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = ConfigurationManager.AppSetting["PythonPath"];
start.Arguments = string.Format($"python_operation_processor.py {accounts[i].client_key_id} {accounts[i].key_sercret_part} {url}");
start.UseShellExecute = false;
start.RedirectStandardOutput = true;
Process process = Process.Start(start);
using (StreamReader reader = process.StandardOutput)
{
accounts[i].auth_key = reader.ReadToEnd().Trim();
}
}
}
And here is my Python script ( python_operation_processor.py )that's making the API requests.
if __name__ == '__main__':
client_key_id = sys.argv[1]
client_secret = sys.argv[2]
API_URL = sys.argv[3]
nonce = str(uuid.uuid4())
d = datetime.datetime.now() - datetime.timedelta(hours=3)
timestamp = d.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'
signature = b64encode(hmac.new(b64decode(client_secret), msg=bytes(client_key_id + nonce + timestamp, 'utf-8'),
digestmod=hashlib.sha256).digest()).decode('utf-8')
r = requests.post(API_URL + '/v1/authenticate',
json={'client_key_id': client_key_id, 'timestamp': timestamp, 'nonce': nonce,
'signature': signature})
if r.status_code != 200:
raise Exception('Failed to authenticate: ' + r.text)
auth_token = r.json()['token']
print(auth_token)
Do you have any idea, how I can wait for the execution of every process, and get the token for every account ?

I recently created something similar and ended up with this because, whilst waiting for the process is easy, it is tricky to get the output stream filled correctly.
The method presented also allow you to display the output into a textblock or similar in your application.
If you use it like this, the token will be written to the StringBuilder, and used as return value.
private async Task<string> RunCommand(string fileName, string args)
{
var timeoutSignal = new CancellationTokenSource(TimeSpan.FromMinutes(3));
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = fileName;
start.Arguments = string.Format("{0}", args);
start.RedirectStandardOutput = true;
start.RedirectStandardError = true;
start.UseShellExecute = false;
start.CreateNoWindow = true;
var sb = new StringBuilder();
using (Process process = new Process())
{
process.StartInfo = start;
process.OutputDataReceived += (sender, eventArgs) =>
{
sb.AppendLine(eventArgs.Data); //allow other stuff as well
};
process.ErrorDataReceived += (sender, eventArgs) => {};
if (process.Start())
{
process.EnableRaisingEvents = true;
process.BeginOutputReadLine();
process.BeginErrorReadLine();
await process.WaitForExitAsync(timeoutSignal.Token);
//allow std out to be flushed
await Task.Delay(100);
}
}
return sb.ToString();
}
To render this to a textblock in a UI application, you'll need to:
implement an event which signals a new line has been read, which means forwarding the process.OutputDataReceived event.
if your thinking about a live feed, make sure you flush the stdio buffer in python setting flush to true: print(""hello world"", flush=True)
If you're using an older .net version; you can implement the WaitForExitAsync as described here: https://stackoverflow.com/a/17936541/2416958 as an extention method:
public static class ProcessHelpers
{
public static Task<bool> WaitForExitAsync(this Process process, TimeSpan timeout)
{
ManualResetEvent processWaitObject = new ManualResetEvent(false);
processWaitObject.SafeWaitHandle = new SafeWaitHandle(process.Handle, false);
TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
RegisteredWaitHandle registeredProcessWaitHandle = null;
registeredProcessWaitHandle = ThreadPool.RegisterWaitForSingleObject(
processWaitObject,
delegate(object state, bool timedOut)
{
if (!timedOut)
{
registeredProcessWaitHandle.Unregister(null);
}
processWaitObject.Dispose();
tcs.SetResult(!timedOut);
},
null /* state */,
timeout,
true /* executeOnlyOnce */);
return tcs.Task;
}
}

gRPC Node Js client: "Method not found"

I have a gRPC server implemented in python and I am calling an RPC from NodeJS but it gives an error "Method not found". When I call using the python client, the request is successful.
stream_csv.proto
syntax = "proto3";
package csv;
service Stream {
rpc csvToObject(CSVDataRequest) returns (stream CSVDataResponse) {};
rpc sayHello(HelloRequest) returns (HelloReply);
}
message CSVDataRequest{
string url = 1;
enum Protocol {
HTTP = 0;
HTTPS = 1;
FTP = 2;
SFTP = 3;
}
Protocol protocol = 2;
}
message CSVDataResponse{
repeated string row = 1;
}
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
client.js
var PROTO_PATH = '../stream_csv.proto';
var grpc = require('grpc');
var protoLoader = require('#grpc/proto-loader');
var packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
var proto = grpc.loadPackageDefinition(packageDefinition).csv;
function main() {
var client = new proto.Stream('localhost:5000',
grpc.credentials.createInsecure());
var user;
if (process.argv.length >= 3) {
user = process.argv[2];
} else {
user = 'world';
}
console.log(user);
client.sayHello({name: user}, function(err, response) {
console.log(err);
});
}
main();
server.py
import grpc
import stream_csv_pb2
import urllib.request
from urllib.error import HTTPError, URLError
from concurrent import futures
class DataService:
def csv_to_object(self, request, context):
url = request.url
protocol = stream_csv_pb2.CSVDataRequest.Protocol.Name(
request.protocol)
fetch_url = protocol.lower() + "://"+url
try:
with urllib.request.urlopen(fetch_url) as data:
for line in data:
decoded_line = line.decode()
val = decoded_line.split(',')
print(val)
print("Data send")
yield stream_csv_pb2.CSVDataResponse(row=val)
print("Sending finished!")
except URLError as e:
context.abort(grpc.StatusCode.UNKNOWN,
'Randomly injected failure.')
# return stream_csv_pb2.CSVDataResponse(row=[], error=e.reason)
def SayHello(self, request, context):
name = request.name
print(name)
return stream_csv_pb2.HelloReply(message='Hello %s' % (name))
def add_DataServicer_to_server(servicer, server):
rpc_method_handlers = {
'CSVToObject': grpc.unary_stream_rpc_method_handler(
servicer.csv_to_object,
request_deserializer=stream_csv_pb2.CSVDataRequest.FromString,
response_serializer=stream_csv_pb2.CSVDataResponse.SerializeToString,
),
'SayHello': grpc.unary_unary_rpc_method_handler(
servicer.SayHello,
request_deserializer=stream_csv_pb2.HelloRequest.FromString,
response_serializer=stream_csv_pb2.HelloReply.SerializeToString,
)
}
generic_handler = grpc.method_handlers_generic_handler(
'stream_csv.Stream', rpc_method_handlers)
server.add_generic_rpc_handlers((generic_handler,))
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
add_DataServicer_to_server(DataService(), server)
server.add_insecure_port('[::]:5000')
server.start()
server.wait_for_termination()
if __name__ == '__main__':
serve()
Error
Error: 12 UNIMPLEMENTED: Method not found!
at Object.exports.createStatusError (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/common.js:91:15)
at Object.onReceiveStatus (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:1209:28)
at InterceptingListener._callNext (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:568:42)
at InterceptingListener.onReceiveStatus (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:618:8)
at callback (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:847:24) {
code: 12,
metadata: Metadata { _internal_repr: {}, flags: 0 },
details: 'Method not found!'
}

I wrote a simpler variant of your Python (and Golang) server implementations.
Both work with your Node.JS client as-is.
I think perhaps your issue is as simple as needing to name the rpc sayHello (rather than SayHello).
from concurrent import futures
import logging
import grpc
import stream_csv_pb2
import stream_csv_pb2_grpc
class Stream(stream_csv_pb2_grpc.StreamServicer):
def sayHello(self, request, context):
logging.info("[sayHello]")
return stream_csv_pb2.HelloReply(message='Hello, %s!' % request.name)
def serve():
logging.info("[serve]")
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
stream_csv_pb2_grpc.add_StreamServicer_to_server(Stream(), server)
server.add_insecure_port('[::]:50051')
server.start()
server.wait_for_termination()
if __name__ == '__main__':
logging.basicConfig()
serve()
And:
node client.js
Greeting: Hello, Freddie!
null
Conventionally (!) gRPC follows Protobufs style guide of CamelCasing service, rpc and message names and using underscores elsewhere see Style Guide. When protoc compiles, the results don't always match perfectly (e.g. Python's CamelCased functions rather than lowercased with underscores).
Conventionally, your proto would be:
service Stream {
rpc CsvToObject(CSVDataRequest) returns (stream CSVDataResponse) {};
rpc SayHello(HelloRequest) returns (HelloReply);
}
...and then the Python generated function would be (!) SayHello too.

How to automatically start and stop a python script in nodejs?

I am developing a temperature monitoring application in a hen house with a web interface. I use two arduinos and a Raspberry.
Arduino 1: I connected a temperature / humidity sensor and an RF433Mhz transmitter.
Arduino 2: An RF433Mhz receiver is connected to it. It receives data from Arduino 1.
Raspberry: Arduino 2 is connected to my raspberry which reads the data received in the serial monitor and send them to the web page via the websockets (package ws of nodejs).
At first I wanted to read this data directly with Nodejs, but I had some problems with the installation of the serial port package.
So I changed my approach: I read the data in the serial monitor with python, write it in files, and Nodejs reads these files and sends the data to the web page.
here are the two codes I use:
Phyton script
import serial
import time
ser = serial.Serial('/dev/ttyACM0', 9600)
while True:
data = ser.readline()
if data:
t = data[0:2]
h = data[6:8]
#decode utf-8
tc = t.decode("utf-8")
hc = h.decode("utf-8")
#write the temperature in the temp file
fileTc=open('temp', 'w')
fileTc.write(str(tc))
fileTc.close
#write the humidity in the hum file
fileHc=open('hum', 'w')
fileHc.write(str(hc))
fileHc.close
#sleep
time.sleep(.1)
Nodejs Script
var express = require("express");
const WebSocket = require('ws');
const wss = new WebSocket.Server({port: 4400});
var path = require("path");
var fs = require("fs");
var sys = require("util");
var exec = require("child_process").exec;
var tempcpu = 0;
var temp = 0;
var hum = 0;
var app = express();
app.set("port", process.env.PORT || 5500);
app.set("views", path.join(__dirname, "views"));
app.set("view engine", "ejs");
app.use('/', express.static('public'));
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('received: %s', message);
});
setInterval(function(){
child1 = exec("cat /sys/class/thermal/thermal_zone0/temp",
function(error, stdout,stderr){
if (error !== null){
console.log('exec error: ' +error);
} else{
tempcpu = parseFloat(stdout)/1000;
}
});
child2 = exec("cat temp", function(error, stdout,stderr){
if (error !== null){
console.log('exec error: ' +error);
} else{
temp = parseFloat(stdout);
}
});
child3 = exec("cat hum", function(error, stdout,stderr){
if (error !== null){
console.log('exec error: ' +error);
} else{
hum = parseFloat(stdout);
}
});
var tempCPU = JSON.stringify(["cpu",tempcpu]);
var temperature = JSON.stringify(["temp",temp]);
var humidity = JSON.stringify(["hum",hum]);
ws.send(tempCPU);
ws.send(temperature);
ws.send(humidity);
}, 5000);
});
app.get("/", function(request, response) {
response.render("dashboard");
});
app.listen(app.get("port"), function() {
console.log("Server started at port " + app.get("port"));
});
for now I have to launch both scripts separately. I would like to run my python script directly from nodejs when I start the node server, and stop it when I stop my nodejs code (CTRL + C).
Do you have an idea of ​​how to do it?

What you want to achieve is spawn a new process in which you execute something from either a Node app or a Python app:
NodeJS approach: Child process
const { spawn } = require('child_process');
const pythonApp = spawn('python', ['my_python_app.py']);
Python approach: Subprocess
import subprocess
node_app = subprocess.Popen(["node","my_node_app.js"], stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
EDIT
Regarding catching the INTERRUPT (CTRL+C) signal, this can also be done in both languages; and leveraged to kill the process you spawned:
With NodeJS:
process.on('SIGINT', () => {
console.log("Caught interrupt signal");
if(pythonApp) pythonApp.exit();
});
With Python:
import sys
try:
# Your app here...
except KeyboardInterrupt:
print("Caught interrupt signal")
if node_app: node_app.kill()
sys.exit()

Calling named pipe of Dropbox to get status: Error: read EPIPE

I have working Python script that checks local Windows Dropbox application sync status that i try to port to nodejs (to use in Electron) .
The problem is that i get:
events.js:141
throw er; // Unhandled 'error' event
^
Error: read EPIPE
at exports._errnoException (util.js:907:11)
at Pipe.onread (net.js:557:26)
when i try my node script.
Here's the python script:
import win32pipe, win32ts, win32api, pywintypes, struct, sys
yourUsername = "YourUsername"
def dropbox_path_status(pathname):
return ['dropbox not running','not in dropbox','up to date','syncronising','sync problem'][dropbox_path_status_code(pathname)+1]
def dropbox_path_status_code(pathname):
processid = win32api.GetCurrentProcessId()
threadid = win32api.GetCurrentThreadId()
request_type = 1
wtf = 0x3048302
pipename = r'\\.\pipe\DropboxPipe_' + str(win32ts.ProcessIdToSessionId(processid))
request = (struct.pack('LLLL', wtf, processid, threadid, request_type) + pathname.encode('utf-16le') + (chr(0)*540))[0:540]
try:
response = win32pipe.CallNamedPipe(pipename, request, 16382, 1000)
except pywintypes.error, err:
if err[0] == 2:
return -1
else:
raise
else:
return int(response[4:-1])
print dropbox_path_status("C:\Users\"+yourUsername+"\Dropbox")
Here's my node script
var net = require('net');
var ffi = require('ffi');
var ref = require('ref');
var BufferStream = require('node-bufferstream')
var PIPE_PATH = "\\\\.\\pipe\\DropboxPipe_1"
var UserName = "YourUsername"
var L = console.log;
var SessionId = ref.alloc("uint");
var kernel32 = new ffi.Library("kernel32",{
GetLastError:['string',[]],
GetCurrentProcessId: ['int',[]] ,
GetCurrentThreadId: ['int',[]],
ProcessIdToSessionId: ['bool',['int','uint *']],
CallNamedPipeA:['bool',[]]
});
var RequestInfo = 0x3048302;
var ProcessId = kernel32.GetCurrentProcessId();
var ThreadId = kernel32.GetCurrentThreadId();
var RequestType =1;
var dropbox = ref.types.void;
if(!kernel32.ProcessIdToSessionId(ProcessId,SessionId )){
console.log(kernel32.GetLastError());
}
else{
console.log(SessionId);
SessionId.type = ref.types.int;
SessionIdInt = SessionId.deref();
console.log(SessionIdInt);
}
var TestBufferIn = new Buffer(16+724);
TestBufferIn.writeUInt32LE(RequestInfo, 0);
TestBufferIn.writeUInt32LE(ProcessId, 4);
TestBufferIn.writeUInt32LE(ThreadId, 8);
TestBufferIn.writeUInt32LE(RequestType, 12);
//TestBufferIn.writeUInt32LE(1234567, 600);
console.log(TestBufferIn.length)
var mypath = "C:/Users/"+UserName+"/Dropbox/"
TestBufferIn.write(mypath,16,162,"utf-16le")
stream = new BufferStream(TestBufferIn);
var L = console.log;
var client = net.createConnection({ path: '\\\\.\\pipe\\DropboxPipe_1'}, function () { console.log('connected');
client.write(TestBufferIn, ()=>{
console.log("write callback")
});
})
client.on('data', function(data) {
L('Client: on data:', data.toString());
client.end('Thanks!');
});
client.on('end', function() {
L('Client: on end');
})
Any idea why the error appears?
EDIT
When i write with string (instead of buffer) i get same error.
var client = net.createConnection({ path: '\\\\.\\pipe\\DropboxPipe_1'}, function () { console.log('connected');
client.write("hello", ()=>{
console.log("write callback"); //this is written in console
});
});