We Keep Coding

Calling a Python script from C# returns an empty output

I am creating a web application using ASP NET that runs on Windows Server 2012 using IIS 10. This application exposes a service at www.domain.com/service/execute which uses a python script executed using Python 3.9.13. Below I report the C# and Python code:
main.py
import pprint
pprint.pprint("Hello World!!")
ServiceController.cs
public class ServiceController : Controller {
public IActionResult Execute() {
var FileName = $#"C:\\path\\to\\python.exe";
var Arguments = $#"C:\\path\\to\\main.py";
var processStartInfo = new ProcessStartInfo() {
FileName = FileName,
Arguments = Arguments,
CreateNoWindow = true,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
};
using(var process = Process.Start(processStartInfo)) {
var taskError = process.StandardError.ReadToEndAsync();
var taskOutput = process.StandardOutput.ReadToEndAsync();
taskError.Wait();
taskOutput.Wait();
var errors = taskError.Result.Trim();
var output = taskOutput.Result.Trim();
return Ok(
$"FileName={FileName}" +
$"\n" +
$"Arguments={Arguments}" +
$"\n" +
$"errors={errors}" +
$"\n" +
$"output={output}"
);
}
}
}
Calling this service I don't get any errors or exceptions but the output is empty:
FileName=C:/path/to/python.exe
Arguments=C:/path/to/main.py
errors=
output=
If in the C# script I insert as Arguments a totally invented path to a non-existent python script I don't get any error/exception and the output is still empty.
I would expect an output like this:
FileName=C:/path/to/python.exe
Arguments=C:/path/to/main.py
errors=
output=Hello World!!

Empty output when calling a python script in C# project

I would like to call a python script in my C# project , I'm using this function to do the job but unfortunately I didn't get any result and the result variable shows always an empty output. I would like to know what's the reason of this
public string RunFromCmd(string rCodeFilePath, string args)
{
string file = rCodeFilePath;
string result = string.Empty;
try
{
var info = new ProcessStartInfo(pythonPath);
info.Arguments = #"C:\Users\MyPc\ExternalScripts\HelloWorld.py" + " " + args;
info.RedirectStandardInput = false;
info.RedirectStandardOutput = true;
info.UseShellExecute = false;
info.CreateNoWindow = true;
using (var proc = new Process())
{
proc.StartInfo = info;
proc.Start();
proc.WaitForExit();
if (proc.ExitCode == 0)
{
result = proc.StandardOutput.ReadToEnd();
}
}
return result;
}
catch (Exception ex)
{
throw new Exception("R Script failed: " + result, ex);
}
}
Click Event ( Calling funtion )
private void Button1_Click(object sender, RoutedEventArgs e)
{
pythonPath = Environment.GetEnvironmentVariable("PYTHON_PATH");
RunFromCmd(pythonPath, "");
}
Python Script :
import sys
def main():
text = "Hello World"
return text
result = main()

I've fixed the issue by setting Copy if newer instead of Do Not Copy to HelloWorld.py Script

AWS Lambda: Input values needs to be concatenated with error message

The query returns the expected result, the request in this forum is regarding the customized error message. Incase of failure the exception needs to stored along with the input values.
This is my first lambda code, please let me know any additional details.
Input
{"EventID":"1246", "DataflowID": "011010"}
Lambda Code (Nodejs), this was nodejs but suggestion in python are also appreciated.
var AWS = require('aws-sdk');
var mydocumentClient = new AWS.DynamoDB.DocumentClient();
exports.handler = function (event, context, callback) {
var params = {
TableName: 'TransactionLog',
KeyConditionExpression : 'EventID = :EventID and Status = :Status',
FilterExpression : 'EventID in (:EventID) and Status in (:Status)',
ExpressionAttributeValues: {":EventID": event.WorkflowDetail.EventID,":Status": "progress"},
ProjectionExpression: "EventID,DataflowID,Status"
};
mydocumentClient.scan(params, function (err, data){
if (err) {
callback(err, null);
}else{
callback(null, data);
}
}
)
}
Actual Example Error Message: Resource not found.
Expected Error Message: Resource not found "EventID":"1246", "DataflowID": "011010"
I tried using different options but no luck. please advise.
console.log(element.Title.S + " (" + element.Subtitle.S + ")");

Declare all the required events inside the parmas. I'm not sure this is the best solution but it meets my requirement. Thanks!
var params = {
...,
....,
Input: "{\"column1\" : \"" + 'type' + "\", \"column2\" : \"" + 'time' +\"}"
};
console.log(params.Input, err);
useful thread: Can execute a step function from a lambda but when I try to pass a value it fails saying input {}

gRPC Node Js client: "Method not found"

I have a gRPC server implemented in python and I am calling an RPC from NodeJS but it gives an error "Method not found". When I call using the python client, the request is successful.
stream_csv.proto
syntax = "proto3";
package csv;
service Stream {
rpc csvToObject(CSVDataRequest) returns (stream CSVDataResponse) {};
rpc sayHello(HelloRequest) returns (HelloReply);
}
message CSVDataRequest{
string url = 1;
enum Protocol {
HTTP = 0;
HTTPS = 1;
FTP = 2;
SFTP = 3;
}
Protocol protocol = 2;
}
message CSVDataResponse{
repeated string row = 1;
}
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
client.js
var PROTO_PATH = '../stream_csv.proto';
var grpc = require('grpc');
var protoLoader = require('#grpc/proto-loader');
var packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
var proto = grpc.loadPackageDefinition(packageDefinition).csv;
function main() {
var client = new proto.Stream('localhost:5000',
grpc.credentials.createInsecure());
var user;
if (process.argv.length >= 3) {
user = process.argv[2];
} else {
user = 'world';
}
console.log(user);
client.sayHello({name: user}, function(err, response) {
console.log(err);
});
}
main();
server.py
import grpc
import stream_csv_pb2
import urllib.request
from urllib.error import HTTPError, URLError
from concurrent import futures
class DataService:
def csv_to_object(self, request, context):
url = request.url
protocol = stream_csv_pb2.CSVDataRequest.Protocol.Name(
request.protocol)
fetch_url = protocol.lower() + "://"+url
try:
with urllib.request.urlopen(fetch_url) as data:
for line in data:
decoded_line = line.decode()
val = decoded_line.split(',')
print(val)
print("Data send")
yield stream_csv_pb2.CSVDataResponse(row=val)
print("Sending finished!")
except URLError as e:
context.abort(grpc.StatusCode.UNKNOWN,
'Randomly injected failure.')
# return stream_csv_pb2.CSVDataResponse(row=[], error=e.reason)
def SayHello(self, request, context):
name = request.name
print(name)
return stream_csv_pb2.HelloReply(message='Hello %s' % (name))
def add_DataServicer_to_server(servicer, server):
rpc_method_handlers = {
'CSVToObject': grpc.unary_stream_rpc_method_handler(
servicer.csv_to_object,
request_deserializer=stream_csv_pb2.CSVDataRequest.FromString,
response_serializer=stream_csv_pb2.CSVDataResponse.SerializeToString,
),
'SayHello': grpc.unary_unary_rpc_method_handler(
servicer.SayHello,
request_deserializer=stream_csv_pb2.HelloRequest.FromString,
response_serializer=stream_csv_pb2.HelloReply.SerializeToString,
)
}
generic_handler = grpc.method_handlers_generic_handler(
'stream_csv.Stream', rpc_method_handlers)
server.add_generic_rpc_handlers((generic_handler,))
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
add_DataServicer_to_server(DataService(), server)
server.add_insecure_port('[::]:5000')
server.start()
server.wait_for_termination()
if __name__ == '__main__':
serve()
Error
Error: 12 UNIMPLEMENTED: Method not found!
at Object.exports.createStatusError (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/common.js:91:15)
at Object.onReceiveStatus (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:1209:28)
at InterceptingListener._callNext (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:568:42)
at InterceptingListener.onReceiveStatus (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:618:8)
at callback (/home/shantam/Documents/grepsr/client/node_modules/grpc/src/client_interceptors.js:847:24) {
code: 12,
metadata: Metadata { _internal_repr: {}, flags: 0 },
details: 'Method not found!'
}

I wrote a simpler variant of your Python (and Golang) server implementations.
Both work with your Node.JS client as-is.
I think perhaps your issue is as simple as needing to name the rpc sayHello (rather than SayHello).
from concurrent import futures
import logging
import grpc
import stream_csv_pb2
import stream_csv_pb2_grpc
class Stream(stream_csv_pb2_grpc.StreamServicer):
def sayHello(self, request, context):
logging.info("[sayHello]")
return stream_csv_pb2.HelloReply(message='Hello, %s!' % request.name)
def serve():
logging.info("[serve]")
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
stream_csv_pb2_grpc.add_StreamServicer_to_server(Stream(), server)
server.add_insecure_port('[::]:50051')
server.start()
server.wait_for_termination()
if __name__ == '__main__':
logging.basicConfig()
serve()
And:
node client.js
Greeting: Hello, Freddie!
null
Conventionally (!) gRPC follows Protobufs style guide of CamelCasing service, rpc and message names and using underscores elsewhere see Style Guide. When protoc compiles, the results don't always match perfectly (e.g. Python's CamelCased functions rather than lowercased with underscores).
Conventionally, your proto would be:
service Stream {
rpc CsvToObject(CSVDataRequest) returns (stream CSVDataResponse) {};
rpc SayHello(HelloRequest) returns (HelloReply);
}
...and then the Python generated function would be (!) SayHello too.

Why gRPC Python does not see first field?

Python gRPC message does not serialize first field. I updated protos and cleaned everything many times but it is not fixed. You can see logs, settings_dict has stop field but after passing these fields to AgentSetting it is not seen in logs and server side. Also I tried to pass stop field manually but it is not seen as well. Annoying thing is that gRPC does not throw any exception it is accepting stop field but it is not sending to server also it is not seen in AgentSetting while printing but stop field is reachable like agent_setting.stop.
This is my proto file:
syntax = 'proto3';
import "base.proto";
message ConnectionCheckRequest {
string host_name = 1;
}
message RecordChunkRequest {
string host_name = 1;
string name = 2;
bytes chunk = 3;
uint32 chunk_index = 4;
uint32 chunk_number = 5;
}
message AgentSetting {
bool stop = 1;
bool connection = 2;
bool registered = 3;
string image_mode = 4;
uint32 sending_fail_limit = 5;
uint64 limit_of_old_records = 6;
string offline_records_exceed_policy = 7;
}
message AgentSettingRequest {
AgentSetting agent_setting = 1;
string host_name = 2;
}
message AgentSettingResponse {
AgentSetting agent_setting = 1;
}
message AgentRegistryRequest {
AgentSetting agent_setting = 1;
string host_name = 2;
}
service Chief {
rpc agent_update_setting(AgentSettingRequest) returns (AgentSettingResponse) {};
rpc agent_registry(AgentRegistryRequest) returns (Response) {};
rpc send (stream RecordChunkRequest) returns (Response) {};
rpc connection_check (ConnectionCheckRequest) returns (Response) {};
}
This is my code snippet:
def register():
try:
with insecure_channel(settings.server_address) as channel:
setting_dict = settings.dict()
logger.info(f'\nSetting Dict: {setting_dict}')
agent_setting = AgentSetting(**setting_dict)
logger.info(f'\nAgent Setting Instance: \n{agent_setting}')
response = ChiefStub(channel).agent_registry(
AgentRegistryRequest(
agent_setting=agent_setting,
host_name=settings.host_name
)
)
return response
except Exception as e:
logger.exception(f'Register Error: {str(e)}')
return Response(success=False, message="failure")
Logs:
|2020-05-05T18:33:56.931140+0300| |5480| |INFO| |reqs:register:28|
Setting Dict: {'stop': False, 'connection': True, 'registered': False, 'image_mode': 'RGB', 'sending_fail_limit': 3, 'limit_of_old_records': 5368709120, 'offline_records_exceed_policy': 'OVERWRITE'}
|2020-05-05T18:33:56.932137+0300| |5480| |INFO| |reqs:register:32|
Agent Setting Instance:
connection: true
image_mode: "RGB"
sending_fail_limit: 3
limit_of_old_records: 5368709120
offline_records_exceed_policy: "OVERWRITE"

In proto3, an unset value and a default value are considered equivalent.
So stop: false is considered equivalent to omitting stop entirely.
See Language Guide (proto3)
Note that for scalar message fields, once a message is parsed there's no way of telling whether a field was explicitly set to the default value (for example whether a boolean was set to false) or just not set at all: you should bear this in mind when defining your message types. For example, don't have a boolean that switches on some behaviour when set to false if you don't want that behaviour to also happen by default. Also note that if a scalar message field is set to its default, the value will not be serialized on the wire.
Note that this is different from proto2.