I am working on a project that just scrapes data from 3 devices (2xserial and 1xssh). I have this part implemented no problem.
I am now heading towards the second part where I need be be able to send the data I need using protobuf to the clients computer where they will receive and display on their own client.
The customer has provided examples of their GRPC servers, and it's written in C#.
Currently, for security reasons, our system uses RedHat 8.3 and I am using a SSH Protocol Library called Paramiko for the SSH part. Paramiko is a Python library. Also the machine I am extracting data from only runs on Linux.
Here are my main questions, and I apologize if I got nowhere.
1.) The developer from the client side provided us with a VM that has a simulator and examples written in C# since their side was written in C#. He says that it's best to use the C# because all examples can be almost re-used as it's all written, etc. While I know it's possible to use C# in Linux these days, I've still have no experience doing so I don't know how complicated/tedious this can get.
2.) I write code in C# and wrap all the python code, which is also something I've never done, but I would be doing this all in RedHat.
3.) I keep it in python because sending protobuf messages works across languages as long as it is sent properly. Also from the client side, I'm not sure if they will need to make adjustments if receiving protobuf messages written in Python(I don't think this is the case because it's just serialized messages, yea?).
Any advice would be appreciated. I am looking to seek more knowledge outside my realm.
Cheers,
Z
If you're happy in Python, I would use option 3. The key thing is to either obtain their .proto schema, or if they've used code-first C# for their server: reverse-engineer the schema (or use tools that generate the schema from code). If you only have C# and don't know how to infer a .proto from that, I can probably help.
That said: if you want to learn some new bits, option 1 (using C# in your system) is also very viable.
IMO option 2 is the worst of all worlds.
Related
A client has asked that I put together a Python program to demonstrate some core concepts of a possible ML app. We agreed I would make a CLI for now just to show how it would work and later we would make a web interface.
I wrote up a proposal and he read it and agreed. I'm now 100% done with this project and would like to collection the 2nd 1/2 of the payment.
Originally he said a video of the functionality was fine, now he is saying it's not and he wants to 'play with it'.
I don't want to give him the code, and I really don't want to do any more work for this client until he pays the rest of the money.
Here's my question: Is there a way to publish a Python CLI online without:
Doing a lot more work (like making a web interface).
Exposing the source code.
If he had to SSH into somewhere that might work.
I have created a SOAP service (using spyne). I have used python on the server side. I have a python client that works perfectly fine. But, I want to reuse the output obtained from the server in a perl script (cannot use python).
One possible way would be to call the python client from within perl and capture STDOUT.
But, it would be nice if I could write a simple perl client to get the results. I wrote a simple perl client for SOAP following this, but my code did not print anything.
my $soap = SOAP::Lite->new();
my $service = $soap -> service($URL);
print Dumper($service -> serverFunction($arg1,$arg2));
Here I found a discussion about perl server and python client.
So, is it possible to have a python server and perl client?
Yes, it is possible, but it's SOAP. See the notice from SOAP::Simple:
Let's face it. SOAP is painfull. It's a dumb idea, the only reason you should ever consider using SOAP is if someone holds a gun to your head or pay you a lot of money for it.
If you want to build an API and have both server and client under your control: Try to avoid SOAP whenever possible. Consider switching to a JSON or basic HTTP API.
If you want to keep SOAP, I suggest to try SOAP::Simple, it might work better than SOAP::Lite. There's also the SOAP module, but it's probably too heavy for your usage case.
PS: You might want to include more details (e.g. code you tried) in your question to get a more specific answer.
Here is what I would like to do, and I want to know how some people with experience in this field do this:
With three POST requests I get from the http server:
widgets and layout
and then app logic (minimal)
data
Or maybe it's better to combine the first two or all three. I'm thinking of using pyqt. I think I can load .ui files. I can parse json data. I just think it would be rather dangerous to pass code over a network to be executed on the client. If someone can hijack the connection, or can change the apps setting to access a bogus server, that is nasty.
I want to do it this way because it keeps all the clients up-to-date. It's sort of like a webapp but simpler because of Qt. Essentially the "thin" app is just a minimal compiled python file that loads data from a server.
How can I do this without introducing security issues on the client? Is https good enough? Is there a way to get pyqt to run in a sandbox of sorts?
PS. I'm not stuck on Qt or python. I do like the concept though. I don't really want to use Java - server or client side.
Your desire to send "app logic" from the server to the client without sending "code" is inherently self-contradictory, though you may not realize that yet -- even if the "logic" you're sending is in some simplified ad-hoc "language" (which you don't even think of as a language;-), to all intents and purposes your Python code will be interpreting that language and thereby execute that code. You may "sandbox" things to some extent, but in the end, that's what you're doing.
To avoid hijackings and other tricks, instead, use HTTPS and validate the server's cert in your client: that will protect you from all the problems you're worrying about (if somebody can edit the app enough to defeat the HTTPS cert validation, they can edit it enough to make it run whatever code they want, without any need to send that code from a server;-).
Once you're using https, having the server send Python modules (in source form if you need to support multiple Python versions on the clients, else bytecode is fine) and the client thereby save them to disk and import / reload them, will be just fine. You'll basically be doing a variant of the classic "plugins architecture" where the "plugins" are in fact being sent from the server (instead of being found on disk in a given location).
Use a web-browser it is a well documented system that does everything you want. It is also relatively fast to create simple graphical applications in a browser. Examples for my reasoning:
The Sage math environment has built their graphical client as an application that runs in a browser, together with a local web-server.
There is the Pyjamas project that compiles Python to Javascript. This is IMHO worth a try.
Edit:
You could try PyPy's sandbox interpreter, as a secure Python interpreter for the code that was transferred over a network.
An then there is the most simple solution: Simply send Python modules over the network, but sign and/or encrypt them. This is the way all Linux distributions work. You store a cryptographic token on the local computer. The server signs/encrypts the code before it sends it, with a matching token. GPG should be able to do it.
I'm building a website where I hook people up so that they can anonymously vent to strangers. You either choose to be a listener, or a talker, and then you get catapulted into a one-on-one chat room.
The reason for the app's construction is because you often can't vent to friends, because your deepest vulnerabilities can often be leveraged against you later on. (Like it or not, this is a part of human nature. Sad.)
I'm looking for some insight into how I should architect everything. I found this neat tutorial, http://giantflyingsaucer.com/blog/?p=875, which suggests using python & stackless + flash. Someone else suggested I should try using p2p sockets, but I don't even know where to begin to look for info on that.
Any other suggestions? I'd like to keep it simple. :^)
Unless you expect super high load, this is simple enough that it doesn't really matter what you use on the backend: just pick something you're comfortable with. PHP, Python, Ruby, Even a bash script using CGI - your skill level with the language is likely to make more difference that the language features themselves.
I would use an XMPP server like ejabberd or OpenFire to power the backend. XMPP contains everything you need for creating chat/real-time applications. You can use a Flex/Flash Actionscript library like Actionscript 3 XIFF to communicate with the XMPP server.
Flash is user-unfriendly for UI (forms, etc) and it is relatively easy to do what you want using HTML and Javascript on the front-end.
One possible approach for reading the messages would be to regularly do an Ajax request from the server for any new messages. Format the new message and insert it into the DOM.
You will probably need to answer at least these questions before you continue, though:
1) Are you recreating IRQ (everyone sees your posts), or is this a random one-to-one chat, like chatroulette?
1a) Is this a way for a specific person to talk to another specific person, or is this more like twitter?
2) What is your plan for scaling up if this idea takes off? Memcached should probably be a method of last-resort ("bandaid over a bullet-hole"). What's your roadmap for eventually handling a large volume of messages?
3) Is there any way to ignore users? Talk to certain users? Hide your rants from users?
Hey Zach I had to create a socket server for a flash game I made. I built my server in C#, but I would use whatever language your familiar with. If you let me know what your most comfortable with I could try to help find a good tutorial.
The one thing I spent many hours on was getting flash to work from a website with a socket server. With the newer versions of Flash you need to send back a policy file. In my case this needed to be the first chunk of data sent back to the client when they connected to the socket server.
Not sure what to tell you about structuring the back end. I need to know a little bit more about your programming experience. I had an array of all user connections, and was placing them in different "Rooms" so they could play each other. So just some simple arrays and understanding how to send messages to the clients would help you here.
If you have any familiarity with C# I would have no problem sending you the source code for my socket server.
I'm writing an application that streams the output (by this I mean both sys.stdout and sys.stderr) of a python script excited on the server, in real time to the browser.
The users on the site will be allowed to select the script to run, excite and kill their chosen script, and change some parameters, so I will need a different thread per user on the site (user A can start, stop and change a script, whilst user B can do the same with a different script).
I know I need to use comet for the web clients, and seeing as the rest of the project is written in python, I'd like to use twisted for the server, however I'm not really sure of what I need to do next!
There are a daunting number of options (Divmod Mantissa, Divmod Nevow, twisted.web, STOMP, etc), and some are better documented that others, making the whole thing rather tricky!
I have a working demo using stompservice on orbited, using Orbited.TCPSocket for the javascript side of things, however I'm starting to think that STOMPs channel model isn't going to work for multithreading, multi-running scripts (unless I open a new channel per run, but that seems like the wrong use of the channel model).
Can anyone point me in the right direction, or some sample code I can learn from?
Thanks!
Nevow Athena is a framework specifically for AJAX and COMET applications and in theory is exactly the sort of thing you are looking for.
However, I am not sure that it is well used or supported at this time - looking at mailing list traffic and google search results suggests that it may not be.
There are a couple of tutorials you could look at to help you decide on it:
one on the 'official' site: http://divmod.org/trac/wiki/DivmodNevow/Athena/Tutorials/LiveElement
and one other that I found:
http://divmodsphinx.funsize.net/nevow/chattutorial/part01/index.html
The code for the latter seems to be included in the Nevow distribution when you download it under /doc/listings/partxx (I think...)
You can implement a very simple "HTTP streaming" by keeping the http connection open and appending javascript chunks that update the dom contents. This works since the browser evaluates the "script" chunks as they arrive.
I wrote a blog entry a while ago with a running example using twisted and very few lines of javascript: Simple HTTP streaming with Twisted & Javascript
You can easily mix this pattern with a publisher/subscriber pattern to make it multiuser, etc. I use this pattern to watch live log streams via web.
An example of serving for long-polling clients with Twisted is slosh. This might not be what you want, but because it's not a large framework, it can help you figure out how to use Twisted.