I'm having a lot of trouble with this question and don't really find a lot of resources with it. I have a Power Automate Flow setup (triggered by a user submitting a form) in which I would like to receive data from my on-premise (on our corporate LAN) SQL (MariaDB) instance on my RHEL. From my understanding, there are three components that I would require: (1) REST API built on my web server which would simply return the data by querying the database, (2) On-Premise Gateway on my RHEL system, and (3) A HTTP Connector in my Flow process.
Developing a simple python/php script for my REST API is quite straightforward, however, I am having a bit of difficulty building my on-premise gateway. Microsoft comes with its own free Gateway however that is only available on Windows. Would I need to set up a Windows VM and from there install the gateway or is there a better solution?
Additionally, is my approach to solve this problem correct or am I going about it wrong?
Related
As part of migrating batch jobs (and used EXEC PGM) to other language (python here), facing challenge in cross server connection.
We are targeting to migrate few of our mainframes batch jobs COBOL programs to python. In this process, some batch jobs will be fully controlled using schedulers and programs will be rewrite in python scripts. But some mainframes programs will remain intact and not be migrated in python for now. As we are targeting partial migration for now, some mainframe batch jobs need to call python scripts on cloud. I am facing challenge here, how to call python scripts from mainframe batch jobs.
I'm assuming in this answer the COBOL applications run on the z/OS operating system on your mainframe, but if that assumption is not correct, please post a follow-up.
Cschneid has a great answer: just run the Python scripts on your mainframe. Python for z/OS is available for download free of charge from Rocket Software here:
https://www.rocketsoftware.com/zos-open-source
You can optionally purchase Python support on z/OS from Rocket Software if you wish. (All Linux distributions for IBM Z machines also include Python, typically supported by the Linux distributor.) Python running on IBM Z can directly operate on IBM Z-based data stores/databases, including well protected, z/OS-encrypted data sets. And you can quite easily create and manage hybrid cloud architectures that include IBM Z resources across all operating systems. That'd be the best arrangement all around since otherwise you're likely to have operational and management issues. You don't have to look very far to find real world instances of organizations that have suffered major, hugely business impactful batch scheduling problems that have completely wrecked their payment processes, for example. (Relatedly, Python is not an enterprise job scheduler.)
OK, that said, if you're still going to proceed down this (probably unwise) path this way, then here are some other options in no particular order:
Configure z/OS Management Facility (included as a base, included, supported feature in z/OS), and use its authorized REST APIs to submit jobs. Details are available here (z/OS 2.4 asssumed, but this feature is available in all currently supported z/OS releases and even prior):
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.4.0/com.ibm.zos.v2r4.izua700/IZUHPINFO_API_RESTJOBS.htm
Make sure you take reasonable, appropriate steps to secure this job submission path since it's quite powerful.
Equip your z/OS installation with IBM's z/OS Connect Enterprise Edition software product, create the REST APIs you need (both easy and powerful), and invoke them from Python. More information on z/OS Connect EE is available here:
https://www.ibm.com/us-en/marketplace/connect-enterprise-edition
If you have MQ for z/OS, then go grab the MQ client, send an appropriately formatted MQ message from Python to an appropriately configured MQ queue on z/OS, and invoke/trigger your programs that way. (MQ Advanced for z/OS is recommended for Advanced Message Security.) The MQ clients are free for unlimited use when connecting to all currently IBM supported, licensed versions of MQ and MQ Advanced for z/OS. Recent releases of MQ and MQ Advanced for z/OS also support REST APIs (and JSON payloads), so you can format your messages that way now. MQ clients are available for download here:
https://developer.ibm.com/messaging/mq-downloads/
At least some of the choices I'm providing on this list can be combined with MQ, which provides assured messaging -- which is quite helpful if you're trying to make this all work robustly.
Go find out what enterprise job scheduler your mainframe has installed (it probably has one), and use its authorized APIs to schedule and to run programs. For example, IBM Z Workload Scheduler provides authorized REST APIs. Refer to this documentation for an introduction:
https://www.ibm.com/support/knowledgecenter/en/SSRULV_9.5.0/com.ibm.tivoli.itws.doc_9.5/common/src_dgd/awsddrestapi.htm
If you click through to the samples you'll find some Python sample code.
....And there are lots of other possible ways, so if for some reason you don't like any of these choices, please post a follow-up.
Cschneid has another reasonable answer: Dovetailed's Co:Z Toolkit ("z/OS Hybrid Batch"). Here are some more possibilities, in no particular order:
The z/OS Client Web Enablement Toolkit, an included, IBM supported feature in the base z/OS operating system. This toolkit allows you to call a REST API from practically any program on z/OS. A COBOL sample is available here:
https://github.com/IBM/zOS-Client-Web-Enablement-Toolkit
z/OS Connect Enterprise Edition, which is bidirectional.
The enterprise job scheduler often installed and hosted on z/OS typically can trigger and manage "remote" tasks on other systems. IBM Z Workload Scheduler (for example) certainly can, and there's a whole manual discussing the topic here:
https://www.ibm.com/support/knowledgecenter/SSRULV_9.5.0/com.ibm.tivoli.itws.doc_9.5/eqqlwmst.pdf
Remote Procedure Calls (RPC), per IETF RFCs 1831 and 1832. If you're using a COBOL program with RPC you'd call the C interfaces, a minor bit of mixed language programming.
Dovetailed Technologies hybrid batch is another product that allows you to execute code residing on remote servers as steps in a batch job, similar to the solutions in the answers posted by #TimothySipples and #KevinMcKenzie.
Without knowing more, this question is impossible to answer.
However, generically speaking, you can issue USS commands from batch, using bpxbatch. So, you could install something like curl or wget from Rocket Software, and then call python via a REST API, or something similar on the cloud end, built in Django or Flask. If you really wanted to do something horrible, you could write a shell script that would ssh in to the cloud system, and issue a command on the remote system.
However, and I realize you probably don't have much say over this, I'd also point to Timothy Sipples' answer, and say this isn't a good idea, and it's going to be fragile. You'll need multiple such scripts, because you'll need to submit work, and then come back later and get the results, and behave appropriately based on the results. You're going to have to build all sorts of error handling capabilities into these batch jobs/shell scripts.
I've seen Google's documentation and a lot of examples over the internet about how to use Firebase within Android / iOS applications, but I want to develop a client Firebase application which runs on a linux machine.
My requirements are:
Client runs on linux environment (either writtern in C++ or python).
Server is written in Javascript (NodeJS).
Server and client should have bi-directional communication between them using firebase realtime database.
I've seen also the firebase REST api, but I assume that is not good enough for me since I haven't seen a client listener api that listens on a something like onValueChanged.
Question:
How can I implement a linux-based app in C++ or python that listens to messages from server (data changes) using a listener, without having to call some get function each and every few seconds (just like NodeJS have the ref.on("child_changed",...) or ref.on("value",...)?
Help is much appreciated!
Unfortunately there is not official library for Realtime Database in Python but there are some third party wrappers around REST API.
Specifically I would look at Pyrebase library which also supports listening to live changes https://github.com/thisbejim/Pyrebase#streaming
For more information, libraries or other languages look at this page https://firebase.google.com/docs/database/rest/start
Google offers a C++ SDK you can download it here.
There is also a tutorial on the bottom of the page, but if you really want to dig into some code, here is the quickstart code on github from google with examples for each category on Firebase.
Google has also implemented a Game for demonstration purposes in C++ for Desktop OSs. here There you can find more advanced features and code samples.
I have only tried it with an XCode project on Mac OS X, and it works fine. I have no code for real time database but you should find the sample project in the github repo. https://github.com/firebase/quickstart-cpp
I hope this helps!
Greetings.
Haven't you try to use a Realtime Database triggers for C++ (that's is currently on beta) that allows you to simulate the same workflow as on JS?
From my point of view, it is the best way to get it to work as you want.
https://firebase.google.com/docs/functions/database-events
I am looking for feasible solutions for my Application to be backed with MongoDB. I am looking to host the MongoDB on the cloud with a python based server to interact with the DB and my app (either mobile/web). I am trying to understand how the architecture should look like.
Either i can host a mongoDB on the AWS cloud and have the server running there only.
I also tried using MongoLab and seemed to be simple accessing it using HTTP requests. but i am not sure if it exposes all the essential features of MongoDB (what ever i can do using a pymongo driver)? Also, should i go for accessing the MongoLab service directly from my application or still i should build a server in-between?
I would prefer to building an server in either case as i want to do some processing before sending the data back to application. but i am not sure in that case how my DB-server-app interaction design should be
Any suggestions?
One thing to consider is that you don't need to use MongoLab's REST API. You can connect directly via a driver as well.
So, if you need to implement business logic (which it sounds like you do), it makes sense to have a three tier architecture with an app server connecting to your MongoLab database via one of the drivers. In your case it sounds like this would be pymongo.
-will
I'm a complete novice in this area, so please excuse my ignorance.
I have three questions:
What's the best (fastest, easiest, headache-free) way of hosting a python program online?
I'm currently looking at Google App Engine and Web Frameworks for Python, but all the options are a bit overwhelming.
Which gui/viz libraries will transfer to a web app environment without problems?
I'm willing to sacrifice some performance for the sake of simplicity.
(Google App Engine can't do C libraries, so this is causing a dilemma.)
Where can I learn more about running a program locally vs. having a program continuously run on a server and taking requests from multiple users?
Currently I have a working Python program that only uses standard Python libraries. It currently uses around 2.7gb of ram, but as I increase my dataset, I'm predicting it will use closer to 6gb. I can run it on my personal machine, and everything is just peachy. I'd like to continue developing on the front end on my home machine and implement the web app later.
Here is a relevant, previous post of mine.
Depending on your knowledge with server administration, you should consider a dedicated server. I was doing running some custom Python modules with Numpy, Scipy, Pandas, etc. on some data on a shared server with Godaddy. One program I wrote took 120 seconds to complete. Recently we switched to a dedicated server and it now takes 2 seconds. The shared environment used CGI to run Python and I installed mod_python on the dedicated server.
Using a dedicated server allows COMPLETE control (including root access) to the server which allows the compilation and/or installation of anything. It is a bit pricy but if you're making money with your stuff it might be worth it.
Another option would be to use something like http://www.dyndns.com/ where you can host a domain on your own machine.
So with that said, perhaps some answers:
It depends on your requirements. ~4gb of RAM might require a dedicated server. What you are asking is not necessarily an easy task so don't be afraid to get your hands dirty.
Not sure what you mean here.
A server is just a computer that responds to requests. On the dedicated server (I keep mentioning) you are operating in a Unix (or Windows) environment just like you would locally. You use SOFTWARE (e.g. Apache web server) to serve client requests. My vote is mod_python.
It's a greater headache than a dedicated server, but it should be much closer to your needs to go with an Amazon EC2 instance.
http://aws.amazon.com/ec2/#instance
Their extra large instance should be more than large enough for what you need to do, and you only turn the instance on when you need it so you don't have the massive bill that you get with a dedicated server that's the same size.
There are some nice javascript based visualization toolkits out there, so you can model your application to return raw (json) data and render that on the client.
I can mention d3.js http://mbostock.github.com/d3/ and the JavaScript InfoVis Toolkit http://thejit.org/
I'm working on a client machine running suse linux and python 2.4.2. I'm not allowed to dowload anything from the net including any external libraries. So, is there any way I can connect to a database (oracle) using only the default libraries?
No. There is nothing in the standard library for connecting to database servers.
You could look at the Apex Listener and/or PL/SQL Embedded Gateway (basically a webserver in the DB). These both provide HTTP access to database resources (eg getting data as JSON). Though obviously this shifts the 'heavy lifting' to the database platform.