I have a python web app that essentially allows 2 computers to talk with one another. If a session ends abruptly the record is still stored in pymongo, I want to be able to run a cron job to clean up old records, but I am not clear on how to do that, can't figure how to use bash to talk to pymongo...
What else could I do, call python from the cron job?
You could write a python script using pymongo (or any other mongodb client library) that does the necessary cleanup and configure cron to run it regularly.
Here is the article on OpenShift on how to get Cron up and running
https://www.redhat.com/openshift/community/blogs/getting-started-with-cron-jobs-on-openshift
Related
I have a web server which I have developed an application on using php and SQL, mainly picked php as I am more comfortable with it.
In short the application automates some of our network tasks .
As part of this I have automated some solarwinds tasks and the library orionsdk doesnt have a php library so I have used python.
It's all working fine but I really need to run these python scripts from my browser .
I have considered using php shell exec and got my python scripts to accept args so I can run them and parse the output.
I know I could also use flask or django but worry I will have a flask app to maintain aswell as a php app.
I think the question is what the best way to achieve this or any way which I haven't mentioned .
Any help would be very much appreciated
So you want PHP to communicate with Python and you've already mentioned using shell commands and http traffic.
I can imagine you could also achieve something similar by connecting up both PHP and Python up to the same database. In that case PHP could write a record in a table and Python could pick that up and do something with the data in there. Python could be either be a long-running process or fired off by a cronjob in this case. If a database seems overkill you could also write a file to some place on disk, which Python can pick up.
Personally I'd go for the shell exec approach if you want to keep it light weight and for a API connection if you want to have a more robust solution which needs to be expanded later on.
I wrote a python script to send Data from a local DB via REST to Kafka.
My goal: I would like this script to run indefinitely, by either restarting in set intervals (i.e. every 5min) or whenever the DB gets new entries. I assume the set Intervals thing would be good enough, easier and safer.
Someone suggested to me to either run it via a cronjob and use a monitoring tool or do it using jenkins (which he considered better).
My Setting: I am not a DevOps engineer, and would like to know about the possibilities and risks setting this Script up. It would be no trouble to recreate the Script in Java if this improves the situation.
My Question: I did try to learn what jenkins is about and i think i understood the CI and CD part. But i don't see how this could help me with my goal. Can someone elaborate on this with some experience on this topic?
If you would suggest a cronjob, what are common methods or tools to monitor such a case? I think the main risks are, failing to send the data due to connection issues on the local machine to REST or the local DB or not beieng started properly at the specified time.
Jobs can be scheduled at regular intervals in Jenkins just like with cron, in fact it uses the same syntax. What's nice about scheduling the job via Jenkins, is that it's very easy to have it send an email if the job exits with a non-zero return code. I've moved all of my cron jobs into Jenkins and it's working well. So by running it via Jenkins you're covering the execution side and the monitoring side at the same time.
I have a python script that connects to Redshift, executes a series of SQL commands, and generates a new derived table.
But for the life of me, I can't figure out a way to have it automatically run every day.
I've tried AWS Data Pipeline but my shell script won't run the first copy statement.
I can't get Lambda or Glue to work because my company's IAM policies are restrictive.
Airflow seems like overkill to just run a single python script daily.
Any suggestions for services to look into?
Cron job?
00 12 * * * /home/scottie/bin/my_python_script.py
Run my_python_script.py at the top of the hour (0th minute), at noon, every day.
use a cron job on an ec2 instance or set up a scheduled event to invoke your aws python lambda function http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html
I use a scheduled task on Windows. Either enter it using the GUI or the at command.
If you are using AWS Glue or have some other reason to install a development endpoint, you can use Apache Zeppelin to run any code from any language (if you have the jar files) on a schedule based on a cron command. Here's the notebook I use to run Redshift nightly maintenance:
Redshift Maintenance in a Zeppelin notebook
I have Python script that is supposed to run once every few days to annotate some data on a remote database.
Which PaaS services (GAE, Heroku, etc.) allows for a stand-alone Python script to be deployed and executed via some sort of cron scheduler?
GAE has a module called cron jobs and Heroku has Heroku Scheduler. Both are fairly easy to use and configure. You can check the documentation of both. As I do not have any other information on what you want to do I don’t know if one would be more suitable to you than the other.
How to create shedule on OpenShift hosting to run python script that parses RSS feeds and will send filtered information to my email? It feature is available? Please help, who works with free version of this hosting. I have script that works fine. But i dont know how to run it every 10 min to catch freelance jobs. Or anyone does know free hosting with python that can create shedule for scripts.
You are looking for the add-on cartridge that is called cron. However, by default the cron cartridge only supports jobs that run every minute or every hour. You would have to write a job that runs minutely to determine if its a 10 minute interval and then execute your script.
Make sense?
rhc cartridge add cron -a yourAppName
Then you will have a cron directory in application directory under .openshift for placing the cron job.
You could so something like this here but setup for 10 minutes instead of 5: https://github.com/openshift-quickstart/openshift-cacti-quickstart/blob/master/.openshift/cron/minutely/cactipoll