We Keep Coding

Python script and Google Cloud Engine [duplicate]

I have two Python scripts on my machine that I want to execute two times a day on specific time period. How do I automate this task? Since I will be away from home and thus my computer for a while, I want to upload them to a site and be executed from there automatic without me doing anything.
How can I do this?

You can use cron for this if you are on a Linux machine. Cron is a system daemon used to execute specific tasks at specific times.
cron works on the principle of crontab, a text file with a list of commands to be run at specified times. It follows a specific format, which can is explained in detail in man 5 crontab
Format for crontab
Each of the sections is separated by a space, with the final section having one or more spaces in it. No spaces are allowed within Sections 1-5, only between them. Sections 1-5 are used to indicate when and how often you want the task to be executed. This is how a cron job is laid out:
minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday), command
01 04 1 1 1 /usr/bin/somedirectory/somecommand
The above example will run /usr/bin/somedirectory/somecommand at 4:01am on January 1st plus every Monday in January. An asterisk (*) can be used so that every instance (every hour, every weekday, every month, etc.) of a time period is used. Code:
01 04 * * * /usr/bin/somedirectory/somecommand
The above example will run /usr/bin/somedirectory/somecommand at 4:01am on every day of every month.
Comma-separated values can be used to run more than one instance of a particular command within a time period. Dash-separated values can be used to run a command continuously. Code:
01,31 04,05 1-15 1,6 * /usr/bin/somedirectory/somecommand
The above example will run /usr/bin/somedirectory/somecommand at 01 and 31 past the hours of 4:00am and 5:00am on the 1st through the 15th of every January and June.
The "/usr/bin/somedirectory/somecommand" text in the above examples indicates the task which will be run at the specified times. It is recommended that you use the full path to the desired commands as shown in the above examples. Enter which somecommand in the terminal to find the full path to somecommand. The crontab will begin running as soon as it is properly edited and saved.
You may want to run a script some number of times per time unit. For example if you want to run it every 10 minutes use the following crontab entry (runs on minutes divisible by 10: 0, 10, 20, 30, etc.)
*/10 * * * * /usr/bin/somedirectory/somecommand
which is also equivalent to the more cumbersome
0,10,20,30,40,50 * * * * /usr/bin/somedirectory/somecommand

In Windows I have come up with two solutions.
First option: Create a .bat file.
Step 1
Create a .bat file to indicate the command you want to run and the script file that will be executed, for instance:
start C:\Users\userX\Python.exe C:\Users\userX\PycharmProjects\Automation_tasks\create_workbook.py
Step 2
Open the Task Scheduler and click on the Task Scheduler Library to see the current tasks that are executed. Click on the Create Task option.
Step 3
In the General tab, put the name of your new task and click on the option Run whether user is logged on or not, check the option Run with highest privileges and make sure to setup the appropriate version of you OS (in my case I picked Windows 7, Windows Server 2008 R2.
Step 4
In the Actions tab, click on the New button and type in the following:
In Program/Scripts you need to look up for the Powershell path that the Task Scheduler will invoke to run the .bat file. In my case, my Powershell path was:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
In Add arguments (optional) you need to type the path of the file that will be executed by Powershell. In my case, the path was:
C:\Users\userX\Desktop\run_the_bat_file.bat
In Start in (optional) you need to type the path of the file but without the name of the .bat file, that is:
C:\Users\userX\Desktop\
Step 5
Click on the Triggers tab and select how often you want to execute this task.
Step 6
Lastly, test your task to see if it truly works by selecting it from the Task Scheduler Library and doing click on the Run option.
Second option: Run the .py file with the Task Scheduler
Step 1
Open the Task Scheduler and click on the Task Scheduler Library to see the current tasks that are executed. Click on the Create Task option.
Step 2
In the General tab, put the name of your new task and click on the option Run whether user is logged on or not, check the option Run with highest privileges and make sure to setup the appropriate version of you OS (in my case I picked Windows 7, Windows Server 2008 R2.
Step 3
In the Actions tab, click on the New button and type in the following:
In Program/Scripts you need to look up for the Python.exe path that the Task Scheduler will invoke to run the python script. In my case, my Python.exe path was:
C:\Users\userX\python.exe
In Add arguments (optional) you need to only type the name of your python script. In my case, the path was:
Permissions_dump.py
In Start in (optional) you need to type the path of the file but without the name of the python script, that is:
C:\Users\userX\PycharmProjects\1099_vendors_costs
Step 4
Click on the Triggers tab and select how often you want to execute this task.
Step 5
Lastly, test your task to see if it truly works by selecting it from the Task Scheduler Library and doing click on the Run option.
Another option (in case you convert a .py to a .exe)
If you use the library Cx_Freeze to convert a .py to a .exe and you want to use the task scheduler to automate this task then you need to follow these steps:
Step 1
Click on Create Task and then click on the Actions tab to type in the following:
In Program/Scripts you need to look up for the C:\Windows\explorer.exe path that the Task Scheduler will invoke to run the .exe script.
In Add arguments (optional) you need to only type the name of your .exe file: CustomerPopulation.exe
In Start in (optional) you need to type the path of the file but without the name of the .exe file, that is:
C:\Users\userX\PycharmProjects\executables
In the General tab, make sure to have selected the Run only when user is logged on and have unchecked the Run with the highest privileges.
If reports stopped working
Make sure to check if your password hasn’t expired, otherwise the reports won’t be sent.
References:
https://gis.stackexchange.com/questions/140110/running-python-script-in-task-scheduler-script-will-not-run?newreg=603bcdbc381b41a283e5d8d0561b835e
https://www.youtube.com/watch?v=oJ4nktysxnE
https://www.youtube.com/watch?v=n2Cr_YRQk7o

If you are using OSX then launchd is the preferred way to schedule tasks. There is a OSX CLI for launchd called launchctl but if you prefer a GUI my preferred one is launchcontrol.

execute and run script python on a server several times

I have a script in python that takes date as an argument in order to get the event in google agenda using google API calendar at that date. Now my problem is as follows: There are three machines: first one is doing some treatment in order to get a date for some reason, second machine is the server that the two other machines are connected to, third machine is where i have to execute the script python in order to get event of its google calendar.
So my question is how can I do such that for every 5 minutes - the first machine send the date (that means every 5 minutes i get a different date ) and each date I get I have to execute that script python to get the event on that date and send the answer to the first machine (doing it dynamically and the server is in the middle between the two other machines ).
In other words, how can I make this process dynamic such that for every 5 minutes the script python has to be executed using for each time a different date and give back the result?
I want to execute the script like this :
python script.py '2017-12-10 10:40:00' #(first time)
python script.py '2017-12-10 10:45:00' #(after 5 minutes)
python script.py '2017-12-10 10:50:00' #(after another 5 minutes)
and so on ....

With a crontab entry like
*/5 * * * * python path/to/script.py "$(date %%F %%T)"
where the path/to is relative to your home directory, and the */5 is a Vixie cron (in practice, more or less Linux only) extension which might not work on other platforms. The crontab format requires the percent signs to be doubled -- maybe put this in an external script if you need complex expressions with many percent signs.
There is no guarantee that this runs exactly on the second, or even on the minute. Maybe use %%H:%%M:00 instead of %%T to force the seconds (forcing the minutes will be slightly more challenging; again, maybe put the logic in an external script if you need this).

Python - Run Multiple Scripts At Same Time Methods

I have a bunch of .py scripts as part of a project. Some of them i want to start and have running in the background whilst the others run through what they need to do.
For example, I have a script which takes a Screenshot every 10 seconds until the script is closed and i wish to have this running in the background whilst the other scripts get called and run through till finish.
Another example is a script which calculates the hash of every file in a designated folder. This has the potential to run for a fair amount of time so it would be good if the rest of the scripts could be kicked off at the same time so they do not have to wait for the Hash script to finish what it is doing before they are invoked.
Is Multiprocessor the right method for this kind of processing, or is there another way to achieve these results which would be better such as this answer: Run multiple python scripts concurrently

You could also use something like Celery to run the tasks async and you'll be able to call tasks from within your python code instead of through the shell.

It depends. With multiprocessing you can create a process manager, so it can spawn the processes the way you want, but there are more flexible ways to do it without coding. Multiprocessing is usually hard.
Check out circus, it's a process manager written in Python that you can use as a library, standalone or via remote API. You can define hooks to model dependencies between processes, see docs.
A simple configuration could be:
[watcher:one-shot-script]
cmd = python script.py
numprocesses = 1
warmup_delay = 30
[watcher:snapshots]
cmd = python snapshots.py
numprocesses = 1
warmup_delay = 30
[watcher:hash]
cmd = python hashing.py
numprocesses = 1

Running a Python Script every n Days in Django App

I am looking to run a Python script every n days as part of a Django app.
I need this script to add new rows to the database for each attribute that a user has.
For example a User has many Sites each that have multiple Metric data points. I need to add a new Metric data point for each Site every n days by running a .py script.
I have already written the script itself, but it just works locally.
Is it the right approach to:
1) Get something like Celery or just a simple cron task running to run the python script every n days.
2) Have the python script run through all the Metrics for each Site and add a new data point by executing a SQL command from the python script itself?

You can use Django's manage commad, then use crontab to run this command circularly.

The first step is to write the script - which you have already done. This script should run the task which is to be repeated.
The second step is to schedule the execution.
The de-facto way of doing that is to write a cron entry for the script. This will let the system's cron daemon trigger the script at the interval you select.
To do so, create an entry in the crontab file for the user that owns the script; the command to do so is crontab -e, which will load a plain text file in your preferred editor.
Next, you need to add an entry to this file. The format is:
minutes hours day-of-month month day-of-week script-to-execute
So if you want to run your script every 5 days:
0 0 */5 * * /bin/bash /home/user/path/to/python.py

Automate Python Script

I'm running a python script manually that fetches data in JSON format.How do I automate this script to run automatically on an hourly basis?
I'm working on Windows7.Can I use tools like Task scheduler?If I can use it,what do I need to put in the batch file?

Can I use tools like Task scheduler?
Yes. Any tool that can run arbitrary programs can run your Python script. Pick the one you like best.
If I can use it,what do I need to put in the batch file?
What batch file? Task Scheduler takes anything that can be run, with arguments—a C program, a .NET program, even a document with a default app associated with it. So, there's no reason you need a batch file. Use C:\Python33\python.exe (or whatever the appropriate path is) as your executable, and your script's path (and its arguments, if any) as the arguments. Just as you do when running the script from the command line.
See Using the Task Scheduler in MSDN for some simple examples, and Task Scheduler Schema Elements or Task Scheduler Scripting Objects for reference (depending on whether you want to create the schedule in XML, or via the scripting interface).
You want to create an ExecAction with Path set to "C:\Python33\python.exe" and Arguments set to "C:\MyStuff\myscript.py", and a RepetitionPattern with Interval set to "PT1H". You should be able to figure out the rest from there.
As sr2222 points out in the comments, often you end up scheduling tasks frequently, and needing to programmatically control their scheduling. If you need this, you can control Task Scheduler's scripting interface from Python, or build something on top of Task Scheduler, or use a different tool that's a bit easier to get at from Python and has more helpful examples online, etc.—but when you get to that point, take a step back and look at whether you're over-using OS task scheduling. (If you start adding delays or tweaking times to make sure the daily foo1.py job never runs until 5 minutes after the most recent hourly foo0.py has finished its job, you're over-using OS task scheduling—but it's not always that obvious.)

May I suggest WinAutomation or AutoMate. These two do the exact same thing, except the UI is a little different. I prefer WinAutomation, because the scripts are a little easier to build.

Yes, you can use the Task Scheduler to run the script on an hourly bases.
To execute a python script via a Batch File, use the following code:
start path_to_python_exe path_to_python_file
Example:
start C:\Users\harshgoyal\AppData\Local\Continuum\Anaconda3\python.exe %UserProfile%\Documents\test_script.py
If python is set as Window’s Environment Window then you can reduce the syntax to:
start python %UserProfile%\Documents\test_script.py
What I generally do is run the batch file once via Task Scheduler and within the python script I call a thread/timer every hour.
class threading.Timer(interval, function, args=None, kwargs=None)