I am trying to make one cron job script in Python . For the start what i did i just add the code to run the cordova and show its version the same python file works if i run it through shell but when its run through cron job it gives me this error
env: node: No such file or directory
Python file has this code
#!/usr/bin/python
import os
import subprocess
subprocess.call('/usr/local/bin/cordova -v',shell=True)
Is there any solution for that ? What i get to understand that maybe when i run code through cronjob the global variables are not accessable . Is there anyway that i can get access to command line terminal through pipe and get access to all the global variables ?
Please use env command from your terminal and cronjob script (e.g.: env > cron_output file), it will give the environment variables available for normal terminal and cronjob. Compare that variables and export the required missing variables in your script.
Related
I have a Python code running on Spyder on our server which needs to run constantly. However, from time to time our server breaks and it is restarted. Unfortunately, my code stops running as well and I need to restart Spyder and the code manually when opening it the next day.
Is there any way to restart the code automatically when the server is restarted?
Thank you a lot!
You can add as a crontab rule what you want to run after reboot.
You can use this line in crontab by using crontab -e command, if you use server running on Linux platform.
#reboot <your_python_file_path>
Simply utilize shell:startup and add your Python location + path there (or create a Batch script for it):
myprogram.bat:
python.exe main.py
Alternatively, if you don't have python location in your PATH env variable, utilize the full path:
myprogram.bat:
"C:\Program Files\Python39\python.exe" main.py
I wrote a Python script which backs up mongoDB, and it works fine when I test run directly in terminal.
However, I get an error from cron saying mongodump: command not found - although the command mongodump works fine when I run the script directly in terminal.
Contents of crontab -e:
* * * * * cd <path-to-script> && python3 script.py
After looking into the post provided by S3DEV's.
Running the full env path of mongodump into the python script worked.
To get the full path of mongodump, in terminal:
which mongodump
>>/usr/local/bin/mongodump
In my case i am using os.system() in my script.
os.system(/usr/local/bin/mongodump [commands])
instead of
os.system(mongodump [commands])
This is because programs started from cron don't get the environment your login shell uses. In particular, PATH is usually quite minimal. The tried and tested way to run scripts from cron is:
Always use an absolute path to a script in the crontab, say /path/to/script.
The beginning of /path/to/script sets and exports PATH and any other variables needed, e.g. with export PATH=$(/usr/bin/getconf PATH):/usr/local/bin
You can test whether any script would run with a reduced environment with
env -i HOME=$HOME /path/to/script
If that runs ok, it is ready for cron.
I am new to Jenkins, recently want to schedule a job to execute a local python script. I do not have a source control yet so I selected "None" in Source Code Management when creating the job in the Jenkins UI.
I did some research about how to execute python scripts in Jenkins UI and I tried using Python Plugin to execute python scripts as build steps. But it failed. (But actually I don't want to use this Plugin since my script takes input arguments so I think I need to select something like "execute shell" in BUILD field -- I tried but also failed) Could anyone help me to find out how to properly run/call a local python script?
PS: I am also not clear about the Jenkins Workspace and how it works? Will appropriate if someone could clarify it for me.
Here is the Console output I got after the fail build:
Started by user Yiming Chen
[EnvInject] - Loading node environment variables.
Building in workspace D:\Application\Jenkins\workspace\downloader
[downloader] $ sh -xe C:\windows\TEMP\hudson3430410121213277597.sh
The system cannot find the file specified
FATAL: command execution failed
java.io.IOException: Cannot run program "sh" (in directory "D:\Application\Jenkins\workspace\downloader"): CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
at hudson.Proc$LocalProc.<init>(Proc.java:245)
at hudson.Proc$LocalProc.<init>(Proc.java:214)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:846)
at hudson.Launcher$ProcStarter.start(Launcher.java:384)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:108)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:65)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1728)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 16 more
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Create a Jenkins job and run your scripts as shell script from jenkins job.
Like this
#!/bin/sh
python <absolute_path_of_python_script>.py
instead of handle local script file on each server, you can actually copy all the python script into the "execute shell" under the Build section.
it has to start with the relevant python shebang. For example:
#!/usr/bin/env python
your script...
you can also add parameters in the job and use environment variables in your python script. for example
parameter1 = os.environ['parameter1']
Another way is creating pipeline and execute sh command, which points to your python script. You also can pass parameters via Jenkins UI as dsaydon mentioned in his answer.
sh command can be as follow (is like you run in command line):
sh 'python.exe myscript.py'
Example pipeline step with creating new virtual environment and run script after installing of all requirements
stage('Running python script'){
sh '''
echo "executing python script"
"'''+python_exec_path+'''" -m venv "'''+venv+'''" && "'''+venv+'''\\Scripts\\python.exe" -m pip install --upgrade pip && "'''+venv+'''\\Scripts\\pip" install -r "'''+pathToScript+'''\\requirements.txt" && "'''+venv+'''\\Scripts\\python.exe" "'''+pathToScript+'''\\my_script.py" --path "'''+PathFromJenkinsUI+'''"
'''
}
where
sh '''
your command here
'''
means multiline shell command (if you really need it)
You also can pass variables from your pipeline (groovy-script) into sh command and, consequently, to your python script as arguments. Use this way '''+argument_value+''' (with three quotes and plus around variable name)
Example: your python script accepts optional argument path and you want to execute it with specific value which you would like to input in your Jenkins UI. Then your shell-command in groovy script should be as follow:
// getting parameter from UI into `pathValue` variable of pipeline script
// and executing shell command with passed `pathValue` variable into it.
pathValue = getProperty('pathValue')
sh '"\\pathTo\\python.exe" "my\\script.py" --path "'''+pathValue+'''"'
To execute a Python script under BUILD option- select execute windows batch command - type these cammands.
I am passing the pythonpath because jenkins was not able to access the environmental variables because of access issues.
set PYTHONPATH=%PYTHONPATH%;C:\Users\ksaha029\AppData\Local\Programs\Python\Python3
python C:\Users\ksaha029\Documents\Python_scripts\first.py
On Mac I just moved the script.py to /Users/Shared/Jenkins/Home/workspace/your_project_name and with chmod 777 /Users/Shared/Jenkins/Home/workspace/your_project_name/script.py I could fix the problem.
Also, I did not need to use :
#!/bin/sh or #!/usr/bin/env python. Just inside jenkins build I used:
python3 /Users/Shared/Jenkins/Home/workspace/your_project_name/script.py
I should mention that, for one day I was trying tu solve this problem, and I have read all the froum related questions. no one really could help :/ .
Simplest implementation is checking the inject environment variables into the build process box. Then define two variables one for the python another for the script.
For example PYTHONPATH = C:/python37/python.exe
TEST1SCRIPT = C:/USERS/USERNAME/Documents/test1.py
Executing the windows batch command.
%PYTHONPATH% %TEST1SCRIPT%
This way you can run a number of scripts inside one or multiple execute windows batch command segments. There are also way to customize. You can create a wrapper that would run the scripts under Jenkins, this way, script result output could be formatted, if emailing results of the whole test suite.
I am using dryscrape in a python script. The python script is called in a bash script, which is run by cron. For those who may not be aware, dryscrape is a headless browser (use QtWebkit in the background - so requires an xsession).
Here are the main points concerning the issue I'm having
When I run the python script from the command line, it works
When I run the bash script from the command line, it works too
I figured out that this may have something to do with different environments between my command prompt and when the cron job is running, so I modified my bash script to source my .profile as follows:
#/bin/bash
. /full/path/to/my/home/directory/.profile
python script_to_run.py
This is what my cronjob crontab entry looks like:
0,55 14-22 * * 1-5 /path/to/script.sh >> $(date "+/path/to/logs/\%Y\%m\%d.mydownload.log" )
By the way, I know that the job is being run (I can see entries in /var/log/syslog, and the script also writes to a log file - which is where I get the error message below):
In all cases, I got the following error message:
Could not connect to X server. Try calling dryscrape.start_xvfb()
before creating a session
I have installed the prerequisites, on my machine (obviously - since it runs at the command line). At the moment, I have run out of ideas.
What is causing the script to run fine at the console, and then fail when run by cron?
[[Relevant Details]]
OS: Linux 16.0.4 LTS
bash: version 4.3.46(1)
cron user: myself (i.e. same user at the command prompt)
dryscrape: version 1.0.1
The solution to this was to call the dryscrape.start_xvfb() method before starting the dryscrape session.
Cron user does not have display, so you cannot run any command which requires a display.
You need to modify the python script to do not use any type of display (check carefully, because some python commands, even though they do not open any display , they internally check for this variable).
The best way to test is to ssh into the machine without Display, and check if you can run it from there without erros.
recently, I want to use python script to set environment in linux.This is one line of my code:
p = subprocess.call(['/bin/csh', '-c', "source setup.csh"])
My setup.csh file is below:
add questa10.2b
add ds5-2013.06
setenv MODELSIM modelsim.ini
But when I run my python, it shows that the files have sourced on screen, but it turns out I have to type myself on command line.
How could I solve these problem? Can any one please help me with this?
You're creating a new csh shell as a subprocess and then running your commands inside that shell, which then terminates. The commands do not run in, or affect, the parent shell within which Python is running. When you just run the commands yourself, they affect the current shell.
If you need these settings to persist in your current shell after Python terminates, your best bet in general is to source setup.csh rather than putting it in a Python script. If other child processes of the Python script need your environment variables, you can alter os.environ.