Schedule a Python script via batch on windows (using Anaconda) - python

I have a script that i run every day and want to make a schedule for it, i have already tried a batch file with:
start C:\Users\name\Miniconda3\python.exe C:\script.py
And im able to run some basic python commands in it, the problem is that my actual script uses some libraries that were installed with Anaconda, and im unable to use them in the script since Anaconda will not load.
Im working on windows and can't find a way to start Anaconda and run my script there automatically every day.

I would be a bit careful in calling python directly through environment as one never knows if the internals for activate function has changed.
I'm just using basic bat-script to help me out.
SET log_file=%cd%\logfile.txt
call C:\Anaconda3\Scripts\activate.bat
cd \script_directory
python script.py arg1 arg2 > %log_file%
This script saves the log-file wherever the bat is run from, calls the right environment through activate (in this case the standard) and directs all the stdout into log-file for further investigation.
Then just point your Windows Task Scheduler to the script and set the home directory where you want the log-file to appear.

I'd recommend creating an Anaconda environment with the packages you need, then using the python from that environment to run your script. Read about Anaconda environments here
For example...
Say you create an environment called cristians_env
conda create --name cristians_env
and you install the packages you need
conda install pandas
Then, all you need to do is this from your batch script (assuming you use Anaconda 2)
start C:\Users\name\Anaconda2\envs\cristians_env\bin\python C:\script.py
and voila! You're using your anaconda environment from your batch script!

I had a similar problem a few days ago.
What I discovered is that anaconda prompt is nothing but your usual cmd prompt after running an 'activate.bat' script which is located in the anaconda 'Scripts' folder.
So to run your python scripts in anaconda all you need to do is write 2 lines in a batch file. (Open notepad and write the lines mentioned below. Save the file with .bat extension)
call C:\....path to anaconda3\Scripts\activate.bat
call python C:\path to your script\Script.py
Then you schedule this batch file to run as you wish and it will run without problems.

Found a solution, i copied the "activate.bat" file in "C:\Users\yo\Miniconda3\Scripts" and renamed it as schedule.bat and added my script (copy pasted it) on the end of the file.
Then i can schedule a task on windows that executes schedule.bat everyday

Related

Unable to debug using gdb with python inside a virtual environment (venv) on windows

I am currently trying to debug a numpy code that I wrote using Gdb. To do that I activate first my virtual environment(on the command prompt) using the following commands (numpy_dev_env is my virtual environment)
numpy_dev_env\Scripts\activate
After that I activate my virtual environment.
gdb
then type "python" to start coding in python (all these steps work good for me).However, I realized that my test were not ran on the virtual environment,but instead on python system available in MINGW. I would appreciate your advice on ways to run the virtual enviroment with gdb.
Thank you,
I tried to configure the .gdbinit file in my home directory.I however did not know how to do with that
edit. clarified that when I run python on gdb the system used is the one available on MINGW and not of my system
Ensure that you are running the correct version of Python: start python in one shell, search for the process id using your task manager, then start gdb and attach the process.
Alternatively, start gdb with the full path to your python inside your environment.

How can I run a python script in anaconda environment with another python script? (not by .bat ro .sh file)

I have a python script which I want to run it in an anaconda environment because it uses some library like NumPy and I'm not going to install them.
I can easily write something like this:
# for Windows:
C:\ProgramData\Anaconda3\Scripts\activate.bat && python script.py arg
# for Linux:
source activate base
python script.py arg
but then I should have two scripts. a batch file and a shell script which is not desirable.
I like to have a single solution and independent of OS.
So I thought maybe it's good to have a python script which activates anaconda environment then run my script.(So I can handle OS there). I could do it by subprocess.call(). However, it gives an additional message which it's not desirable and I couldn't solve it even with a StackOverflow question.
Now I'm asking do you have a solution to run a python script in anaconda environment with another python script? (which of course doesn't give a notification like my solution.)

How to run the python shell inside a terminal in Windows?

I've scrolled the internet all over and theres no simple, clean explanation for this. How can I open the Python shell, but inside a terminal window such as CMD? I have an application for it, but want to run it inside a terminal, so I can practice Python code inside my IDE like VS code.
If you have a reasonably new version of Python you need to run py.exe (or just py) instead of python.exe, because the real python executable (python.exe) is not in your path environment variable by default, whereas py.exe is in the /windows directory and thus automatically in path.
So to run your script type py yourscript.py from the directory of your script. If you just run py it will start the interactive interpreter.
Look e.g. at https://www.python-course.eu/python3_execute_script.php
Under the title Start a Python script (almost at the beginning)
there is an instruction. Just write from the command prompt:
python,
a space,
and the filename containing your script, with .py extension.
As in each command, press Enter and your script will be executed.
Of course, I assume that you have installed Python on your computer.
If this is not the case, start from installing it.
I have quite a new Python version installed (3.6.5) and start Python
writing just python. Writing just py also does the job, but (IMHO)
it is not necessary.
If you want to start Python interpreter and play around with some
Python commands, write just python and Enter. The interpreter will
start and display its own command prompt (>>>).

Running Python Scripts with Jenkins

I am looking to schedule my python script runs with jenkins. The issue is, my scripts use a lot of libs like pandas etc that are installed on my mac terminal.
Is there a way to allow Jenkins to pick up these modules (or run the scripts as if it was terminal)? Also is there a way to run Python3 in jenkins?
I have already configured Jenkins to execute from custom workspace and have tried both shell and plugin executions.
The answer is yes, but it is detailed so I can only give you high level steps here. Jenkins can execute command line statements, and python modules can be run from the command line.
I would start by using the begins library to create a python file to run from the command line with arguments. Get it working on your local machine that way.
You will want to use either virtualenv or venv, and do all your pip installs using that virtual environment. Then you can copy the virtual environment to your Jenkins machine, or create a new one. Look into the freeze tool.
When calling your python from jenkins, you must first activate your virtual environment just as if you were working on it yourself.
You have a lot of research to do, but is very doable. I can help with follow up questions if needed.

How can I execute Python scripts using Anaconda's version of Python?

I recently downloaded the Anaconda distribution for Python. I noticed that if I write and execute a Python script (by double-clicking on its icon), my computer (running on Windows 8) will execute it using my old version of Python rather than Anaconda's version. So for example, if my script contains import matplotlib, I will receive an error. Is there a way to get my scripts to use Anaconda's version of Python instead?
I know that I can just open Anaconda's version of Python in the command prompt and manually import it, but I'd like to set things us so that I can just double-click on a .py file and Anaconda's version of Python is automatically used.
I know this is old, but none of the answers here is a real solution if you want to be able to double-click Python files and have the correct interpreter used without modifying your PYTHONPATH or PATH every time you want to use a different interpreter. Sure, from the command line, activate my-environment works, but OP specifically asked about double-clicking.
In this case, the correct thing to do is use the Python launcher for Windows. Then, all you have to do is add #! path\to\interpreter\python.exe to the top of your script. Unfortunately, although the launcher comes standard with Python 3.3+, it is not included with Anaconda (see Python & Windows: Where is the python launcher?), and the simplest thing to do is to install it separately from here.
Set your python path to the Anaconda version instead
Windows has a built-in dialog for changing environment variables (following guide applies to XP classical view): Right-click the icon for your machine (usually located on your Desktop and called “My Computer”) and choose Properties there. Then, open the Advanced tab and click the Environment Variables button.
In short, your path is:
My Computer ‣ Properties ‣ Advanced ‣ Environment Variables
In this dialog, you can add or modify User and System variables. To change System variables, you need non-restricted access to your machine (i.e. Administrator rights).
Find your PATH variable and add the location of your Anaconda directory.
Example of someone doing it here: How to add to the PYTHONPATH in Windows, so it finds my modules/packages? Make sure that you sub path out for the Anaconda file though.
The instructions in the official Python documentation worked for me: https://docs.python.org/2/using/windows.html#executing-scripts
Launch a command prompt.
Associate the correct file group with .py scripts:
assoc .py=Python.File
Redirect all Python files to the new executable:
ftype Python.File=C:\Path\to\pythonw.exe "%1" %*
The example shows how to associate the .py extension with the .pyw executable, but it works if you want to associate the .py extension with the Anaconda Python executable. You need administrative rights. The name "Python.File" could be anything, you just have to make sure is the same name in the ftype command. When you finish and before you try double-clicking the .py file, you must change the "Open with" in the file properties. The file type will be now ".py" and it is opened with the Anaconda python.exe.
I like to run a "bare-bones" version of Python 2 to verify scripts that I create for other people without an advanced python setup. But Anaconda and Python 3 have a lot of nice features. To enjoy both things on the same computer I do this on my Windows computer which allows me to easily switch.
C:\Users>python --version
Python 2.7.11
C:\Users>conda create --name p3 python=3
C:\Users>conda info --envs
Using Anaconda Cloud api site https://api.anaconda.org
# conda environments:
#
p3 C:\Anaconda3\envs\p3
root * C:\Anaconda3
C:\Users>activate p3
Deactivating environment "C:\Anaconda3"...
Activating environment "C:\Anaconda3\envs\p3"...
[p3] C:\Users>python --version
Python 3.5.1 :: Continuum Analytics, Inc.
For more info: http://conda.pydata.org/docs/test-drive.html
I know this is an old post, but I recently came across with the same problem. However, adding Anaconda to PYTHONPATH wasn't working for me. What got it fixed was the following:
Added Anaconda to the PYTHONPATH and remove any other distribution of Python from any paths.
Opened the command prompt and started python (Here I had to verify that it was indeed running under the Anaconda dist)
Ran the following lines inside anaconda
>>> import sys
>>> sys.path
['','C:\\Anaconda','C:\\Anaconda\\Scripts','C:\\Anaconda\\python27.zip','C:\\Anaconda\\DLLs','C:\\Anaconda\\lib','C:\\Anaconda\\lib\\plat-win','C:\\Anaconda\\lib\\lib-tk','C:\\Anaconda\\lib\\site-packages','C:\\Anaconda\\lib\\site-packages\\PIL','C:\\Anaconda\\lib\\site-packages\\Sphinx-1.2.3-py2.7.egg','C:\\Anaconda\\lib\\site-packages\\win32', 'C:\\Anaconda\\lib\\site-packages\\win32\\lib', 'C:\\Anaconda\\lib\\site-packages\\Pythonwin','C:\\Anaconda\\lib\\site-packages\\runipy-0.1.1-py2.7.egg','C:\\Anaconda\\lib\\site-packages\\setuptools-5.8-py2.7.egg']
Copied the displayed path
Within the script that I'm trying to execute on double click, changed the path to the previously copied one.
import sys
sys.path =['','C:\\Anaconda','C:\\Anaconda\\Scripts','C:\\Anaconda\\python27.zip','C:\\Anaconda\\DLLs','C:\\Anaconda\\lib','C:\\Anaconda\\lib\\plat-win','C:\\Anaconda\\lib\\lib-tk','C:\\Anaconda\\lib\\site-packages','C:\\Anaconda\\lib\\site-packages\\PIL','C:\\Anaconda\\lib\\site-packages\\Sphinx-1.2.3-py2.7.egg','C:\\Anaconda\\lib\\site-packages\\win32', 'C:\\Anaconda\\lib\\site-packages\\win32\\lib', 'C:\\Anaconda\\lib\\site-packages\\Pythonwin','C:\\Anaconda\\lib\\site-packages\\runipy-0.1.1-py2.7.egg','C:\\Anaconda\\lib\\site-packages\\setuptools-5.8-py2.7.egg']
Changed the default application for the script to 'python'
After doing this, my scripts are working on double click.
This solution is not perfectly answering the question because it modifies your .py file into a .bat file but it serves the save purpose, so I think it can help few people out there.
To launch a python script with anaconda's environment : change your .py extension into a .bat extension and add this command line in the beginning of your script :
#echo off & call conda activate <env name> & python -x "%~f0" %* & goto :eof
It will launch a .bat file that will activate your conda environment and execute your python script. This allows you to have a single file that you can copy/paste anywhere and just double click on it to launch it.
Here is an example of the content of the ".bat" file :
#echo off & call conda activate my_environment & python -x "%~f0" %* & goto :eof
print("Hello World!")
input()
import pandas as pd
print('succes')
input()
Hope this helps some people ! Cheers
Note:
If anaconda is not in yourr PATH environment variable, just replace :
conda activate with the path of anaconda activate .bat file, it should look like this C:\ProgramData\Anaconda3\Scripts\activate.bat.
You can also replace the environment name with it's directory, it often looks like that :
C:\Users\<user name>\.conda\envs\<environment name>
Many thanks to this post that helped me figure out this solution:
Python command line -x option
don't know windows 8 but you can probably set the default prog for a specific extension, for example on windows 7 you do right click => open with, then you select the prog you want and select 'use this prog as default', or you can remove your old version of python from your path and add the one of the anaconda
You can try to change the default .py program via policy management. Go to windows, search for regedit, right click it. And then run as administrator. Then, you can search the key word "python.exe" And change your Python27 path to you Anaconda path.
I know that this is an old question, but still there is no accepted answer.
Here is what I'm doing to start a python script with a double click in a specific anaconda environment.
I have found a very useful script in the anaconda3 folder cwp.py. This doesn't activate an environment, but it adds the environment prefix to the path.
You should use it as follow:
python cwp.py PREFIX ARGS
So actually, assuming that your anaconda installation is in C:\Users\User\anaconda3 and your environment is in C:\Users\User\anaconda3\env\myenv then what you have to do is to write the following line in a windows shortcut as a target:
C:\Users\User\anaconda3\pythonw.exe C:\Users\User\anaconda3\cwp.py C:\Users\User\anaconda3\env\myenv C:\Users\User\anaconda3\env\myenv\pythonw.exe path_to_your_script\your-script.py
A double click on the shortcut will start your script!

Categories