What is the best way to ignore IPython magic when running scripts using the python interpreter?
I often include IPython magic in my script files because it work with the code interactively. For example, with the autoreload magic, I don't have to keep reload-ing the modules after I make some changes and fix bugs:
%load_ext autoreload
%autoreload 2
However, when I try to run this script using a usual python interpreter, I get an error:
File "<string>", line 1
%load_ext autoreload
^
SyntaxError: invalid syntax
Wrapping IPython magic inside an if statement does not work, because incorrect syntax is detected before the file is actually ran.
So what is the best way to get python to ignore IPython magic?
It's annoying to have to change your scripts whenever you want to run then in python, pdb, sphinx, etc.
For all tools that can read from standard input you could use grep to remove any magic lines and pipe the result into python:
grep -v '^%' magicscript.ipy | python
Works well as a bash alias:
alias pynomagic='( grep -v "^%" | python ) < '
pynomagic magicscript.ipy
Tools like pdb that only accept filenames could be called like this (bash again):
pdb <(grep -v '^%' magicscript.ipy)
In case this helps anyone.
At least for Databricks, when syncing a notebook with a .py file in Github, a magic function can be specified with a specially formatted comment.
Like this:
# MAGIC %run ./my_external_file
You should load such magic in your config file, not in your scripts! It is just not valid Python.
Put the following in your ~/.ipython/profile_default/ipython_config.py:
c = get_config()
c.InteractiveShellApp.extensions = ['autoreload']
c.InteractiveShellApp.exec_lines = ['%autoreload 2']
c.InteractiveShellApp.exec_lines.append('print("Warning: disable autoreload in ipython_config.py to improve performance.")')
Create a template file named simplepython.tpl. Copy the below statements.
{% extends 'python.tpl'%}
{% block codecell %}
{{ super().replace('get_ipython','#get_ipython') if "get_ipython" in super() else super() }}
{% endblock codecell %}
Save simplepython.tpl.
Type in command line:
jupyter nbconvert --to python 'IPY Notebook' --template=simplepython.tpl --stdout
Spyder gives warning (as given in the picture below), when a coder use this type of code and says that it is not a valid Python code.
So, in order to use IPython magics, saving files with the .ipy extension may be a solution.
Spyder screenshot
Related
I have a source file having environment variables (few environmental variables are conditional based as well).
source file is like (filename: global_setup)
if( -f /tools/tool/bin) then
setenv SHELL /bin/csh
endif
setenv LICENCE_FILE 2457#abc
setenv ...........
I want to source this file using python script. Code is like this:
import subprocess
import os
if os.path.isfile('global_setup'):
subprocess.call(['/bin/csh', '-c', 'source ~/global_setup'], shell=True)
else:
print("file not found");
But this is not working. Showing error as:
/home/global_setup: line 53: syntax error: unexpected end of file
Also, I tried to remove this EOF error by changing file format to Unix.
Even it works, i doubt setup will be done in some other terminal as i am using subprocess.call() in python.
Sourcing a file with subprocess.call() won't have an effect on the environment variables in Python, since subprocess runs, well, a subprocess.
You will need to parse the file yourself if you wish to affect os.environ within your Python interpreter.
Since your file is csh-style, the otherwise very nice envparse module won't do here.
Something you could do, though, is invoke
env = subprocess.check_output(['/bin/csh', '-c', 'source ~/global_setup; printenv'], shell=True)
and parse the resulting env variable line by line and add the contents into os.environ. This way the conditionals within global_setup would also work as expected.
Instead of doing it this way, you should be doing one of two things:
Source your files in the linux shell before invoking your script.
Convert your file into a dotenv file that Python can read using the Python-dotenv module.
Sourcing it before is easier IMO, instead of sourcing it during runtime using subprocess.
I'm in IPython and want to run a simple python script that I've saved in a file called "test.py".
I'd like to use the %run test.py command to execute it inside IPython, but I don't know to which folder I need to save my test.py.
Also, how can I change that default folder to something else, for example C:\Users\user\foldername ?
I tried with the .ipython folder (original installation folder) but that's not working.
I found the answer:
import os
filepath = 'C:\\Users\\user\\foldername'
os.chdir(filepath)
%run test.py
%run myprogram works for Python scripts/programs.
To run any arbitrary programs, use ! as a prefix, e.g. !myprogram.
Many common shell commands/programs (cd, ls, less, ...) are also registered as IPython magic commands (run via %cd, %ls, ...), and also have registered aliases, so you can directly run them without any prefix, just as cd, ls, less, ...
I'd like to be able to use %cd "default_dir" and %matplotlib whenever I call ipython from my terminal. I tried writing this in a .py file in .ipython/profile_default/startup/file.py but it results in the following error:
[TerminalIPythonApp] WARNING | Unknown error in handling startup files:
File "/Users/<name>/Dropbox/.ipython/profile_default/startup/startup.py", line 18
%cd "~/Dropbox/"
^
SyntaxError: invalid syntax
You just need to use the magic in your startup scripts:
get_ipython().magic('cd ~/Dropbox')
get_ipython().magic('matplotlib')
Put that in the contents of your startup script and it should do the magic you need ✨🔮✨
I just wanted to elaborate the Wayne's answer, but do not have enough reputation to do a comment. You can have the following in the start up script to run the required magic commands
from IPython.core import getipython
getipython.get_ipython().magic(u"%reload_ext autoreload")
getipython.get_ipython().magic(u"%autoreload 2")
Module reference is here Ipython module
To run the above start up at terminal, do this
ipython -i startup.py
I'm just getting my feet wet with Python v3.5.2. I've installed IPython via Anaconda and am now attempting to run a simple program.
I wrote a simple print("Hello World!") script in a text editor and saved it as "C:\Python code\python_practice_code.py".
I've attempted various ways to execute python_practice_code.py, with and without quotes, and I get one of two errors:
In [34]: %run C:\Python code\python_practice_code.py
ERROR: File `'C:\Python/py'` not found.
or
In [35]: ipython 'C:\Python code\python_practice_code.py'
File "<ipython-input-35-30b39bc825d7>", line 1
ipython 'C:\Python code\python_practice_code.py'
SyntaxError: invalid syntax
What am I doing wrong?
You don't need to change the spaces in your path - as #MadPhysicist said, sometimes you don't have control over that. Instead, you can surround your path in quotes:
In [42]: %run "C:\Python code\python_practice_code.py"
It seems backslash or double backslash will not work in windows, similarly when used double or single quotes, shows error like
File
'\'\'"\'"\'C:/Users/xxx.yyy/.ipython/profile_default/startup/50-middle.py\'"\'"\'\'.py'
not found
. Forward slash no quote path works, for example:
%run -i C:/Users/xxx.yyy/.ipython/profile_default/startup/50-middle.py
But need to change all magic command to function call first such as %matplotlib inline to get_ipython().run_line_magic('matplotlib', 'inline'), %load_ext autoreload to get_ipython().run_line_magic('load_ext', 'autoreload') and etc.
I'm running IPython in an emacs shell using:
;; Set IPython interpreter in my init.el
(defvar python-shell-interpreter "ipython")
(defvar python-shell-interpreter-args "-i")
Then:
Start IPython with M-x run-python
Run a program within IPython with %run myprog.py. myprog.py imports a module called mymodule.
I make changes to mymodule but when I run %run myprog.py again, it runs the original mymodule, not the changed code.
FWIW, I'm using emacs 24.5 prelude, on Windows 10 with Anaconda and Python 3.5.
It turns out that IPython's %run command does not reload modules.
My current workaround is:
Add the code below to ~/.ipython/profile_default/ipython_config.py
Use $run myprog.py args
.
# this code in `~/.ipython/profile_default/ipython_config.py`
# get_config() is injected into the global namespace whilst
# config files are loaded
c = get_config()
# Autoreload modules
c.InteractiveShellApp.extensions = ['autoreload']
c.InteractiveShellApp.exec_lines = ['%autoreload 2']
I didn't realise that %run does not reload modules because I'm used to using Spyder's runfile command, which does. It's nuts that %run doesn't and I'd like to submit a patch to fix it at some point.
On Windows, the HOME environment variable must be set so that the run-python command in emacs can read the IPython profile. If HOME is not set, you can add this to your init.el:
(add-hook 'inferior-python-mode-hook (lambda ()
(progn
(python-shell-send-string-no-output "%load_ext autoreload")
(python-shell-send-string-no-output "%autoreload 2"))))