Question 1:
I am using jupyter 4 with python and I would need my script to do a relaunch all the cells above when a crash occurs.
Is this possible ?
Question 2:
If I need to relaunch all some cells, can I ask python to execute them according to some cell-id? I could then create a list of the cells id which have to be re-executed when catching an exception...
You can always relaunch all cells above the active cell using Cell > Run All Above. But when it comes to doing so programmatically and reliably, I've got both good and bad news for you.
Let's get the bad news regarding question 2 out of the way: NO
...at least not very reliably, because any ID of a cell would change if you insert or remove any other cell.
According to Execute specific cells through widgets and conditions on github:
We don't have the Ids of of cell in order to handle them
programatically.
And further down on the same post:
There are some APIs which can run cells identified by numbers, but
unfortunately the numbers change if you insert or delete a cell
somewhere above.
And now to the good news about the first question: YES
...but it's not 100% certain that it will solve your error handling needs as per the details in your question. But we'll get to that in a bit. Because the good news is that the answer to the question as it stands in the title
How to relaunch all cells above when a crash occurs?
is YES WE CAN!
The hard (maybe even impossible) part of this question is to implement it as a robust error handling method. If you're only interested in that, skip to the section The hard part at the end of my answer. For now, let's go on with the easy part that is to programmatically run the menu option Cell > Run All (as described in the answer by Nic Cottrell). You have two options:
Option 1 - Run all cells above by executing a cell:
If you insert the following snippet in a cell and run it, all cells above will be executed:
from IPython.display import Javascript
display(Javascript('IPython.notebook.execute_cells_above()'))
Option 2 - Run all cells above by clicking a button:
If you insert the following snippet in a cell and run it, all cells above will be executed when you click the appearing button:
Snippet:
from IPython.core.display import display, HTML
HTML('''<script> </script> <form action="javascript:IPython.notebook.execute_cells_above()"><input type="submit" id="toggleButton" value="Run all"></form>''')
Output:
THE HARD PART
So, how can we set this up to handle an error when a crash occurs? I'm not an expert on this, but I think I've been able to make a setup that will work for you. But it will most likely depend on the type of error in question and the rest of your work flow.
The following example builds on two different error messages. The first is a NameError that occurs when you try to assign a value to a variable that does not exist. And this will be useful since re-running some cells after an error will need an iterator that resets only when the notebook is restarted completely, and not when a cell is re-run as part of an error handling method. The name error will only occur when the kernel is restarted upon a fresh restart of your notebook. As part of the error handling, the value 0 is assigned to x1. When the cell is only re-run x1 will increase by 1.
The second error will serve as a proxy for your error, and is an AssignmentError that occurs each time you try to delete an element from a list that does not exist. And this leads us to the real challenge, since if your error handler re-runs all cells above every time the error is triggered, you'll quickly end up in a bad loop. But we'll handle that with a counter that exits the looping execution of cells after a few runs.
It's also a bit problematic that there does not seem to exist a functionality to rerun your existing cell, or the cell from where the run cells above functionality is initialized. But we'll handle that with another suggestion from the same github post as earlier:
Doing the following helps me to execute the cell right below the code
cell. You can also change the values to get cells in other parts of
the notebook.
display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1, IPython.notebook.get_selected_index()+2)'))
Notebook with suggested work-flow:
Insert the four following snippets below in four cells. Click the menu option Cell > Run all once, and we're good to go!
Snippet 1 - Imports and setup
import sys
import os
from IPython.core.display import display, HTML
from IPython.display import Javascript
from random import randint
# Trigger to randomly raise en error in the next cell
ErrorTrigger = randint(0, 9)
# Assignment of variables at first run of the Norebook
try: x1
except NameError: x1 = None
if x1 is None:
%qtconsole # opens a qtconsole (for variable inspection and debugging)
x1 = 0 # counter for NameError
x2 = 0 # counter for assignment error (used in cells below)
mr = 0 # counter for manual relaunch by button
ErrorTriggers=[] # container for ErroTriggers
print('NameErrors = ', x1)
else:
x1 = x1 + 1
ErrorTriggers.append(ErrorTrigger)
#print('Executions:', x1, '||', 'Triggers:', ErrorTriggers)
Snippet 2 - Proxy for your error
# PROXY ERROR => INSERT YOUR CODE FROM HERE ################################################################
list1 = [1,2,3,4]
# 80 % chance of raising an error trying to delete an element that does not exist in the list
if ErrorTrigger > 2:
elemDelete = 8 # error
else:
elemDelete = 0 # not error
try:
del list1[elemDelete]
print('Executions:', x1, '||', 'Triggers:', ErrorTriggers)
print('Routine success on attempt', x2 + 1)
print('Error mesg: None')
ErrorTriggers=[]
x2 = 0 # reset error counter
# TO HERE #################################################################################################
except Exception:
x2 = x2 + 1
# Will end error handler after 5 attempts
if x2 < 3:
# As long as we're UNDER the attempt limit, the next cell executed by:
display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1,'+
' IPython.notebook.get_selected_index()+2)'))
else:
# If we're OVER the attempt limit, it all ends here. The next cell is NOT run.
# And NEITHER is the last cell with the button to relaunch the whole thing.
print('Executions:', x1, '||', 'Triggers:', ErrorTriggers)
print('Routine aborted after attempt', x2)
print('Error msg:', sys.exc_info()[1]) # Returns a message describing the error
# reset variables
ErrorTriggers = []
x2 = 0
Snippet 3 - Cell to rerun all cells above as error handler
display(Javascript('IPython.notebook.execute_cells_above()'))
Snippet 4 - Cell to rerun the whole thing with en error probability of 20%
HTML('''<script> </script> <form action="javascript:IPython.notebook.execute_cells_above()"><input type="submit" id="toggleButton" value="Run again!"></form>''')
Screenshot after a few test runs:
I'll gladly add more details if the comments in the snippets are unclear.
But if you run the notebook a few times by clicking Run Again! and at the same time have a look at the output of cell 3, you'll quickly grasp how the whole thing is put together:
I'm running Notebook server 5.4.0 and I have an option Cell > Run All Above which seems to do exactly this.
In Jupyter, click in each of the cells you want to re-run after error and go to View > Cell Toolbar > Tags. Type 'raises-exception' (no quotes) in the box at the top of the cell and Add Tag.
In the next cell, put the following:
from IPython.display import Javascript
Javascript("IPython.notebook.execute_all_cells()")
Then select Cell > Run All.
This should catch the errors and run all the cells on an infinite loop until interrupted.
This is for jupyterlab
go to: Settings->Advanced Settings Editor->Keyboard Shortcuts
Paste the below code in the User Preferences window:
{
"shortcuts": [
{
"command": "notebook:run-all-above",
"keys": [
"Shift Backspace"
],
"selector": ".jp-Notebook.jp-mod-editMode"
}
]
}
click on save (top right of the user-preferences window)
This will be effective immediately. Here, two consecutive shift + backspace presses runs all cells above the selected line.
Notably, system defaults has empty templates for all menu commands, including this code (search for run-all-above).
Related
It happens to me that when reading/reviewing the code, I becomes easier if I can see the 'look' of the variable a function is processing.
For that, I'd like to display a 'static' version of an instance of that variable (as a visual aid).
That variable may not be there on another run of the notebook, that's why it has to be text, not output.
This is also useful when creating documentation within the notebook.
With this little function
#----------------------------------
def vdisplay(var):
"""Converts the var to a pretty string and inserts
it on a new cell just below the present one.
Then you have to change that 'next cell' type to Markdown and execute it.
"""
# To print the var nicely.
from pprint import pformat as pf
string_to_insert=f"""
This is how it looks like:
```
{pf(var)}
```
"""
# Create a code cell and insert a string in it
get_ipython().set_next_input(string_to_insert)
return
#----------------------------------
You can do this
# This is the data we want to show
x={i:str(i)*i for i in range(10)}
# Show it!
vdisplay(x)
Visually:
I use the mouse intentionally so you can see the steps. Using keyboard shortcuts is much quicker.
Story: I explored several venues. The last one was a combination of
%store var f.txt and %load f.txt but that involved some manual
work. The evolution of that method is the one above.
I am trying to have pyautogui move the mouse whenever it detects a color but for some reason whenever I try running it keeps on prompting this error, I have run this code before and it worked perfectly fine. pls help
Code
Output
You are getting that error because "locateAllOnScreen" returns a "generator" which can be looped through which contains all instances of the image. You may be looking for "locateOnScreen".
Here are some example on how to use the two different functions:
# Will loop through all instances of 'img.png'
for pos in pyautogui.locateAllOnScreen('img.png')
# Code here...
# Will find one instance and return the position
pyautogui.locateOnScreen('img.png')
This link has some good information on the different methods
In Pycharm, there's "code structurure" side bar which provides a tree to navigate through the code, but, it is only useful when the code has classes and methods and objects. If nothing of that is in code then it is useless.
My question is: is there any way in which I dictate that this is a block, and I want to be able to collapse it and expand it? Something similar to Jupyter where the code is inherently divided to cells.
Currently, I'm doing this:
# ---------------------------------- chunck x blah blah -----------------------
EDIT:
Most of comments say that I'm dumb and I don't know how to code efficiently and that I should use functions and classes. Guys, I know how to use those, that's not my question. Thanks.
Turns out that the answer is very simple:
Select the code, right click, do custom folding
PyCharm allows you to define 'code cells' when you have 'Scientific Mode' enabled. These code cells are collapsible and expandable. To quote from the PyCharm website:
A “code cell” is a block of lines to be executed all at once in the
integrated Python console. You can define cells simply by adding
inline comments #%% to your regular Python files. PyCharm detects
these comments and shows you a special run icon in the left gutter.
Clicking this icon triggers the execution of a cell:
The only catch is that Scientific Mode and its code cell functionality are only available in PyCharm Professional Edition.
You can select a region, and press ctr+alt+t, then select <editor-fold...>. This will surround the region with a comment that makes the region collapsible. You can also do this manually by adding the following around the region:
# <editor-fold desc="This text is shown when collapsed">
# </editor-fold>
I sometimes use True conditional statements to create collapsible blocks in PyCharm and other IDEs. This also helps me to visually relate all indented code, access it when needed, and collapse it, when I'm focusing on other parts of my code.
if True:
# block code goes here
A fancier way is to use a descriptive string in the condition. The description stays visible for a collapsed block. You can also disable these with negation anytime, if needed.
if 'Define similarities':
Dot = lambda x, y: x # y
CosSim = lambda x, y: x # y / (x # x)**0.5 / (y # y)**0.5
I have a variable that I use in one cell of the jupyter but when I call it in the next cell it says the variable cannot be found
I've tried creating an instance of the variable before reading the data from the son file. I've tried creating another variable the same as the first and using that one in the next cell.
if os.path.isfile("cointegration.json"):
with open("cointegration.json", "r") as f:
cointegration_mv = json.load(f)
else:
cointegration_mv = {}
for i in range(0, len(usd_pairs)):
pair_i = usd_pairs[i]
print(pair_i)
for j in range(0, i):
pair_j = usd_pairs[j]
print("\t+",pair_j)
if pair_i != pair_j:
code = pair_i + '_' + pair_j
print("\t\t=",code)
cointegration_mv[code] = get_cointegration(pair_i, pair_j)
json = json.dumps(cointegration_mv)
f = open("cointegration.json","w")
f.write(json)
f.close()
print(cointegration_mv.keys())
#This does print the keys
#The next cell
print(cointegration_mv.keys())
The error is:
NameError Traceback (most recent call last)
<ipython-input-15-7abb854e8cc9> in <module>
----> 1 print(cointegration_mv.keys())
NameError: name 'cointegration_mv' is not defined
This happens even when I create an instance of the variable before reading in the data (so I'd expect the keys to just be empty), and when I try duplicating the variable and passing the duplicate.
I know this question is 2 years old but for anyone new to Jupyter notebook having this problem, you most likely have to run the previous cell in order for Jupyter to initialize/declare/store those previous variables. Whenever you run a cell it DOES NOT automatically run the other cells and gets those previous variables. As the guide below explains it best:
Be aware that it is the order of execution of cells that is important in a Jupyter notebook, not the order in which they appear. Python will remember all the code that was run previously, including any variables you have defined, irrespective of the order in the notebook. Therefore if you define variables lower down the notebook and then (re)run cells further up, those defined further down will still be present
Jupyter/Python Guide
Make sure you´ve run all the cells.
For example, if a variable x is defined in the first cell you have to also run this cell. Only then the variable will be defined, ready to be used across all cells. Otherwise, it won´t exist at all.
May be that variable declared in first cell is in local scope.Thats why when calling in second cell giving not defined as error.
try putting your code in function in first cell and then call that function in second cell
Consider the following trivial PyMEL script. It basically calls an existing MEL function.
from pymel import *
message = "Are you sure you want to clean the scene? (Nodes will be removed.)"
do_clean = core.windows.confirmBox("Scene clean", message)
if do_clean:
result = mel.eval("MLdeleteUnused()")
print(result)
"result" contains the number of nodes removed from the scene, rather than which nodes were deleted. However, in Script Editor I see the following output when I invoke the script:
delete "edit_example_20160209_1"; # < can I get this output?
delete "edit_screenshot_1";
delete "place2dTexture1";
delete "place2dTexture2";
4 # < output from my print statement
Can I access this output so I can display it to the user (without them having to look in the Script Editor themselves)?
You can look at the way MLdeleteUnused() works in <path_to_your_maya_install/scripts/others/MLdeleteUnused.mel It looks like the UI version of the command is invoking a progress function that does the actual printing of results, but it doesnt return the values -- it's basically just printing them out as it goes