I have a question for the jupyter notebook.
When I copied and pasted 663 lines of a python code to the jupyter notebook,
it shows the much lower response than the notebook which has just a few code lines.
Have anyone experienced this issue?
Anyone knows the solution?
Without any information about your code is really difficult to give you an answer.
However try to keep you output under control. Too much output to generate with a single run can overkill the kernel.
Moreover, it makes not much sense to run in a single cell almost 700 lines of code, are you sure you're using the right tool?
Sometimes a piece of code could slow all the session, if you split your execution in smaller pieces, over multiple cells you will find what is really your bottleneck.
Add this to your notebook and then click on the link after you execute the cell. Then you can track progress of what's running and see which statements are causing it to be slow. You could also split the code up into multiple cells to see where slow down is occurring.
from IPython.core.display import display, HTML
#sc = SparkContext.getOrCreate()
from pyspark import SparkContext
sc =SparkContext()
spark_url = sc.uiWebUrl
display(HTML('''
<p>
<br />Spark connection is ready! Use this URL to monitor your Spark application!
</p>
<p>
{spark_url}
</p>'''.format(spark_url=spark_url)))
Related
I made a function in the jupyter notebook and the cell got deleted. I want to know the code in the function. I cannot use undo deleted cell because it got deleted a long time ago. The function is not deleted from the notebook memory as I can run it. Is there a way to get the code written inside the function?
You might still be able to get the source code of the function, if you still have it in memory like you say.
This post gives a way to try and get it through the inspect library:
https://stackoverflow.com/a/427533/14486144
import inspect
lines = inspect.getsource(foo)
print(lines)
Try give that a shot, I tried it out in jupyter notebooks, although I wasn't exactly sure how to recreate your situation that you can't undo deleted cells. Try that out and see it if works.
Context:
I started teaching myself a few new libraries using Jupyter Lab. I know showing emotion on SO is strictly forbidden and this will get edited, but WOW, Jupyter notebooks are cool!
Anyway, I'm taking notes in markdown as I work through code examples. It gave me the idea of writing my own little textbook as I learn.
For example, in notebook 1, I talk about (teach myself) linear regression. It take notes on vocabulary, show some mathy formulas then work through some code examples. End section.
In notebook 2, I start the conversation about different metrics to show how effective the regression model was. Then I want to execute some code to calculate those metrics... but all the code for the regression model is in the last notebook and I can't access it.
Question:
Is there a way to link these two notebooks together so that I don't have to re-write the code from the first one?
My attempt:
It seems like the closest thing to what I want to do is to use
%run notebook_01.ipynb
However, this throws an error. Note that it appears to search for a .py file to run:
ERROR:root:File 'linear_regression01.ipynb.py' not found.
I have found some questions/answers where this appears to work for other users, but it is not for me.
Edit: I got the magic command %run to work, however it runs AND prints the entire first notebook into the second. I'ts good to know how to do this and it does achieve the goal of not having to re-code, but it re-prints absolutely everything, which I do not want.
If you run this from the command line :
jupyter nbconvert --to script first_notebook.iynb
It will create a python file from your first notebook called 'first_notebook.py'. After that you can import from that file into your second notebook with:
import first_notebook
Ok, I found the answer by way of suppressing outputs:
Just put this at the top of your second notebook:
from IPython.utils import io
with io.capture_output() as captured:
%run your_linked_notebook.ipynb
This will cause the notebook you want to link to run, allowing you to use any of the data from it, but without having to see all of the outputs and visualizations from it.
This is probably not a great way to go if you are working with a lot of data or linking a lot of notebooks that perform expensive computations, but will likely do the trick for most people.
If there is an answer that does not involve running the notebook linked, I'd be excited to see it.
Everytime I try to display a complete DataFrame in my jupyter notebook, the note book crashes. The file wont start up so I had to make a new jupyter notebook file. When i do display(df) it only shows a couple of the rows, when I need to show 57623 rows. I need to show results for all of these rows and put them into an html file.
I tried setting the max rows and max columns, but the entire dataframe would not print out without the notebook crashing
''' python
pd.set_option('display.max_columns', 24)
"pd.set_option('display.max_rows', 57623)
'''
The expected results were for the entire DataFrame to print out, but instead the notebook would have an hourglass next to it and nothing would load.
I think your machine is not powerful enough to handle so much data. I can display 57623 rows for my data (with 16 columns) without any problem. But my machine has 252GB memory, and it still took about 1 minute to display these data in jupyter notebook.
If I tried to scroll through the data, it's slow, and sometimes stuck for a while.
On the other hand, what do you want to achieve by displaying all data here? There's definitely another way to achieve the same thing as you are doing now.
Is it possible to execute ONLY the highlighted code in a Jupyter notebook cell? This is possible in Spyder and RStudio.
I find this to be quite useful for trouble-shooting code as you write.
If a cell contains:
a=13
b=17
c=42
a=a*c
I'd like to be able to highlight and run only the desired lines (e.g. variable assignmemnts), but not the final line.
I use this frequently in Spyder and RStudio, would love to do in Jupyter as well. I find I am constantly splitting and re-combining cells in order to troubleshoot a single line of code, where for example, I've indexed into something incorrectly. Highlighting and printing the variable allows me to see what I've actually assigned it to be and is throwing an error, vs. what I had intended.
There is no such thing in Jupyter as 'highligh and run'. At least I am not aware of it.
Run the cell after commenting the other lines out using CTRL + /, split cells and execute only the chosen ones or use a debugger (e.g. pudb, it works in Jupyter) to change variables values on the fly (while debugging).
It seems now it is available in python notebook as well.
https://github.com/jupyterlab/jupyterlab/pull/2191
If I open a python notebook in Kaggle (www.kaggle.com) and select a text, it lets me run only the highlighted part.
I'm starting to learn Python and I'm currently following a MOOC using Jupyter, which seems to be a great tool. It's composed of several questions, each one requiring to write a piece of code.
I submit my code on a regulary basis so I know that this particular notebook works. But now I have an orange "Method not allowed" flag top right on my browser:
When I click on it it disappears for a while and then come back. All code chunks are executed without any error.
How could I get informations about how to solve this issue ?
EDIT :
It wasn't about the code written in the notebook. See answer below.
Thanks :)
I finally figured out what it was for my case :
This flag meant that some command from Jupyter menu somehow failed/was rejected (e.g. save & checkpoint at a moment my login session had expired)
When using automatic commands (e.g. auto-save), it can appear randomly and be hard to understand
It had no link with code in the notebook
This issue might have been caused by the fact I used Jupyter locally for personal purposes, at the same time I used MOOC's Jupyter.
Reloading the page fixed the issue but I lost some pieces of the code I wrote and I wasn't able to download the file before reloading (it failed too), so beware!
I had the same problem (the orange 'method not allowed' message was appearing, and autosave and save were failing). Seems like it might be caused by an expired session as suggested by kdallaporta.
A solution that avoids reloading (as suggested by kdallaporta) and possible data loss is the following (this worked for me in a coursera notebook): log out and then log in from another window, and then open a new copy of the notebook (without closing the original notebook). The original notebook should work again without the problems above, and the new copy can be closed.