Running Jupyter/IPython document on Zepplin - python

I have a large set of existing documents running on Jupyter. I want to move to Zeppelin.
I assume Jupyter pages run in Zeppelin, but I can't find any documentation. Has anyone tried this? Does it work? Anyone have a useful link?
Thanks
Peter

You can use the new "JupyterUtility" introduced by Pull Request 2393 to convert one jupyter notebook (.ipynb) to apache zeppelin notebook (.json). It's not yet released and the documentation is still in progress (see ZEPPELIN-2616). But you can either take the 0.8.0-SNAPSHOT build or checkout the sources and build/use it with maven.
E.g.: checkout -> switch to folder "zeppelin-jupyter":
mvn exec:java -Dexec.mainClass="org.apache.zeppelin.jupyter.JupyterUtil" -Dexec.args="-i INPUTFILE.ipynb -o OUTPUTFILE.json"

Solved in Zeppelin 0.10.0. This is the pull request closed ZEPPELIN-3033
Check the version at the menu "About Zepplin".
Go to "Import Note" from the main page
Then you can select the ipynb file to import.
Maybe the interpreter won't be ok. You'd have to export the file, use a text editor to remplace all the interpreter by default and then import again.
If you are using Zeppelin with HortonWorks or Cloudera be careful. Zeppelin's version is 0.8.0. Here is the doc
Right now, I didn't find offical documentation about how to import a jupyter file into zeppelin. I think I doesn't exist because the pull request is open ZEPPELIN-2616

Jupyter's ipynb is json file. You can find The Jupyter Notebook Format
Zeppelin's note.json is also json format. Each notebook has an folder(notebook id) and note.json.
It should be possible to convert with a small application. I think most important information shall be.
cells(jupyter) -> paragraphs(Zeppelin)
cell -> paragraph
cell_type -> %... in text
source -> text

Related

Disable Jedi - no ipython_config.py?

I have Anaconda distribution of Python on Windows.
I have issues with auto-complete in Jupyter Notebook.
I think auto-complete was working before but it doesn't quite work now. I don't get it.
So I am trying to disable Jedi as it messes up my auto-complete in Jupyter Notebook (that's what other folks say who had the same issue).
This is described on several web pages as a possible solution.
See also
Kernel taking too long to autocomplete (tab) in Jupyter notebook
I follow this suggestion
https://github.com/ipython/ipython/issues/10493#issuecomment-298968328
I can locate my profile directory but there's no ipython_config.py there.
C:\Users\username\.ipython> dir ipython_config.py /s
Volume in drive C is OS
Volume Serial Number is BC61-492E
File Not Found
C:\Users\username\.ipython>
Why so? What am I missing?
Should I create the file myself if it's missing?
Maybe under the profile_default subfolder of C:\Users\username\.ipython?
Just in case you need it on the fly for a single notebook in jupyter:
%config IPCompleter.use_jedi=False
does the magic trick, too.
If the file ipython_config.py is missing then:
Run ipython profile create in the profile_default sub-folder
Append this property c.IPCompleter.use_jedi = False
to the newly created ipython_config.py file

Run from and save to .py file from Jupyter Notebook

On my old computer, I was able to run .py files from Jupyter Notebook, edit them, and run them. The .py file was effectively a notebook file for all intents and purposes. I updated to the latest version of notebook, and I am no longer able to do this. How do I use .py files on my notebook?
I know there are roundabout ways to do this. I am looking for the method where, when you are in notebook, instead of opening a .ipynb file, you select a .py file which is opened, and behaves like a .ipnyb. When you save it, it writes to .py.
A text file can be loaded in a notebook cell with the magic command %load.
If you execute a cell containing:
%loadpy filename.py
The content of filename.py will be loaded in the next cell. You can edit and execute it as usual.
To save the cell content back into a file add the cell-magic
%%writefile filename.py at the beginning of the cell and run it.
To see the help for any magic command add a ?: like %loadpy? or %%writefile?.
%COMMAND-NAME?
i.e. %run?
For list of available magic function use %lsmagic.
Alternatively there is also another method magic function called %save-f but I would not recommend that, it's an indirect way of saving files.
Also see -
1. Magic Functions docs
2.this nbviewer for further explanation with examples. Hope this helps.
This is not the exact answer. At one point, I was able to open .py files using python notebook and work on it as if it were a notebook file.
However, I have been able to replicate this behavior using VScode.
https://code.visualstudio.com/docs/python/jupyter-support-py
Using VScode, you can export all your .ipynb files into .py files, then run code blocks. Code blocks are separated by # %%.
I have not used it sufficiently long enough to decide if it is better than python notebook, but this seems to be the best solution so far. I previously tried using Atom/Hydrogen and did not enjoy the experience.
You can save individual cells as files using the following code: %%writefile some_file_name.py.
You can run that code straight from the terming or from another notebook using the following code: %run some_file_name.py
Some editors (like spyder and vscode) have jupyter notebook functionality. These can be used if jupyter in installed in the python environment.
You can use it by add #%% on top of the block of code. (in vscode the button 'run cell' will automatically appear)
Also it is possible to import .ipynb as .py which can be run in to fancy decrypt above.
I just found this package p2j and tested it with a .py file with functions, comments and normal code.
I used it as indicated in this answer by doing the following:
pip install p2j
p2j -o script.py -t new_file.ipynb
You can also add -o flag to overwrite the original file.
With this, I got a working Jupiter Notebook with each block of code in a cell and the comments as markdown.
Example:
Original .py script
Converted .ipynb

What is the difference between a .py file and .ipynb file?

I have read about .py and .ipy, also the difference between python, ipython and notebook.
But the question is: what is the real difference between .py and .ipynb file?
Is .ipynb file just more convenient to be run on jupyter notebook, or anything more? I am wondering because I am thinking about which format to be used for publishing on GitHub.
Thanks
.py is a regular python file. It's plain text and contains just your code.
.ipynb is a python notebook and it contains the notebook code, the execution results and other internal settings in a specific format. You can just run .ipynb on the jupyter environment.
Better way to understand the difference: open each file using a regular text editor like notepad (on Windows) or gedit (on Linux).
Save on git the .ipynb if you want to show the results of your script for didatic purposes, for example. But if you are going to run your code on a server, just save the .py
Adding #Josir answer, the below information is very useful for open .ipynb file using PyCharm.
Create a new Python project in Pycharm
Specify a virtual environment, and install the jupyter package(pip install jupyterlab).
Run the server using the jupyter-lab command.
Browser will open the jupyter notebook like below, there you can execute the .ipynp file.
Here is documentation https://jupyterlab.readthedocs.io/en/latest/
py means PYthon
ipynb means Interactive PYthon NoteBook - which is now known as Jupyter notebook.
The latter one is merely a Python script with descriptive contents - you describe what your data is doing by means of Python script and some funny texts. That's pretty much it - and also, you need a specific editor e.g. PyCharm or Google Collab to open and run it.
I think the answer here might help you: https://stackoverflow.com/a/32029027/11924650
.ipy indicates that it's an IPython script. The only difference between IPython scripts and normal Python scripts is that IPython scripts can use IPython magics, e.g. %timeit, and run system commands as !echo Hi.

Using VSCode Python extension with Jupyter and RemoteFS?

I use the RemoteFS extension in VSCode to connect to my remote SSH server. When I open a .py file on the remote server in VSCode, and then add #%% comment to the .py file, I don't get the option to run a Jupyter cell like I would locally.
Has anybody gotten VSCode's Python extension working with it's built-in Jupyter support and the RemoteFS extension?
We had an overly restrictive file check for when we allowed our "Run Cell" commands to show up and it was limiting it to local files only. I've fixed that issue in the following PR here:
https://github.com/Microsoft/vscode-python/pull/4191
I verified that I was seeing the cell commands using Remote FS after that. Sadly this just missed the cutoff for our recent January release, so it won't show up in the extension until later in February. If you want to check out the fix you can access our daily development build here:
https://github.com/Microsoft/vscode-python/blob/master/CONTRIBUTING.md#development-build
That build has the fix already, but it's not the full public release.

Links between IPython notebooks

Is it possible to link one IPython notebook to another with a hyperlink in a Markdown cell? If I try
Link to [Notebook 2](files/notebook2.ipynb)
or
Link to Notebook 2
A new tab is opened with raw unformatted contents of the ipynb file. Is there a way to get IPython to open another notebook for use in a new tab via a hyperlink?
Since IPython 2 you may use exactly the syntax you first tried:
Link to [Notebook 2](notebook2.ipynb)
It is now possible to do this with Ipython 1.0+ at least.
Just do:
localhost:8888/My Notebook.ipynb
Here is the documentation for this feature.
https://github.com/ipython/ipython/pull/3058
From http://python.6.n6.nabble.com/where-is-the-code-to-generate-IPython-Notebook-URL-for-a-new-ipynb-file-td4996991.html:
You can access a json version of all the notebooks from url: $host/notebooks
Here is a snippet that worked for me:
import urllib2
import json
data = urllib2.urlopen("http://127.0.0.1:8888/notebooks")
json_data=data.read()
list_o_dicts=json.loads(json_data)
for d in list_o_dicts:
if d['name'] == 'test':
print d['notebook_id']
Modify this according to your need.
** on further reading, I just realized OP was also seeking new notebook creation, keeping my answer anyway as way to work with linking existing notebooks.
One way to try for OP's goal is to run a script which will create a new notebook.ipynb file into the ipython folder where ipython notebook was started from. That .ipynb file can be templated from a new ipython notebook created from dashboard, with the name and id of the notebook replaced with whatever you are trying to link from your existing notebook. I have not tried this, but should work since dropping a .ipynb extension file into ipython folder does show it up in the dashboard.
Remember that if your file name has spaces you will need to replace those with %20
eg:
[Numpy](Numpy%20For%20Python.ipynb)
In addition to akim's suggestion - you can link to any (py or ipynb) file using a relative link, starting with "edit", and and then from the directory where you started the server.
E.g. in a markdown cell, if I want to reference a file whose relative location (relative to my git repo, which is also where I launched the notebook server) is "./path/to/source.py", I'd add:
[link to source](/edit/path/to/source.py)
Unfortunately, this is not practically possible.
The link would need to be to the notebook ID
(e.g. /a1e2a88f-3b91-4a4e-8ca1-d4fd7240f750 for the one I'm working on right now).
This is an UUID created at startup by the IPython server.
So you can copy the link from IPython Dashboard, but it will be valid only until you restart.

Categories