I am currently creating many tables in python and most of the cells need their own SQL query. I am ending up with a long list of SQL querys. Ideally I would put the table creation of each table in its own file and load the files from a main notebook.
Is there a way to run sub-notebooks (containing the tables) from one main notebook? Mathematica has the EvaulateNotebook[] function which does exactly this. For python I found the nbconvert package which seems to come close to what I need. However, nbconvert seems to run in its own kernel. I would need the sub-notebook to have access to all variables in the main-notebook and subesquently the main-notebook to have acess to the variables in the sub-notebook.
In the end I would like to have the convenience of the notebooks in terms of editing, while not having to scroll through each and every table if I only need to change one table. So I am a bit reluctant to build a standard python package which I import, but would rather have a way to run a sub-ipynb in a way as if the cells where in the main-notebook.
I don't believe this is currently possible in Jupyter to have "child" notebooks in the same way you can with knitr.
Fernando Perez wrote a script to merge together multiple notebooks which may be helpful in your case to avoid having to scroll through a lot of text.
It looks like this may also eventually be incorporated into the main nbconvert codebase (see issue here).
Related
I'm using Databricks to run my Python notebook, and often use %md to create cells containing section titles and annotations in my code (Markdown cells). Is there some way to create Raw NBConvert cells using a % command? Raw NBConvert is available in JupyterLab in drop down menu:
but not in Databricks.
No, magics can't do that. The fact that a cell is a "raw" cell is independent of the kernel that execute code. There might be some hacks possible, but they will likely not be added in core Jupyter, and will probably break often.
If you are using Databrick, you want to contact support for that.
Is it possible to execute ONLY the highlighted code in a Jupyter notebook cell? This is possible in Spyder and RStudio.
I find this to be quite useful for trouble-shooting code as you write.
If a cell contains:
a=13
b=17
c=42
a=a*c
I'd like to be able to highlight and run only the desired lines (e.g. variable assignmemnts), but not the final line.
I use this frequently in Spyder and RStudio, would love to do in Jupyter as well. I find I am constantly splitting and re-combining cells in order to troubleshoot a single line of code, where for example, I've indexed into something incorrectly. Highlighting and printing the variable allows me to see what I've actually assigned it to be and is throwing an error, vs. what I had intended.
There is no such thing in Jupyter as 'highligh and run'. At least I am not aware of it.
Run the cell after commenting the other lines out using CTRL + /, split cells and execute only the chosen ones or use a debugger (e.g. pudb, it works in Jupyter) to change variables values on the fly (while debugging).
It seems now it is available in python notebook as well.
https://github.com/jupyterlab/jupyterlab/pull/2191
If I open a python notebook in Kaggle (www.kaggle.com) and select a text, it lets me run only the highlighted part.
Does anyone know how to export RCloud notebooks (not RStudio) to a common format? I’m using it for a project to try to predict if a car will statistically be a good purchase or a lemon, and I’m learning - so I’m trying a lot of new/different code and packages to make graphs, regression charts etc.
I’m new to RCloud so I want to save this notebook as a reference document/cheat sheet on my laptop so I can ‘reuse’ the common R commands I used (e.g. how to use "lapply" command to change vectors to numeric like “mycarsub[, 1:6] <- lapply(mycarsub[, 1:6], as.numeric”, "na.omit", etc. I just want a reference to use for other projects or notebooks in Rcloud, RStudio etc.
So I’m wondering if anyone knows how to export it in Text format that is searchable or easily read with common apps (outside of RCloud, or RStudio)? Like export to Word/Libreoffice, HTML etc?
I tried “Share” at the top but think it only exports R file types, I’m probably doing it wrong. Or if you have another way to accomplish what i'm trying to do. I cut and pasted but doesn't work all the time (user error?). I searched Stack Overflow but only got RStudio or R developer code exporting via API's etc. Hope this is enough info, first post.
RCloud was created for making it easier to share code and for others to learn from existing code so it includes the ability to search code using Lucene search syntax. Rather that creating another document to keep track of, I would suggest opening multiple RCloud tabs - use one to search, cut and paste from and the other to code in; you can create multiple tabs by copying and pasting any notebook URL into a new tab.
If you prefer to have a separate document, you can export the RCloud notebooks as an R Source file or a Rmarkdown file using the Advanced menu located in the navigation bar at the far right.
I would like to access the textual contents of another cell in the notebook from Python so that I can feed it to the unit testing script, regardless of whether it has been executed or not. It seems like this should be possible, but I can't find the right API to do it in the IPython kernel. Thoughts?
This question asks the same question, but I don't really want to use magics unless I have to. Ideally the workflow would be "select widget choice" followed by "click widget" to cause the tests to run.
Background
I'm working with some students to teach them Python, and in the past when I've done this I've set up a bunch of unit tests and then provided instructions on how to run the tests via a shell script. However, I'm working with some students that don't have access to computers at home, so I decided to try and use an Jupyter notebook environment (via mybinder.org) to allow them to do the same thing. I've got most of it working already via some ipywidgets and a helper script that runs the unit tests on some arbitrary set of code.
As far as the cells that have been run, the input and output caching system described at https://ipython.readthedocs.io/en/stable/interactive/reference.html#input-caching-system might be useful. (Examples of its use at https://stackoverflow.com/a/27952661/8508004 ). It works in Jupyter notebooks as shown below. (The corresponding notebook can be viewed/accessed here.)
Because #A. Donda raised the issue of markdown in the comments below, I'll add here that nbformat provides related abilities that works with saved notebook files. Reading a saved notebook file with nbformat allowa getting cells and content, no matter if it is code or markdown, and sorting whether the cells are markdown or code, etc.. I have posted a number of examples of using nbformat on the Jupyter Discourse forum that can be seen listed via this search here. It offers more utility than the related json.load(open('test.ipynb','r')) command, highlighted in a comment here to read a notebook file because of the additional notebook context automatically included.
If you want to capture the contents of a specific cell to access it from another one, one workaround seems to bee to write the contents of the first cell to file when executing it and later loading the resulting file (i.e. the text giving the former cell's content) inside the latter cell, where the content of the former cell is required.
Cell's contents can be saved to file (when executing the respective cell) via the command
%%writefile foo.py
which has to be placed at the beginning of a cell. This results in the cell's content (in which the upper command is executed) being saved to the file foo.py and it's just a matter of later reading it in, again.
The output of a cell can be made available more easily:
Just place %%capture output in the first line of a cell. Then, the output of the cell (after execution) is going to be saved as a string to the variable output and can be used like any standard python string-variable.
References:
Programmatically get current Ipython notebook cell output? and
https://nbviewer.jupyter.org/github/ipython/ipython/blob/1.x/examples/notebooks/Cell%20Magics.ipynb
I found a solution to this that can get the output of any cell. It requires running some Javascript.
Here is the code that you can put in a cell and run it an it will generate a Python variable called cell_outputs which will be an array of the cell outputs in the same order as they appear on the page. The output of the cell executing this code, will have an empty string.
%%js
{
let outputs=[...document.querySelectorAll(".cell")].map(
cell=> {
let output=cell.querySelector(".output_text")
if(output) return output.innerText
output=cell.querySelector(".rendered_html")
if(output) return output.innerHTML
return ""
}
)
IPython.notebook.kernel.execute("cell_outputs="+JSON.stringify(outputs))
}
If you need the output of a specific cell, just use its index, eg: cell_outputs[2] in your Python code to access the output of cell #3.
I tested this on my Jupyter notebook 6.0.3 (recent install via the Anaconda community edition) on Google Chrome. The above could should work fine on any modern browser (Chrome, Firefox or Edge).
I'm currently working on a project that requires me to write a (seemingly endless) series of macros to reproduce pivot tables designed by one of our Analysts. The details change, but the code is largely the same from example to example.
I would like to programmaticly generate the vba code based on a handful of options and then add the macro to the given worksheet. Is it possible to create an excel macro with python using win32com or other? Is it possible to create an excel macro from another excel macro? Any other ideas?
Project Background, I have a python script that does the following:
pulls Google Analytic data
does various analysis
writes the results to excel
triggers a pre-written macro to turn the data into a beautifully formatted pivot table
emails it off to people who probably don't actually read it
Yes you can create an Excel macro with another Excel macro. For that to work, you need to tell Excel to Trust Access to the VBA Project Object model. (The setting is found in Macro Options in the Trust Center.) I don't know if you can do it from Python, but if you can, you probably also need to tell Excel it is ok.
For ease of coding, if you are doing this in Excel, add a reference to Micorsoft Visual Basic for Applications Extensibility.
You might also want to check out MZ-Tools 3.0 I find it very helpful for adding default\common code to a project.
On the other hand, your project sounds ripe for code reuse. If the common pivot table code is in one class/module, it is really easy to copy it from one open Excel project to another. (Just click and drag in the Project Explorer window.) You can also export it out to a text file and import it back in to another project later.