I have a Jupyter notebook that scrapes web data and saves a dataframe to a csv. I would like to run this every day automatically. I am using a mac for this project.
I have looked around a lot (including here: how to run a python jupyter notebook daily automatically), but as of yet I have not found a clear enough answer. I am quite new to all this, so I am looking for a step-by-step: like how you'd explain it to someone with no knowledge on cron etc.
Any advice would be much appreciated! Thank you!
Why don't you just convert the jupyter notebook into a raw python file? You can use this command: jupyter nbconvert --to script "[YOUR_NOTEBOOK].ipynb" (replace [YOUR_NOTEBOOK] with your notebook name)
EDIT: You could also use Juptext, as pointed out by #Wayne in the comments below.
If you need a jupyter notebook:
Use datalab or papermill
Use the SeekWall Chrome Extenstion
Create a custom python script to launch jupyter notebook, and run that python script using the Automator app in your Mac
Related
Hi guys I have a great problem: when i develop my model on jupyter notebook, i'm used to write the command:
%qtconsole
in order to test a little pieces of notebook before implementing them. I had to install a new jupyter environment as always done, but in this case if i run this cell nothing happen. If i write !jupyter console work, but I cannot write on the jupyter notebook because it appears as it is bsy with the console. How can i call a console connected to my jupyter notebook to test a little pieces of code? Thank you for the attention and the patience. Have a good day
I imagine this question is rather silly.
I'm using VS code and Jupyter notebooks within it. Is it possible to have the output sent to the interactive python interpreter instead/at the same time as the output beneath the Jupyter notebook cells? I tried searching, google but failed.
The answer in short is unfortunately 'no'. The Interactive window can be used with the #%% magic line or using the 'Jupiter.execSelectionInteractive' command (⇧ + ↩ on Mac). However, the later is not available in a notebook, only in .py files.
Peeking at Jupyter notebooks is a long process, need to cd to the file location and need to run jupyter notebook to start the server and get things done, but I don't need to edit or work with it. Is there any other possible way to just take a look at notebooks quickly?
Looking for the Linux equivalent of this tuxu/nbviewer
PS: Already know about a Firefox extension but it is broken now.
.ipynb(s) cannot be previewed, instead if you convert your notebook to html, it can be previewed right away, use jupyter nbconvert --to html yournotebook.ipynb and preview it with a double click. You can write your own code to automate this since you are using python, or you can install this nb-viewer which basically lets you open .ipynb(s) in your linux machine with a double-click.
within my code I use a Python package that calls external programs which write their output to the same console that IPython is running in. If I now run the same code from an IPython notebook, the output is not written to the notebook output cells, but into the console that the IPython notebook server is running in.
Do you know if it is possible to capture this output somehow, and then display it within the IPython notebook? I'm on Xubuntu 14.04, by the way.
Thank you very much for your help!
This gives you nice and comprehensive description of your problem and potential solutions. You need to try it out for yourself with your specific setup.
Been using the safe and easy confines of PyCharm for a bit now, but I'm trying to get more familiar with using a text editor and the terminal together, so I've forced myself to start using iPython Notebook and Emacs. Aaaaand I have some really dumb questions.
after firing up ipython notebook from terminal with the command 'ipython notebook', it pops up on my browser and lets me code. but can i not use terminal while it's connected to ipython notebook server?
after writing my code in ipython notebook, i'm left with a something.ipynb file. How do I run this file from terminal? If it was a .py file, i know i could execute it by tying python something.py from the command line; but it doesn't work if i type python something.ipynb in the command line. And of course, I assume I hit Control-C to quit out of the running server in terminal first? or do I run the command without quitting that? Or am i doomed to test it in iPython and then copy and paste it to a different txt editor like Emacs and save it in a .py file to run it?
what good is the .ipynb file if i can't run it in terminal or outside the iPython Notebook browser? Could I get my code in a .py from iPython Notebook if I wanted to? (i assume I'll be able to easily run it in terminal by tying something.py then)
thanks in advance. I'm still very much trying to figure out how to use this thing and there aren't many answers out there for questions this elementary.
Yes, you can not use the same terminal. Solutions: open another terminal or run ipython notebook inside screen. If you use Windows you might want to take a look into this question
Notebook documents (ipynb files) can be converted to a range of static formats including LaTeX, HTML, PDF and Python. Read more about converting notebooks in manual
Notebooks are great, because you can show other people your interactive sessions accompanied with text, which may have rich formatting. And if someone can run notebook server he can easily reproduce your computations and maybe modify them. Check out awesome notebook on traveling salesperson problem by Peter Norvig as an example of what you can do with ipynb. Or this notebook. More examples are available here
You can run your IPython notebook process in background.
On Unix platforms you can perform that with (note the leading &):
ipython notebook &
Or after a "normal" run, hit [Control+z] and run the bg command
(some lecture).
you can convert .ipynb file into .py file using nbconvert
with Ipython notebook 2.x (some lecture):
ipython nbconvert --to python mynotebook.ipynb
with Ipython notebook 3.x (some lecture):
ipython nbconvert --to script mynotebook.ipynb
.ipynb contains your script AND results AND formatted text.
You can compare it to a literate script (but using "machine programming language").