We Keep Coding

Auto-generate title from filename in Pelican

1. Summary
I can’t automatically generate the correct title from filenames of articles/pages.
For example, I can’t automatically generate metadata key title Kira Goddess from article Kira-Goddess.md
2. Argumentation
DRY, automation. I don’t want to manually write the title every time for each article and page if I can do it automatically.
An exception — files with words, that contain hyphens — “well-known”, “English-speaking”. In this case, I must explicitly specify title in the metadata of my articles. But words with hyphens are rare in filenames of my articles.
3. MCVE
3.1. Data
You can see it in my KiraTitleFromFilename branch of my repository for Pelican debugging.
pelicanconf.py:
"""MCVE."""
AUTHOR = 'Sasha Chernykh'
SITENAME = 'SashaPelicanDebugging'
SITEURL = '.'
PATH = 'content'
TIMEZONE = 'Europe/Moscow'
DEFAULT_LANG = 'en'
# [INFO] Use article name when preserve the slug:
# https://docs.getpelican.com/en/stable/settings.html#url-settings
SLUGIFY_SOURCE = 'basename'
# [INFO] Preserve case of article filename
SLUGIFY_PRESERVE_CASE = True
# [INFO] Get title from article filename:
# https://docs.getpelican.com/en/stable/settings.html#metadata
# https://github.com/getpelican/pelican/issues/2107
# https://github.com/getpelican/pelican/commit/2e82a53cdf3f1f9d66557850cc2811479d5bb645
FILENAME_METADATA = '(?P<title>.*)'
Kira-Goddess.md:
Date: 2020-09-24 18:57:33
Kira Goddess!
Another Pelican files generated by pelican-quickstart.
Simplified part of base.html:
<title>{{ article.title }}</title>
3.2. Steps to reproduce
See .travis.yml:
Run Pelican build:
pelican content -s pelicanconf.py --fatal warnings --verbose
Finding content of <title> tag:
grep -E "<title>.*</title>" output/Kira-Goddess.html
3.3. Behavior
3.3.1. Current
See Travis build:
<title>Kira-Goddess</title>
3.3.2. Desired
It would be nice, if:
<title>{{ article.title }}</title>
will transform to:
<title>Kira Goddess</title>
4. Not helped
In the description of EXTRA_PATH_METADATA variable I read, that Pelican used Python group name notation syntax (?P<name>…). I couldn’t find, how I can make substitutions in Python <class 'str'> (print(type(FILENAME_METADATA)) → <class 'str'>). I tried variants as:
import re
KIRA_DEFAULT_FILENAME_REGEX = '(?P<title>.*)'
FILENAME_METADATA = re.sub(KIRA_DEFAULT_FILENAME_REGEX, "-", " ")
or
KIRA_DEFAULT_FILENAME_REGEX = '(?P<title>.*)'
FILENAME_METADATA = KIRA_DEFAULT_FILENAME_REGEX.replace("-", "")
It doesn’t work.
5. Don’t offer
5.1. Use Jinja2 filters in templates
5.1.1. Suggestion
Use replace() filter in your template files like this:
<title>{{ article.title|replace('-', " ") }}</title>
5.1.2. Why is it not good
Pelican plugins (e.g. Pelican Open Graph) still will use article.title. Unwanted data as Kira-Goddess, not Kira Goddess still will pass to plugins.
5.2. Use spaces in your Markdown
5.2.1. Suggestion
For example, name your file Kira Goddess.md, not Kira-Goddess.md.
5.2.2. Why is it not good
Whitespaces in filenames is a bad practice — 1, 2, 3, 4, 5.

It seems like pelican doesn't provide a way to implement what you want.
FILENAME_METADATA regex can only select a name from a filename, but you can't substitute - with whitespaces there.
So I think the current best way for you is to specify title tag manually in each file.

How do I export images of graphs to a pdf in Jupyter?

I do not wish to export the entire notebook as a pdf - I've searched and found solutions for that problem. I want to only export the plots in my notebook to a pdf. Is there a Python library does allows for this?

The Jupyter nbconvert command allows specifying a custom template.
Michael Goerz has written a full custom template for LaTeX/PDFs here:
https://gist.github.com/goerz/d5019bedacf5956bcf03ca8683dc5217
To only print the graphs, you could modify it to blank out any sections other than the output cells, like so:
% Tell the templating engine what output template we want to use.
((* extends 'article.tplx' *))
% Template will setup imports, etc. as normal unless we override these sections.
% Leave title blank
((* block title -*))
((*- endblock title *))
% Leave author blank
((* block author -*))
((* endblock author *))
% Etc.
((* block maketitle *))
((* endblock maketitle *))
% Don't show "input" prompt
((*- block in_prompt -*))
((*- endblock in_prompt -*))
% Hide input cells
((*- block input -*))
((*- endblock input -*))
% Don't show "output" prompt
((*- block output_prompt -*))
((*- endblock output_prompt -*))
% Let template render output cells as usual
To generate a LaTeX file, save the above as custom_article.tplx and run:
jupyter nbconvert --to=latex --template=custom_article.tplx file.ipynb
To generate the LaTeX file and PDF in a single command:
jupyter nbconvert --to=pdf --template=custom_article.tplx file.ipynb

This is probably not the most elegant answer, but it's also extremely flexible in case you want to do more than just put each plot on a page. You can use LaTeX to collect all graphs in a single pdf after exporting them as images. Here's an example where we save the graphs as report/imgs/*.png, then write a report/report.tex file and compile it with pdflatex into a final report/report.pdf.
import numpy as np
import matplotlib.pyplot as plt
Create and save two images:
plt.bar(np.arange(5), np.arange(5)*2)
plt.savefig('report/imgs/fig1.png')
plt.bar(np.arange(6), np.arange(6)**2 - 1, color = 'g')
plt.savefig('report/imgs/fig2.png')
Write a .tex file to display both images:
img_data = ['fig1', 'fig2']
latex_src = '\\documentclass{article}\n'
latex_src += '\\usepackage{graphicx}\n'
latex_src += '\\graphicspath{{./imgs/}}\n'
latex_src += '\\begin{document}\n'
for image in img_data:
latex_src += '\t\\begin{figure}[h]\n'
latex_src += f'\t\t\\includegraphics{{{image}}}\n'
latex_src += '\t\\end{figure}\n'
latex_src += '\\end{document}'
with open('report/report.tex', 'w', encoding = 'utf-8') as handle:
handle.write(latex_src)
print(latex_src)
\documentclass{article}
\usepackage{graphicx}
\graphicspath{{./imgs/}}
\begin{document}
\begin{figure}[h]
\includegraphics{fig1}
\end{figure}
\begin{figure}[h]
\includegraphics{fig2}
\end{figure}
\end{document}
And finally compile:
!cd report && pdflatex report.tex

Can jupyter notebook web UI be used with custom templates?

I have seen many posts on how to use a custom template when rendering LaTeX from a jupyter notebook (eg hide code in jupyter notebook) and have successfully implemented the answer described here (custom template location) eg:
~/dev/notebooks$ jupyter nbconvert --to=latex filename.ipynb --Application.log_level='DEBUG'
works fine (and applies the custom template specified in my ~/.jupyter/jupyter_nbconvert_config.py).
But how can I set the jupyter web UI to function the same way using:
File->Download as->LaTeX (.tex)
Thanks in advance.

I found the solution jupyter_notebook_config.py. Basically, the jupyter application determines the applied template, which is why it was working fine with:
jupyter nbconvert ...
but was not working with:
File->Download as->LaTeX (.tex)
So, by using this ~/.jupyter/jupyter_notebook_config.py:
import os
c = get_config()
c.TemplateExporter.template_path = [os.path.expanduser('~/.jupyter/templates'), \
'/usr/local/bin/miniconda3/envs/MyEnv/lib/python3.6/site-\
packages/jupyter_contrib_nbextensions/templates', '.']
c.LatexExporter.template_file = 'custom_latex.tplx'
and placing a 'custom_latex.tplx' in the base template directory. I can set a default template (that does nothing) and then each user can override that default template as they choose (this is for a jupyter hub installation). If you don't need this generic behavior (ie you aren't on a jupyter hub), you can ignore the default template portion entirely.
For completeness...
default template:
((= Nbconvert custom style for LaTeX export =))
((*- extends 'article.tplx' -*))
%==============================================================================\
=
% Custom definitions
%==============================================================================\
=
((* block definitions *))
((( super() )))
% Pygments definitions
((( resources.latex.pygments_definitions )))
% Exact colors from NB
\definecolor{incolor}{rgb}{0.0, 0.0, 0.5}
\definecolor{outcolor}{rgb}{0.545, 0.0, 0.0}
% Don't number sections
\renewcommand{\thesection}{\hspace*{-0.5em}}
\renewcommand{\thesubsection}{\hspace*{-0.5em}}
((* endblock definitions *))
% No title
((* block maketitle *))((* endblock maketitle *))
%==============================================================================\
=
% Latex Article
%==============================================================================\
=
% You can customize your LaTeX document here, e.g. you can
% - use a different documentclass like
% \documentclass{report}
% - add/remove packages (like ngerman)
((* block docclass *))
% !TeX spellcheck = de_DE
% !TeX encoding = UTF-8
\documentclass{article}
((* endblock docclass *))
personal template:
((= Nbconvert custom style for LaTeX export =))
((*- extends 'nbextensions.tplx' -*))
%==============================================================================\
=
% Custom definitions
%==============================================================================\
=
((* block definitions *))
((( super() )))
% Pygments definitions
((( resources.latex.pygments_definitions )))
% Exact colors from NB
\definecolor{incolor}{rgb}{0.0, 0.0, 0.5}
\definecolor{outcolor}{rgb}{0.545, 0.0, 0.0}
% Don't number sections
\renewcommand{\thesection}{\hspace*{-0.5em}}
\renewcommand{\thesubsection}{\hspace*{-0.5em}}
((* endblock definitions *))
% No title
((* block maketitle *))((* endblock maketitle *))
%==============================================================================\
=
% Latex Article
%==============================================================================\
=
% You can customize your LaTeX document here, e.g. you can
% - use a different documentclass like
% \documentclass{report}
% - add/remove packages (like ngerman)
((* block docclass *))
% !TeX spellcheck = de_DE
% !TeX encoding = UTF-8
\documentclass{article}
% this is all unnecessary when extending nbextensions.tplx + hide_input_all
\usepackage{verbatim} %for {comment}
\usepackage{fancyvrb}
\renewenvironment{Verbatim}{\comment}{\endcomment}
((* endblock docclass *))
Hope this helps someone else - I don't think the documentation is clear on this point at all.

How to combine two mechanism for localization site written on Pelican?

I use two mechanisms for localization site:
1. I use the standard template tag {{ gettext 'some_text'}} in my index.html
2. I wrote custom jinja extension that takes the content of markdown file according to language that used on the site.
And I use Babel to create messages.pot file and then to create massages.po file.
I have this babel configuration in babel.cfg :
[jinja2: theme/templates/index.html]
silent = false
And this is my custom jinja extension - custom_jinja_extension.py :
from jinja2 import nodes
from jinja2.ext import Extension
from markdown import Markdown
class IncludeMarkdownExtension(Extension):
"""
a set of names that trigger the extension.
"""
tags = set(['include_markdown'])
def __init__(self, environment):
super(IncludeMarkdownExtension, self).__init__(environment)
def parse(self, parser):
tag = parser.stream.__next__()
ctx_ref = nodes.ContextReference()
if tag.value == 'include_markdown':
args = [ctx_ref, parser.parse_expression(), nodes.Const(tag.lineno)]
body = parser.parse_statements(['name:endinclude_markdown'], drop_needle=True)
callback = self.call_method('convert', args)
return nodes.CallBlock(callback, [], [], body).set_lineno(tag.lineno)
def convert(self, context, tagname, linenum, caller):
"""
Function for converting markdown to html
:param tagname: name of converting file
:return: converting html
"""
for item in context['extra_siteurls']:
if item == context['main_lang']:
input_file = open('content/{}/{}'.format('en', tagname))
else:
input_file = open('content/{}/{}'.format(context['main_lang'], tagname))
text = input_file.read()
html = Markdown().convert(text)
return html
I use this template tag - {% include_markdown 'slide3.md' %}{% endinclude_markdown %}
In my pelicanconf.py I add such strings for jinja extensions:
# Jinja2 extensions
JINJA_ENVIRONMENT = {
'extensions': [
'jinja2_markdown.MarkdownExtension',
'jinja2.ext.i18n',
'custom_jinja_extension.IncludeMarkdownExtension'
]
}
When I run the command:
pybabel extract --mapping babel.cfg --output messages.pot ./
I get this error
jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag
'include_markdown'. Jinja was looking for the following tags:
'endblock'. The innermost block that needs to be closed is 'block'.
When I delete all using of custom template tag gettext work well. What I do wrong?

Trouble was in the path. Babel looking jinja extension in virtualenv in jinja folder, but my custom jinja extension was in the project folder.
That's why I run this command in the terminal
export PYTHONPATH=$PYTHONPATH:/local/path/to/the/project/
and change my babel.cfg :
[jinja2: theme/templates/index.html]
extensions=jinja2.ext.i18n, **custom_jinja_extension.IncludeMarkdownExtension**
silent = false
After this changes babel found my custom extension custom_jinja_extension and created messages.pot file correctly!

export notebook to pdf without code [duplicate]

This question already has answers here:
How to hide code from cells in ipython notebook visualized with nbviewer?
(23 answers)
Closed 4 years ago.
I have a large notebook with a lot of figures and text. I want to convert it to a html file. However, I don't want to export the code. I am using the following command
ipython nbconvert --to html notebook.ipynb
But this option also exports the code. Is there a way to convert the notebook to html without the code?

I found this article interesting it explains how to remove the input columns :
you have to create a template file named "hidecode.tplx" in same directory as the notebook you want to convert and add those line in it :
((*- extends 'article.tplx' -*))
((* block input_group *))
((*- if cell.metadata.get('nbconvert', {}).get('show_code', False) -*))
((( super() )))
((*- endif -*))
((* endblock input_group *))
And after run this command it will use pdfLatex to convert the notebook in pdf via latex:
jupyter nbconvert --to pdf --template hidecode Example.ipynb
or if you want to edit you can convert it to a .tex document and use pdfLatex to put it in pdf :
jupyter nbconvert --to latex --template hidecode Example.ipynb
EDIT Sept 2018:
ipython nbconvert is deprecated. It will be replaced by jupyter nbconvert: So we replace the command ipython with jupyter
EDIT Feb 2021: (This is my best answer here, so let me take care of it)
Following #Mrule comment adding --no-input flag will make it work without the template...
jupyter nbconvert --to latex --no-input Example.ipynb
PS: If you are getting issue saying :
LaTeX error related to tcolorbox.sty not found
Please refer to this guide to update your tex installation and this question

I was seeking the same question in SO and finally turned out to a very straightforward way:
Assuming using Firefox(57) + Win7
Run Jupyter notebook and download the notebook in the browser: File->Download as->HTML and you will get a html page with code and output.
Open the exported HTML with browser and activate the browser console with key F12
Run following command in the console:
document.querySelectorAll("div.input").forEach(function(a){a.remove()})
The code removes all input div DOM. Then right mouse button and chose "Save Page As" and Save the "Complete page" (not single page).
You will get a page with an associated folder in windows. Use a trick by zip the html page and then extract to unbind the associated. The folder is useless.
Now it is a single html page without code. You can re-distribute it or print it as PDF.
If you are not using Firefox or Windows, please adjust the above 3-6 steps.

Here is how I do it:
I simply download my notebook as html.
Then run this python script to convert that html file so that prompts and code cells are gone:
FILE = "/somewhere/myHTMLFile.html"
with open(FILE, 'r') as html_file:
content = html_file.read()
# Get rid off prompts and source code
content = content.replace("div.input_area {","div.input_area {\n\tdisplay: none;")
content = content.replace(".prompt {",".prompt {\n\tdisplay: none;")
f = open(FILE, 'w')
f.write(content)
f.close()
That script bascially adds the CSS 'display: none' attribute for all divs of class 'prompt' or 'input_area'.

As a generalization of this answer, so that hidecode template could be accessible from multiple locations:
Go into your home directory:
cd ~/.jupyter
Create jupyter_nbconvert_config.py under this directory.
Write the following into the .py file (Change user_name to your user name):
c = get_config()
c.TemplateExporter.template_path = ['.', "~/.jupyter" ]
c.LatexExporter.template_path = ['.', "~/.jupyter"]
Create a template file under this directory, named hidecode.tplx or hidecode.tpl:
((*- extends 'article.tplx' -*))
((* block input_group *))
((*- if cell.metadata.get('nbconvert', {}).get('show_code', False) -*))
((( super() )))
((*- endif -*))
((* endblock input_group *))
Then, the following should generate a PDF without .ipynb files' codes (Change file_name to your file name):
jupyter nbconvert --to pdf '<file_name>.ipynb' --template=hidecode.tpl

You can add this css to your page and then print into PDF from the browser. Note that the code is only hidden, not removed.
div.output_prompt { visibility: hidden; }
*{ font-family: "Palatino Linotype", "Book Antiqua", Palatino, serif !important }
.input,#header { display: none; }
To make sure you remove the code you should use something like
$(".input").remove()
As far as I know there is no way to generate via latex a PDF that does not include the code. It would be great to have a jupyter notebook extension that does it though...

nbconvert uses templates to convert the notebook's JSON into a document; for example, in the markdown template that comes with nbconvert, markdown.tpl:
{% block input %}
{% if nb.metadata.language_info %}{{ nb.metadata.language_info.name }}{% endif %}
{{ cell.source}}
{% endblock input %}
which prints each cell's source in the output. If you delete {{ cell.source}} then the cell source will not print out. There are corresponding blocks in the HTML and Latex templates as well.
Better yet, make your own template that does exactly what you want :)