We Keep Coding

Cross-reference a python code chunk in Rmarkdown with reticulate

I have a Rmarkdown document containing some python code chunks using the reticulate library. The code executes output perfectly; however, how would I cross-reference the generated plots in the text using its label? I am using bookdown::pdf_documents2 etc, and have no issue with inline reference of R chunks using the standard \#ref(fig:my-plot).
An MWE would be:
```{python, my-plot}
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(10)
y = np.arange(10)
plt.figure()
plt.plot(x, y)
plt.show()
```
In Figure \#ref(fig:my-plot)....
I have tried without prepending the chunk type (fig, etc.)
I understand I can save the image to file and include it with knitr as a subsequent R chunk, but it would be preferable to do it through the python chunk alone.

I must have been making an error when previously attempting fig.cap within the chunk, as Daniel correctly suggests in the question comments. Attempting with this again, I can cross-reference perfectly with the python chunk across all HTML, PDF and word outputs. The updated MWE is:
```{python, my-plot, fig.cap="My Caption"}
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(10)
y = np.arange(10)
plt.figure()
plt.plot(x, y)
plt.show()
```
In Figure \#ref(fig:my-plot)...
Each of the outputs in the YAML header required the varying bookdown:: options. For PDF (bookdown::pdf_document2), for word (bookdown::word_document2) and Rmdformats in my case, the option of use_bookdone: true.

Writing create_tree_digraph plot to a png file in Python

I want the tree of my lightgbm model to save to a .png format. I have tried two plotting methods from lightgbm API - plot_tree and create_tree_diagraph.
import lightgbm as lgb
from sklearn.datasets import load_iris
X, y = load_iris(True)
clf = lgb.LGBMClassifier()
clf.fit(X, y)
When I use plot_tree, it displays the tree but in place of values there are small blank boxes
lgb.plot_tree(clf, tree_index=0)
When I try the create_tree_diagraph, I get the graph but I cant save it as it is.
lgb.create_tree_digraph(clf)
I used the below code to save it a file but that gets saved as the first plot (using plot_tree)
import graphviz
s = graphviz.Source(graph_b.source, filename = "test1.gv", format = "png")
s.view()
Any suggestions to save the plot as an image. I ultimately want to write these tree plots to excel.
I am using graphviz version 0.8.3
Thanks,

I suppose you are using a jupyter notebook or something like that.
This worked for me, but surely is not the best way.
ax = lgb.create_tree_digraph(clf)
with open('fst.svg', 'w') as f:
f.write(ax._repr_svg_())

Printing image in r-markdown using matplotlib and python code

I am trying to run the python code using the R-Markdown file (RMarkdown to pdf).
What I achieved till now -
1- I am able to configure my python engine using knitr and reticulate library
2- I am able to execute my python codes.
What I tried -
1- I tried all the methods which are discussed in this forum, but nothing is working out.
2- I also tried to save the image,(as one of the posts here suggests), but that also is not working.
My problem -
1- When I am trying to plot a graph using matlplotlib and command plt.imshow() and plt.show(), it's not printing the image in the output. Rather it's showing the image in a separate window. You can see my results in the attached image.
Result_of_my_code
Here is my code
```{r setup, include=FALSE}
library(knitr)
library(reticulate)
knitr::knit_engines$set(python = reticulate::eng_python)
```
```{python}
import numpy as np
import os
import torch
import torchvision.datasets as dsets
import matplotlib.pyplot as plt
print(os.getcwd())
os.chdir('D:\\1st year\\Python codes\\CIFR Analysis\\self contained analysis')
print(os.getcwd())
train_mnist = dsets.MNIST("../data", train=True)
test_mnist = dsets.MNIST("../data", train= False)
print(len(train_mnist))
#print(train_mnist[0][0])
plt.imshow(train_mnist[0][0], cmap="gray")
#plt.savefig("trainzero.png")
plt.show()
```
Kindly, help me to fix this issue, as I want to compile my python codes using the R markdown file.
thanks

So with R Markdown, you have to do some things a little differently. In the following, I have a dataframe with two series created by concatenating them. The original plotting code in the Jupyter Notebook is as follows and just printed out the series.
# make a plot of model fit
train.plot(figsize=(16,8), legend=True)
backtest.plot(legend=True);
However, it does not work with way with R Markdown. Then with plotting, you always have to assign them, and with the code below, you get the same plot.
dfreg = pd.concat([reg, backtest], axis = 1)
ax = dfreg.plot(figsize=(16,8), legend = True)
ax1 = predictions.plot(legend=True)
plt.show()
This is common with other plotting functions like plot_acf() too.

Any example/idea of how to plot the U-MATRIX using python (working on SOM)?

Working on SOM implementation using Python ? I want to know how to generate the u-matrix

One implementation can be found in the SUSI package (pip3 install susi). You can use it like that:
import susi
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
# get data (replace this part with your data)
X, y = make_blobs(n_samples=100, n_features=2, centers=3)
# initialize and fit SOM
som = susi.SOMClustering()
som.fit(X)
u_matrix = som.get_u_matrix()
plt.imshow(np.squeeze(u_matrix), cmap="Greys")
plt.colorbar()
plt.show()
which results in this plot:
The code and the plot are taken from susi/SOMClustering.ipynb. You find the implementation of the u-matrix there as well.

Strange plot by using sklearn.linear_model

I typically use MATLAB, but want to push myself to learn something about Python. I tried a code of linear regression that introduced by a youtuber. Here is the code:
import pandas as pd
from sklearn import linear_model
import matplotlib.pyplot as plt
#read data
dataframe = pd.read_fwf('brain_body.txt')
x_values = dataframe[['Brain']]
y_values = dataframe[['Body']]
#train model on data
body_reg = linear_model.LinearRegression()
body_reg.fit(x_values,y_values)
#visualize results
plt.scatter(x_values,y_values)
plt.plot(x_values,body_reg.predict(x_values))
plt.show()
But I ended up with a very strange plot (I use Python 3.6):
1
here is part of details:
2
Apparently, something is missing or wrong.
The data of brain_body.txt can be found in https://github.com/llSourcell/linear_regression_demo/blob/master/brain_body.txt
Any suggestion or advice is welcome.
Update
I tried sera's code, and here is what I get:
3
It's funny and weird. it occurred to me that something is wrong with my data file, or something missing in my Python, but I just copied and pasted the raw data into the notepad and saved as .txt; I tried Python 3.6 and 2.7 as well as Pycharm and Spyder...so I have no idea...
BTW, the youtube video is here
#sascha #Moritz #sera I asked my friend to run the same code and data file, and everything is fine. In other words, there is something wrong with my Python and I don't know why. Let me try another computer and/or try an earlier version of python.
I tried, but nothing changed. Here are two different approaches I used to install Python:
1. Install Python (e.g. ver. 3.6); install Pycharm; install packages Pandas, scikit-learn...
2. Install Anaconda
Solved
Thanks for #Marc Bataillou 's suggestion. This is a problem associated with different versions of matplotlib. The problem was found in version 2.1.0. I tried 2.0.2 and found that the original code works fine in the older version; apparently, some changes are made from 2.0.2 to 2.1.0. Thanks for all your efforts.

You should use
plt.scatter(x_values.values,y_values.values)
instead of
plt.scatter(x_values,y_values)
I hope it works !

You can visualize the results using the following code. I use cross validation for the predictions. If the model was perfect, then all the dots would be on the plotted line.
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import cross_val_predict
from sklearn import linear_model
#read data
dataframe = pd.read_fwf('brain_body.txt')
x_values = dataframe[['Brain']]
y_values = dataframe[['Body']]
#model on data
body_reg = linear_model.LinearRegression()
# cross_val_predict returns an array of the same size as `y` where each entry
# is a prediction obtained by cross validation:
predicted = cross_val_predict(body_reg, x_values, y_values, cv=10)
fig, ax = plt.subplots()
ax.scatter(y_values, predicted, edgecolors=(0, 0, 0))
ax.plot([y_values.min(), y_values.max()], [y_values.min(), y_values.max()], 'k--', lw=4)
ax.set_xlabel('Measured')
ax.set_ylabel('Predicted')
plt.show()
Results:
Data
https://ufile.io/p7x0r