We Keep Coding

Printing image in r-markdown using matplotlib and python code

I am trying to run the python code using the R-Markdown file (RMarkdown to pdf).
What I achieved till now -
1- I am able to configure my python engine using knitr and reticulate library
2- I am able to execute my python codes.
What I tried -
1- I tried all the methods which are discussed in this forum, but nothing is working out.
2- I also tried to save the image,(as one of the posts here suggests), but that also is not working.
My problem -
1- When I am trying to plot a graph using matlplotlib and command plt.imshow() and plt.show(), it's not printing the image in the output. Rather it's showing the image in a separate window. You can see my results in the attached image.
Result_of_my_code
Here is my code
```{r setup, include=FALSE}
library(knitr)
library(reticulate)
knitr::knit_engines$set(python = reticulate::eng_python)
```
```{python}
import numpy as np
import os
import torch
import torchvision.datasets as dsets
import matplotlib.pyplot as plt
print(os.getcwd())
os.chdir('D:\\1st year\\Python codes\\CIFR Analysis\\self contained analysis')
print(os.getcwd())
train_mnist = dsets.MNIST("../data", train=True)
test_mnist = dsets.MNIST("../data", train= False)
print(len(train_mnist))
#print(train_mnist[0][0])
plt.imshow(train_mnist[0][0], cmap="gray")
#plt.savefig("trainzero.png")
plt.show()
```
Kindly, help me to fix this issue, as I want to compile my python codes using the R markdown file.
thanks

So with R Markdown, you have to do some things a little differently. In the following, I have a dataframe with two series created by concatenating them. The original plotting code in the Jupyter Notebook is as follows and just printed out the series.
# make a plot of model fit
train.plot(figsize=(16,8), legend=True)
backtest.plot(legend=True);
However, it does not work with way with R Markdown. Then with plotting, you always have to assign them, and with the code below, you get the same plot.
dfreg = pd.concat([reg, backtest], axis = 1)
ax = dfreg.plot(figsize=(16,8), legend = True)
ax1 = predictions.plot(legend=True)
plt.show()
This is common with other plotting functions like plot_acf() too.

Line Graphs in Anaconda

I am writing a Python program to generate line graphs of cryptocurrency prices.
The goal is to be able to overlay multiple arbitrary pairs, i.e., BTC/USD + ETH/BTC, or BTC/USD + BCH/LTC + XRP/BTG, across the same horizontal timeline, similar to https://coinmarketcap.com/currencies/ethereum/
The bells and whistles in the above example are not required. It doesn't need click-to-zoom technology, the log-scale button, or fancy tool tips, if these features increase the implementation difficulty. Separate colors for each line would be helpful. This is the format of the data I have to work with:
https://min-api.cryptocompare.com/data/histominute?fsym=BTC&tsym=USD&limit=60&aggregate=1&e=Coinbase
https://min-api.cryptocompare.com/data/histominute?fsym=ETH&tsym=BCH&limit=30&aggregate=1&e=CCCAGG
Currently, I have Anaconda 3.6 on Visual Studio, which according to https://docs.anaconda.com/anaconda/packages/py3.6_win-64.html should include multiple packages to plot this data without needing to manually install third party code through a command line. However, when I try to import any of them (i.e., matplotlib, bokeh, seaborn), I receive a "ModuleNotFoundError" message, so I'm not sure if my Anaconda is functioning properly. What is the easiest way to graph this data with Anaconda?

The python interpreter in VSCode is probably not using your Anaconda installation. In the VSCode python shell, type import sys and then sys.version and it should tell you the version of python you are using.
Here is some code to get you started with doing this in bokeh, a library that comes with Anaconda. Instead of using Visual Studio (or Visual Studio Code--I'm not sure which one you are referring to), I used jupyter notebook, and some of the import here are specific to that environment (to show the bokeh graph inline). You may want to format the date differently.
from bokeh.plotting import figure, output_file, show
from bokeh.io import output_notebook
import numpy as np
output_notebook()
import requests
import datetime
from math import pi
def format_date(utc_time):
time = datetime.datetime.fromtimestamp(int(utc_time))
return time
url1 = "https://min-api.cryptocompare.com/data/histominute?fsym=BTC&tsym=USD&limit=60&aggregate=1&e=Coinbase"
url2 = "https://min-api.cryptocompare.com/data/histominute?fsym=ETH&tsym=BCH&limit=30&aggregate=1&e=CCCAGG"
r1 = requests.get(url1)
r2 = requests.get(url2)
r1_source = r1.json()["Data"]
r2_source = r2.json()["Data"]
r1_data = [i["close"] for i in r1_source]
r1_time = [format_date(i["time"]) for i in r1_source]
# r2_data = [i["close"] for i in r2_source]
# r2_time = [i["time"] for i in r2_source]
p = figure(plot_width=800, plot_height=400)
p.line(r1_time,r1_data, line_width=2)
# p.line(r2_time, r2_data, line_width=2, line_color="red")
p.xaxis.major_label_orientation = pi/4

Python ipyleaflet export map as PNG or JPG or SVG

I have tried to export a visualisation of data with ipyleaflet as PNG or any other file format but i could not find a method that is working. For example in folium there is map.save(path). Is there a library or method in ipyleaflet that i have missed in my research which helps me to accomplish my goal?
here is some example code to generate a map
from ipyleaflet import *
center = [34.6252978589571, -77.34580993652344]
zoom = 10
m = Map(default_tiles=TileLayer(opacity=1.0), center=center, zoom=zoom)
m
I'd like to export this map as an image file without taking a screenshot manually.
I found two sources that allow to export javascript leaflet maps:
https://github.com/aratcliffe/Leaflet.print and https://github.com/mapbox/leaflet-image
Unfortunately i was not able to make use of them in python.

My colleague and I found a decent work around for ipyleaflet (python) image export. Here is how it works. The folium library is required for an export. The GeoJson data in this example is already prepared with style properties:
import folium
map = folium.Map([51., 12.], zoom_start=6,control_scale=True)
folium.GeoJson(data).add_to(map)
map.save('map.html')
This is how the result looks:
The html file can be further processed in python (windows) with subprocess calls to make a PDF or PNG out of it. I hope this helps as the ipyleaflet doc for python is almost non existant.

For generating html, you can use ipywidgets
from ipywidgets.embed import embed_minimal_html
embed_minimal_html('map.html', views=[m])
If you want to make a PNG, you can use ipywebrtc, more specifically:
https://ipywebrtc.readthedocs.io/en/latest/ImageRecorder.html
https://ipywebrtc.readthedocs.io/en/latest/WidgetStream.html
Or in code:
from ipywebrtc import WidgetStream, ImageRecorder
widget_stream = WidgetStream(widget=m, max_fps=1)
image_recorder = ImageRecorder(stream=widget_stream)
display(image_recorder)
Saving the PNG:
with open('map.png', 'wb') as f:
f.write(image_recorder.image.value)
Or converting to pillow image for preprocessing:
import PIL.Image
import io
im = PIL.Image.open(io.BytesIO(image_recorder.image.value))

ipyleaflet supports saving as html. Export of svg and png does not seem to be supported.
https://ipyleaflet.readthedocs.io/en/latest/map_and_basemaps/map.html#save-to-html
m.save('output.html')
I created an issue ticket for ipyleaflet:
https://github.com/jupyter-widgets/ipyleaflet/issues/1083

Convert SVG/PDF to EMF

I am looking for a way to save a matplotlib figure as an EMF file. Matplotlib allows me to save as either a PDF or SVG vector file but not as EMF.
After a long search I still cannot seem to find a way to do this with python. Hopefully anyone has an idea.
My workaround is to call inkscape using subprocess but this is far from ideal as I would like to avoid the use of external programs.
I'm running python 2.7.5 and matplotlib 1.3.0 using the wx backend.

For anyone who still needs this, I wrote a basic function that can let you save a file as an emf from matplotlib, as long as you have inkscape installed.
I know the op didn't want inkscape, but people who find this post later just want to make it work.
import matplotlib.pyplot as plt
import subprocess
import os
inkscapePath = r"path\to\inkscape.exe"
savePath= r"path\to\images\folder"
def exportEmf(savePath, plotName, fig=None, keepSVG=False):
"""Save a figure as an emf file
Parameters
----------
savePath : str, the path to the directory you want the image saved in
plotName : str, the name of the image
fig : matplotlib figure, (optional, default uses gca)
keepSVG : bool, whether to keep the interim svg file
"""
figFolder = savePath + r"\{}.{}"
svgFile = figFolder.format(plotName,"svg")
emfFile = figFolder.format(plotName,"emf")
if fig:
use=fig
else:
use=plt
use.savefig(svgFile)
subprocess.run([inkscapePath, svgFile, '-M', emfFile])
if not keepSVG:
os.system('del "{}"'.format(svgFile))
#Example Usage
import numpy as np
tt = np.linspace(0, 2*3.14159)
plt.plot(tt, np.sin(tt))
exportEmf(r"C:\Users\userName", 'FileName')

I think the function is cool but inkscape syntax seems not working in my case. I search in other post and find it as:
inkscape filename.svg --export-filename filename.emf
So if I replace -M by --export-filename within the subprocess argument, everything works fine.

Save an "interactive figure" with matplotlib [duplicate]

Is there a way to save a Matplotlib figure such that it can be re-opened and have typical interaction restored? (Like the .fig format in MATLAB?)
I find myself running the same scripts many times to generate these interactive figures. Or I'm sending my colleagues multiple static PNG files to show different aspects of a plot. I'd rather send the figure object and have them interact with it themselves.

I just found out how to do this. The "experimental pickle support" mentioned by #pelson works quite well.
Try this:
# Plot something
import matplotlib.pyplot as plt
fig,ax = plt.subplots()
ax.plot([1,2,3],[10,-10,30])
After your interactive tweaking, save the figure object as a binary file:
import pickle
pickle.dump(fig, open('FigureObject.fig.pickle', 'wb')) # This is for Python 3 - py2 may need `file` instead of `open`
Later, open the figure and the tweaks should be saved and GUI interactivity should be present:
import pickle
figx = pickle.load(open('FigureObject.fig.pickle', 'rb'))
figx.show() # Show the figure, edit it, etc.!
You can even extract the data from the plots:
data = figx.axes[0].lines[0].get_data()
(It works for lines, pcolor & imshow - pcolormesh works with some tricks to reconstruct the flattened data.)
I got the excellent tip from Saving Matplotlib Figures Using Pickle.

As of Matplotlib 1.2, we now have experimental pickle support. Give that a go and see if it works well for your case. If you have any issues, please let us know on the Matplotlib mailing list or by opening an issue on github.com/matplotlib/matplotlib.

This would be a great feature, but AFAIK it isn't implemented in Matplotlib and likely would be difficult to implement yourself due to the way figures are stored.
I'd suggest either (a) separate processing the data from generating the figure (which saves data with a unique name) and write a figure generating script (loading a specified file of the saved data) and editing as you see fit or (b) save as PDF/SVG/PostScript format and edit in some fancy figure editor like Adobe Illustrator (or Inkscape).
EDIT post Fall 2012: As others pointed out below (though mentioning here as this is the accepted answer), Matplotlib since version 1.2 allowed you to pickle figures. As the release notes state, it is an experimental feature and does not support saving a figure in one matplotlib version and opening in another. It's also generally unsecure to restore a pickle from an untrusted source.
For sharing/later editing plots (that require significant data processing first and may need to be tweaked months later say during peer review for a scientific publication), I still recommend the workflow of (1) have a data processing script that before generating a plot saves the processed data (that goes into your plot) into a file, and (2) have a separate plot generation script (that you adjust as necessary) to recreate the plot. This way for each plot you can quickly run a script and re-generate it (and quickly copy over your plot settings with new data). That said, pickling a figure could be convenient for short term/interactive/exploratory data analysis.

Why not just send the Python script? MATLAB's .fig files require the recipient to have MATLAB to display them, so that's about equivalent to sending a Python script that requires Matplotlib to display.
Alternatively (disclaimer: I haven't tried this yet), you could try pickling the figure:
import pickle
output = open('interactive figure.pickle', 'wb')
pickle.dump(gcf(), output)
output.close()

Good question. Here is the doc text from pylab.save:
pylab no longer provides a save function, though the old pylab
function is still available as matplotlib.mlab.save (you can still
refer to it in pylab as "mlab.save"). However, for plain text
files, we recommend numpy.savetxt. For saving numpy arrays,
we recommend numpy.save, and its analog numpy.load, which are
available in pylab as np.save and np.load.

I figured out a relatively simple way (yet slightly unconventional) to save my matplotlib figures. It works like this:
import libscript
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2*np.pi*t)
#<plot>
plt.plot(t, s)
plt.xlabel('time (s)')
plt.ylabel('voltage (mV)')
plt.title('About as simple as it gets, folks')
plt.grid(True)
plt.show()
#</plot>
save_plot(fileName='plot_01.py',obj=sys.argv[0],sel='plot',ctx=libscript.get_ctx(ctx_global=globals(),ctx_local=locals()))
with function save_plot defined like this (simple version to understand the logic):
def save_plot(fileName='',obj=None,sel='',ctx={}):
"""
Save of matplolib plot to a stand alone python script containing all the data and configuration instructions to regenerate the interactive matplotlib figure.
Parameters
----------
fileName : [string] Path of the python script file to be created.
obj : [object] Function or python object containing the lines of code to create and configure the plot to be saved.
sel : [string] Name of the tag enclosing the lines of code to create and configure the plot to be saved.
ctx : [dict] Dictionary containing the execution context. Values for variables not defined in the lines of code for the plot will be fetched from the context.
Returns
-------
Return ``'done'`` once the plot has been saved to a python script file. This file contains all the input data and configuration to re-create the original interactive matplotlib figure.
"""
import os
import libscript
N_indent=4
src=libscript.get_src(obj=obj,sel=sel)
src=libscript.prepend_ctx(src=src,ctx=ctx,debug=False)
src='\n'.join([' '*N_indent+line for line in src.split('\n')])
if(os.path.isfile(fileName)): os.remove(fileName)
with open(fileName,'w') as f:
f.write('import sys\n')
f.write('sys.dont_write_bytecode=True\n')
f.write('def main():\n')
f.write(src+'\n')
f.write('if(__name__=="__main__"):\n')
f.write(' '*N_indent+'main()\n')
return 'done'
or defining function save_plot like this (better version using zip compression to produce lighter figure files):
def save_plot(fileName='',obj=None,sel='',ctx={}):
import os
import json
import zlib
import base64
import libscript
N_indent=4
level=9#0 to 9, default: 6
src=libscript.get_src(obj=obj,sel=sel)
obj=libscript.load_obj(src=src,ctx=ctx,debug=False)
bin=base64.b64encode(zlib.compress(json.dumps(obj),level))
if(os.path.isfile(fileName)): os.remove(fileName)
with open(fileName,'w') as f:
f.write('import sys\n')
f.write('sys.dont_write_bytecode=True\n')
f.write('def main():\n')
f.write(' '*N_indent+'import base64\n')
f.write(' '*N_indent+'import zlib\n')
f.write(' '*N_indent+'import json\n')
f.write(' '*N_indent+'import libscript\n')
f.write(' '*N_indent+'bin="'+str(bin)+'"\n')
f.write(' '*N_indent+'obj=json.loads(zlib.decompress(base64.b64decode(bin)))\n')
f.write(' '*N_indent+'libscript.exec_obj(obj=obj,tempfile=False)\n')
f.write('if(__name__=="__main__"):\n')
f.write(' '*N_indent+'main()\n')
return 'done'
This makes use a module libscript of my own, which mostly relies on modules inspect and ast. I can try to share it on Github if interest is expressed (it would first require some cleanup and me to get started with Github).
The idea behind this save_plot function and libscript module is to fetch the python instructions that create the figure (using module inspect), analyze them (using module ast) to extract all variables, functions and modules import it relies on, extract these from the execution context and serialize them as python instructions (code for variables will be like t=[0.0,2.0,0.01] ... and code for modules will be like import matplotlib.pyplot as plt ...) prepended to the figure instructions. The resulting python instructions are saved as a python script whose execution will re-build the original matplotlib figure.
As you can imagine, this works well for most (if not all) matplotlib figures.

If you are looking to save python plots as an interactive figure to modify and share with others like MATLAB .fig file then you can try to use the following code. Here z_data.values is just a numpy ndarray and so you can use the same code to plot and save your own data. No need of using pandas then.
The file generated here can be opened and interactively modified by anyone with or without python just by clicking on it and opening in browsers like Chrome/Firefox/Edge etc.
import plotly.graph_objects as go
import pandas as pd
z_data=pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/api_docs/mt_bruno_elevation.csv')
fig = go.Figure(data=[go.Surface(z=z_data.values)])
fig.update_layout(title='Mt Bruno Elevation', autosize=False,
width=500, height=500,
margin=dict(l=65, r=50, b=65, t=90))
fig.show()
fig.write_html("testfile.html")