Generate python code for execution (auto correct """) - python

I write a python code generator
as an input it has a source code: source
Part of the output I need to generate is execute(source_code)
When source_code is a string representing source .
If I write "execute({0})".format(source) for input source = "import sys"
i'll get execute(import sys).
So I tried : execute(\"\"\"{0}\"\"\")format(source). Is it ok? I tried to test it...Sometimes it is ok....The problem occurs when inside the source there are"""
For example:
from IPython.display import HTML
HTML("""
<script>
console.log("hello");
</script>
<b>HTML</b>
""")
my code turns to be
execute("""from IPython.display import HTML
HTML("""
<script>
console.log("hello");
</script>
<b>HTML</b>
""")""")
UPD:
Changing the code to
execute('''{0}''').format(source)
doesn`t solve the problem, the issue will be encountered with :
def tojson(data):
'''Shorten the code to respond a little bit.'''
print(json.dumps(data))


Using single triple quotes should help:
execute('''from IPython.display import HTML
HTML("""
<script>
console.log("hello");
</script>
<b>HTML</b>
""")''')
Running in a Notebook, you need to use eval() to actually display the HTML:
exec('''from IPython.display import HTML''')
eval('''HTML("""
<script>
console.log("hello");
</script>
<b>HTML</b>
""")''')
In your case probably:
execute('''{0}''').format(source)
Works also if there are ''' inside the string:
source = """
def add(a, b):
'''Add'''
return a + b
print(add(1, 2))
"""
exec('''{0}'''.format(source))
Output:
3

Related

Python - Scrape Javascript using bs4 and print out the value

So I have been trying to create a script where there is a countdown with epoch time which I later gonna convert it.
The html is following:
<script type="text/javascript">
new Countdown('countdown_timer', '1547161260', 'https://stackoverflow.com/');
</script>
and I started to scrape it which I managed to scrape using:
try:
time_countdown_tag = bs4.find_all('script', {'type': 'text/javascript'})
except Exception:
time_countdown_tag = []
for countdown in time_countdown_tag:
if 'new Countdown' in countdown.text.strip():
print(countdown)
which my output is:
<script type="text/javascript">
new Countdown('countdown_timer', '1547161260', 'https://stackoverflow.com/');
</script>
However what I want to print out in this case is the number inside the params which is 1547161260 - I would appreciate all kind of help to be able to be able to only print out the number (epoch) if it is possible?

You can use regular expressions to match the portion of the JS that contains a positive integer:
import re
output = """<script type="text/javascript">
new Countdown('countdown_timer', '1547161260', 'https://stackoverflow.com/');
</script>"""
re.findall("\d+", output)

Error in converting from markdown to editor in flask

I am using markdown library in python to display some markdown in my flask app.
I am getting an error during display of the output as it is showing the markdown content without converting it to HTML.
This is my python code.
import markdown
from flask import Flask
#import some other libraries
#app.route('/md')
def md():
content = """
<h1>Hello</h1>
Chapter
=======
Section
-------
* Item 1
* Item 2
**Ishaan**
"""
content = Markup(markdown.markdown(content))
return render_template('md.html', **locals())
This is my html code.
<html>
<head>
<title>Markdown Snippet</title>
</head>
<body>
{{ content }}
</body>
</html>
I am following the code from here
I know I am doing some blunder but I'll be thankful if anybody help me.
Thanks in advance.

Dedent your lines of Markdown.
Anything inside Python triple-quotes is interpreted by Python literally. That includes the indentation. Therefore, the text passed to Markdown is indented by one level causing Markdown to interpret the entire document as a code block. Remove the indentation and Markdown will recognize the text properly:
#app.route('/md')
def md():
content = """
<h1>Hello</h1>
Chapter
=======
Section
-------
* Item 1
* Item 2
**Ishaan**
"""
Note that the example you are copying from also does not indent the triple-quoted text. Of course, that makes your Python code less readable. Therefore, the Python Standard Library includes the textwrap.dedent() function, which will remove the indentation programmatically:
from textwrap import dedent
#app.route('/md')
def md():
content = """
<h1>Hello</h1>
Chapter
=======
Section
-------
* Item 1
* Item 2
**Ishaan**
"""
content = Markup(markdown.markdown(dedent(content))) # <= dedent here
Note that content is passed through dedent before being passed to Markdown.

save plotly plot to local file and insert into html

I am using python and plotly to product interactive html report. This post gives a nice framework.
If I produce the plot(via plotly) online, and insert the url into the html file, it works but refreshing the charts takes a long time. I wonder if I could produce the chart offline and have it embedded in the html report, so that loading speed is not a problem.
I find plot offline would generate a html for the chart, but I don't know how to embed it in another html. Anyone could help?

There is a better alternative as of right now, that is to do offline plotting into a div, rather than a full html.
This solution does not involve any hacks.
If you call:
plotly.offline.plot(data, filename='file.html')
It creates a file named file.html and opens it up in your web browser. However, if you do:
plotly.offline.plot(data, include_plotlyjs=False, output_type='div')
the call will return a string with only the div required to create the data, for example:
<div id="82072c0d-ba8d-4e86-b000-0892be065ca8" style="height: 100%; width: 100%;" class="plotly-graph-div"></div>
<script type="text/javascript">window.PLOTLYENV=window.PLOTLYENV || {};window.PLOTLYENV.BASE_URL="https://plot.ly";Plotly.newPlot("82072c0d-ba8d-4e86-b000-0892be065ca8",
[{"y": ..bunch of data..., "x": ..lots of data.., {"showlegend": true, "title": "the title", "xaxis": {"zeroline": true, "showline": true},
"yaxis": {"zeroline": true, "showline": true, "range": [0, 22.63852380952382]}}, {"linkText": "Export to plot.ly", "showLink": true})</script>
Notice how its just a tiny portion of an html that you are supposed to embed in a bigger page. For that I use a standard template engine like Jinga2.
With this you can create one html page with several charts arranged the way you want, and even return it as a server response to an ajax call, pretty sweet.
Update:
Remember that you'll need to include the plotly js file for all these charts to work.
You could include
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
just before putting the div you got. If you put this js at the bottom of the page, the charts won't work.

Option 1: Use plotly's offline functionality in your Jupyter Notebook (I suppose you are using a Jupyter Notebook from the link you are providing). You can simply save the whole notebook as a HTML file. When I do this, the only external reference is to JQuery; plotly.js will be inlined in the HTML source.
Option 2: The best way is probably to code directly against plotly's JavaScript library. Documentation for this can be found here: https://plot.ly/javascript/
Update: Calling an internal function has never been a good idea. I recommend to use the approach given by #Fermin Silva. In newer versions, there now is also a dedicated function for this: plotly.io.to_html (see https://plotly.com/python-api-reference/generated/plotly.io.to_html.html)
Hacky Option 3 (original version for reference only): If you really want to continue using Python, you can use some hack to extract the HTML it generates. You need some recent version of plotly (I tested it with plotly.__version__ == '1.9.6'). Now, you can use an internal function to get the generated HTML:
from plotly.offline.offline import _plot_html
data_or_figure = [{"x": [1, 2, 3], "y": [3, 1, 6]}]
plot_html, plotdivid, width, height = _plot_html(
data_or_figure, False, "", True, '100%', 525)
print(plot_html)
You can simply paste the output somewhere in the body of your HTML document. Just make sure that you include a reference to plotly in the head:
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
Alternatively, you can also reference the exact plotly version you used to generate the HTML or inline the JavaScript source (which removes any external dependencies; be aware of the legal aspects however).
You end up with some HTML code like this:
<html>
<head>
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
</head>
<body>
<!-- Output from the Python script above: -->
<div id="7979e646-13e6-4f44-8d32-d8effc3816df" style="height: 525; width: 100%;" class="plotly-graph-div"></div><script type="text/javascript">window.PLOTLYENV=window.PLOTLYENV || {};window.PLOTLYENV.BASE_URL="https://plot.ly";Plotly.newPlot("7979e646-13e6-4f44-8d32-d8effc3816df", [{"x": [1, 2, 3], "y": [3, 1, 6]}], {}, {"showLink": false, "linkText": ""})</script>
</body>
</html>
Note: The underscore at the beginning of the function's name suggests that _plot_html is not meant to be called from external code. So it is likely that this code will break with future versions of plotly.

In addition to the other answers, another possible solution is to use to_json() on the plotly figure.
No need for custom JSON serializer or use of internal solutions.
import plotly
# create a simple plot
bar = plotly.graph_objs.Bar(x=['giraffes', 'orangutans', 'monkeys'],
y=[20, 14, 23])
layout = plotly.graph_objs.Layout()
fig = plotly.graph_objs.Figure([bar], layout)
# convert it to JSON
fig_json = fig.to_json()
# a simple HTML template
template = """<html>
<head>
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
</head>
<body>
<div id='divPlotly'></div>
<script>
var plotly_data = {}
Plotly.react('divPlotly', plotly_data.data, plotly_data.layout);
</script>
</body>
</html>"""
# write the JSON to the HTML template
with open('new_plot.html', 'w') as f:
f.write(template.format(fig_json))

Thought the answer needs an update based on recent version. (plotly==4.9.0)
fig.write_html("path/to/file.html")
reference: plotly documentation

I wasn't able to get any of these solutions to work. My goal was to generate plots in one notebook and publish them in another, so persisting the plot HTML wasn't as important for me as simply having some method for serializing the plot to disk to be rebuilt somewhere else.
The solution I came up with is to serialize the fig object to JSON, and then use plotly's "json chart schema" to build the plot from JSON. This demo is all python, but it should be trivial to build a plot in HTML using this JSON-serialization strategy and invoking the plotly javascript library directly, if that's what you need.
import numpy as np
import json
from plotly.utils import PlotlyJSONEncoder
from plotly.offline import download_plotlyjs, init_notebook_mode, iplot
import plotly.graph_objs as go
init_notebook_mode()
def plotlyfig2json(fig, fpath=None):
"""
Serialize a plotly figure object to JSON so it can be persisted to disk.
Figures persisted as JSON can be rebuilt using the plotly JSON chart API:
http://help.plot.ly/json-chart-schema/
If `fpath` is provided, JSON is written to file.
Modified from https://github.com/nteract/nteract/issues/1229
"""
redata = json.loads(json.dumps(fig.data, cls=PlotlyJSONEncoder))
relayout = json.loads(json.dumps(fig.layout, cls=PlotlyJSONEncoder))
fig_json=json.dumps({'data': redata,'layout': relayout})
if fpath:
with open(fpath, 'wb') as f:
f.write(fig_json)
else:
return fig_json
def plotlyfromjson(fpath):
"""Render a plotly figure from a json file"""
with open(fpath, 'r') as f:
v = json.loads(f.read())
fig = go.Figure(data=v['data'], layout=v['layout'])
iplot(fig, show_link=False)
## Minimial demo ##
n = 1000
trace = go.Scatter(
x = np.random.randn(n),
y = np.random.randn(n),
mode = 'markers')
fig = go.Figure(data=[trace])
#iplot(fig)
plotlyfig2json(fig, 'myfile.json')
plotlyfromjson('myfile.json')
EDIT: Per discussion on the associated github issue, this is probably the current best approach.

Easiest way is to use pio from plotly
import plotly.io as pio
pio.write_html(fig, file='Name.html', auto_open=True)
I have used it a ton of lots to store my graphs before important meetings

You can also use an iframe according to an answer to the question how to embed html into ipython output?
plotly.offline.plot(fig, filename='figure.html',validate=False)
from IPython.display import IFrame
IFrame(src='./figure.html', width=1000, height=600)

To improvise the below code you could just call, to_plotly_json() for ex:,
def plotlyfig2json(fig, fpath=None):
"""
Serialize a plotly figure object to JSON so it can be persisted to disk.
Figures persisted as JSON can be rebuilt using the plotly JSON chart API:
http://help.plot.ly/json-chart-schema/
If `fpath` is provided, JSON is written to file.
Modified from https://github.com/nteract/nteract/issues/1229
"""
redata = json.loads(json.dumps(fig.data, cls=PlotlyJSONEncoder))
relayout = json.loads(json.dumps(fig.layout, cls=PlotlyJSONEncoder))
fig_json=json.dumps({'data': redata,'layout': relayout})
if fpath:
with open(fpath, 'wb') as f:
f.write(fig_json)
else:
return fig_json
--------------------------------------------
Simple way:
fig = go.Figure(data=['data'], layout=['layout'])
fig.to_plotly_json()

In Plotly 5 (and likely in 4 as well), BaseFigure.to_html has include_plotlyjs argument that is True by default.
Specifies how the plotly.js library is included/loaded in the output div string.
If True, a script tag containing the plotly.js source code (~3MB) is included in the output. HTML files generated with this option are fully self-contained and can be used offline.
Funnily enough Plotly figure HTML embedding can be illustrated with Pyodide right here.
const plotlyScript = `
import plotly.express as px
fig = px.bar(y=[0, 1, 1, 2, 3, 5, 8, 13, 21, 34])
fig.to_html(full_html=False)
`
async function main() {
const pyodideUrl = 'https://cdn.jsdelivr.net/pyodide/v0.17.0/full/'
const pyodide = await loadPyodide({'indexURL': pyodideUrl})
await pyodide.loadPackage(['pandas']) // needed for plotly
const html = await pyodide.runPythonAsync(plotlyScript)
const fragment = document.createRange().createContextualFragment(html)
const output = document.getElementById('plotly-output')
output.textContent = ''
output.append(fragment)
}
main()
<script type="text/javascript" src="https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.js"></script>
<div id="plotly-output">Loading pyodide... It may take a minute.</div>

to embed it in another page just use
<iframe width="560" height="315" title="Plot1" src="./scatter1.html"></iframe>
But why go with the trouble when you can not save it to a file and instead it, embed it directly
import plotly.offline as pyo
import plotly.graph_objs as go
import numpy as np
#generate the Plot
np.random.seed(42)
random_x = np.random.randint(1,101,100)
random_y = np.random.randint(1,101,100)
data = [go.Scatter(
x = random_x,
y = random_y,
mode = 'markers',
)]
the_plot= pyo.plot(data, include_plotlyjs=False, output_type='div')
print(the_plot)
num=11
strTable = '<html><head><title>Plots and Reports</title><script src="https://cdn.plot.ly/plotly-latest.min.js"></script></head><body><table style="width:100%"><tr><th>Plot</th><th>Something else</th></tr>'
strRW = "<tr><td>"+the_plot+ "</td><td>"+str(num)+"</td></tr>"
strTable = strTable+strRW
strTable = strTable+"</table></body></html>"
hs = open("theReport.html", 'w')
hs.write(strTable)

I have recently needed to export Plotly Chart to HTML files.
Here is the simple proper way to do it in 2019.
import plotly.offline
plot(figure, "file.html")

If you want to insert fig (go.Figure) in HTML, maybe you should try use it.
Note: where the fig is one instance of go.Figure and fig contain your plot.
import json
import plotly.graph_objects as go
# step 1 convert fig to json
data = fig.to_json() #return one string
# step 2, save data in the file
filename = 'name_file.json'
with open ('' + filename + 'w') as file:
json.dump(data, file, indent=4)
After you will be able to insert the json file in HTML code, but you should insert dependencies.
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.6/d3.min.js"></script>
<script src="https://code.jquery.com/jquery-2.1.4.min.js"></script>
<!-- Plotly.js -->
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>

Just adding to this thread for anyone who is looking to render a plotly figure into a HTML template as a HTML readable string. I recently had to do this and wanted to share...
Basically, if you have a HTML file where you want to place a plotly figure as a static image, one method is to insert an img class and add a string representation of the figure to the src kwarg. For example:
<img
class="max-width"
border="0"
style="display:block; padding:0px; line-height: 0px; font-size: 0px; border:0px; color:#000000;text-decoration:none;max-width:100% !important; width:100%;"
align="top"
height="350"
width="1000"
alt=""
src="" <---- this is where you can add a string rep of figure
>
To generate the string rep of any figure from plotly you can use the following function:
def render_base64_image_from_plotly_fig(fig):
"""Render a base64 encoded image from a plotly figure.
Args:
fig (dict): Plotly figure information (data and layout).
Returns:
str: base64 encoded string of plotly graph.
"""
# render figure to .png image
img_bytes = fig.to_image(format="png")
# encode into b64
encoding = b64encode(img_bytes).decode()
# create b64 image placeholder
img_b64 = "data:image/png;base64," + encoding
return img_b64
The output string can be inserting into the src="" and you will have a HTML rendered version of your plotly graph.
NOTE: watch out for truncation in the output string if you are copying straight from an IDE... this caught me out and will cause a headache. Basically a truncated base64 string will truncate with a '.' but '.' is a forbidden char in a base64 string. If you want to guaranteed that this doesn't happen, just write the output string to a .txt file on your machine and then copy and paste from it. i.e.
with open('html.txt', 'w') as f:
f.write(img_b64)
f.close()
I hope that this helps someone!

As alternative to Plotly, you can use utilmy which save HTML
on disk without extra config:
!pip install utilmy
from utilmy.viz import vizhtml as vi
import pandas as pd
url='https://raw.githubusercontent.com/AlexAdvent/high_charts/main/data/stock_data.csv'
df = pd.read_csv(url)
doc.h2('Stock Market Analysis')
doc.h4('plot Data in table format')
doc.table(df, table_id="test", custom_css_class='intro',use_datatable=True)
doc.hr()
doc.h4('Stock tseries graph')
doc.plot_tseries(
df,coldate = 'Date',
date_format = '%m/%d/%Y',
coly1 = ['Open', 'High', 'Low', 'Close'],
coly2 = ['Turnover (Lacs)'],title = "Stock",
)
doc.save('stock market analysis.html')

Embedding Python Game Into HTML Using Skulpt

I have written a game in Python using the PyGame library that I am trying to embed into an HTML page to allow me to play in a web browser.
I am attempting to do this using the JavaScript library Skulpt. I have attached a test script below that successfully outputs the print statement below.
skulpt.html
<html>
<head>
<script src="assets/skulpt/skulpt.js" type="text/javascript"></script>
</head>
<body>
<textarea id="pythonCode">
print "I am python."
</textarea><br />
<pre id="output"></pre>
<script type="text/javascript">
function outf(text) {
var mypre = document.getElementById("output");
mypre.innerHTML = mypre.innerHTML + text;
}
var code = document.getElementById("pythonCode").value;
Sk.configure({output:outf});
eval(Sk.importMainWithBody("<stdin>",false,code));
</script>
</body>
</html>
Output of skulpt.html:
The issue that I am having is that when I use my game code instead of the simple print statement shown above it produces the error seen below;
I have included all relevant images to my web servers' directory at the correct path. I am unsure of why this error is being produced. Any help would be much appreciated, thanks!
Also, here is the attached Python game code (and a live demo of the error):
http://nicolasward.com/portfolio/skulpt.html

You have a lot of indentation on line 1 -> remember, in python, indentation always matters. Take away all those spaces/tabs on the first line and it should run.

Display multiple mpld3 exports on a single HTML page

I've found the mpld3 package to be brilliant for exporting a matplolib plot to HTML and displaying this via a flask app.
Each export comes with a lot of JS which seems unnecessary duplication if you want to display multiple plots within a single page. However I'm not well enough versed in JS to extract the relevant components and then loop through them. The .fig_to_dict method gives the necessary JSON to display each chart but then I'm left wondering what JS/ template work is needed to display each chart in turn.
I considered just stacking each plot into a single big figure but I'd like to layout the charts in separate DIVs and so this isn't the right solution.
I think I can see what the JS is doing to display them but I don't have enough knowledge to modify the functions to suit the purpose.
I haven't included code as I'm expecting this only to be relevant to someone with mpld3 experience but can supply some sample code if needed.
Sample HTML output for mpld3.fig_to_html(fig, template_type="simple"):
<script type="text/javascript" src="http://d3js.org/d3.v3.min.js"></script>
<script type="text/javascript" src="http://mpld3.github.io/js/mpld3.v0.1.js"></script>
<style>
</style>
<div id="fig136845463888164451663379"></div>
<script type="text/javascript">
var spec136845463888164451663379 = { <snip JSON code> };
var fig136845463888164451663379 = mpld3.draw_figure("fig136845463888164451663379", spec136845463888164451663379);
</script>
I'd thought it would be as simple as linking the two core scripts from the template header and then creating a new script for each JSON export. But that hasn't worked for me.

You're half-way there with your answer. I think what you want to do is something like this, which will embed three figures on the page:
<script type="text/javascript" src="http://d3js.org/d3.v3.min.js"></script>
<script type="text/javascript" src="http://mpld3.github.io/js/mpld3.v0.1.js"></script>
<style>
</style>
<div id="fig01"></div>
<div id="fig02"></div>
<div id="fig03"></div>
<script type="text/javascript">
var json01 = { <snip JSON code> };
var json02 = { <snip JSON code> };
var json03 = { <snip JSON code> };
mpld3.draw_figure("fig01", json01);
mpld3.draw_figure("fig02", json02);
mpld3.draw_figure("fig03", json03);
</script>
The json code for each figure can be created in Python by running
import json
# ... create matplotlib figure
json01 = json.dumps(mpld3.fig_to_dict(fig))
Embed this string at the appropriate place in the HTML document you're creating, and you should be good to go. I hope that helps!

Note that since jakevdp's answer was posted mpld3 has had a new release. As of today (September 2014) the mpld3 include has to be:
<script type="text/javascript" src="http://mpld3.github.io/js/mpld3.v0.2.js"></script>

Categories