Plotting barcode displays differently in plot window than in saved .pdf - python

I am having some trouble saving my pdf properly. I am trying to plot a barcode label and subsequently save it as a pdf, as in the following code. I have installed the code128.ttf font on my windows. Also, I have tried setting the .savefig dpi argument to fig.dpi, as argued in this post.
import os
import matplotlib.pyplot as plt
from matplotlib import font_manager as fm
def draw_label(label, label_dimensions_x=3.8189, label_dimensions_y=1.41732):
# import barcode code128 font
fpath = os.path.join("path", "to", "font", "code128.ttf")
prop = fm.FontProperties(fname=fpath, size=58)
fig, ax = plt.subplots(1, figsize=(label_dimensions_x,
label_dimensions_y))
plt.axis('off')
plt.xticks([], [])
plt.yticks([], [])
plt.tight_layout()
plt.xlim(0, label_dimensions_x)
plt.ylim(0, label_dimensions_y)
# plot barcode
plt.text(label_dimensions_x / 2, label_dimensions_y / 2, label,
ha='center', va='bottom',
fontproperties=prop)
plt.show()
try:
plt.savefig(os.path.join("path", "to", "output", label + '.pdf'),
dpi=plt.gcf().dpi)
except PermissionError:
logging.warning("Close the current label pdf's before running this script.")
plt.close()
return
draw_label('123456789')
This is what is output in the plot window.
This is what is output in the .pdf saved file, and this happens for all kinds of labels - it's not as if the numbers 1 to 9 except 8 are not printable.
EDIT: If I substitute a normal text font (in this case Frutiger Roman) for the code128.ttf, and set plt.axis('on') the text is not clipped, see this. Admitted, it's not pretty and doesn't fit too well, but it should be readable still.

Sam,
First, your barcode won't scan, as is. The string requires a start character, a checksum and a stop character to be added for Code128B. So, there's that.
I recommend changing to Code 39 font (which, doesn't require checksum, and start and stop characters are the same: "*") or writing the code to produce the checksum and learning a little more about Code 128 at Code 128 Wiki.
Second, I suspect there are issues with the bounding box for the graphic during the conversion to PDF. That small section of barcode being converted looks more like a piece of the number nine in the string. I suspect there is some image clipping going on.
Try substituting a regular text font to make sure the barcode image isn't being lost in the conversion.
Edited answer to include suggestion to use PNG instead of PDF.
I managed to get the software to work if you output to PNG format. I know, now the problem becomes how to convert PNG to PDF. You can start by investigating some of the libraries mentioned here: Create PDF from a list of images
In short I recommend you create graphics files and then embed them in document files.
I also added the code you need to build the barcode with the start, checksum and stop characters:
import os
import matplotlib.pyplot as plt
from matplotlib import font_manager as fm
def draw_label(label, label_dimensions_x=3.8189, label_dimensions_y=1.41732):
# import barcode code128 font
fpath = os.path.join("./", "code128.ttf")
prop = fm.FontProperties(fname=fpath, size=32)
fig, ax = plt.subplots(1, figsize=(label_dimensions_x,
label_dimensions_y))
plt.axis('off')
plt.xticks([], [])
plt.yticks([], [])
plt.tight_layout()
plt.xlim(0, label_dimensions_x)
plt.ylim(0, label_dimensions_y)
# calc checksum THEN plot barcode
weight = 1
chksum = 104
for x in label:
chksum = chksum + weight*(ord(x)-32)
weight = weight + 1
chksum = chksum % 103
chkchar = chr(chksum+32)
label128 = "%s%s%s%s" % ('Ñ', label, chkchar, 'Ó')
plt.text(label_dimensions_x / 2, label_dimensions_y / 2, label128,
ha='center', va='bottom',
fontproperties=prop)
try:
plt.savefig(os.path.join("./", label + '.png'))
except PermissionError:
logging.warning("Close the current label pdf's before running this script.")
return
draw_label('123456789')
draw_label('987654321')
draw_label('Test&Show')

You're over-complicating things by using matplotlib and a font. Generating an image directly and saving it to a PDF file is not much more complicated, and far more reliable.
As noted by Brian Anderson, it's not enough to encode the characters in your string. You need to add a start code, a checksum, and a stop code to make a complete barcode. The function code128_codes below does this, leaving the conversion to an image as a separate step.
from PIL import Image
def list_join(seq):
''' Join a sequence of lists into a single list, much like str.join
will join a sequence of strings into a single string.
'''
return [x for sub in seq for x in sub]
_code128B_mapping = dict((chr(c), [98, c+64] if c < 32 else [c-32]) for c in range(128))
_code128C_mapping = dict([(u'%02d' % i, [i]) for i in range(100)] + [(u'%d' % i, [100, 16+i]) for i in range(10)])
def code128_codes(s):
''' Code 128 conversion to a list of raw integer codes.
Only encodes ASCII characters, does not take advantage of
FNC4 for bytes with the upper bit set. Control characters
are not optimized and expand to 2 characters each.
Coded for https://stackoverflow.com/q/52710760/5987
'''
if s.isdigit() and len(s) >= 2:
# use Code 128C, pairs of digits
codes = [105] + list_join(_code128C_mapping[s[i:i+2]] for i in range(0, len(s), 2))
else:
# use Code 128B and shift for Code 128A
codes = [104] + list_join(_code128B_mapping[c] for c in s)
check_digit = (codes[0] + sum(i * x for i,x in enumerate(codes))) % 103
codes.append(check_digit)
codes.append(106) # stop code
return codes
_code128_patterns = '''
11011001100 11001101100 11001100110 10010011000 10010001100 10001001100
10011001000 10011000100 10001100100 11001001000 11001000100 11000100100
10110011100 10011011100 10011001110 10111001100 10011101100 10011100110
11001110010 11001011100 11001001110 11011100100 11001110100 11101101110
11101001100 11100101100 11100100110 11101100100 11100110100 11100110010
11011011000 11011000110 11000110110 10100011000 10001011000 10001000110
10110001000 10001101000 10001100010 11010001000 11000101000 11000100010
10110111000 10110001110 10001101110 10111011000 10111000110 10001110110
11101110110 11010001110 11000101110 11011101000 11011100010 11011101110
11101011000 11101000110 11100010110 11101101000 11101100010 11100011010
11101111010 11001000010 11110001010 10100110000 10100001100 10010110000
10010000110 10000101100 10000100110 10110010000 10110000100 10011010000
10011000010 10000110100 10000110010 11000010010 11001010000 11110111010
11000010100 10001111010 10100111100 10010111100 10010011110 10111100100
10011110100 10011110010 11110100100 11110010100 11110010010 11011011110
11011110110 11110110110 10101111000 10100011110 10001011110 10111101000
10111100010 11110101000 11110100010 10111011110 10111101110 11101011110
11110101110 11010000100 11010010000 11010011100 1100011101011'''.split()
def code128_img(s, height=100, bar_width=1):
''' Generate a Code 128 barcode image.
Coded for https://stackoverflow.com/q/52968042/5987
'''
codes = code128_codes(s)
pattern = ''.join(_code128_patterns[c] for c in codes)
pattern = '00000000000' + pattern + '00000000000'
width = bar_width * len(pattern)
color, bg = (0, 0, 0), (255, 255, 255)
im = Image.new('RGB', (width, height), bg)
ld = im.load()
for i, bar in enumerate(pattern):
if bar == '1':
for y in range(height):
for x in range(i * bar_width, (i + 1) * bar_width):
ld[x, y] = color
return im
>>> im = code128_img('AM-H-10-01-1')
>>> im.save(r'c:\temp\temp.pdf')

Related

Create a curve which adjust the contrast and brightness respectively of an image that is loaded

I have a series of images and was wondering if there is a possibility to write something in python to apply a contrast and brightness curve Like in the images below.
.
You should refer to, for instance, the OpenCV website (you can learn a lot of interesting methods and have a lot of examples on CV problems), here you have some documentation to check:
LUT method
And after the last link Changing the contrast and brightness of an image!
How to apply a lookup table
examples
other question
I hope it's helpful.
As stated by Fred in the comments, you probably want to create a Catmull-Rom spline from your points and then apply that using a LUT within OpenCV.
#!/usr/bin/env python3
# from https://splines.readthedocs.io/en/latest/euclidean/piecewise-monotone.html#Examples
import matplotlib.pyplot as plt
import numpy as np
import splines
import cv2
# Just helper code for plotting - you don't need this
def grid_lines(x=None, y=None, ax=None):
if ax is None:
ax = plt.gca()
if x is not None:
ax.set_xticks(x)
ax.xaxis.grid(True)
if y is not None:
ax.set_yticks(y)
ax.yaxis.grid(True)
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(False)
ax.spines['left'].set_visible(False)
ax.xaxis.set_ticks_position('none')
ax.yaxis.set_ticks_position('none')
# Just helper code for plotting - you don't need this
def plot_spline_1d(spline, samples=100, **kwargs):
'Plot a one-dimensional spline.'
ax = plt.gca()
times = np.linspace(spline.grid[0], spline.grid[-1], samples)
ax.plot(times, spline.evaluate(times), **kwargs)
ax.scatter(spline.grid, spline.evaluate(spline.grid))
The following simulates an S-shaped contrast curve similar to your 5 points:
xvals = 0, 64, 128, 192, 255
yvals = 0, 32, 128, 206, 255
# Just code for plotting the curve - you don't need this
plot_spline_1d(splines.CatmullRom(yvals, xvals), label='Catmull–Rom')
plt.legend()
grid_lines(xvals)
plt.show()
The actual code you need only really starts here:
# Derive Catmull-Rom spline for our X,Y
res = splines.CatmullRom(yvals, xvals)
# Make LUT (Lookup Table) from spline
LUT = np.uint8(res.evaluate(range(0,256)))
# Load ramp image as greyscale
im = cv2.imread('ramp.png', cv2.IMREAD_GRAYSCALE)
# Apply LUT to image
stretched = cv2.LUT(im, LUT)
# Save result
cv2.imwrite('result.png', stretched)
That turns this ramp:
into this:
Note that I artificially added a thin red border so you can see the extent of the image on StackOverflow's annoying background.
As regards reading a Photoshop Curves file (ACV or .acv), I wrote some code a while back that parses them but I haven't integrated that with the code above - it shouldn't be too hard - you basically save the points from the ACV file and use them to generate the spline for the above code. I leave it below for anyone interested to play with:
"""
################################################################################
fauxtoshop - does some things like Adobe Photoshop
Mark Setchell (mark#thesetchells.com)
Reads, interprets and possibly applies Photoshop files:
- Curves files (*.acv)
- Levels files (*.alv)
- Filter Kernel files (*.acf)
- Hue Saturation files (*.ahu)
################################################################################
"""
import sys
import numpy
from struct import unpack
def loadFilter(filename):
if filename.lower().endswith('.acv'):
loadCurvesFilter(filename)
return
if filename.lower().endswith('.alv'):
loadLevelsFilter(filename)
return
if filename.lower().endswith('.acf'):
loadKernelFilter(filename)
return
if filename.lower().endswith('.ahu'):
loadHSLFilter(filename)
return
sys.exit(f'ERROR: Unknown file extension {filename}')
def loadCurvesFilter(filename):
with open(filename, 'rb') as f:
version, ncurves = unpack('>HH', f.read(4))
print(f'File: {filename}')
print(f'Version: {version}')
if version != 4:
sys.exit('ERROR: Cowardly refusing to read version other than 4')
print(f'Curve count: {ncurves}')
curves = []
for c in range(ncurves):
npoints, = unpack('>H', f.read(2))
print(f'Curve: {c}, {npoints} points follow:')
curve = []
for p in range(npoints):
y, x = unpack('>HH', f.read(4))
print(f'Curve: {c}, point: {p}, x={x}, y={y}')
curve.append((x,y))
curves.append(curve)
return curves
def loadLevelsFilter(filename):
sys.exit("ERROR: Levels filter not yet implemeted")
def loadKernelFilter(filename):
sys.exit("ERROR: Kernel filter not yet implemeted")
def loadHSLFilter(filename):
with open(filename, 'rb') as f:
version, usage, pad = unpack('>HBB', f.read(4))
print(f'File: {filename}')
print(f'Version: {version}')
if version != 2:
sys.exit('ERROR: Cowardly refusing to read version other than 2')
if usage == 0:
print('Usage: Hue adjustment')
else:
print('Usage: Colorization')
sys.exit(f'ERROR: Cowardly refusing to apply colorization rather than Hue adjustment')
MasterHue, MasterSaturation, MasterLightness = unpack('>HHH', f.read(6))
print(f'Master Hue: {MasterHue}')
print(f'Master Saturation: {MasterSaturation}')
print(f'Master Lightness: {MasterLightness}')
# There follow 6 hextants, each with 4 range values and 3 settings values
for h in range(6):
ranges = unpack('>HHHH',f.read(8))
settings = unpack('>HHH', f.read(6))
print(f'Hextant: {h}, ranges: {ranges}, settings: {settings}')
################################################################################
# main
################################################################################
if __name__ == '__main__':
if len(sys.argv) not in set([2,3]):
print('Usage: {sys.argv[0]} filter.[acv|ahu|alv|acf] [image]', file=sys.stderr)
sys.exit(1)
if len(sys.argv) == 2:
loadFilter(sys.argv[1])
Note also that you can apply a Photoshop Curves (ACV) file using ffmpeg like this:
ffmpeg -i input.jpg -vf curves=psfile="curves.acv" output.jpg

Using a Data Converter to Display 3D Volume as Images

I would like to write a data converter tool. I need analyze the bitstream in a file to display the 2D cross-sections of a 3D volume.
The dataset I am trying to view can be found here: https://figshare.com/articles/SSOCT_test_dataset_for_OCTproZ/12356705.
It's the file titled: burned_wood_with_tape_1664x512x256_12bit.raw (832 MB)
Would extremely appreciate some direction. Willing to award a bounty if I could get some software to display the dataset as images using a data conversion.
As I'm totally new to this concept, I don't have code to show for this problem. However, here's a little something I tried using inspiration from other questions on SO:
import rawpy
import imageio
path = "Datasets/burned_wood_with_tape_1664x512x256_12bit.raw"
for item in path:
item_path = path + item
raw = rawpy.imread(item_path)
rgb = raw.postprocess()
rawpy.imshow(rgb)
Down below I implemented next visualization.
Example RAW file burned_wood_with_tape_1664x512x256_12bit.raw consists of 1664 samples per A-scan, 512 A-scans per B-scan, 16 B-scans per buffer, 16 buffers per volume, and 2 volumes in this file, each sample is encoded as 2-bytes unsigned integer in little endian order, only 12 higher bits are used, lower 4 bits contain zeros. Samples are centered approximately around 2^15, to be precise data has these stats min 0 max 47648 mean 32757 standard deviation 454.5.
I draw gray images of size 1664 x 512, there are total 16 * 16 * 2 = 512 such images (frames) in a file. I draw animated frames on screen using matplotlib library, also rendering these animation into GIF file. One example of rendered GIF at reduced quality is located after code.
To render/draw images of different resulting resolution you need to change code line with plt.rcParams['figure.figsize'], this fig size contains (widht_in_inches, height_in_inches), by default DPI (dots per inch) equals to 100, meaning that if you want to have resulting GIF of resolution 720x265 then you need to set this figure size to (7.2, 2.65). Also resulting GIF contains animation of a bit smaller resolution because axes and padding is included into resulting figure size.
My next code needs pip modules to be installed one time by command python -m pip install numpy matplotlib.
Try it online!
# Needs: python -m pip install numpy matplotlib
def oct_show(file, *, begin = 0, end = None):
import os, numpy as np, matplotlib, matplotlib.pyplot as plt, matplotlib.animation
plt.rcParams['figure.figsize'] = (7.2, 2.65) # (4.8, 1.75) (7.2, 2.65) (9.6, 3.5)
sizeX, sizeY, cnt, bits = 1664, 512, 16 * 16 * 2, 12
stepX, stepY = 16, 8
fps = 5
try:
fsize, opened_here = None, False
if type(file) is str:
fsize = os.path.getsize(file)
file, opened_here = open(file, 'rb'), True
by = (bits + 7) // 8
if end is None and fsize is not None:
end = fsize // (sizeX * sizeY * by)
imgs = []
file.seek(begin * sizeY * sizeX * by)
a = file.read((end - begin) * sizeY * sizeX * by)
a = np.frombuffer(a, dtype = np.uint16)
a = a.reshape(end - begin, sizeY, sizeX)
amin, amax, amean, stdd = np.amin(a), np.amax(a), np.mean(a), np.std(a)
print('min', amin, 'max', amax, 'mean', round(amean, 1), 'std_dev', round(stdd, 3))
a = (a.astype(np.float32) - amean) / stdd
a = np.maximum(0.1, np.minimum(a * 128 + 128.5, 255.1)).astype(np.uint8)
a = a[:, :, :, None].repeat(3, axis = -1)
fig, ax = plt.subplots()
plt.subplots_adjust(left = 0.08, right = 0.99, bottom = 0.06, top = 0.97)
for i in range(a.shape[0]):
title = ax.text(
0.5, 1.02, f'Frame {i}',
size = plt.rcParams['axes.titlesize'],
ha = 'center', transform = ax.transAxes,
)
imgs.append([ax.imshow(a[i], interpolation = 'antialiased'), title])
ani = matplotlib.animation.ArtistAnimation(plt.gcf(), imgs, interval = 1000 // fps)
print('Saving animated frames to GIF...', flush = True)
ani.save(file.name + '.gif', writer = 'imagemagick', fps = fps)
print('Showing animated frames on screen...', flush = True)
plt.show()
finally:
if opened_here:
file.close()
oct_show('burned_wood_with_tape_1664x512x256_12bit.raw')
Example output GIF:
I don't think it's a valid RAW file at all.
If you try this code:
import rawpy
import imageio
path = 'Datasets/burned_wood_with_tape_1664x512x256_12bit.raw'
raw = rawpy.imread(path)
rgb = raw.postprocess()
You will get a following error:
----> 5 raw = rawpy.imread(path)
6 rgb = raw.postprocess()
~\Anaconda3\envs\py37tf2gpu\lib\site-packages\rawpy\__init__.py in imread(pathOrFile)
18 d.open_buffer(pathOrFile)
19 else:
---> 20 d.open_file(pathOrFile)
21 return d
rawpy\_rawpy.pyx in rawpy._rawpy.RawPy.open_file()
rawpy\_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()
LibRawFileUnsupportedError: b'Unsupported file format or not RAW file'

Change the scale of the graph image

I try to generate a graph and save an image of the graph in python. Although the "plotting" of the values seems ok and I can get my picture, the scale of the graph is badly shifted.
If you compare the correct graph from tutorial example with my bad graph generated from different dataset, the curves are cut at the bottom to early: Y-axis should start just above the highest values and I should also see the curves for the highest X-values (in my case around 10^3).
But honestly, I think that problem is the scale of the y-axis, but actually do not know what parameteres should I change to fix it. I tried to play with some numbers (see below script), but without any good results.
This is the code for calculation and generation of the graph image:
import numpy as np
hic_data = load_hic_data_from_reads('/home/besy/Hi-C/MOREX/TCC35_parsedV2/TCC35_V2_interaction_filtered.tsv', resolution=100000)
min_diff = 1
max_diff = 500
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(12, 12))
for cnum, c in enumerate(hic_data.chromosomes):
if c in ['ChrUn']:
continue
dist_intr = []
for diff in xrange(min_diff, min((max_diff, 1 + hic_data.chromosomes[c]))):
beg, end = hic_data.section_pos[c]
dist_intr.append([])
for i in xrange(beg, end - diff):
dist_intr[-1].append(hic_data[i, i + diff])
mean_intrp = []
for d in dist_intr:
if len(d):
mean_intrp.append(float(np.nansum(d)) / len(d))
else:
mean_intrp.append(0.0)
xp, yp = range(min_diff, max_diff), mean_intrp
x = []
y = []
for k in xrange(len(xp)):
if yp[k]:
x.append(xp[k])
y.append(yp[k])
l = plt.plot(x, y, '-', label=c, alpha=0.8)
plt.hlines(mean_intrp[2], 3, 5.25 + np.exp(cnum / 4.3), color=l[0].get_color(),
linestyle='--', alpha=0.5)
plt.text(5.25 + np.exp(cnum / 4.3), mean_intrp[2], c, color=l[0].get_color())
plt.plot(3, mean_intrp[2], '+', color=l[0].get_color())
plt.xscale('log')
plt.yscale('log')
plt.ylabel('number of interactions')
plt.xlabel('Distance between bins (in 100 kb bins)')
plt.grid()
plt.ylim(2, 250)
_ = plt.xlim(1, 110)
fig.savefig('/home/besy/Hi-C/MOREX/TCC35_V2_results/filtered/TCC35_V2_decay.png', dpi=fig.dpi)
I think that problem is in scale I need y-axis to start from 10^-1 (0.1), in order to change this I tried this:
min_diff = 0.1
.
.
.
dist_intr = []
for diff in xrange(min_diff, min((max_diff, 0.1 + hic_data.chromosomes[c]))):
.
.
.
plt.ylim((0.1, 20))
But this values return: "integer argument expected, got float"
I also tried to play with:
max_diff, plt.ylim and plt.xlim parameters little bit, but nothing changed to much.
I would like to ask you what parameter/s and how I need change to generate image of the correctly focused graph. Thank you in advance.

Multiple route mapping to different matplotlib graphs in flask app

I have this "flask app" with two links, each mapping to different matplotlib visualizations, for example: localhost:5000/line_chart and localhost:5000/bar_chart.
When I start the server, and click the a route (any of them), I see what I expect.
localhost:5000/bar_chart
When I go back and view the other link, both graphs break.
localhost:5000/line_chart
localhost:5000/bar_chart
I can reproduce this every time by closing the server then running the "run.py" script again. Seems to be an overwriting conflict with the in-memory buffer. Has anyone had this issue before?
app/views.py
import matplotlib
matplotlib.use('Agg') # this allows PNG plotting
import matplotlib.pyplot as plt
import base64
from flask import render_template
from app import app
from io import BytesIO
#app.route('/')
#app.route('/index')
def index():
res = ''
navigation = [['Line Chart','line_chart'],['Bar Chart','bar_chart']]
res = res + '<h1>Matplotlib Chart Examples</h1>'
res = res + '<ul>'
for item in navigation:
name = item[0]
link = item[1]
res = res + '<li>'+ name +'</li>'
res = res +'</ul>'
return res
#app.route('/bar_chart')
def bar_chart():
movies = ["Annie Hall", "Ben-Hur", "Casablanca", "Gandhi", "West Side Story"]
num_oscars = [5, 11, 3, 8, 10]
# bars are by default width 0.8, so we'll add 0.1 to the left coordinates
# so that each bar is centered
xs = [i + 0.1 for i, _ in enumerate(movies)]
# plot bars with left x-coordinates [xs], heights [num_oscars]
plt.bar(xs, num_oscars)
plt.ylabel("# of Academy Awards")
plt.title("My Favorite Movies")
# label x-axis with movie names at bar centers
plt.xticks([i + 0.5 for i, _ in enumerate(movies)], movies)
return compute(plt)
#app.route('/line_chart')
def line_chart():
years = [1950, 1960, 1970, 1980, 1990, 2000, 2010]
gdp = [300.2, 543.3, 1075.9, 2862.5, 5979.6, 10289.7, 14958.3]
# create a line chart, years on x-axis, gdp on y-axis
plt.plot(years, gdp, color='green', marker='o', linestyle='solid')
# add a title
plt.title("Nominal GDP")
# add a label to the y-axis
plt.ylabel("Billions of $")
return compute(plt)
def compute(plt):
# run plt.plot, plt.title, etc.
figfile = BytesIO()
plt.savefig(figfile, format='png')
figfile.seek(0) # rewind to beginning of file
#figfile.getvalue() extracts string (stream of bytes)
figdata_png = base64.b64encode(figfile.getvalue())
return render_template('index.html',
title='matplotlib chart',
results=figdata_png)
Thank you for your time.
I guess you need two figures, test this code and tell what happened:
#app.route('/bar_chart')
def bar_chart():
movies = ["Annie Hall", "Ben-Hur", "Casablanca", "Gandhi", "West Side Story"]
num_oscars = [5, 11, 3, 8, 10]
# bars are by default width 0.8, so we'll add 0.1 to the left coordinates
# so that each bar is centered
xs = [i + 0.1 for i, _ in enumerate(movies)]
# plot bars with left x-coordinates [xs], heights [num_oscars]
plt.figure(1)
plt.bar(xs, num_oscars)
plt.ylabel("# of Academy Awards")
plt.title("My Favorite Movies")
# label x-axis with movie names at bar centers
plt.xticks([i + 0.5 for i, _ in enumerate(movies)], movies)
return compute(plt, 1)
#app.route('/line_chart')
def line_chart():
years = [1950, 1960, 1970, 1980, 1990, 2000, 2010]
gdp = [300.2, 543.3, 1075.9, 2862.5, 5979.6, 10289.7, 14958.3]
# create a line chart, years on x-axis, gdp on y-axis
plt.figure(2)
plt.plot(years, gdp, color='green', marker='o', linestyle='solid')
# add a title
plt.title("Nominal GDP")
# add a label to the y-axis
plt.ylabel("Billions of $")
return compute(plt,2)
def compute(plt, fignum):
# run plt.plot, plt.title, etc.
plt.figure(fignum)
figfile = BytesIO()
plt.savefig(figfile, format='png')
figfile.seek(0) # rewind to beginning of file
#figfile.getvalue() extracts string (stream of bytes)
figdata_png = base64.b64encode(figfile.getvalue())
return render_template('index.html',
title='matplotlib chart',
results=figdata_png)
In my case, that solution didn't work. It seems that there is a race condition when trying to access plot. I first tried to use a lock from a library, but that didn't work, so instead I sort of engineered out a lock. In my case, I wanted to create n images using the same function on the same view, so I started by creating a list in the following way:
queue = [False for i in range(n)]
Then, my flask app look something like this:
#app.route('/vis/<j>')
def vis(j):
global queue
# We check that it's image's #j turn, as if it was single threaded
j = int(j)
if j == 0:
for i in range(len(queue)):
queue[i] = False
else:
while not queue[j-1]:
# If it's not, we sleep for a short time (from time import sleep)
sleep(0.5)
# This is not important, it's how I was plotting some random figures
# (from random import seed) (from datetime import datetime)
seed(datetime.now())
n = 10
p1 = [randint(0, 10) for _ in range(n)]
p2 = [randint(0, 10) for _ in range(n)]
t = [i for i in range(n)]
fig = plt.figure(j)
plt.clf()
plt.plot(t, p1, color='blue')
plt.plot(t, p2, color='orange')
plt.xlabel('Time')
plt.ylabel('Value')
# Save the plot
img = BytesIO()
fig.savefig(img, dpi=128)
img.seek(0)
# We finished using everything related to plot, so we free the "lock"
queue[j] = True
# Return the object as a file that can be accessed
return send_file(img, mimetype='image/png')
Finally, when wanting to display this in my flask app, all I had to do was using this <img src="/vis/1"> in my html file.
Edit: I forgot one of the most important part! For some reason, this would still create some unrelated thread issue. I looked it up and that's when I came with the full solution. The threading issue was solved by adding at the beginning of the file:
import matplotlib
import matplotlib.pyplot as plt
matplotlib.use('Agg')
For some reason, using that Agg backend solved the second threading I was having. I don't really have a good explanation for that, but it does work, so it's enough for me.
Alternatively, what also worked was running the app disabling threads by adding:
if __name__ == '__main__':
app.run(threading=False, debug=True)
I don't know however, at the moment, whether this works in production, so I preferred the other solution. :)
I hope this helps if you had the same issue!

Plot really big file in python (5GB) with x axis offset

I am trying to plot a very big file (~5 GB) using python and matplotlib. I am able to load the whole file in memory (the total available in the machine is 16 GB) but when I plot it using simple imshow I get a segmentation fault. This is most probable to the ulimit which I have set to 15000 but I cannot set higher. I have come to the conclusion that I need to plot my array in batches and therefore made a simple code to do that. My main isue is that when I plot a batch of the big array the x coordinates start always from 0 and there is no way I can overlay the images to create a final big one. If you have any suggestion please let me know. Also I am not able to install new packages like "Image" on this machine due to administrative rights. Here is a sample of the code that reads the first 12 lines of my array and make 3 plots.
import os
import sys
import scipy
import numpy as np
import pylab as pl
import matplotlib as mpl
import matplotlib.cm as cm
from optparse import OptionParser
from scipy import fftpack
from scipy.fftpack import *
from cmath import *
from pylab import *
import pp
import fileinput
import matplotlib.pylab as plt
import pickle
def readalllines(file1,rows,freqs):
file = open(file1,'r')
sizer = int(rows*freqs)
i = 0
q = np.zeros(sizer,'float')
for i in range(rows*freqs):
s =file.readline()
s = s.split()
#print s[4],q[i]
q[i] = float(s[4])
if i%262144 == 0:
print '\r ',int(i*100.0/(337*262144)),' percent complete',
i += 1
file.close()
return q
parser = OptionParser()
parser.add_option('-f',dest="filename",help="Read dynamic spectrum from FILE",metavar="FILE")
parser.add_option('-t',dest="dtime",help="The time integration used in seconds, default 10",default=10)
parser.add_option('-n',dest="dfreq",help="The bandwidth of each frequency channel in Hz",default=11.92092896)
parser.add_option('-w',dest="reduce",help="The chuncker divider in frequency channels, integer default 16",default=16)
(opts,args) = parser.parse_args()
rows=12
freqs = 262144
file1 = opts.filename
s = readalllines(file1,rows,freqs)
s = np.reshape(s,(rows,freqs))
s = s.T
print s.shape
#raw_input()
#s_shift = scipy.fftpack.fftshift(s)
#fig = plt.figure()
#fig.patch.set_alpha(0.0)
#axes = plt.axes()
#axes.patch.set_alpha(0.0)
###plt.ylim(0,8)
plt.ion()
i = 0
for o in range(0,rows,4):
fig = plt.figure()
#plt.clf()
plt.imshow(s[:,o:o+4],interpolation='nearest',aspect='auto', cmap=cm.gray_r, origin='lower')
if o == 0:
axis([0,rows,0,freqs])
fdf, fdff = xticks()
print fdf
xticks(fdf+o)
print xticks()
#axis([o,o+4,0,freqs])
plt.draw()
#w, h = fig.canvas.get_width_height()
#buf = np.fromstring(fig.canvas.tostring_argb(), dtype=np.uint8)
#buf.shape = (w,h,4)
#buf = np.rol(buf, 3, axis=2)
#w,h,_ = buf.shape
#img = Image.fromstring("RGBA", (w,h),buf.tostring())
#if prev:
# prev.paste(img)
# del prev
#prev = img
i += 1
pl.colorbar()
pl.show()
If you plot any array with more than ~2k pixels across something in your graphics chain will down sample the image in some way to display it on your monitor. I would recommend down sampling in a controlled way, something like
data = convert_raw_data_to_fft(args) # make sure data is row major
def ds_decimate(row,step = 100):
return row[::step]
def ds_sum(row,step):
return np.sum(row[:step*(len(row)//step)].reshape(-1,step),1)
# as per suggestion from tom10 in comments
def ds_max(row,step):
return np.max(row[:step*(len(row)//step)].reshape(-1,step),1)
data_plotable = [ds_sum(d) for d in data] # plug in which ever function you want
or interpolation.
Matplotlib is pretty memory-inefficient when plotting images. It creates several full-resolution intermediate arrays, which is probably why your program is crashing.
One solution is to downsample the image before feeding it into matplotlib, as #tcaswell suggests.
I also wrote some wrapper code to do this downsampling automatically, based on your screen resolution. It's at https://github.com/ChrisBeaumont/mpl-modest-image, if it's useful. It also has the advantage that the image is resampled on the fly, so you can still pan and zoom without sacrificing resolution where you need it.
I think you're just missing the extent=(left, right, bottom, top) keyword argument in plt.imshow.
x = np.random.randn(2, 10)
y = np.ones((4, 10))
x[0] = 0 # To make it clear which side is up, etc
y[0] = -1
plt.imshow(x, extent=(0, 10, 0, 2))
plt.imshow(y, extent=(0, 10, 2, 6))
# This is necessary, else the plot gets scaled and only shows the last array
plt.ylim(0, 6)
plt.colorbar()
plt.show()

Categories