We Keep Coding

How to properly use dvioptions in preview functon from sympy? - python

Form preview documentation,
If the value of ‘output’ is different from ‘dvi’ then command line options can be set (‘dvioptions’ argument) for the execution of the ‘dvi’+output conversion tool. These options have to be in the form of a list of strings (see subprocess.Popen).
I couldn't manage to find any doc regarding valid arguments of dvioptions except some examples like this.
from sympy import preview
preview(r'$$H_{2}O$$', viewer='file', filename='water.png', euler=False, dvioptions=["-T", "tight", "-z", "0", "--truecolor", "-D 600", "-bg", "Transparent"])
what do these "-z", "0", "--truecolor" mean? or, better, if you could just point me to appropriate doc.

These look like the command line options of dvipng. You can see a full list if you run man dvipng:
DVIPNG(1) User commands DVIPNG(1)
NAME
dvipng - A DVI-to-PNG translator
SYNOPSIS
dvipng [options] filename
dvipng [options] [filename] -
DESCRIPTION
This program makes PNG and/or GIF graphics from DVI files as obtained
from TeX and its relatives.
If GIF support is enabled, GIF output is chosen by using the dvigif
binary or with the --gif option.
The benefits of dvipng/dvigif include
o Speed. It is a very fast bitmap-rendering code for DVI files, which
makes it suitable for generating large amounts of images on-the-
fly, as needed in preview-latex, WeBWorK and others.
o It does not read the postamble, so it can be started before TeX
finishes. There is a --follow switch that makes dvipng wait at end-
of-file for further output, unless it finds the POST marker that
indicates the end of the DVI.
o Interactive query of options. dvipng can read options interactively
through stdin, and all options are usable. It is even possible to
change the input file through this interface.
o Supports PK, VF, PostScript Type1, and TrueType fonts, subfonts
(i.e., as used in CJK-LaTeX), color specials, and inclusion of
PostScript, PNG, JPEG or GIF images.
o and more...
OPTIONS
Many of the parameterless options listed here can be turned off by
suffixing the option with a zero (0); for instance, to turn off page
reversal, use -r0. Such options are marked with a trailing *.
- Read additional options from standard input after processing the
command line.
--help
Print a usage message and exit.
--version
Print the version number and exit.
-bd num
-bd color_spec
-bd 'num color_spec'
Set the pixel width of the transparent border (default 0). Using
this option will make the image edges transparent, but it only
affects pixels with the background color. Giving a color_spec will
set the fallback color, to be used in viewers that cannot handle
transparency (the default is the background color). The color spec
should be in TeX color \special syntax, e.g., 'rgb 1.0 0.0 0.0'.
Setting the fallback color makes the default border width 1 px.
--bdpi num
This option only has an effect when using bitmapped (PK) fonts. The
option sets the base (Metafont) resolution, both horizontal and
vertical, to num dpi (dots per inch). This option is necessary when
manually selecting Metafont mode with the --mode option (see
below).
-bg color_spec
Choose background color for the images. This option will be ignored
if there is a background color \special in the DVI. The color spec
should be in TeX color \special syntax, e.g., 'rgb 1.0 0.0 0.0'.
You can also specify 'Transparent' or 'transparent' which will give
you a transparent background with the normal background as a
fallback color. A capitalized 'Transparent' will give a full-alpha
transparency, while an all-lowercase 'transparent' will give a
simple fully transparent background with non-transparent
antialiased pixels. The latter would be suitable for viewers who
cannot cope with a true alpha channel. GIF images do not support
full alpha transparency, so in case of GIF output, both variants
will use the latter behaviour.
-d num
Set the debug flags, showing what dvipng (thinks it) is doing. This
will work unless dvipng has been compiled without the "DEBUG"
option (not recommended). Set the flags as you need them, use -d -1
as the first option for maximum output.
-D num
Set the output resolution, both horizontal and vertical, to num dpi
(dots per inch).
One may want to adjust this to fit a certain text font size (e.g.,
on a web page), and for a text font height of font_px pixels (in
Mozilla) the correct formula is
<dpi> = <font_px> * 72.27 / 10 [px * TeXpt/in / TeXpt]
The last division by ten is due to the standard font height 10pt in
your document, if you use 12pt, divide by 12. Unfortunately, some
proprietary browsers have font height in pt (points), not pixels.
You have to rescale that to pixels, using the screen resolution
(default is usually 96 dpi) which means the formula is
<font_px> = <font_pt> * 96 / 72 [pt * px/in / (pt/in)]
On some high-res screens, the value is instead 120 dpi. Good luck!
--depth*
Report the depth of the image. This only works reliably when the
LaTeX style preview.sty from preview-latex is used with the active
option. It reports the number of pixels from the bottom of the
image to the baseline of the image. This can be used for vertical
positioning of the image in, e.g., web documents, where one would
use (Cascading StyleSheets 1)
<IMG SRC="<filename.png>" STYLE="vertical-align: -<depth>px">
The depth is a negative offset in this case, so the minus sign is
necessary, and the unit is pixels (px).
--dvinum*
Set this option to make the output page number be the TeX page
numbers rather than the physical page number. See the -o switch.
-fg color_spec
Choose foreground color for the images. This option will be ignored
if there is a foreground color \special in the DVI. The color spec
should be in TeX color \special syntax, e.g., 'rgb 1.0 0.0 0.0'.
--follow*
Wait for data at end-of-file. One of the benefits of dvipng is that
it does not read the postamble, so it can be started before TeX
finishes. This switch makes dvipng wait at end-of-file for further
output, unless it finds the POST marker that indicates the end of
the DVI. This is similar to tail -f but for DVI-to-PNG conversion.
--freetype*
Enable/disable FreeType font rendering (default on). This option is
available if the FreeType2 font library was present at compilation
time. If this is the case, dvipng will have direct support for
PostScript Type1 and TrueType fonts internally, rather than using
gsftopk for rendering the fonts. If you have PostScript versions of
Computer Modern installed, there will be no need to generate
bitmapped (PK) variants on disk of these. Then, you can render
images at different (and unusual) resolutions without cluttering
the disk with lots of bitmapped fonts. One reason to disable
FreeType font rendering would be to generate identical output on
different platforms, since FreeType uses the native renderer and
therefore can give slightly different output on each platform.
--gamma num
Control the interpolation of colors in the greyscale anti-aliasing
color palette. Default value is 1.0. For 0 < num < 1, the fonts
will be lighter (more like the background), and for num > 1, the
fonts will be darker (more like the foreground).
--gif*
The images are output in the GIF format, if GIF support is enabled.
This is the default for the dvigif binary, which only will be
available when GIF support is enabled. GIF images are palette
images (see the --palette option) and does not support true alpha
channels (see the --bg option). See also the --png option.
--height*
Report the height of the image. This only works reliably when the
LaTeX style preview.sty from preview-latex is used with the active
option. It reports the number of pixels from the top of the image
to the baseline of the image. The total height of the image is
obtained as the sum of the values reported from --height and
--depth.
-l [=]num
The last page printed will be the first one numbered num. Default
is the last page in the document. If num is prefixed by an equals
sign, then it (and the argument to the -p option, if specified) is
treated as a physical (absolute) page number, rather than a value
to compare with the TeX \count0 values stored in the DVI file.
Thus, using -l =9 will end with the ninth page of the document, no
matter what the pages are actually numbered.
--mode mode
This option only has an effect when using bitmapped (PK) fonts. Use
mode as the Metafont device name for the PK fonts (both for path
searching and font generation). This needs to be augmented with the
base device resolution, given with the --bdpi option. See the file
<ftp://ftp.tug.org/tex/modes.mf> for a list of resolutions and mode
names for most devices.
-M* This option only has an effect when using bitmapped (PK) fonts. It
turns off automatic PK font generation (mktexpk).
--nogs*
This switch prohibits the internal call to GhostScript for
displaying PostScript specials. --nogs0 turns the call back on.
--nogssafer*
Normally, if GhostScript is used to render PostScript specials, the
GhostScript interpreter is run with the option -dSAFER. The
--nogssafer option runs GhostScript without -dSAFER. The -dSAFER
option in Ghostscript disables PostScript operators such as
deletefile, to prevent possibly malicious PostScript programs from
having any effect.
--norawps*
Some packages generate raw PostScript specials, even non-rendering
such specials. This switch turns off the internal call to
GhostScript intended to display these raw PostScript specials.
--norawps0 turns the call back on.
-o name
Send output to the file name. A single occurrence of %d or %01d,
..., %09d will be exchanged for the physical page number (this can
be changed, see the --dvinum switch). The default output filename
is file%d.png where the input DVI file was file.dvi.
-O x-offset,y-offset
Move the origin by x-offset,y-offset, a comma-separated pair of
dimensions such as .1in,-.3cm. The origin of the page is shifted
from the default position (of one inch down, one inch to the right
from the upper left corner of the paper) by this amount.
-p [=]num
The first page printed will be the first one numbered num. Default
is the first page in the document. If num is prefixed by an equals
sign, then it (and the argument to the -l option, if specified) is
treated as a physical (absolute) page number, rather than a value
to compare with the TeX \count0 values stored in the DVI file.
Thus, using -p =3 will start with the third page of the document,
no matter what the pages are actually numbered.
--palette*
When an external image is included, dvipng will automatically
switch to truecolor mode, to avoid unnecessary delay and quality
reduction, and enable the EPS translator to draw on a transparent
background and outside of the boundingbox. This switch will force
palette (256-color) output and make dvipng revert to opaque clipped
image inclusion. This will also override the --truecolor switch if
present.
--picky*
No images are output when a warning occurs. Normally, dvipng will
output an image in spite of a warning, but there may be something
missing in this image. One reason to use this option would be if
you have a more complete but slower fallback converter. Mainly,
this is useful for failed figure inclusion and unknown \special
occurrences, but warnings will also occur for missing or unknown
color specs and missing PK fonts.
--png*
The images are output in the PNG format. This is the default for
the dvipng binary. See also the --gif option.
-pp firstpage-lastpage
Print pages firstpage through lastpage; but not quite equivalent to
-p firstpage -l lastpage. For example, when rendering a book, there
may be several instances of a page in the DVI file (one in
"\frontmatter", one in "\mainmatter", and one in "\backmatter"). In
case of several pages matching, -pp firstpage-lastpage will render
all pages that matches the specified range, while -p firstpage -l
lastpage will render the pages from the first occurrence of
firstpage to the first occurrence of lastpage. This is the
(undocumented) behaviour of dvips. In dvipng you can give both
kinds of options, in which case you get all pages that matches the
range in -pp between the pages from -p to -l. Also multiple -pp
options accumulate, unlike -p and -l. The - separator can also be
:. Note that -pp -1 will be interpreted as "all pages up to and
including 1", if you want a page numbered -1 (only the table of
contents, say) put -pp -1--1, or more readable, -pp -1:-1.
-q* Run quietly. Don't chatter about pages converted, etc. to standard
output; report no warnings (only errors) to standard error.
-Q num
Set the quality to num. That is, choose the number of antialiasing
levels for bitmapped fonts (PK), to be num*num+1. The default value
is 4 which gives 17 levels of antialiasing for antialiased fonts
from these two. If FreeType is available, its rendering is
unaffected by this option.
-r* Toggle output of pages in reverse/forward order. By default, the
first page in the DVI is output first.
--strict*
The program exits when a warning occurs. Normally, dvipng will
output an image in spite of a warning, but there may be something
missing in this image. One reason to use this option would be if
you have a more complete but slower fallback converter. See the
--picky option above for a list of when warnings occur.
-T image_size
Set the image size to image_size which can be either of bbox,
tight, or a comma-separated pair of dimensions hsize,vsize such as
.1in,.3cm. The default is bbox which produces a PNG that includes
all ink put on the page and in addition the DVI origin, located 1in
from the top and 1in from the left edge of the paper. This usually
gives whitespace above and to the left in the produced image. The
value tight will make dvipng only include all ink put on the page,
producing neat images.
--truecolor*
This will make dvipng generate truecolor output. Note that
truecolor output is automatic if you include an external image in
your DVI, e.g., via a PostScript special (i.e., the graphics or
graphicx package). This switch is overridden by the --palette
switch.
-v* Enable verbose operation. This will currently indicate what fonts
is used, in addition to the usual output.
--width*
Report the width of the image. See also --height and --depth.
-x num
This option is deprecated; it should not be used. It is much better
to select the output resolution directly with the -D option. This
option sets the magnification ratio to num/1000 and overrides the
magnification specified in the DVI file. Must be between 10 and
100000. It is recommended that you use standard magstep values
(1095, 1200, 1440, 1728, 2074, 2488, 2986, and so on) to help
reduce the total number of PK files generated. num may be a real
number, not an integer, for increased precision.
-z num
Set the PNG compression level to num. This option is enabled if
your libgd is new enough. The default compression level is 1, which
selects maximum speed at the price of slightly larger PNGs. For an
older libgd, the hard-soldered value 5 is used. The include file
png.h says "Currently, valid values range from 0 - 9, corresponding
directly to the zlib compression levels 0 - 9 (0 - no compression,
9 - "maximal" compression). Note that tests have shown that zlib
compression levels 3-6 usually perform as well as level 9 for PNG
images, and do considerably fewer calculations. In the future,
these values may not correspond directly to the zlib compression
levels."
NOTES
The full manual is accessible in info format, on most systems by typing
info dvipng
COPYRIGHT
This program is released under the GNU Lesser General Public License
version 3, see the COPYING file in the dvipng distribution or
<http://www.gnu.org/licenses/gpl.html>.
Copyright (c) 2002-2015, 2019 Jan-AAke Larsson
dvipng (TeX Live) 1.17 2021-02-26 DVIPNG(1)

Related

I'm trying to minimize file size of PNG images written with pyvips.Image.pngsave(). Original files written with just .pngsave(output) are at https://github.com/CDDA-Tilesets/UltimateCataclysm and we'll look at giant.png which is 119536 bytes.
ImgBot was able to reduce file size to 50672.
pngsave(output, compression=9, palette=True, strip=True) to 58722
But the convert command from ImageMagick is still able to reduce file size further after the latter, to 42833 with default options:
$ convert giant_pyvips_c9.png giant_pyvips_magick.png
The question is whether it's possible to fit the same image into 42833 bytes using only pyvips to avoid adding another step to our workflow?
Update: Warning
palette size is limited to 256 colors and pyvips doesn't warn you if conversion becomes lossy.

Try turning off filtering:
$ vips copy giant.png x.png[palette,compression=9,strip,filter=0]
$ ls -l x.png
-rw-r--r-- 1 john john 41147 Feb 14 10:58 x.png
Background: PNG filters put the image though a difference filter before compression. Compressing differences to neighbouring pixels rather than absolute pixel values can boost the compression ratio if there is some local pattern in values. pyvips uses an adaptive filter by default.
Palette images encode an index into a look up table rather than anything related to luminance, so there is much less local correlation. In this case, filtering actually hurts compression.
http://www.w3.org/TR/PNG-Filters.html
You can see the values allowed for the filter= parameter here:
https://github.com/libvips/libvips/blob/master/libvips/include/vips/foreign.h#L579-L598

I'm using sphinx.ext.imgmath to include mathematical expressions in my Python project documentation. It's working, but the resolution of the produced images is quite low. How can I make it produce images with higher resolution?

With imgmath you can use either the png or svg format. In my experience, svg has a better resolution by default. You can enable it in your Sphinx config.py by defining the variable
imgmath_image_format = 'svg'
You may need to adjust the math font size. This can be done using
imgmath_font_size = 14 # for font size 14
If you really need png, then you can override the default settings for the dvipng application which is used by imgmath. The corresponding variable for conf.py is
imgmath_dvipng_args = ['-gamma', '1.5', '-D', '110', '-bg', 'Transparent']
which is the default. Those are key value pairs for dvipng. -D means output resolution, and its value is 110. You can increase it, but that makes the whole displayed image bigger as well. So you have to fiddle with the font size. For other dvipng options, use its command line help.
All in all, I recommend using svg. :-)

I want to use Gotham Cond SSm XLight in MATLAB's text command to draw several digits, letters and symbols in the font above. However, it seems, according to the official documentation, FontWeight in text only allows normal|bold, but not XLight for Extra Light style.
Here is the fonts I have installed in my system.
$ ls GothamCondensed/OpenType/
GothamCondSSm-Black.otf GothamCondSSm-Book.otf GothamCondSSm-Medium.otf
GothamCondSSm-BlackItalic.otf GothamCondSSm-BookItalic.otf GothamCondSSm-MediumItalic.otf
GothamCondSSm-Bold.otf GothamCondSSm-Light.otf GothamCondSSm-XLight.otf
GothamCondSSm-BoldItalic.otf GothamCondSSm-LightItalic.otf GothamCondSSm-XLightItalic.otf
Here is what I used for MATLAB R2015b:
% draw symbol `$' in Gotham Cond SSm XLight
text(0,0,'$','FontName','Gotham Cond SSm','FontWeight', 'XLight');
% also try this, just in case
text(0,0,'$','FontName','Gotham Cond SSm XLight');
text(0,0,'$','FontName','Gotham Cond SSm XLight','FontWeight', 'XLight');
If it is not supported in MATLAB, even by some undocumented java hack, then it would still be great to be able to produce these symbols (e.g. `$') in any language (preferably Python or R) in high resolution by specifying size in pixels, then saving them as grayscale image files. I could then read those image files into my MATLAB struct.

I am generating SVG image in python (pure, no external libs yet). I want to know what will be a text element size, before I place it properly. Any good idea how to make it? I checked pysvg library but I saw nothing like getTextSize()

This can be be pretty complicated. To start with, you'll have to familiarize yourself with chapter on text of the SVG specification. Assuming you want to get the width of plain text elements, and not textpath elements, at a minimum you'd have to:
Parse the font selection properties, spacing properties and read the xml:space attibute, as well as the writing-mode property (can also be top-bottom instead of just left-to-right and right-to-left).
Based on the above, open the correct font, and read the glyph data and extract the widths and heights of the glyphs in your text string. Alone finding the font can be a big task, seeing the multiple places where font files can hide.
(optionally) Look through the string for possible ligatures (depending on the language), and replace them with the correct glyph if it exists in the font.
Add the widths for all the characters and spaces, the latter depending on the spacing properties and (optionally) possible kerning pairs.
A possible solution would be to use the pango library. You can find python bindings for it in py-gtk. Unfortunately, except from some examples, the documentation for the python bindings is pretty scarce. But it would take care of the details of font loading and determining the extents of a Layout.
Another way is to study the SVG renderer in your browser. But e.g. the support for SVG text in Firefox is limited.
Also instructive is to study how TeX does it, especially the concept (pdf) of boxes (for letters) and glue (for spacing).

I had this exact same problem, but I had a variable width font. I solved it by taking the text element (correct font and content) I wanted, wrote it to a svg file, and I used Inkscape installed on my PC to render the drawing to a temporary png file. I then read back the dimensions of the png file (extracted from the header), removed the temp svg and png files and used the result to place the text where I wanted and elements around it.
I found that rendering to a drawing, using a DPI of 90 seemed to give me the exact numbers I needed, or the native numbers used in svgwrite as a whole. -D is the flag to use so that only the drawable element, i.e. the text, is rendered.
os.cmd(/cygdrive/c/Program\ Files\ \(x86\)/Inkscape/inkscape.exe -f work_temp.svg -e work_temp.png -d 90 -D)
I used these functions to extract the png numbers, found at this link, note mine is corrected slightly for python3 (still working in python2)
def is_png(data):
return (data[:8] == b'\x89PNG\r\n\x1a\n'and (data[12:16] == b'IHDR'))
def get_image_info(data):
if is_png(data):
w, h = struct.unpack('>LL', data[16:24])
width = int(w)
height = int(h)
else:
raise Exception('not a png image')
return width, height
if __name__ == '__main__':
with open('foo.png', 'rb') as f:
data = f.read()
print is_png(data)
print get_image_info(data)
It's clunky, but it worked

I'm writing a CAD application that outputs PDF files using the Cairo graphics library. A lot of the unit testing does not require actually generating the PDF files, such as computing the expected bounding boxes of the objects. However, I want to make sure that the generated PDF files "look" correct after I change the code. Is there an automated way to do this? How can I automate as much as possible? Do I need to visually inspect each generated PDF? How can I solve this problem without pulling my hair out?

(See also update below!)
I'm doing the same thing using a shell script on Linux that wraps
ImageMagick's compare command
the pdftk utility
Ghostscript (optionally)
(It would be rather easy to port this to a .bat Batch file for DOS/Windows.)
I have a few reference PDFs created by my application which are "known good". Newly generated PDFs after code changes are compared to these reference PDFs. The comparison is done pixel by pixel and is saved as a new PDF. In this PDF, all unchanged pixels are painted in white, while all differing pixels are painted in red.
Here are the building blocks:
pdftk
Use this command to split multipage PDF files into multiple singlepage PDFs:
pdftk reference.pdf burst output somewhere/reference_page_%03d.pdf
pdftk comparison.pdf burst output somewhere/comparison_page_%03d.pdf
compare
Use this command to create a "diff" PDF page for each of the pages:
compare \
-verbose \
-debug coder -log "%u %m:%l %e" \
somewhere/reference_page_001.pdf \
somewhere/comparison_page_001.pdf \
-compose src \
somewhereelse/reference_diff_page_001.pdf
Ghostscript
Because of automatically inserted meta data (such as the current date+time), PDF output is not working well for MD5hash-based file comparisons.
If you want to automatically discover all cases which consist of purely white pages, you could also convert to a meta-data free bitmap format using the bmp256 output device. You can do that for the original PDFs (reference and comparison), or for the diff-PDF pages:
gs \
-o reference_diff_page_001.bmp \
-r72 \
-g595x842 \
-sDEVICE=bmp256 \
reference_diff_page_001.pdf
md5sum reference_diff_page_001.bmp
If the MD5sum is what you expect for an all-white page of 595x842 PostScript points, then your unit test passed.
Update:
I don't know why I didn't previously think of generating a histogram output from the ImageMagick compare...
The following is a command pipeline chaining 2 different commands:
the first one is the same as the above compare which generates the 'white pixels are equal, red pixels are differences'-format, only it outputs the ImageMagick internal miff format. It doesn't write to a file, but to stdout.
the second one uses convert to read stdin, generate a histogram and output the result in text form. There will be two lines:
one indicating the number of white pixels
the other one indicating the number of red pixels.
Here it goes:
compare \
reference.pdf \
current.pdf \
-compose src \
miff:- \
| \
convert \
- \
-define histogram:unique-colors=true \
-format %c \
histogram:info:-
Sample output:
56934: (61937, 0, 7710,52428) #F1F100001E1ECCCC srgba(241,0,30,0.8)
444056: (65535,65535,65535,52428) #FFFFFFFFFFFFCCCC srgba(255,255,255,0.8)
(Sample output was generated by using these reference.pdf and current.pdf files.)
I think this type of output is really well suited for automatic unit testing. If you evaluate the two numbers, you can easily compute the "red pixel" percentage and you could even decide to return PASSED or FAILED based on a certain threshold (if you don't necessarily need "zero red" for some reason).

You could capture the PDF as a bitmap (or at least a losslessly-compressed) image, and then compare the image generated by each test with a reference image of what it's supposed to look like. Any differences would be flagged as an error for the test.

The first idea that pops in my head is to use a diff utility. These are generally used to compare texts of documents but they might also compare the layout of the PDF. Using it, you can compare the expected output with the output supplied.
The first result google gives me is this. Altough it is commercial, there might be other free/open source alternatives.

I would try this using xpresser - (https://wiki.ubuntu.com/Xpresser ) You can try to match images to similar images not exact copies - which is the problem in these cases.
I don't know if xpresser is being ctively developed, or if it can be used with stand alone image files (I think so) -- anyway it takes its ideas from teh Sikuli project (which is Java with a Jython front end, while xpresser is Python).

I wrote a tool in Python to validate PDFs for my employer's documentation. It has the capability to compare individual pages to master images. I used a library I found called swftools to export the page to PNG, then used the Python Imaging Library to compare it with the master.
The relevant code looks something like this (this won't run as there are some dependencies on other parts of the script, but you should get the idea):
# exporting
gfxpdf = gfx.open("pdf", self.pdfpath)
if os.path.isfile(pngPath):
os.remove(pngPath)
page = gfxpdf.getPage(pagenum)
img = gfx.ImageList()
img.startpage(page.width, page.height)
page.render(img)
img.endpage()
img.save(pngPath)
return os.path.isfile(pngPath)
# comparing
outPng = os.path.join(outpath, pngname)
masterPng = os.path.join(outpath, "_master", pngname)
if os.path.isfile(masterPng):
output = Image.open(outPng).convert("RGB") # discard alpha channel, if any
master = Image.open(masterPng).convert("RGB")
mismatch = any(x[1] for x in ImageChops.difference(output, master).getextrema())