We Keep Coding

Python: SVG to PNG converting issue

UPDATE: I tried increasing size in the chess.svg.board and it somehow cleared all the rendering issues at size = 900 1800
I tried using the svglib and reportlab to make .png files from .svg, and here is how the code looks:
import sys
import chess.svg
import chess
from svglib.svglib import svg2rlg
from reportlab.graphics import renderPM
board = chess.Board("rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR")
drawing = chess.svg.board(board, size=350)
f = open('file.svg', 'w')
f.write(drawing)
drawing = svg2rlg("file.svg")
renderPM.drawToFile(drawing, "file.png", fmt="png")
If you try to open file.png there is a lot of missing parts of the image, which i guess are rendering issues. How can you fix this?
Sidenote: also getting a lot of 'x_order_2: colinear!' messages when running this on a discord bot, but I am not sure if this affects anything yet.

THIS!! I am having the same error with the same libraries... I didn't find a solution but just a workaround which probably won't help too much in your case, where the shapes generating the bands are not very sparse vertically.
I'll try playing with the file dimensions too, but so far this is what I got. Note that my svg consists of black shapes on a white background (hence the 255 - x in the following code)
Since the appearance of the bands is extremely random, and processing the same file several times in a row produces different results, I decided to take advantage of randomness: what I do is I export the same svg a few times into different pngs, import them all into a list and then only take those pixels that are white in all the exported images, something like:
images_files = [my_convert_function(svgfile=file, index=i) for i in range(3)]
images = [255 - imageio.imread(x) for x in images_files]
result = reduce(lambda a,b: a & b, images)
imageio.imwrite(<your filename here>, result)
[os.remove(x) for x in images_files]
where my_convert_function contains your same svg2rlg and renderPM.drawToFile, and returns the name of the png file being written. The index 'i' is to save several copies of the same png with different names.
It's some very crude code but I hope it can help other people with the same issue

The format parameter has to be in uppercase
renderPM.drawToFile(drawing, "file.png", fmt="PNG")

detect and select nonblack images in a folder

I am currently working on a media project. We've shooted looong clips, mainly dark if not black. I have decomposed these clips into their frames (>500k single frames) and put them in some folders. Now, my goal is to find out and select those frames that are not black or mainly dark: it's around a thousand out of the total.
This seems a job that a simple Python script can handle without too much effort. I know that scikit-image is quite common to work with images, but don't know how to come up with a script that does the job neatly. I have some experience with scientific programming but this with images manipulation is a bit out of my field.
For example, this image should be reported as black and thus ignored, while this other one, although in low light, should be kept as good.
Ideally, it would be optimal to have a script that uses one or more criteria to determine if an image is totally dark or not, and in the latter case put it into another folder for human (me) inspection.
Any help is exteremely appreciated!

You can get the mean of each image very simply without writing any code using ImageMagick which is available for Windows, Linux and macOS.
Like this:
magick identify -format '%[fx:mean*255] %f\r\n' black.jpg
1.01936 black.jpg
and:
magick identify -format '%[fx:mean*255] %f\r\n' nonblack.jpg
1.72921 nonblack.jpg
To improve performance, I would use GNU Parallel on macOS or Linux, but in Windows, I would open a new command prompt for each directory and run several scripts in parallel, or start one script processing all the files ending in 0 or 1, a second one processing files ending in 2 or 3, a third one processing files ending in 4,5 or 6 and a final one processing files ending in 7,8 or 9.
If I was doing it in Python I would use a multiprocessing pool to speed things up, by the way.

Opencv is enough to solve this problem.
use np.mean(image, axis=2) to get mean of different channels, then you can easily check the black ones.

As pointed out in the replies, taking a 'mean' of the image helped. After reading in the image, I compute np.mean(img, axis = 2).mean() so that I have the mean of the three colour channels. If this mean is low (<2) then the image is discarded, otherwise the file is copied to another folder.
The code is not really time efficient as it takes ~3 hours for 200k files, but does the trick!

You'll probably want to use PIL (Python Image Library).
I did a quick search for code that calculates the average of an image and found this snippet:
Image Average Color
import Image
def get_average_color((x,y), n, image):
""" Returns a 3-tuple containing the RGB value of the average color of the
given square bounded area of length = n whose origin (top left corner)
is (x, y) in the given image"""
r, g, b = 0, 0, 0
count = 0
for s in range(x, x+n+1):
for t in range(y, y+n+1):
pixlr, pixlg, pixlb = image[s, t]
r += pixlr
g += pixlg
b += pixlb
count += 1
return ((r/count), (g/count), (b/count))
image = Image.open('test.png').load()
r, g, b = get_average_color((24,290), 50, image)
print r,g,b
Maybe you could just iterate through all of the images in your folder and log (or copy) ones that are above a certain values.
There's probably a more elegant way to do this using PIL but maybe this will get you started.
Hope it helps!

Python: What's the fastest way to load a video file into memory?

First some background
I am trying to write my own set of tools for video analysis, mainly for detecting render errors like flashing frames and possibly some other stuff in the future.
The (obvious) goal is to write a script, that is faster and more accurate than me watching the file in real time.
Using OpenCV, I have something that looks like this:
import cv2
vid = cv2.VideoCapture("Video/OpenCV_Testfile.mov", cv2.CAP_FFMPEG)
width = 1024
height = 576
length = vid.get(cv2.CAP_PROP_FRAME_COUNT)
for f in range(length):
blue_values = []
vid.set(cv2.CAP_PROP_POS_FRAMES, f)
is_read, frame = vid.read()
if is_read:
for row in range(height):
for col in range(width):
blue_values.append(frame[row][col][0])
print(blue_values)
vid.release()
This just prints out a list of all blue values of every frame.
- Just for simplicity (My actual script compares a few values across each frame and only saves the frame number when all are equal)
Although this works, it is not a very fast operation. (Nested loops, but most important, the read() method has to be called for every frame, which is rather slow.
I tried to use multiprocessing but basically ended up having the same crashes as described here:
how to get frames from video in parallel using cv2 & multiprocessing in python
I have a 20s long 1024x576#25fps Testfile which performs as follows:
mov, ProRes: 15s
mp4, h.264: 30s (too slow)
My machine is capable of playing back h.264 in 1920x1080#50fps with mplayer (which uses ffmpeg to decode). So, I should be able to get more out of this. Which leads me to
my Question
How can I decode a video and simply dump all pixel values into a list for further (possibly multithreaded) operations? Speed is really all that matters. Note: I'm not fixated on OpenCV. Whatever works best.
Thanks!

moviepy resize not working in some sizes

I have a mp4 video of 720x1280, and I want it in different sizes like:
0.66%, 0.5% and 0.33%.
For each of these sizes I use:
clip = mp.VideoFileClip(file)
clip_resized1 = clip.resize(height=int(clip.h * float(0.66666)))
clip_resized1.write_videofile(name + '-2x' + ext)
I do this for each of the sizes but some of them work and some not. The 0.66 not works, just like the 0.33. The 0.5% works just fine.
It creates the files for every size, but they are corrupt, and can't open them (except 0.5 as I said, which works ok).
Any clue on this? Any better solution for resizing in Python?

The issue I believe is that most video player cannot play mp4 if one of the dimensions of the clip is an odd number. For instance 720x1280 works on all players but 721x1280 will only play on some video players like VLC.
So make sure that clip.h and clip.w are both even before writing to a video file. There are several ways you can do that, either indicate the new dimensions of the clip yourself, like clip.resize((844, 476)), or redimension the clip of 66% and add a 1px black margin at the top, like clip.resize(0.66).margin(top=1)

Working with truncated images with PIL

I am trying to get the Python 2.7 PIL Library to work with JPEG images that are only available as a stream coming from a HDD image and are not complete.
I have set the option:
ImageFile.LOAD_TRUNCATED_IMAGES = True
And load the stream as far as it is available (or better said: as far as I am 100% sure that this data is still a image, not some other file type). I have tested different things and as far as I can tell (for JPEGs) PIL only accepts it as a valid JPEG Image if it finds the 0xFFDA (Start of Scan Marker). This is a short example of how I load the data:
from PIL import Image
from StringIO import StringIO
ImageFile.LOAD_TRUNCATED_IMAGES = True
with open("/path/to/image.raw", 'rb') as fp:
fp.seek("""jump to position in image where JPEG starts""")
data = fp.read("""number of bytes I know that those belong to that jpeg""")
img = Image.open(StringIO(data)) # This would throw exception if the data does
# not contain the 0xffda marker
pixel = img.load() # Would throw exception if LOAD_TRUNCATED_IMAGES = false
height,width = img.size
for i in range(height):
for j in range(width):
print pixel[i,j]
On the very last line I expected (or hoped) to see at least the read pixel data to be displayed. But for every pixel it returns (0,0,0).
The Question: Is what I am trying here not possible with PIL?
Some weeks ago I tried the same with a image file I truncated myself, simply by cutting data from it with an editor. It worked for the pixel-data that was available. As soon as it reached a pixel that I cut off, the program threw an exception (I will try this again later today to make sure that I am not remembering wrong).
If somebody is wondering why I am doing this: I need to make sure that the image/picture inside that hdd image is in consecutive blocks/clusters and is not fragmented. To make sure of this I wanted to use pixel matching.
EDIT:
I have tried it again and this is what I have seen.
I opened a truncated image in GIMP and it showed me a few pixel lines in the upper part, but PIL was not able to at least give me the RGB values of those pixels. It always returns (0,0,0).
I made the image slightly bigger such that the lower 4/5 of the image was not visible, but that was enough for PIL to show me the RGB values that were available. Everything else was (0,0,0).
I am still not 100% sure whether PIL can show me the RGB values, even if only view pixel-data is available.

I would try it with an uncompressed format like TGA. JPG being a compressed format may not make any sense to extract pixels from an incomplete image. JPEG actually stores the parameters for equations that describe the image, not pixel values. When you query a JPEG for a pixel value it evaluates the equations at that point and returns the result.

I have the same problem with Pillow==9.2.0
Let's downgrade to Pillow==8.3.2 and it works.

I don't really know about streaming, but I think that you simply cannot access rgb value the way you do.
Try:
rgb_im = img.convert('RGB')
r, g, b = rgb_im.getpixel((i, j))