Cleaning Image for Reading Numbers Pytesseract - python

I'm trying to read some entries from a table of data filled with a name and then columns of numbers. Here's the original picture:
Between binarizing, converting to black/white, and just inverting, I found that inverting the image led to the best results.
image = PIL.ImageOps.invert(image
This lets me process roughly 90%+ of the columns I have as I scroll down to more images, but I'm still failing on a bunch of them. Sometimes, the parenthesis in the columns merge the two numbers I have in each column. Is there any way I can fix issues with parenthesis being mixed with numbers, or maybe remove all of the green text?

Resizing the image seemed to be the option that fixed the problems.
image = pyautogui.screenshot(region=(550, 354, 964, 552))
width, height = image.size
image = image.resize((args.resize*width, args.resize*height))
I resized to at least 3x the original size. I guess that increased the distance between characters, making it simpler to recognize the end of one digit and the parenthesis that followed.
Alternatively, the following is an even larger improvement:
image = cv2.imread(output)
image = cv2.bitwise_not(image)
image = cv2.resize(image, None, fx=1.5, fy=1.7,
interpolation=cv2.INTER_CUBIC) # scale
cv2.imwrite(output, image
The scaling is not linear and if you skew it a bit it works better.

Related

Using PIL to draw individual pixels, but the image is blurry

I am trying to create an image made up of coloured squares. I only need each square to be one pixel large, as it is just a single block colour. However, when I use this code, the image generated is extremely blurry. Is there anyway to make the boarders sharp?
def fancycolnw2(seq,m):
data=numbwall(seq,m)
#print(data)
for i in range(len(data)):
for j in range(len(data[i])):
if data[i][j]==' ':
data[i][j]=-1
im = Image.new('RGBA', (len(data[0]),len(data))) # create the Image of size 1 pixel
#print(data)
for i in range(len(data)-1):
for j in range(len(data[i])-1):
#print(i,j)
if data[i][j]==-1:
im.putpixel((j,i), ImageColor.getcolor('black', 'RGBA'))
if data[i][j]==0:
#print('howdy')
im.putpixel((j,i), ImageColor.getcolor('red', 'RGBA'))
if data[i][j]==1:
im.putpixel((j,i), ImageColor.getcolor('blue', 'RGBA'))
if data[i][j]==2:
im.putpixel((j,i), ImageColor.getcolor('grey', 'RGBA'))
im.show()
im.save('simplePixel.png') # or any image format
The result I get looks like this:
Image
It is the correct image, I just wish the boundaries between pixels were sharp. Any help would be greatly appreciated!
The image is perfectly sharp, but rather small. I suspect that you are "zooming in" to view it clearer, and that whatever program you are zooming with is filtering the image, because with most images this looks better. You need to find a viewing program that uses "nearest neighbour" resampling when zooming in, or generate a larger image to start with, for example by setting a 4-by-4 pixel block rather than individual pixels.
(Also, the code says "# or any other image format". Don’t use JPEG for this, as the lossy compression will likely wreck your image.)

OpenCV Python Stitch unordered images

I currently have built a stitching program that is able to stitch any number of images however they have to be of specific order which is a problem for my specific image sets (working with 1452 images). My goal is to create an unordered stitcher program. I am convinced that the issue takes place when I am physically (well virtually) stitching the images to one another.
Here is code for my stitching (assume keypoints found are accurate as well as homography)
def stitchMatches(self,image1,image2,homography):
#gather x and y axis of images that will be stitched
height1, width1 = image1.shape[0], image1.shape[1]
height2, width2 = image2.shape[0], image2.shape[1]
#create blank image that will be large enough to hold stitched image
blank_image = np.zeros(((width1 + width2),(height1 + height2),3),np.uint8)
#stitch image two into the resulting image while using blank_image
#to create a large enough frame for images
result = cv2.warpPerspective((image1),homography,blank_image.shape[0:2])
#numpy notation for slicing a matrix together allows you to see the image
result[0:image2.shape[0], 0:image2.shape[1]] = image2
return result
I believe the issue is with these two lines.
result = cv2.warpPerspective((image1),homography,blank_image.shape[0:2])
result[0:image2.shape[0], 0:image2.shape[1]] = image2
When I run the code on an ordered image (image 1 is north east most)set this is the result.
However when I run the images in the reverse order I get this result.
To my understanding, this is because I have set the resulting image shape to map to image 2 but haven't been able to develop a work around.
Here is a photo of multiple images showing the same issue of the images getting cropped out.
Any help is appreciated.

From Raw binary image data to PNG in Python

After searching for a few hours, I ended up on this link. A little background information follows.
I'm capturing live frames of a running embedded device via a hardware debugger. The captured frames are stored as raw binary files, without headers or format. After looking at the above link and understanding, albeit perfunctorily, the NumPY and Matplotlib, I was able to convert the raw binary data to an image successfully. This is important because I'm not sure if the link to the raw binary file will help any one.
I use this code:
import matplotlib.pyplot as plt # study documentation
import numpy as np # " "
iFile = "FramebufferL0_0.bin" # Layer-A
shape = (430, 430) # length and width of the image
dtype = np.dtype('<u2') # unsigned 16 bit little-endian.
oFile = "FramebufferL0_0.png"
fid = open(iFile, 'rb')
data = np.fromfile(fid, dtype)
image = data.reshape(shape)
plt.imshow(image, cmap = "gray")
plt.savefig(oFile)
plt.show()
Now, the image I'm showing is black and white because the color map is gray-scale (right?). The actual captured frame is NOT black and white. That is, the image I see on my embedded device is "colorful".
My question is, how can I calculate actual color of each pixel from the raw binary file? Is there a way I can get the actual color map of the image from the raw binary? I looked into this example and I'm sure that, if I'm able to calculate the R, G and B channels (and Alpha too), I'll be able to recreate the exact image. An example code would be of much help.
An RGBA image has 4 channels, one for each color and one for the alpha value. The binary file seems to have a single channel, as you don't report an error when performing the data.reshape(shape) operation (the shape for the corresponding RGBA image would be (430, 430, 4)).
I see two potential reasons:
The image actual does have colour information but when you are grabbing the data you are only grabbing one of the four channels.
The image is actually a gray-scale image, but the embedded device shows a pseudocolor image, creating the illusion of colour information. Without knowing what the colourmap is being used, it is hard to help you, other than point you towards matplotlib.pyplot.colormaps(), which lists all already available colour maps in matplotlib.
Could you
a) explain the exact source / type of imaging modality, and
b) show a photo of the output of the embedded device?
PS: Also, at least in my hands, the pasted binary file seems to have a size of 122629, which is incongruent with an image shape of (430,430).

Analyzing pixel RGB values with PIL

I'm working on a project where I need to find the RGB values of each pixel in a picture. How could I do this using PIL? I know that Pillow is better, but since I only need to do this one thing I thought I could just use PIL. If this won't work as well please tell me.
from PIL import Image
img = Image.open("filename.png")
pixels = img.load()
#get the B value of the pixel at x=23, y=42
print pixels[23, 42][2]
The previous answer is a good solution but just suggesting another way which is one line:
from scipy import misc;
imgData = misc.imread('./image.png');
You can then easily get the colors at every pixels you need.
Kevin got it pretty much spot on, you can also use getdata() to return a list of tuples.
I may have got this totally wrong, but load() might work better if you need particular pixels, and getdata() if you need all of them. Also, it's a good idea to convert to RGB if it's just a normal image, I've had errors before by not doing that.
image = Image.open('filename').convert('RGB')
width, height = image.size
#Get pixels in a list of tuples
pixels = image_input.getdata()
#If you need a flat list containing all the colours
bytes = [j for i in pixels for j in i]
If you needed to do stuff to the pixels and rebuild the image after, that's where the image size comes in useful.

Working with truncated images with PIL

I am trying to get the Python 2.7 PIL Library to work with JPEG images that are only available as a stream coming from a HDD image and are not complete.
I have set the option:
ImageFile.LOAD_TRUNCATED_IMAGES = True
And load the stream as far as it is available (or better said: as far as I am 100% sure that this data is still a image, not some other file type). I have tested different things and as far as I can tell (for JPEGs) PIL only accepts it as a valid JPEG Image if it finds the 0xFFDA (Start of Scan Marker). This is a short example of how I load the data:
from PIL import Image
from StringIO import StringIO
ImageFile.LOAD_TRUNCATED_IMAGES = True
with open("/path/to/image.raw", 'rb') as fp:
fp.seek("""jump to position in image where JPEG starts""")
data = fp.read("""number of bytes I know that those belong to that jpeg""")
img = Image.open(StringIO(data)) # This would throw exception if the data does
# not contain the 0xffda marker
pixel = img.load() # Would throw exception if LOAD_TRUNCATED_IMAGES = false
height,width = img.size
for i in range(height):
for j in range(width):
print pixel[i,j]
On the very last line I expected (or hoped) to see at least the read pixel data to be displayed. But for every pixel it returns (0,0,0).
The Question: Is what I am trying here not possible with PIL?
Some weeks ago I tried the same with a image file I truncated myself, simply by cutting data from it with an editor. It worked for the pixel-data that was available. As soon as it reached a pixel that I cut off, the program threw an exception (I will try this again later today to make sure that I am not remembering wrong).
If somebody is wondering why I am doing this: I need to make sure that the image/picture inside that hdd image is in consecutive blocks/clusters and is not fragmented. To make sure of this I wanted to use pixel matching.
EDIT:
I have tried it again and this is what I have seen.
I opened a truncated image in GIMP and it showed me a few pixel lines in the upper part, but PIL was not able to at least give me the RGB values of those pixels. It always returns (0,0,0).
I made the image slightly bigger such that the lower 4/5 of the image was not visible, but that was enough for PIL to show me the RGB values that were available. Everything else was (0,0,0).
I am still not 100% sure whether PIL can show me the RGB values, even if only view pixel-data is available.
I would try it with an uncompressed format like TGA. JPG being a compressed format may not make any sense to extract pixels from an incomplete image. JPEG actually stores the parameters for equations that describe the image, not pixel values. When you query a JPEG for a pixel value it evaluates the equations at that point and returns the result.
I have the same problem with Pillow==9.2.0
Let's downgrade to Pillow==8.3.2 and it works.
I don't really know about streaming, but I think that you simply cannot access rgb value the way you do.
Try:
rgb_im = img.convert('RGB')
r, g, b = rgb_im.getpixel((i, j))

Categories