We Keep Coding

Looking for a Python procedure to extract table information from image

We have paper invoices coming in, which are in paper format. We take images of these invoices, and wish to extract the information contained within the cells of the tabular region(s), and export them as CSV or similar.
The tables include multiple columns, and the cells contain numbers and words.
I have been searching around for ML-based Python procedures to have this performed, expecting this to be a relatively straightforward task (or maybe I'm mistaken), yet not much luck in coming across a procedure.
I can detect the horizontal and vertical lines, and combine them to locate the cells. But retrieving the information contained within the cells seems to be problematic.
Could I please get help?
I followed one procedure from this reference, yet came across an error with "bitnot":
import pytesseract
extract=[]
for i in range(len(order)):
for j in range(len(order[i])):
inside=''
if(len(order[i][j])==0):
extract.append(' ')
else:
for k in range(len(order[i][j])):
side1,side2,width,height = order[i][j][k][0],order[i][j][k][1], order[i][j][k][2],order[i][j][k][3]
final_extract = bitnot[side2:side2+h, side1:side1+width]
final_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2, 1))
get_border = cv2.copyMakeBorder(final_extract,2,2,2,2, cv2.BORDER_CONSTANT,value=[255,255])
resize = cv2.resize(get_border, None, fx=2, fy=2, interpolation=cv2.INTER_CUBIC)
dil = cv2.dilate(resize, final_kernel,iterations=1)
ero = cv2.erode(dil, final_kernel,iterations=2)
ocr = pytesseract.image_to_string(ero)
if(len(ocr)==0):
ocr = pytesseract.image_to_string(ero, config='--psm 3')
inside = inside +" "+ ocr
extract.append(inside)
a = np.array(extract)
dataset = pd.DataFrame(a.reshape(len(hor), total))
dataset.to_excel("output1.xlsx")
The error I get is this:
final_extract = bitnot[side2:side2+h, side1:side1+width]
NameError: name 'bitnot' is not defined`

How to tell if matchTemplate succeeds? [duplicate]

I'm attempting to find an image in another.
im = cv.LoadImage('1.png', cv.CV_LOAD_IMAGE_UNCHANGED)
tmp = cv.LoadImage('e1.png', cv.CV_LOAD_IMAGE_UNCHANGED)
w,h = cv.GetSize(im)
W,H = cv.GetSize(tmp)
width = w-W+1
height = h-H+1
result = cv.CreateImage((width, height), 32, 1)
cv.MatchTemplate(im, tmp, result, cv.CV_TM_SQDIFF)
print result
When I run this, everything executes just fine, no errors get thrown. But I'm unsure what to do from here. The doc says that result stores "A map of comparison results". I tried printing it, but it gives me width, height, and step.
How do I use this information to find whether or not one image is in another/where it is located?

This might work for you! :)
def FindSubImage(im1, im2):
needle = cv2.imread(im1)
haystack = cv2.imread(im2)
result = cv2.matchTemplate(needle,haystack,cv2.TM_CCOEFF_NORMED)
y,x = np.unravel_index(result.argmax(), result.shape)
return x,y
CCOEFF_NORMED is just one of many comparison methoeds.
See: http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html
for full list.
Not sure if this is the best method, but is fast, and works just fine for me! :)

MatchTemplate returns a similarity map and not a location.
You can then use this map to find a location.
If you are only looking for a single match you could do something like this to get a location:
minVal,maxVal,minLoc,maxLoc = cv.MinMaxLoc(result)
Then minLoc has the location of the best match and minVal describes how well the template fits. You need to come up with a threshold for minVal to determine whether you consider this result a match or not.
If you are looking for more than one match per image you need to use algorithms like non-maximum supression.

How to iterate over and download each image in an image collection from the Google Earth Engine python api

I am new to google earth engine and was trying to understand how to use the Google Earth Engine python api. I can create an image collection, but apparently the getdownloadurl() method operates only on individual images. So I am trying to understand how to iterate over and download all of the images in the collection.
Here is my basic code. I broke it out in great detail for some other work I am doing.
import ee
ee.Initialize()
col = ee.ImageCollection('LANDSAT/LC08/C01/T1')
col.filterDate('1/1/2015', '4/30/2015')
pt = ee.Geometry.Point([-2.40986111110000012, 26.76033333330000019])
buff = pt.buffer(300)
region = ee.Feature.bounds(buff)
col.filterBounds(region)
So I pulled the Landsat collection, filtered by date and a buffer geometry. So I should have something like 7-8 images in the collection (with all bands).
However, I could not seem to get iteration to work over the collection.
for example:
for i in col:
print(i)
The error indicates TypeError: 'ImageCollection' object is not iterable
So if the collection is not iterable, how can I access the individual images?
Once I have an image, I should be able to use the usual
path = col[i].getDownloadUrl({
'scale': 30,
'crs': 'EPSG:4326',
'region': region
})

It's a good idea to use ee.batch.Export for this. Also, it's good practice to avoid mixing client and server functions (reference). For that reason, a for-loop can be used, since Export is a client function. Here's a simple example to get you started:
import ee
ee.Initialize()
rectangle = ee.Geometry.Rectangle([-1, -1, 1, 1])
sillyCollection = ee.ImageCollection([ee.Image(1), ee.Image(2), ee.Image(3)])
# This is OK for small collections
collectionList = sillyCollection.toList(sillyCollection.size())
collectionSize = collectionList.size().getInfo()
for i in xrange(collectionSize):
ee.batch.Export.image.toDrive(
image = ee.Image(collectionList.get(i)).clip(rectangle),
fileNamePrefix = 'foo' + str(i + 1),
dimensions = '128x128').start()
Note that converting a collection to a list in this manner is also dangerous for large collections (reference). However, this is probably the most scalable method if you really need to download.

Here is my solution:
import ee
ee.Initialize()
pt = ee.Geometry.Point([-2.40986111110000012, 26.76033333330000019])
region = pt.buffer(10)
col = ee.ImageCollection('LANDSAT/LC08/C01/T1')\
.filterDate('2015-01-01','2015-04-30')\
.filterBounds(region)
bands = ['B4','B5'] #Change it!
def accumulate(image,img):
name_image = image.get('system:index')
image = image.select([0],[name_image])
cumm = ee.Image(img).addBands(image)
return cumm
for band in bands:
col_band = col.map(lambda img: img.select(band)\
.set('system:time_start', img.get('system:time_start'))\
.set('system:index', img.get('system:index')))
# ImageCollection to List
col_list = col_band.toList(col_band.size())
# Define the initial value for iterate.
base = ee.Image(col_list.get(0))
base_name = base.get('system:index')
base = base.select([0], [base_name])
# Eliminate the image 'base'.
new_col = ee.ImageCollection(col_list.splice(0,1))
img_cummulative = ee.Image(new_col.iterate(accumulate,base))
task = ee.batch.Export.image.toDrive(
image = img_cummulative.clip(region),
folder = 'landsat',
fileNamePrefix = band,
scale = 30).start()
print('Export Image '+ band+ ' was submitted, please wait ...')
img_cummulative.bandNames().getInfo()
A reproducible example can you found it here: https://colab.research.google.com/drive/1Nv8-l20l82nIQ946WR1iOkr-4b_QhISu

You could possibly use ee.ImageCollection.iterate() with a function that gets the image and adds it to a list.
import ee
def accumluate_images(image, images):
images.append(image)
return images
for img in col.iterate(accumulate_images, []):
url = img.getDownloadURL(dict(scale=30, crs='EPSG:4326', region=region))
Unfortunately I am not able to test this code as I do not have access to the API, but it might help you arrive at a solution.

I have a similar problem and was not able o solve with presented solutions. Then I have elaborated a sample code for this purpose. It iterates over an image collection in client side, then it is not affected by limitations (server side only) of .map() or .iterate().
It is possible to download the code and see its explanation here
It basically transform the ImageCollection into a list (ic.toList()). Then it performs a standard loop, and for each individual image it is possible to convert it back to ee.Image(list.get(i)), and then process one by one taking all images in the collection.
In your particular case, to download each image, the function to be called within the loop could be: getDOwnloadURL() or getThumbURL():
var url = imgNew.getDownloadURL({
region: geometry,
});
var thumbURL = imgNew.getThumbURL({region: geometry,dimensions: 512, format: 'png'});

Python for x in list basic question

I am trying to create a function which will load a whole lot of images and map them to appropriate names in PyGame. I'm not all that great with python and this really has me stuck. My current code is this:
tile1 = pygame.image.load("/one.bmp")
tile2 = pygame.image.load("/two.bmp")
tile3 = pygame.image.load("/three.bmp")
and it keeps going on for about 20 tiles. The thing is I just found out that I need a lot more and was wondering how I could do this using a for x in y loop. My basic idea was:
tile = ['/one.bmp', '/two.bmp', '/three.bmp']
tilelist = [1,2,3]
for tile in tile:
tilelist[x] = pygame.image.load(tile)
or something like that but I'm not quite there. I was also wondering if it could be done using dictionaries.
Any help would be appreciated, thanks :)

List comprehensions to the rescue.
tiles = ['/one.bmp', '/two.bmp', '/three.bmp']
tilelist = [pygame.img.load(tile) for tile in tiles]
As #isakkarlsson commented,
...or easier(?) tilelist = map(pygame.img.load, tiles)

To load the data
tile = ['/one.bmp', '/two.bmp', '/three.bmp']
imageMap = {}
for t in tile:
imageMap[t] = pygame.img.load(t)
Then you have all the data in a dictionary and can loop through the file names using imageMap.keys() or the index directly into the dictionary to get a particular image.

Python Image Library: How to combine 4 images into a 2 x 2 grid?

I have 4 directories with images for an animation. I would like to take the set of images and generate a single image with the 4 images arranged into a 2x2 grid for each frame of the animation.
My code so far is:
import Image
fluid64 = "Fluid64_half_size/00"
fluid128 = "Fluid128_half_size/00"
fluid512 = "Fluid512_half_size/00"
fluid1024 = "Fluid1024_half_size/00"
out_image = "Fluid_all/00"
for pic in range(1, 26):
blank_image = Image.open("blank.jpg")
if pic < 10:
image_num = "0"+str(pic)
else:
image_num = str(pic)
image64 = Image.open(fluid64+image_num+".jpg")
image128 = Image.open(fluid128+image_num+".jpg")
image512 = Image.open(fluid512+image_num+".jpg")
image1024 = Image.open(fluid1024+image_num+".jpg")
out = out_image + image_num + ".jpg"
blank_image.paste(image64, (0,0)).paste(fluid128, (400,0)).paste(fluid512, (0,300)).paste(fluid1024, (400,300)).save(out)
Not sure why it's not working. I'm getting the error:
Traceback (most recent call last):
File "C:\Users\Casey\Desktop\Image_composite.py", line 24, in <module>
blank_image.paste(image64, (0,0)).paste(fluid128, (400,0)).paste(fluid512, (
ste(fluid1024, (400,300)).save(out)
AttributeError: 'NoneType' object has no attribute 'paste'
shell returned 1
Any help would be awesome. Thanks!

The only problem there is that "paste" does not return an image object - it rather modifies the "blank" image inplace.
So, when the second paste is called (the one that uses the fuild128 image), it tries to be applied on "None" - which is the return value of the first image.
If that is the only problem you are having, just make one paste call per line, like this:
blank_image.paste(image64, (0,0))
blank_image.paste(fluid128, (400,0))
blank_image.paste(fluid512, (0,300))
blank_image.paste(fluid1024, (400,300))
blank_image.save(out)
Although it looks likely you'd need to scale each image so that their format match as well.
And your code for the "image_num" variable is unecessary. Python is really good with strings - just do something like this:
image64 = Image.open(fluid64 + "%02d.jpg" % pic)

You may want to be using something along the lines of :
blank_image = Image.new("RGB", (800, 600))
This will create a new area in memory in which you can generate your image. You should then be able to paste you images into that.
Then you'll need to save it out again later on with:
blank_image.save("blank.jpg")

Read the error message:
AttributeError: 'NoneType' object has no attribute 'paste'
This means you tried to call .paste on something that was of type NoneType, i.e. on the None object.
Image.paste returns None. You can't "chain" together calls like that except when the functions are specifically designed to support it, and Image.paste is not. (Support for this sort of thing is accomplished by having the function return self. You get an error that talks about NoneType because the function is written not to return anything, and everything in Python returns None by default if nothing else is returned explicitly.) This is considered Pythonic: methods either return a new value, or modify self and return None. Thus, so-called "fluent interfaces" are not used when the functions have side effects - Pythonistas consider that harmful. Returning None is a warning that the function has side effects. :)
Just do four separate .paste calls.

Tiling figures in a 2-by-2 grid would be easy to achieve with the append_images function defined in this reply
https://stackoverflow.com/a/46623632/8738113
For example:
img1 = append_images([image64, image128], direction='horizontal')
img2 = append_images([image512, image1024], direction='horizontal')
final = append_images([img1, img2], direction='vertical')
final.save("Fluid_all/00.jpg")

Unlike PIL APIs copy, crop, resize or rotate which return an Image object, paste returns None which prevents chained method calls. Not so convenient API design.