We Keep Coding

Why isn't Python writing elements from one 3-D list to another?

I'm trying to create a program that creates an average out of an image by looping through each RGBA value in two images to average them out and create one composite image, but the right value isn't being written to my list comp_img, which contains all the new image data.
I'm using these 256x256 images for debugging.
But it just creates this as output:
While this is the composite color, the 64 gets wiped out entirely. Help is very much appreciated, thank you.
from PIL import Image
import numpy as np
from math import ceil
from time import sleep
red = Image.open("64red.png")
grn = Image.open("64green.png")
comp_img = []
temp = [0,0,0,0] #temp list used for appending
#temp is a blank pixel template
for i in range(red.width):
comp_img.append(temp)
temp = comp_img
#temp should now be a row template composed of pixel templates
#2d to 3d array code go here
comp_img = []
for i in range(red.height):
comp_img.append(temp)
reddata = np.asarray(red)
grndata = np.asarray(grn)
reddata = reddata.tolist() #its uncanny how easy it is
grndata = grndata.tolist()
for row, elm in enumerate(reddata):
for pxl, subelm in enumerate(elm):
for vlu_index, vlu in enumerate(subelm):
comp_img[row][pxl][vlu_index] = ceil((reddata[row][pxl][vlu_index] + grndata[row][pxl][vlu_index])/2)
#These print statements dramatically slowdown the otherwise remarkably quick program, and are solely for debugging/demonstration.
output = np.array(comp_img, dtype=np.uint8) #the ostensible troublemaker
outputImg = Image.fromarray(output)
outputImg.save("output.png")

You could simply do
comp_img = np.ceil((reddata + grndata) / 2)
This gives me
To get correct values it needs to work with 16bit values - because for uint8 it works only with values 0..255 and 255+255 gives 254 instead of 510 (it calculates it modulo 256 and (255+255) % 256 gives 254)
reddata = np.asarray(red, dtype=np.uint16)
grndata = np.asarray(grn, dtype=np.uint16)
and then it gives
from PIL import Image
import numpy as np
red = Image.open("64red.png")
grn = Image.open("64green.png")
reddata = np.asarray(red, dtype=np.uint16)
grndata = np.asarray(grn, dtype=np.uint16)
#print(reddata[128,128])
#print(grndata[128,128])
#comp_img = (reddata + grndata) // 2
comp_img = np.ceil((reddata + grndata) / 2)
#print(comp_img[128,128])
output = np.array(comp_img, dtype=np.uint8)
outputImg = Image.fromarray(output)
outputImg.save("output.png")

You really should work with just NumPy arrays and functions, but I'll explain the bug here. It's rooted in how you make the nested list. Let's look at the first level:
temp = [0,0,0,0] #temp list used for appending
#temp is a blank pixel template
for i in range(red.width):
comp_img.append(temp)
At this stage, comp_img is a list with size red.width whose every element/pixel references the same RGBA-list [0,0,0,0]. I don't just mean the values are the same, it's one RGBA-list object. When you edit the values of that one RGBA-list, you edit all the pixels' colors at once.
Just fixing that step isn't enough. You also make the same error in the next step of expanding comp_img to a 2D matrix of RGBA-lists; every row is the same list.
If you really want to make a blank comp_img first, you should just make a NumPy array of a numerical scalar dtype; there you can guarantee every element is independent:
comp_img = np.zeros((red.height, red.width, 4), dtype = np.uint8)
If you really want a nested list, you have to properly instantiate (make new) lists at every level for them to be independent. A list comprehension is easy to write:
comp_img = [[[0,0,0,0] for i in range(red.width)] for j in range(red.height)]

how to iterate through names of lists

I have multiple lists (bit0, bit1, bit2, etc), and I want to iterate through them in a loop like this:
for i in range(3):
view_image(bit1[i])
view_image(bit2[i])
view_image(bit3[i])
view_image(bit4[i])
view_image(bit5[i])
How can I avoid repetition of the view_image statements by constructing the name of its argument inside the loop?
Something like this: view_image('bit' + str(i+1)[i])
These lists represent collections of images as numpy arrays. So for example, bit1[0] is the first image in the bit1 collection.

I would it do like
arrays = (bit1, bit2, bit3, bit4, bit5)
for i in range(3):
for arr in arrays:
view_image(arr[i])
As you are storing several images of different resolution, and have a maybe runtime-determined number, you can resort to the following:
For creation of the list, you can have:
resolutions = ('low', 'medium', 'high', 'hq')
images = {} # empty dict
for res in resolutions:
if pic_present(res):
images[res] = get_image(res)
# Now you can proceed as above:
for i in range(3):
for res, img in images.iteritems():
announce(res, i) # sth. like print "Showing component", i, "of image", res
view_image(img[i])
If the number of images is always the same, you can use a list:
images = [] # empty list
for index, res in enumerate(resolutions):
images[index] = get_image(res)
# Now you can proceed as above:
for i in range(3):
for index, img in enumerate(images):
announce(resolutions[index], i) # sth. like print "Showing component", i, "of image", res
view_image(img[i])
The same thing a little bit different, if you have 256 levels of resolution, unnamed. In this case, you just replace
resolutions = ('low', 'medium', 'high', 'hq')
with
resolutions = range(256) # gives you the numbers of 0 to 255 to iterate over
and proceed as above. No need to hassle aroung with 256 different variable names, but the data is neatly put into a dict or list and that's it.

If possible, at the beginning set it up as
bit = [[],[],[],[],[]]
and build up each part of bit. Then you can loop through as
for i in xrange(3):
for bitarray in bit:
view_image(bitarray)
If they really can't be put into a list of lists that you can enumerate over, then what glglgl suggested looks best.

This is the answer I was looking for (thanks to tmrlvi):
for i in range(3):
for k in range(16):
view_image(locals()['bit'+str(k+1)][i])
I looked at this blog post:
http://stupidpythonideas.blogspot.de/2013/05/why-you-dont-want-to-dynamically-create.html
but I still don't understand why this code would be a bad idea in my case.
If you can explain what can go wrong, I would appreciate it!

Flipping Images

I am trying to complete an image editing task in my learning Python book. I need help with the horizontal flip.
The instructions are: Write a function called "flip_horizontal" which will flip the picture horizontally. That is, the pixel that is on the far right end of the row ends up on the far left of the row and vice versa (remember to preserve RGB order!).
My code does not flip the image horizontally when I open it. Also, how can I write my effects to different files (use the original file and apply one function the original file and output it, and then apply another function to the original file and output it to another file). Please, keep in mind that I am only 11 years old and have a very basic understanding of Python and image editing, it is just an interest of mine.
class PPM(object):
def __init__(self, infile, outfile):
self.infile=infile
self.outfile=outfile
#Read in data of image
data= open(self.infile,"r")
datain=data.read()
splits=datain.split()
#Header info
self.type=splits[0]
self.columns=splits[1]
self.row=splits[2]
self.colour=splits[3]
self.pixels=splits[4:]
def negate_red(self):
for b in range (0, (len(self.pixels)) , 3):
remainder=255-self.colour
def writetofile(self):
dataout= open(self.outfile,"w")
dataout.write(self.type +"\n" + self.columns + "\n" + self.row +"\n"+ self.colour +"\n"+ " ".join (self.pixels))
def grey_scale(self):
if int(self.columns) > 1000:
return "ERROR!! Number of columns is larger than what can be held in a buffer."
else:
for b in range(0, (len(self.pixels)) , 3):
sum = int(self.pixels[b]) + int(self.pixels[b+1]) + int(self.pixels[b+2])
avg = int(sum/3)
self.pixels[b] = str(avg)
self.pixels[b+1] = str(avg)
self.pixels[b+2] = str(avg)
def flatten_red(self):
for colour in range (0,len(self.pixels),3):
self.pixels [colour]=str(0)
#Create a 2d lists with the smaller lists containing the rgb values and append lists of lists
def horizontal_flip(self):
if int(self.columns) > 1000:
return "ERROR!! Number of columns is larger than what can be held in a buffer."
else:
temp_list = []
for b in range(int(self.row)):
column_list = []
column_list += self.pixels[0:int(self.columns) * 3]
self.pixels = self.pixels[int(self.columns) * 3 : ]
temp_list.append(column_list)
#print temp_list
new_list = []
for i in range(int(len(temp_list))):
new_list.append (temp_list[i][0])
new_list.append (temp_list[i][1])
new_list.append (temp_list[i][2])
temp_list[i] = temp_list[i][::-1]
sample= PPM("cake.ppm", "Replica.ppm")
sample.flatten_red()
sample.horizontal_flip()
sample.greyscale()
sample.negate_red()

Imagine a row of pixels.
i.imgur.com/1iZesZL.jpg
Now, what we want to do is to flip it so that the right-most pixel is on the left-most place, right?
So if we have the pixel on the far-left with the coordinates (x,y) then the pixel on the far-right has the coordinates (x+n, y) where n = the width of the picture in pixels.
i.imgur.com/EE7Qj5r.jpg
Now, a horizontal flip would look like this, right?
i.imgur.com/fbNLCuX.jpg
So what we do is we go from the far right and the far left, swap the values of the current pixels and go one step to the right and one step to the left until they meet in the middle.
In pseudo-code this might look something like this:
n = width
x = 0
y = whatever row we're on currently
while n != width/2
temporary = (x,y) # (x,y) refers to a specific pixel in the picture
(x,y) = (n, y)
(n, y) = temporary
n = n-1
x = x+1
Do you think that's enough to solve the rest yourself? Wouldn't want to take the fun out of it :-)

Are you really 11 years old?
It looks like each element of your temp_list is a column of the image to reverse the order of the columns you just have to do temp_list = temp_list[::-1], but you're doing temp_list[i] = temp_list[i][::-1] which I think flips the image up-down (I might have it backwards though). Either way, once you get the flip, you'll need to flatten the image again and replace self.pixels, something like:
pixels = []
for column in temp_list:
pixels.extend(column)
self.pixels = pixels
You're not doing much with new_list, I don't think you need it. Also if you want to save the image to different files, take the filename as an argument to writetofile, if you do that you won't need to have it in __init__, so something like:
class PPM(object):
def __init__(self, infile):
self.infile=infile
# More code here
def writetofile(self, outfile):
dataout = open(outfile,"w")
dataout.write(self.type +"\n" + self.columns + "\n" + self.row +"\n"+ self.colour +"\n"+ " ".join (self.pixels))
dataout.close()
sample = PPM("cake.ppm")
sample.horizontal_flip()
sample.writetofile("Replica1.ppm")
sample.negate_red()
sample.writetofile("Replica2.ppm")

Maybe not for you since you want to practice but I came here looking for a solution to the same Problem after Research I found this and wanted to share the following:
OpenCV provides a function to flip an  image.
void flip(array src, array dst, int flipCode)
Flips a 2D array around vertical, horizontal or both axes.
Parameters:
src – The source Array
dst – The destination array; will have the same size and same type as src
flipCode – Specifies how to flip the array: 0 means flipping around the x-axis, positive (e.g., 1) means flipping around y-axis, and negative (e.g., -1) means flipping around both axes.The function flip flips the array in one of three different ways (row and column indices are 0-based).
Example code:
cv.flip(original_image,flip_image,1);

Size-Incremental Numpy Array in Python

I just came across the need of an incremental Numpy array in Python, and since I haven't found anything I implemented it. I'm just wondering if my way is the best way or you can come up with other ideas.
So, the problem is that I have a 2D array (the program handles nD arrays) for which the size is not known in advance and variable amount of data need to be concatenated to the array in one direction (let's say that I've to call np.vstak a lot of times). Every time I concatenate data, I need to take the array, sort it along axis 0 and do other stuff, so I cannot construct a long list of arrays and then np.vstak the list at once.
Since memory allocation is expensive, I turned to incremental arrays, where I increment the size of the array of a quantity bigger than the size I need (I use 50% increments), so that I minimize the number of allocations.
I coded this up and you can see it in the following code:
class ExpandingArray:
__DEFAULT_ALLOC_INIT_DIM = 10 # default initial dimension for all the axis is nothing is given by the user
__DEFAULT_MAX_INCREMENT = 10 # default value in order to limit the increment of memory allocation
__MAX_INCREMENT = [] # Max increment
__ALLOC_DIMS = [] # Dimensions of the allocated np.array
__DIMS = [] # Dimensions of the view with data on the allocated np.array (__DIMS <= __ALLOC_DIMS)
__ARRAY = [] # Allocated array
def __init__(self,initData,allocInitDim=None,dtype=np.float64,maxIncrement=None):
self.__DIMS = np.array(initData.shape)
self.__MAX_INCREMENT = maxIncrement
if self.__MAX_INCREMENT == None:
self.__MAX_INCREMENT = self.__DEFAULT_MAX_INCREMENT
# Compute the allocation dimensions based on user's input
if allocInitDim == None:
allocInitDim = self.__DIMS.copy()
while np.any( allocInitDim < self.__DIMS ) or np.any(allocInitDim == 0):
for i in range(len(self.__DIMS)):
if allocInitDim[i] == 0:
allocInitDim[i] = self.__DEFAULT_ALLOC_INIT_DIM
if allocInitDim[i] < self.__DIMS[i]:
allocInitDim[i] += min(allocInitDim[i]/2, self.__MAX_INCREMENT)
# Allocate memory
self.__ALLOC_DIMS = allocInitDim
self.__ARRAY = np.zeros(self.__ALLOC_DIMS,dtype=dtype)
# Set initData
sliceIdxs = [slice(self.__DIMS[i]) for i in range(len(self.__DIMS))]
self.__ARRAY[sliceIdxs] = initData
def shape(self):
return tuple(self.__DIMS)
def getAllocArray(self):
return self.__ARRAY
def getDataArray(self):
"""
Get the view of the array with data
"""
sliceIdxs = [slice(self.__DIMS[i]) for i in range(len(self.__DIMS))]
return self.__ARRAY[sliceIdxs]
def concatenate(self,X,axis=0):
if axis > len(self.__DIMS):
print "Error: axis number exceed the number of dimensions"
return
# Check dimensions for remaining axis
for i in range(len(self.__DIMS)):
if i != axis:
if X.shape[i] != self.shape()[i]:
print "Error: Dimensions of the input array are not consistent in the axis %d" % i
return
# Check whether allocated memory is enough
needAlloc = False
while self.__ALLOC_DIMS[axis] < self.__DIMS[axis] + X.shape[axis]:
needAlloc = True
# Increase the __ALLOC_DIMS
self.__ALLOC_DIMS[axis] += min(self.__ALLOC_DIMS[axis]/2,self.__MAX_INCREMENT)
# Reallocate memory and copy old data
if needAlloc:
# Allocate
newArray = np.zeros(self.__ALLOC_DIMS)
# Copy
sliceIdxs = [slice(self.__DIMS[i]) for i in range(len(self.__DIMS))]
newArray[sliceIdxs] = self.__ARRAY[sliceIdxs]
self.__ARRAY = newArray
# Concatenate new data
sliceIdxs = []
for i in range(len(self.__DIMS)):
if i != axis:
sliceIdxs.append(slice(self.__DIMS[i]))
else:
sliceIdxs.append(slice(self.__DIMS[i],self.__DIMS[i]+X.shape[i]))
self.__ARRAY[sliceIdxs] = X
self.__DIMS[axis] += X.shape[axis]
The code shows considerably better performances than vstack/hstack several random sized concatenations.
What I'm wondering about is: is it the best way? Is there anything that do this already in numpy?
Further it would be nice to be able to overload the slice assignment operator of np.array, so that as soon as the user assign anything outside the actual dimensions, an ExpandingArray.concatenate() is performed. How to do such overloading?
Testing code: I post here also some code I used to make comparison between vstack and my method. I add up random chunk of data of maximum length 100.
import time
N = 10000
def performEA(N):
EA = ExpandingArray(np.zeros((0,2)),maxIncrement=1000)
for i in range(N):
nNew = np.random.random_integers(low=1,high=100,size=1)
X = np.random.rand(nNew,2)
EA.concatenate(X,axis=0)
# Perform operations on EA.getDataArray()
return EA
def performVStack(N):
A = np.zeros((0,2))
for i in range(N):
nNew = np.random.random_integers(low=1,high=100,size=1)
X = np.random.rand(nNew,2)
A = np.vstack((A,X))
# Perform operations on A
return A
start_EA = time.clock()
EA = performEA(N)
stop_EA = time.clock()
start_VS = time.clock()
VS = performVStack(N)
stop_VS = time.clock()
print "Elapsed Time EA: %.2f" % (stop_EA-start_EA)
print "Elapsed Time VS: %.2f" % (stop_VS-start_VS)

I think the most common design pattern for these things is to just use a list for the small arrays. Sure you could do things like dynamic resizing (if you want to do crazy things, you can try to use the resize array method too). I think a typical method is to always double the size, when you really don't know how large things will be. Of course if you know how large the array will grow to, just allocating the full thing up front is simplest.
def performVStack_fromlist(N):
l = []
for i in range(N):
nNew = np.random.random_integers(low=1,high=100,size=1)
X = np.random.rand(nNew,2)
l.append(X)
return np.vstack(l)
I am sure there are some use cases where an expanding array could be useful (for example when the appending arrays are all very small), but this loop seems better handled with the above pattern. The optimization is mostly about how often you need to copy everything around, and doing a list like this (other then the list itself) this is exactly once here. So it is much faster normally.

When I faced a similar problem, I used ndarray.resize() (http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.resize.html#numpy.ndarray.resize). Most of the time, it will avoid reallocation+copying altogether. I can't guarantee it would prove to be faster (it probably would), but it's so much simpler.
As for your second question, I think overriding slice assignment for extending purposes is not a good idea. That operator is meant for assigning to existing items/slices. If you want to change that, it's not immediately clear how you'd want it to behave in some cases, e.g.:
a = MyExtendableArray(np.arange(100))
a[200] = 6 # resize to 200? pad [100:200] with what?
a[90:110] = 7 # assign to existing items AND automagically-allocated items?
a[::-1][200] = 6 # ...
My suggestion is that slice-assignment and data appending should remain separate.

Horizontal flip of Image on Python with pypng

I am trying to horizontally flip an image (from left to right) on Python using PyPNG, I have written the following codes but it does not seem to work, anybody have any idea what I'm doing wrong here?
def horizontal_flip(image):
rows = len(image)
cols = len(image[0])
new_image = []
for r in range(rows):
new_row = []
for c in range(0,cols,3):
if c != cols/2:
image[c:c+3], image[-c-3: -c] = image[-c-3: -c], image[c:c+3]
new_row.append(image[r][c])
new_image.append(new_row)
return new_image

new_row.append(image[r][c]) should be outside of the if.
Also, you're flipping the image horizontally... twice. Make your for loop use range(0,cols/2,3). (That may also eliminate the need for that if.)
You're also modifying the original image in-place; are you sure you want to do that?
Seems a simpler solution might be to loop through each row in reverse, appending to a row for the new image.

The inner loop logic is wrong but particularly, this line:
image[c:c+3], image[-c-3: -c] = image[-c-3: -c], image[c:c+3]
You are changing the image variable in-place, but you seemed to forget the row variable r. So right now, you are changing rows. And your negative slicing is a bit off. For c=0, you'll get image[-3:0] and this is not a valid slice and it will return [].
But judging from your code you don't mean to change image in-place, you rather want to create new_image. What you should be doing is inserting slices at the end of new_row:
def horizontal_flip(image):
rows = len(image)
cols = len(image[0])
new_image = []
for r in range(rows):
new_row = []
for c in range(0,cols,3):
new_row = image[r][c:c+3] + new_row
new_image.append(new_row)
return new_image
By the way, you can also change the image in-place, but be careful. As you are passing a list, you should copy it before changing so that original is unchanged. Here is that version:
def horizontal_flip(image):
cols = len(image[0])/3
#make a copy so that original image is not altered
image = [row[:] for row in image]
for row in image:
for c in range(int(cols/2)): # int() is not needed for Python 2.x, since integer division yields integer
# This also takes care of odd n cases, middle chunk is not changed.
row[3*c:3*c+3], row[3*(cols-c-1):3*(cols-c-1)+3] = row[3*(cols-c-1):3*(cols-c-1)+3], row[3*c:3*c+3]
return image
This can also be done with a list comprehension in single line, but it will be less readable. If you like, here is how you can do it:
from itertools import chain
flipped_image = [list(chain(*[row[3*i:3*i+3] for i in range(len(image[0])/3-1,-1,-1)])) for row in image]