We Keep Coding

Solving a non-linear equation using fsolve

I have this non-linear equation:
Ax + Bxy - C = 0 where A, B and C are all constants. I have a bunch of different values for A, B and C, like about 10000.
My questions is how can i use the different set of A, B and C values I have to generate the perfect solution to the system of non-linear equations, what starting point to use and also how to plot them in matplotlib finally. Thanks for your help!.
I tried to use fsolve like this.
Note: The coeffs is a dataframe which stored the values for A and B and merged_sensoren is another dataframe which stores the value for C in the first column.
Here are a snippet of the dataframe coeffs:
coeffs = {'coeffX0': [0, 0, 0, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3], 'coeffX1': [0, 0, 0, 58, 60, 58, 60, 87, 90, 87, 90, 87, 90, 87, 90]}
Some values of the coeffecient C are = [0, 0, 0, 0, 0, 4, 4, 0, 0, 1, 1, 15, 15]
from scipy.optimize import fsolve
def f(x):
return [coeffs.iloc[10,0] * x[0] + coeffs.iloc[10,1] * x[0] * x[1] - merged_sensoren.iloc[10,0],
coeffs.iloc[11,0] * x[0] + coeffs.iloc[11,1] * x[0] * x[1] - merged_sensoren.iloc[11,0]]
root = fsolve(f, [1, 1])
root
This gave me the answer as
array([1., 1.])
Can someone explain what does 1., 1. mean for a solution. Is this even a number?
however, root = fsolve(f, [0, 0]) gave me array([ 1.40333333e+02, -3.32541568e-02]).

How to vectorize performing pairwise sums given two numpy arrays?

I have two numpy arrays which look like this:
x = [v1, v2, v3, ..., vm]
y = [w1, w2, w3, ..., wn]
where vi, wj are numpy arrays of length 3.
I want to perform a pairwise summation of v's and w's and get a final array
z = [v1+w1, v1+w2,...,v1+wn,v2+w1, ..., vi+wj, ..., vm+wn]
A simple way of obtaining z is as follows:
z = np.zeros ((m*n, 3))
for i in range(m):
for j in range(n):
z[n*i+j] = x[i] + y[j]
This computation is not feasible is m, n are very large.
I know scipy.spatial has methods to enumerate pairwise distances using distance_matrix in a vectorized fashion.
I want to ask if there is a vectorized version of performing such pairwise additions for numpy arrays?

You can take advantage of broadcasting, creating a 2D array, then you can easily get z[i,j] = x[i] + y[j]
x = np.reshape(x, (-1, 1)) # shape (N, 1)
y = np.reshape(y, (-1, 1)) # shape (N, 1)
z = x + y.T # shape (N, N)
If you want to have z as a 1D array you can do z.reshape(-1).

If x is mx3 matrix, y is a nx3
x.shape # (m,3)
y.shape # (n,3)
x1 = x.reshape(m,1,3)
y1 = y.reshape(1,n,3)
z = x1 + y1 # shape (m,n,3)
z1 = z.reshape(-1,3) # (m*n, 3)
equivalently
z = x[:,None]+y
test:
In [263]: x=np.arange(12).reshape(4,3); y=np.arange(6).reshape(2,3)
In [264]: z = x[:,None]+y
In [265]: z.shape
Out[265]: (4, 2, 3)
In [266]: z
Out[266]:
array([[[ 0, 2, 4],
[ 3, 5, 7]],
[[ 3, 5, 7],
[ 6, 8, 10]],
[[ 6, 8, 10],
[ 9, 11, 13]],
[[ 9, 11, 13],
[12, 14, 16]]])

Solving System of linear equation using Cramer's method in Python

I am trying to solve the following system of linear equations:
10x1+ 40x2+ 70x3= 300
20x1+ 50x2+ 80x3= 360
30x1+ 60x2+ 80x3= 390
by using Cramer's method implementing a function by scratch:
def cramer(mat, constant): # takes the matrix and the costants
D = np.linalg.det(mat) # calculating the determinant of the original matrix
# substitution of the column with costant and creating new matrix
mat1 = np.array([constant, mat[1], mat[2]])
mat2 = np.array([mat[0], constant, mat[2]])
mat3 = np.array([mat[0], mat[1], constant])
#calculatin determinant of the matrix
D1 = np.linalg.det(mat1)
D2 = np.linalg.det(mat2)
D3 = np.linalg.det(mat3)
#finding the X1, X2, X3
X1 = D1/D
X2 = D2/D
X3 = D3/D
print(X1, X2, X3)
By using the above function on the system
a = np.array([[10, 40, 70],
[20, 50, 80],
[30, 60, 80]])
b = np.array([300, 360, 390])
I get the following result:
cramer(a,b)
-22.99999999999996 21.999999999999964 2.999999999999993
I have solved the system using the numpy function np.linalg.solve and I get another result:
x = np.linalg.solve(a, b)
[1. 2. 3.]
I cannot spot the formula error in the function I have witten. What should I adjust in the fuction in order to make it working properly?

The main problem is how you select the columns of a, i.e. you are actually selecting the rows of a rather than the columns. You can fix it by changing the matrix initializations to this:
mat1 = np.array([constant, mat[:,1], mat[:,2]])
mat2 = np.array([mat[:,0], constant, mat[:,2]])
mat3 = np.array([mat[:,0], mat[:,1], constant])
Basically mat[:,1] is saying something like mat[all rows, column 1].

TL;DR Optimal solution at the bottom.
To fix your current solution you need to use the second dimensions and pass all three matrices to compute determinants together (this way you will get stable floating point values):
def cramer(mat, constant):
D = np.linalg.det(mat)
mat1 = np.array([constant, mat[:, 1], mat[:, 2]])
mat2 = np.array([mat[:, 0], constant, mat[:, 2]])
mat3 = np.array([mat[:, 0], mat[:, 1], constant])
Dx = np.linalg.det([mat1, mat2, mat3])
X = Dx/D
print(X)
However, you don't need to create all these matrices one by one either. Instead, use a series of numpy manipulations described below.
First, create the mask to so you can then use it to replace values in a by values from b:
>>> mask = np.broadcast_to(np.diag([1,1,1]), [3, 3, 3]).swapaxes(0, 1)
array([[[1, 0, 0],
[1, 0, 0],
[1, 0, 0]],
[[0, 1, 0],
[0, 1, 0],
[0, 1, 0]],
[[0, 0, 1],
[0, 0, 1],
[0, 0, 1]]])
Then use np.where to get three matrices, each with one column replaced by b:
>>> Ms = np.where(mask, np.repeat(b, 3).reshape(3, 3), a)
array([[[300, 40, 70],
[360, 50, 80],
[390, 60, 80]],
[[ 10, 300, 70],
[ 20, 360, 80],
[ 30, 390, 80]],
[[ 10, 40, 300],
[ 20, 50, 360],
[ 30, 60, 390]]])
Then, compute three determinants and divide the determinant of a itself:
>>> np.linalg.det(Ms) / np.linalg.det(a)
array([1., 2., 3.])
Putting it all together:
def cramer(a, b):
mask = np.broadcast_to(np.diag([1,1,1]), [3, 3, 3]).swapaxes(0, 1)
Ms = np.where(mask, np.repeat(b, 3).reshape(3, 3), a)
return np.linalg.det(Ms) / np.linalg.det(a)

Create matrix with numpy meshgrid

I want to create a matrix with M rows and N columns. The increment along the columns are always 1, whilst the increment in rows are a constant value, c. For example, to create this matrix:
The number of rows are 4, the number of columns are 2 and the shift between rows: c = 8. One way to perform this could be:
# Indices of columns
coord_x = np.arange(0, 2)
# Indices of rows
coord_y = np.arange(1, 37, 9)
# Creates 2 matrices with the coordinates
x, y = np.meshgrid(coord_x, coord_y)
# To perform the shift between columns
idx_left = x + y
And the output is:
print(idx_left)
[[ 1 2]
[10 11]
[19 20]
[28 29]]
Can I perform this without the adding idx_left = x + y?. I've already seen other functions but I don't find any that considers a shift along the rows and columns...

Stride tricks
You can use np.lib.stride_tricks for this purpose.
arr = np.arange(1,100)
shape = (4,2)
strides = (arr.strides[0]*9,arr.strides[0]*1) #8 bytes with 9 steps on axis=0, 8bytes with 1 step on axis=1
np.lib.stride_tricks.as_strided(arr, shape=shape, strides=strides)
array([[ 1, 2],
[10, 11],
[19, 20],
[28, 29]])
Another example with 2 shift on axis=0 and 3 shift on axis=1.
arr = np.arange(1,100)
shape = (4,2)
strides = (arr.strides[0]*2,arr.strides[0]*3) #8bytes * 2shift on axis=0, 8bytes*3shift on axis=1
np.lib.stride_tricks.as_strided(arr, shape=shape, strides=strides)
array([[ 1, 4],
[ 3, 6],
[ 5, 8],
[ 7, 10]])
Broadcasting
You could simply do this the same way you are doing without meshgrids only using broadcasting as well -
#Your original example
coord_x = np.arange(0, 2, 1) #start, stop, step
coord_y = np.arange(1, 37, 9)
coord_x[None,:] + coord_y[:,None]
array([[ 1, 2],
[10, 11],
[19, 20],
[28, 29]])
Linspace
You could use linspace if you have extreme limits. So, in your case, you can create a 4 row array from (1,2) to (28,29).
np.linspace((1,2),(28,29),4)
array([[ 1., 2.],
[10., 11.],
[19., 20.],
[28., 29.]])
Mgrid
Mgrid is more convenient than mesh grid for your purpose. You can do -
np.mgrid[0:2:1, 1:37:9].sum(0).T
array([[ 1, 2],
[10, 11],
[19, 20],
[28, 29]])

Pandas: conversion to/from cvxopt matrices

Given a Pandas dataframe, what is the best way (readability OR execution speed) to convert to a cvxopt matrix or vice versa?
Currently I am doing:
cvxMat = matrix(pdObj.as_matrix())
pdObj[:]=np.array(cvxMat)
Also, is there a reasonably readable way of doing vector or matrix algebra using a mixture of cvxopt matrices and pandas dataframes without converting the objects?
The following is a vector dot product (pdObj & cvxMat are column vectors) that is far from readable:
(matrix(pdObj.as_matrix()).T*cvxMat)[0]
Any advice?
Follow-up to waitingkuo's answer:
Just for illustration with pandas dataframes:
>>> m1 = cvxopt.matrix([[1, 2, 3], [2, 3, 4]])
>>> m2 = pd.DataFrame(np.array(m1)).T
>>> m1
<3x2 matrix, tc='i'>
>>> m2.shape
(2, 3)
>>> np.dot(m1,m2)
array([[ 5, 8, 11],
[ 8, 13, 18],
[11, 18, 25]])
But note:
>>> m1 * m2
0 1 2
0 1 4 9
1 4 9 16
[2 rows x 3 columns]

You can get the numpy array from pandas by pdObj.values
You can do matrix multiplication between the cvxopt matrix and numpy matrix directly
In [90]: m1 = cvxopt.matrix([[1, 2, 3], [2, 3, 4]])
In [91]: m2 = np.matrix([[1, 2, 3], [2, 3, 4]])
In [92]: m1
Out[92]: <3x2 matrix, tc='i'>
In [94]: m2.shape
Out[94]: (2, 3)
In [95]: m1 * m2
Out[95]:
matrix([[ 5, 8, 11],
[ 8, 13, 18],
[11, 18, 25]])

An alternative to messing with cvxopt __init__ is to define your own dot;
A or B can be numpy arrays, or array-like, or anything with .value or .values:
def dot( A, B ):
""" np.dot .value or .values if they exist """
for val in "value values" .split():
A = getattr( A, val, A ) # A.val or A
B = getattr( B, val, B )
A = np.asanyarray( A )
B = np.asanyarray( B )
try:
np.dot( A, B )
except ValueError:
print >>sys.stderr, "error: can't dot shapes %s x %s" % (A.shape, B.shape)
raise
(Bytheway I avoid matrices, stick to numpy arrays and vecs -- a separate issue.)