We Keep Coding

I am trying to get inverse of a matrix from a file in python, but the format is in string and even after changing it into float I am unable to do it

I am trying to write a code which reads this file and gives the inverse of square root of each term in a matrix. This is the file I am using:
1.659999999999999963e-04
3.970000000000000005e-04
-8.014499999999999402e-02
-2.274299999999999933e-02
-7.559999999999999880e-03
-3.156229999999999869e-01
5.650100000000000261e-02
2.350100000000000106e-02
-4.383999999999999876e-03
-4.878299999999999997e-02
1.207599999999999993e-02
-5.254199999999999843e-02
1.123500000000000019e-02
1.614240000000000119e-01
1.954900000000000040e-02
-2.614100000000000104e-02
1.534899999999999980e-02
5.446000000000000320e-03
-6.210299999999999848e-02
-9.615000000000000283e-03
1.687800000000000064e-02
6.460999999999999729e-03
-9.490999999999999437e-03
1.676700000000000065e-02
-2.308000000000000156e-03
-1.412399999999999940e-02
8.978899999999999382e-02
1.848960000000000048e-01
5.956000000000000356e-03
-5.592300000000000049e-02
1.114599999999999966e-02
-5.689600000000000213e-02
-6.731000000000000004e-03
2.572999999999999940e-02
1.512000000000000106e-03
-3.237999999999999993e-03
-4.068999999999999700e-03
-1.234000000000000071e-03
2.378109999999999946e-01
-1.128000000000000096e-03
-3.534999999999999948e-03
-4.550000000000000008e-04
1.479999999999999925e-04
5.220000000000000031e-04
3.718099999999999877e-02
1.104580000000000006e-01
1.965000000000000167e-03
4.266999999999999960e-03
-5.140999999999999737e-03
1.648640000000000105e-01
1.776220000000000021e-01
1.922000000000000097e-03
3.250600000000000017e-02
4.402899999999999869e-02
-8.430999999999999259e-03
4.409999999999999858e-04
1.389999999999999905e-04
1.374209999999999876e-01
-2.431860000000000133e-01
-1.727000000000000019e-03
-2.280000000000000126e-04
8.100000000000000375e-05
-7.480999999999999803e-03
8.000000000000000654e-05
-3.939999999999999817e-04
1.441000000000000007e-03
-7.290000000000000473e-04
-3.663000000000000284e-02
-1.657999999999999969e-03
-8.369999999999999619e-04
-6.904999999999999680e-03
1.593100000000000072e-02
-3.393000000000000183e-03
1.495999999999999934e-03
-7.368999999999999682e-03
1.436199999999999977e-02
-1.319700000000000040e-02
-4.557000000000000287e-03
8.123700000000000365e-02
2.447399999999999923e-02
-1.295199999999999997e-02
-8.722100000000000686e-02
-5.232999999999999804e-03
-1.255940000000000112e-01
1.291999999999999963e-03
-1.382999999999999898e-03
4.989999999999999644e-03
1.508000000000000009e-03
2.304399999999999851e-02
2.819400000000000031e-02
3.119999999999999944e-04
-8.781999999999999876e-03
6.794500000000000539e-02
6.198999999999999649e-03
-2.058879999999999877e-01
9.219999999999999680e-04
-1.618800000000000100e-02
-3.415860000000000007e-01
-1.660999999999999933e-03
-4.889999999999999599e-04
1.759999999999999954e-04
-3.763999999999999985e-03
-6.566600000000000215e-02
-7.680000000000000195e-04
-1.231799999999999978e-01
7.047999999999999578e-03
1.425000000000000051e-02
-2.900799999999999906e-02
1.187499999999999944e-01
-1.449199999999999933e-01
-1.106999999999999911e-03
-1.557999999999999923e-03
-2.236999999999999839e-03
-7.270699999999999386e-02
-5.140000000000000254e-04
2.246999999999999865e-03
-1.778949999999999976e-01
1.669599999999999904e-02
-1.277799999999999943e-02
-2.379040000000000044e-01
-2.207999999999999893e-03
1.925000000000000062e-03
7.750100000000000044e-02
-5.004100000000000215e-02
1.704999999999999918e-03
3.272400000000000309e-02
1.957499999999999865e-02
-1.514620000000000133e-01
-3.288999999999999823e-03
-3.605699999999999877e-02
3.648999999999999900e-03
3.459799999999999681e-02
-1.859999999999999945e-04
3.300000000000000253e-05
3.000000000000000076e-06
9.999999999999999547e-07
4.800000000000000122e-05
1.361999999999999929e-03
-6.057300000000000184e-02
5.689999999999999529e-04
-5.000000000000000409e-06
2.984699999999999853e-02
6.999999999999999387e-05
4.600000000000000004e-05
-1.294499999999999991e-02
-2.318000000000000182e-03
-0.000000000000000000e+00
1.858200000000000129e-01
-5.969959999999999711e-01
6.000000000000000152e-06
The code I have tried to write is this:
from ast import Num
from cmath import sqrt
import math
import numpy as np
from numpy import append
f1= open('diagonal.txt', 'r')
k=f1.readlines()
#def mkstr(s):
# str1=" "
# return (str1.join(s))
#print(float(mkstr(k)))
#for j in f1:
#inverse_root(j)
#def mkstr2():
# f1=open("diagonal.txt", "r")
# k=f1.readlines()
# fl=[float(item) for item in k]
# inverse_root(fl)
#for j in mkstr2():
#inverse_root(j)
f=[float(x) for x in k]
sa=np.array(f)
ca=sa.astype(np.float)
def inverse_root(j):
return [1/(sqrt(j))]
j=[inverse_root(ca)]
I am getting the output like this:
DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
ca=sa.astype(np.float)
Traceback (most recent call last):
File "/home/gian-2018/prateek/python/read_file/27/5/rmwpd.py", line 34, in <module>
j=[inverse_root(ca)]
File "/home/gian-2018/prateek/python/read_file/27/5/rmwpd.py", line 33, in inverse_root
return [1/(sqrt(j))]
TypeError: only length-1 arrays can be converted to Python scalars
So to get inverse of sqrt for each term from the matrix file: I tried to convert the list of these strings into float list, but even after I get error for trying to get list of these numbers into the formula of inverse sqrt instead of using a real number.
Now I am trying to create the inverse formula suitable for an array which means creating a new array which converts each term in the previous matrix into their inverse sqrt and saves the value into the new array.
By any way possible please either tell me how to convert this list into real numbers or convert each term of this matrix into its inverse sqrt and save it into new matrix.
Or just tell me how to implement this task in any way possible if you have understood it.

Convert a sympy poly with imaginary powers to an mpmath mpc

I have a sympy poly that looks like:
Poly(0.764635937801645*I**4 + 7.14650839258644*I**3 - 0.667712176660315*I**2 - 2.81663805543677*I - 0.623299856233272, I, domain='RR')
I'm converting to mpc using the following code:
a = val.subs('I',1.0j)
b = sy.re(a)
c = sy.im(a)
d = mpmath.mpc(b,c)
Two questions.
Assuming my mpc and sympy type have equal precision (of eg 100 dps) is there a precision loss using this conversion from a to d?
Is there a better way to convert?
Aside: sympy seems to treat I just like a symbol here. How do I get sympy to simplify this polynomial?
Edit: Ive also noticed that the following works in place of a above:
a = val.args[0]

Strings and expressions
Root cause of the issue is seen in val.subs('I', 1.0j) -- you appear to pass strings as arguments to SymPy functions. There are some valid uses for this (such as creation of high-precision floats), but when symbols are concerned, using strings is a recipe for confusion. The string 'I' gets implicitly converted to SymPy expression Symbol('I'), which is different from SymPy expression I. So the answer to
How do I get sympy to simplify this polynomial?
is to revisit the process of creation of that polynomial, and fix that. If you really need to create it from a string, then use locals parameter:
>>> S('3.3*I**2 + 2*I', locals={'I': I})
-3.3 + 2*I
Polynomials and expressions
If the Poly structure is not needed, use the method as_expr() of Poly to get an expression from it.
Conversion to mpmath and precision loss
is there a precision loss using this conversion from a to d?
Yes, splitting into real and imaginary and then recombining can lead to precision loss. Pass a SymPy object directly to mpc if you know it's a complex number. Or to mpmathify if you want mpmath to decide what type it should have. An example:
>>> val = S('1.111111111111111111111111111111111111111111111111')*I**3 - 2
>>> val
-2 - 1.111111111111111111111111111111111111111111111111*I
>>> import mpmath
>>> mpmath.mp.dps = 40
>>> mpmath.mpc(val)
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111111')
>>> mpmath.mpmathify(val)
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111111')
>>> mpmath.mpc(re(val), im(val))
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111114')
Observations:
When I is actual imaginary unit, I**3 evaluates fo -I, you don't have to do anything for it to happen.
A string representation of high-precision decimal is used to create such a float in SymPy. Here S stands for sympify. One can also be more direct and use Float('1.1111111111111111111111111')
Direct conversion of a SymPy complex number to an mpmath complex number is preferable to splitting in real/complex and recombining.
Conclusion
Most of the above is just talking around an XY problem. Your expression with I was not what you think it was, so you tried to do strange things that were not needed, and my answer is mostly a waste of time.

I'm adding my own answer here, as FTP's answer, although relevant and very helpful, did not (directly) resolve my issue (which wasn't that clear from the question tbh). When I ran the code in his example I got the following:
>>> from sympy import *
>>> import mpmath
>>> val = S('1.111111111111111111111111111111111111111111111111')*I**3 - 2
>>> val
-2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mp.dps = 40
>>> mpmath.mpc(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 373, in __new__
real = cls.context.mpf(real)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 77, in __new__
v._mpf_ = mpf_pos(cls.mpf_convert_arg(val, prec, rounding), prec, rounding)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 96, in mpf_convert_arg
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mpmathify(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 662, in convert
return ctx._convert_fallback(x, strings)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp.py", line 614, in _convert_fallback
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mpc(re(val), im(val))
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111114')
>>> mpmath.mpmathify(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 662, in convert
return ctx._convert_fallback(x, strings)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp.py", line 614, in _convert_fallback
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
Updating my sympy (1.0->1.1.1) and mpmath (0.19->1.0.0) fixed the exceptions. I did not test which of these upgrades actually resolved the issue.

Simple example of biopython clustering

I would like get some basic understanding how to use biopython for clustering genes.
Lets say i have genes that i would like to group. How to feed them to the algorithm, and how to give a cutoff point under which size and amount of cluster would depend?
I've tried straightforward approach:
from Bio.Cluster import kcluster
list1 = [
'ADHAMKCAIROSURBANDJVUGLOBALIZATIONANDURBANFANTASIESPLA',
'AGGESTAMKTHEARABSTATEANDNEOLIBERALGLOBALIZATIONTHEARAB',
'AGGESTAMKTHEARABSTATEANDNEOLIBERALGLOBALIZATIONTHEARAB',
'AGGESTAMKTHEARABSTATEANDNEOLIBERALGLOBALIZATIONTHEARAB'
]
list2 = [Seq(gen, IUPAC.extended_protein) for gen in list1]
clusterid, error, nfound = kcluster(list2)
but it just brought me an error:
Traceback (most recent call last):
File "./test.py", line 9, in <module>
clusterid, error, nfound = kcluster(list2)
TypeError: data cannot be converted to needed array.

The kcluster function takes a data matrix as input and not Seq instances.
You need to convert your sequences to a matrix and provide that to the kcluster function.
One way of converting the data to a matrix containing numerical elements only is by using the numpy.fromstring function. It basically translates each letter in a sequence to it's ASCII counterpart.
This creates a 2D array of encoded sequences that the kcluster function recognized and uses to cluster your sequences.
>>> from Bio.Cluster import kcluster
>>> import numpy as np
>>> sequences = [
... 'ADHAMKCAIROSURBANDJVUGLOBALIZATIONANDURBANFANTASIESPLA',
... 'AGGESTAMKTHEARABSTATEANDNEOLIBERALGLOBALIZATIONTHEARAB',
... 'AGGESTAMKTHEARABSTATEANDNEOLIBERALGLOBALIZATIONTHEARAB',
... 'AGGESTAMKTHEARABSTATEANDNEOLIBERALGLOBALIZATIONTHEARAB'
... ]
>>> matrix = np.asarray([np.fromstring(s, dtype=np.uint8) for s in sequences])
>>> clusterid, error, nfound = kcluster(matrix)
>>> print(clusterid)
[1, 0, 0, 0]

receiving "ValueError: setting an array element with a sequence."

I've been having some problems with this code, trying to end up with an inner product of two 1-D arrays. The code of interest looks like this:
def find_percents(i):
percents=[]
median=1.5/(6+2*int(i/12))
b=2*median
m=b/(7+2*int(i/12))
for j in xrange (1,6+2*int(i/12)):
percents.append(float((b-m*j)))
percentlist=numpy.asarray(percents, dtype=float)
#print percentlist
total=sum(percentlist)
return total, percentlist
def playerlister(i):
players=[]
for i in xrange(i+1,i+6+2*int(i/12)):
position=sheet.cell(i,2)
points=sheet.cell(i,24)
if re.findall('RB', str(position.value)):
vbd=points.value-rbs[24]
players.append(vbd)
else:
pass
playerlist=numpy.asarray(players, dtype=float)
return playerlist
def others(i,percentlist,playerlist,total):
alternatives=[]
playerlist=playerlister(i)
percentlist=find_percents(i)
players=numpy.dot(playerlist,percentlist)
I am receiving the following error in response to the very last line of this attached code:
ValueError: setting an array element with a sequence.
In most other examples of this error, I have found the error to be because of incorrect data types in the arrays percentlist and playerlist, but mine should be float type. If it helps at all, I call these functions a little later in the program, like so:
for i in xrange(1,30):
total, percentlist= find_percents(i)
playerlist= playerlister(i)
print type(playerlist[i])
draft_score= others(i,percentlist,playerlist,total)
Can anyone help me figure out why I am setting an array element with a sequence? Please let me know if any more information might be helpful! Also for clarity, the playerlister is making use of the xlrd module to extract data from a spreadsheet, but the data are numerical and testing has shown that that both lists have a type of numpy.float64.
The shape and contents of each of these for one iteration of i is
<type 'numpy.float64'>
(5,)
[ 73.7 -94.4 140.9 44.8 130.9]
(5,)
[ 0.42857143 0.35714286 0.28571429 0.21428571 0.14285714]

Your function find_percents returns a two-element tuple.
When you call it in others, you are binding that tuple to the variable named percentlist, which you then try to use in a dot-product.
My guess is that by writing this in others it is fixed:
def others(i,percentlist,playerlist,total):
playerlist = playerlister(i)
_, percentlist = find_percents(i)
players = numpy.dot(playerlist,percentlist)
provided of course playerlist and percentlist always have the same number of elements (which we can't check because of the missing spreadsheet).
To verify, the following gives you the exact error message and the minimum of code needed to reproduce it:
>>> import numpy as np
>>> a = np.arange(5)
>>> np.dot(a, (2, a))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: setting an array element with a sequence.

numpy.savetxt fmt='%f', appending concatenated arrays to a list

I have a function inside a class that it's called inside a while loop.
This function gets 2 6x6 matrices and returns a vector magnified by the dimensions specified in np.linspace, i.e. 6x91. I want to store these three arrays in a file (using savetxt) by concatenating them as float numbers. Below there's a sample of the function run separately from the class, which works fine.
The problem comes when I declare the list output before the loop and by updating EulArr via a Tkinter window, append the arrays to the list. I can only save the arrays with %s format, but not anymore by %f. Why?
LegForces(self, Li, ti)
import numpy as np
from math import pi
M=100. #total estimated mass to move (upper platform+upper legs+payload)
g=9.81
nsteps= 91
Li=np.matrix([[-10.64569774, -93.1416122, 116.35191853],\
[-10.68368329, 93.17236065, 116.35542498],\
[85.985087,37.35196994, 116.20350534],[-75.34703551,-55.83790049, 115.44528196],\
[-75.33938926, 55.78964226, 115.44457613],[86.0307188,-37.33446016, 116.19929305]])
ti=np.matrix([[88.15843159,88.04450508,50.10006323, -138.28903445, -138.26610178,50.2369224],\
[-108.75675186, 108.84408749, 130.72504635, 21.82594871, -21.97569549,-130.6774372],\
[ 119.40585161, 119.40170883, 119.23577854, 118.41560138, 118.41643529,119.24075525]])
ti=ti.T #transpose the ti for compatible format
x_cm= 1.
y_cm= -1.
z_cm=87.752
z=np.linspace(0,90,nsteps)
Fx=-M*g*np.cos(pi*z/180)
Fy= M*g*np.sin(pi*z/180)
Fz=50 # including braking forces [N]
#specify here the centre of mass coordinates retrieved from Inventor
Mx=Fz*y_cm-Fy*z_cm
My=-Fz*x_cm+Fx*z_cm
Mz=Fy*x_cm-Fx*y_cm
mc=np.zeros((6,3),'float')
ex_forces=np.array([Fx,Fy,Fz,Mx,My,Mz])
for i in range(6):
mc[i,:]=np.cross(ti[i,:],Li[i,:])
r1=[Li[0,0],Li[1,0],Li[2,0],Li[3,0],Li[4,0],Li[5,0]]
r2=[Li[0,1],Li[1,1],Li[2,1],Li[3,1],Li[4,1],Li[5,1]]
r3=[Li[0,2],Li[1,2],Li[2,2],Li[3,2],Li[4,2],Li[5,2]]
r4=[mc[0,0],mc[1,0],mc[2,0],mc[3,0],mc[4,0],mc[5,0]]
r5=[mc[0,1],mc[1,1],mc[2,1],mc[3,1],mc[4,1],mc[5,1]]
r6=[mc[0,2],mc[1,2],mc[2,2],mc[3,2],mc[4,2],mc[5,2]]
DMatrix=np.vstack([r1,r2,r3,r4,r5,r6])
print 'DMatrix form:\n', DMatrix
invD=np.linalg.inv(DMatrix)
print ' inv(DMatrix) is:\n', invD
legF=np.dot(ex_forces,invD)
#slice it!
legF=legF.tolist()
a,b=np.shape(legF)
print 'check identity matrix:\n', np.dot(invD,DMatrix)
print 'leg forces:\n',legF, type(legF)
newlegF=np.reshape(legF,(1,a*b))
strokeValues= np.array([[-0.3595, .1450483, -0.3131,0.4210,-0.0825,.19124]])
print 'strokeValues shape:\n', np.shape(strokeValues)
print 'leg forces vector shape:', np.shape(newlegF)
EulArr=np.array([[0.12,0.2,0,-3.,-1.,15.]])
output=np.concatenate((strokeValues,EulArr,newlegF),axis=1)
np.savetxt('leg_forces.dat', output,fmt=' %f')
print output ,np.shape(output)
The class will look like this:
class Hexapod(self,....):
output=[]
while 1:
...
LegAxialF=self.LegForces(Li,ti)
#create a list for EUl parameters which can be refreshed
EulArr=np.zeros((6,1),'float')
EulArr[0]=180/pi*self.EulPhi
EulArr[1]=180/pi*self.EulTheta
EulArr[2]=180/pi*self.EulPsi
EulArr[3]=self.EulX
EulArr[4]=self.EulY
EulArr[5]=self.EulZ-self.height
#print meaningful values to the specified file
EulArr=np.reshape(EulArr,(1,6))
EulArrList=EulArr.tolist()
strokes=np.reshape(strokeValues,(1,6))
strokeList=strokes.tolist()
output.append(np.concatenate((strokeList,\
EulArrList,LegFList),axis=1))
np.savetxt('act_lengths.dat', output, fmt='%f')
def LegForces(self, Li, ti):
I get the following error:
np.savetxt('act_lengths.dat', output, fmt='%f')
File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 1047, in savetxt
fh.write(asbytes(format % tuple(row) + newline))
TypeError: float argument required, not numpy.ndarray

Output is a list containing ndarrays. I think in your code, the elements of output must be ndarrays of varying size and internally NumPy is failing to convert this into a float array due to the ragged data, instead creating an array of objects*. I can reproduce your error with
>>> output = [np.random.rand(3), np.random.rand(4)]
>>> np.savetxt("test", output, fmt='%f')
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Users/yotam/anaconda/lib/python2.7/site-packages/numpy/lib/npyio.py", line 1083, in savetxt
fh.write(asbytes(format % tuple(row) + newline))
TypeError: float argument required, not numpy.ndarray
>>>
Whereas if I instead use output = [np.random.rand(3), np.random.rand(3)] it works.
(*) i.e. you are getting
>>> np.asarray(output)
array([array([ 0.87346791, 0.10046296, 0.60304887]),
array([ 0.25116526, 0.29174373, 0.26067348, 0.68317986])], dtype=object)