scipy and matlab fft results differ - python

I am working with a large amount of data and am trying to use the fftpack in scipy to help me out. In Matlab, I have code that looks like:
val1 = fft(val1,fft_window_size);
where fft_window_size is 2^19.
This outputted:
val1
ans = -5.5162
ans = 4.5001 - 0.0263i
ans = -2.4261 + 0.0256i
ans = 0.8575 - 0.0233i
ans = -0.2189 + 0.0531i
For the first 5 indices of Val1.
In Python, I used:
val1=scipy.fftpack.fft(val1,fft_window_size)
where the fft_window_size was the same as listed above (2**19) and got completely different answers for the first 5 indices:
('val1', array([-7.6888 +0j, 5.2122 - 0.07556j,
-1.4928+0.02275j, 0.15854 +0.01481j, -0.07528+0.03379j]))
I've looked at as many examples as I could find and couldn't find a good answer as to why they are so vastly different. I checked val1 all the way up to this command and the two matched perfectly (with Python having more decimal places). I don't think this is a rounding issue, but I'm not sure what else to look at.
Any thoughts would be good.

Related

Python multiple series multiplication simplified

Guys I have the following short version of my code.
ab10=10
ab11=78
ab12=68
bc10=91
bc11=73
bc12=54
df10=87
df11=81
df12=90
b1=ab10/df10
b2=ab11/df11
b3=ab12/df12
c1=bc10/df10
c2=bc11/df11
c3=bc12/df12
m1=bc10/ab10
m2=bc11/ab11
m3=bc12/ab12
Isthere shorter way to make such multiplications as I have more and more such variables to calculate by years from 10-12 ?
I tried for i in range (10,12) but not working. In my case variables ab10, ab11 and so on I used other variables. Doing everything manually takes lots of time provided there are more years not limited to 10,11,12 at least 10 years. I will be happy to hear any way to simplify the process.
I would. appreciate any thoughts or codes shares to direct me and make my work efficient.
You could also consider using numpy if the amount of numbers is very high, the numbers are saved as vectors to which you can perform operations:
import numpy as np
ab1 = np.array([10,78,68])
bc1 = np.array([91,73,54])
df1 = np.array([87,81,90])
b = ab1/df1
c = bc1/df1
m = bc1/ab1
You can use dictionary for this kind of problems:
my_dict = {'ab10': 10, 'ab11':78, 'df10':87}
b1 = my_dict.ab10 / my_dict.df10 or my_dict['ab10] / my_dict['df10]

Getting the non-trivial solution to a set of linear equations

I'm trying to write a program that will allow me to solve a system of equations using numpy, however, I want the solution to be non-trivial (not all zeros). Obviously the program is just going to set everything to 0, and boom, problem solved. I attempted to use a while loop (like below), but quickly found out it's going to continue to spit 0 back at me. I don't care if I end up using numpy, I'm open to other solutions if it's more elegant.
I actually haven't solved this particular set by hand, maybe the trivial solution is the only solution. If so, the principle still applies. Numpy seems to always spit 0 back.
Any help would be appreciated! Thanks.
x1 = .5
x2 = .3
x3 = .2
x4 = .05
a = np.array([[x1,x2],[x3,x4]])
b = np.array([0,0])
ans = np.linalg.solve(a,b)
while ans[0] == 0 and ans[1] == 0:
print ("got here")
ans = np.linalg.solve(a,b)
print(ans)
In your case, the matrix a is invertible. Therefore your system of linear equations has only one solution and the solution is [0, 0]. Are you wondering why you only get that unique solution?
Check out Sympy and it's use of solve and matrix calculations. Here are the pages for both.
http://docs.sympy.org/latest/tutorial/matrices.html
http://docs.sympy.org/latest/tutorial/solvers.html

Returning incorrect values with complex numbers in Python

I'm working a physics problem with complex numbers and think I'm setting everything up correctly but am obviously not doing something right along the way. It could be that I'm either not using the right equations or that I'm unfamiliar with how Python's handling the math, and I'm pretty sure I'm using the right stuff. I've already worked a problem using the same kind of procedure and got the correct value, so substituting my new values should
Given f = 1000, SWR = -5.9, L = 0.081, I apparently should be getting z = 1.4 - 0.23j.
Here's what I'm doing:
import numpy as np
import cmath
f = 1000 #frequency
SWR = -5.9
L = 0.081
w = 2*f*np.pi #angular frequency
c = 343 #speed of sound in air
k = w/c #wavenumber
BA = (SWR-1)/(SWR+1) #given
theta = 2*k*L-np.pi #given
z = (1+BA*np.e**(1j*theta))/(1-BA*np.e**(1j*theta)) #given
print(z)
This gives me z = (-4.699946746470462-2.3316919882323677j), obviously not what I'm being told is the correct value.
I've gone over this multiple times now and can't find anything wrong. I just again worked through the problem I already got correct and made the minor substitutions to fit these given values, and I'm still getting the returned value of z. I don't want to tell my professor his "check that your code is giving the correct results" result is wrong, but...
Am I missing something?
E: Apologies for the rough display, but I'm not sure I can type in LaTeX here. The following are what I'm working with. Furthermore, the final image shows that I worked basically the same problem correctly and that I should be able to just make some substitutions to work this one. Also note that in my code, z is actually z divided by the rhocS quantity. I'm after that, don't need to know their values.
Equation for z, BA, theta, and the worked similar problem

How to handle number overflow?

I am calculating a trend line slope using numpy:
xs = []
ys = []
my_x = 0
for i in range(2000):
my_x += 1
ys.append(5*my_x+random.rand())
xs.append(my_x)
A = matrix(xs).T;
b = matrix(ys).T;
N = A.T*A
U = A.T*b
print N,U
a = (N.I*U)[0,0]
print a
The result I get is a=-8.2053307679 instead of the expected 5. Probably it happends beacuse the number in variable N is too big.
How to overcome this problem ? any help will be appreciated.
When I run the code, the answer is as you would expect:
[[2668667000]] [[ 1.33443472e+10]]
5.00037927592
It's probably due to the fact that you're on a 32-bit system, and I'm on a 64-bit system. Instead, you can use
A = matrix(xs, dtype='float64').T;
b = matrix(ys, dtype='float64').T;
Just FYI, when using numpy you'll be much more efficient if you work on vectorizing your algorithms. For example, you could replace the first several lines with this:
xs = np.arange(2000)
ys = 5 * xs + np.random.rand(2000)
Edit – one more thing: numerically, it is a bad idea to explicitly invert matrices when doing computations like these. It would be better to use something like a = np.linalg.solve(N, U)[0, 0] in your algorithm. It won't make a big difference here, but if you move to more complicated problems it definitely will! For some discussion this, take a look at this article.
:) The problem solved by using:
A = matrix(xs,float64).T;
b = matrix(ys,float64).T;

Pitfalls in R for Python programmers

I have mostly programmed in Python, but I am now learning the statistical programming language R. I have noticed some difference between the languages that tend to trip me.
Suppose v is a vector/array with the integers from 1 to 5 inclusive.
v[3] # in R: gives me the 3rd element of the vector: 3
# in Python: is zero-based, gives me the integer 4
v[-1] # in R: removes the element with that index
# in Python: gives me the last element in the array
Are there any other pitfalls I have to watch out for?
Having written tens of thousands of lines of code in both languages, R is just a lot more idiosyncratic and less consistent than Python. It's really nice for doing quick plots and investigation on a small to medium size dataset, mainly because its built-in dataframe object is nicer than the numpy/scipy equivalent, but you'll find all kinds of weirdness as you do things more complicated than one liners. My advice is to use rpy2 (which unfortunately has a much worse UI than its predecessor, rpy) and just do as little as possible in R with the rest in Python.
For example, consider the following matrix code:
> u = matrix(1:9,nrow=3,ncol=3)
> v = u[,1:2]
> v[1,1]
[2] 1
> w = u[,1]
> w[1,1]
Error in w[1, 1] : incorrect number of dimensions
How did that fail? The reason is that if you select a submatrix from a matrix which has only one column along any given axis, R "helpfully" drops that column and changes the type of the variable. So w is a vector of integers rather than a matrix:
> class(v)
[1] "matrix"
> class(u)
[1] "matrix"
> class(w)
[1] "integer"
To avoid this, you need to actually pass an obscure keyword parameter:
> w2 = u[,1,drop=FALSE]
> w2[1,1]
[3] 1
> class(w2)
[1] "matrix"
There's a lot of nooks and crannies like that. Your best friend at the beginning will be introspection and online help tools like str,class,example, and of course help. Also, make sure to look at the example code on the R Graph Gallery and in Ripley's Modern Applied Statistics with S-Plus book.
EDIT: Here's another great example with factors.
> xx = factor(c(3,2,3,4))
> xx
[1] 3 2 3 4
Levels: 2 3 4
> yy = as.numeric(xx)
> yy
[1] 2 1 2 3
Holy cow! Converting something from a factor back to a numeric didn't actually do the conversion you thought it would. Instead it's doing it on the internal enumerated type of the factor. This is a source of hard-to-find bugs for people who aren't aware of this, because it's still returning integers and will in fact actually work some of the time (when the input is already numerically ordered).
This is what you actually need to do
> as.numeric(levels(xx))[xx]
[1] 3 2 3 4
Yeah, sure, that fact is on the factor help page, but you only land up there when you've lost a few hours to this bug. This is another example of how R does not do what you intend. Be very, very careful with anything involving type conversions or accessing elements of arrays and lists.
This isn't specifically addressing the Python vs. R background, but the R inferno is a great resource for programmers coming to R.
The accepted answer to this post is possibly a bit outdated. The Pandas Python library now provides amazing R-like DataFrame support.
There may be... but before you embark on that have you tried some of the available Python extensions? Scipy has a list.

Categories