I'm attempting to map out coordinates for a circle and export those values to excel to then plot the values. My output CSV file returns the correct values but with brackets around the values ex x,y, [0.],[9.]. I would like to learn how to remove those brackets with my current code attempt if possible.
import numpy as np
from itertools import zip_longest
r = 9
h = 1
k = 1
# r = int(input('Radius'))
# h = int(input('h'))
# k = int(input('k'))
deg = np.arange(0, 361, 1)[:, np.newaxis]
theta = (np.pi * deg * 2)/360
x = (r * np.sin(theta) + h)
y = (r * np.cos(theta) + k)
d = x, y
Points = zip_longest(*d, fillvalue = '')
with open('test.csv', 'w', encoding="ISO-8859-1", newline='') as myfile:
wr = csv.writer(myfile)
wr.writerow(('x','y','r','h','k'))
wr.writerows(Points)
myfile.close()
x and y are arrays with shape (361,1). When you iterate over such an array (as zip_longest will do), you get a stream of arrays of shape (1,) which is where your surplus brackets come from.
One solution would be to make your arrays (361,) instead (i.e, one-dimensional). Then iterating would give scalars.
Another solution would be to use np.hstack([x,y]) instead of zip, and tossing the entire resulting (361,2) array to writerows.
(Off topic: You do not need myfile.close(); the with ... construct takes care of that for you — that's what it's for!)
Related
I'm using Rasterio to work with a satellite image, and I need to iterate through the entire file. and applying the formula to each pixel. This process takes a long time and makes it difficult for me to try out different modifications because it takes a long time to see the results each time.
any suggestions to improve time execution ?
And is it better to work on this project locally or via Jupiter, Google Colab, or other tools?
def dn_to_radiance(data_array, band_number):
# getting the G value
channel_gain = float(Landsat8_mlt_dict['RADIANCE_MULT_BAND_' + str(band_number) + ' '])
# Getting the B value
channel_offset = float(Landsat8_mlt_dict['RADIANCE_ADD_BAND_' + str(band_number) + ' '])
# creating a temp array to store the radiance value
# np.empty_like Return a new array with the same shape and type as a given array.
new_data_array = np.empty_like(data_array)
# loooping through the image
for i, row in enumerate(data_array):
for j, col in enumerate(row):
# checking if the pixel value is not nan, to avoid background correction
if data_array[i][j].all() != np.nan:
new_data_array[i][j] = data_array[i][j] * channel_gain + channel_offset
print(f'Radiance calculated for band {band_number}')
return new_data_array
Landsat8_mlt_dict = {}
with open('LC08_L2SP_190037_20190619_20200827_02_T1_MTL.txt', 'r') as _:
# print(type(_))
for line in _:
line = line.strip()
if line != 'END':
key, value = line.split('=')
Landsat8_mlt_dict[key] = value
# print(Landsat8_mlt_dict)
def radiance_to_reflectance(arr, ESUN, ):
# getting the d value
d = float(Landsat8_mlt_dict['EARTH_SUN_DISTANCE '])
# calculating rh phi value from theta
phi = 90 - float(Landsat8_mlt_dict['SUN_ELEVATION '])
# creating the temp array
new_data_array = np.empty_like(arr)
# loop to finf the reflectance
for i, row in enumerate(arr):
for j, col in enumerate(row):
if arr[i][j].all() != np.nan:
new_data_array[i][j] = np.pi * arr[i][j] * d ** 2 / (ESUN * cos(phi * math.pi / 180))
print(f"Reflectance of Band calculated")
return new_data_array
You could use thir-party libraries such as EOReader to convert Landsat bands to reflectance for you.
from eoreader.reader import Reader
from eoreader.bands import RED, GREEN
prod = Reader().open(r"LC08_L1TP_200030_20201220_20210310_02_T1.tar")
# Load those bands as a dict of xarray.DataArray
bands = prod.load([RED, GREEN])
green = bands[GREEN]
red = bands[RED]
Disclaimer: I am the maintener of EOReader
If you want to do that yourself, you should do some tutorials on how to handle arrays in Python.
Never ever loop over them! You should instead vectorize your computations: it will go way faster!
I am trying to save a large matrix, 1000x1000 which follows log-normal distribution. But the saved file turns out to be empty. What am I doing incorrectly here?
import numpy as np
import csv
with open('Radius.csv', 'w') as f:
shape = 1000,1000
zmin, zmax = 0.2,0.8
n = np.prod(shape)
zc = np.array([])
while True:
z = np.random.lognormal(mean=0.2, sigma=0.5, size=n * 100)
z = z[(zmin <= z) & (z < zmax)]
z = np.r_[zc, z]
if len(z) >= n:
break
inv_r = z[:n].reshape(shape)
print("1/r =",[inv_r])
writer = csv.writer(f)
writer.writerows(zip(1,[inv_r]))
It has to do with the way you are writing to rows, the zip function takes in two iterables, you passed in an int and an iterable [list]
the while loop also only ever will go through once as it stands. If you run this:
import numpy as np
import csv
with open('Radius.csv', 'w+') as f:
shape = 1000,1000
zmin, zmax = 0.2,0.8
n = np.prod(shape)
zc = np.array([])
z = np.random.lognormal(mean=0.2, sigma=0.5, size=n * 100)
z = z[(zmin <= z) & (z < zmax)]
z = np.r_[zc, z]
inv_r = z[:n].reshape(shape)
print("1/r =",[inv_r])
writer = csv.writer(f)
writer.writerows(inv_r)
it will at least log to the csv, definitely check your zip function to make sure it does what you want it to!
I have a numpy array of arrays, say 400x80. I want to turn it into an array 400x160 so that each item would be formed like this:
Here each frame of 80 is copied into the beginning of the next frame and the first frame gest 80 zeroes. So how to do such thing in numpy? Is there a mechanism that can generalize to three or more frames?
Lets assume that your data is in X, then
np.hstack((np.vstack((np.zeros(X.shape[1]), X[:-1])), X))
where:
np.vstack((np.zeros(X.shape[1]), X[:-1]))
creates the first column, we add a row of zeros, and cut the last row
and then with hstack we just combine the two "columns" together.
import numpy as np
X = np.random.normal(size=(400, 80))
print(np.hstack((np.vstack((np.zeros(X.shape[1]), X[:-1])), X)).shape)
gives (400, 160) as expected.
Or you can do things manually:
Y = []
previos = np.zeros(X.shape[1])
for row in X:
Y.append(np.vstack((previous, row)))
previous = row
Y = np.array(Y)
You are trying to create a sliding window. If you want a view that looks into the original buffer, indexing the same memory locations multiple times, you can make some adjustments.
m, n = a.shape
p = 2 * n
x = np.lib.stride_tricks.as_strided(a, shape=(m * ((n - 1) + p) // p, p), strides=a.strides)
This is the totally general approach. If you're guaranteed p % n == 0, then for k = p // n, you can do
x = np.lib.stride_tricks.as_strided(a, shape=(m - k + 1, n * k), strides=a.strides)
In either case, to avoid memory issues, you can use x.copy()
I'm developing Python software for someone and they specifically requested that I use their DFT function, written in MATLAB, in my program. My translation is just plain not working, tested with sin(2*pi*r).
The MATLAB function below:
function X=dft(t,x,f)
% Compute DFT (Discrete Fourier Transform) at frequencies given
% in f, given samples x taken at times t:
% X(f) = sum { x(k) * e**(2*pi*j*t(k)*f) }
% k
shape = size(f);
t = t(:); % Format 't' into a column vector
x = x(:); % Format 'x' into a column vector
f = f(:); % Format 'f' into a column vector
W = exp(-2*pi*j * f*t');
X = W * x;
X = reshape(X,shape);
And my Python interpretation:
def dft(t, x, f):
i = 1j #might not have to set it to a variable but better safe than sorry!
w1 = f * t
w2 = -2 * math.pi * i
W = exp(w1 * w2)
newArr = W * x
return newArr
Why am I having issues? The MATLAB code works fine but the Python translation outputs a weird increasing sine curve instead of a Fourier transform. I get the feeling Python is handling the calculations slightly differently but I don't know why or how to fix this.
Here's your MATLAB code -
t = 0:0.005:10-0.005;
x = sin(2*pi*t);
f = 30*(rand(size(t))+0.225);
shape = size(f);
t = t(:); % Format 't' into a column vector
x = x(:); % Format 'x' into a column vector
f = f(:); % Format 'f' into a column vector
W = exp(-2*pi*1j * f*t'); %//'
X = W * x;
X = reshape(X,shape);
figure,plot(f,X,'ro')
And here's one version of numpy ported code might look like -
import numpy as np
from numpy import math
import matplotlib.pyplot as plt
t = np.arange(0, 10, 0.005)
x = np.sin(2*np.pi*t)
f = 30*(np.random.rand(t.size)+0.225)
N = t.size
i = 1j
W = np.exp((-2 * math.pi * i)*np.dot(f.reshape(N,1),t.reshape(1,N)))
X = np.dot(W,x.reshape(N,1))
out = X.reshape(f.shape).T
plt.plot(f, out, 'ro')
MATLAB Plot -
Numpy Plot -
Numpy arrays do element wise multiplication with *.
You need np.dot(w1,w2) for matrix multiplication using numpy arrays (not the case for numpy matrices)
Make sure you are clear on the distinction between Numpy arrays and matrices. There is a good help page "Numpy for Matlab Users":
http://wiki.scipy.org/NumPy_for_Matlab_Users
Doesn't appear to be working at present so here is a temporary link.
Also, use t.T to transpose a numpy array called t.
I have written the following code for creating a 2D array and filing the first element of each row. I am new to numpy. Is there a better way to do this?
y=np.zeros(N*T1).reshape(N,T1)
x = np.linspace(0,L,num = N)
for k in range(0,N):
y[k][0] = np.sin(PI*x[k]/L)
Yes, since numpy vectorizes operations, you can just do:
y[:,0] = np.sin(np.pi * x / L)
Note that y[:,0] grabs the first column of y (the : in the first coordinate essentially means "grab all rows", and the 0 in the second coordinate means "from the column at index 0" (ie the first column)). Since np.sin(np.pi * x / L) is also an array, you can assign the latter to the former directly.
This question is rather for codereview#stackexchange, but this snippet works!
import numpy as np
N = 1000 # arbitrary
T1 = 1000 # arbitrary
L = 10 # arbitrary
x = np.linspace(0,L,num = N)
# you don't need reshape here, give the size as a tuple!
y = np.zeros((N,T1))
# use a vectorized call here:
y[:,0] = np.sin(np.pi*x/L)