I am following a book which has the following code:
import numpy as np
np.random.seed(1)
streetlights = np.array([[1, 0, 1], [0, 1, 1], [0, 0, 1], [1, 1, 1]])
walk_vs_stop = np.array([[1, 1, 0, 0]]).T
def relu(x):
return (x > 0) * x
def relu2deriv(output):
return output > 0
alpha = 0.2
hidden_layer_size = 4
# random weights from the first layer to the second
weights_0_1 = 2*np.random.random((3, hidden_layer_size)) -1
# random weights from the second layer to the output
weights_1_2 = 2*np.random.random((hidden_layer_size, 1)) -1
for iteration in range(60):
layer_2_error = 0
for i in range(len(streetlights)):
layer_0 = streetlights[i : i + 1]
layer_1 = relu(np.dot(layer_0, weights_0_1))
layer_2 = relu(np.dot(layer_1, weights_1_2))
layer_2_error += np.sum((layer_2 - walk_vs_stop[i : i + 1])) ** 2
layer_2_delta = layer_2 - walk_vs_stop[i : i + 1]
layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1)
weights_1_2 -= alpha * layer_1.T.dot(layer_2_delta)
weights_0_1 -= alpha * layer_0.T.dot(layer_1_delta)
if iteration % 10 == 9:
print(f"Error: {layer_2_error}")
Which outputs:
# Error: 0.6342311598444467
# Error: 0.35838407676317513
# Error: 0.0830183113303298
# Error: 0.006467054957103705
# Error: 0.0003292669000750734
# Error: 1.5055622665134859e-05
I understand everything but this part is not explained and I am not sure why it is the way it is:
weights_0_1 = 2*np.random.random((3, hidden_layer_size)) -1
weights_1_2 = 2*np.random.random((hidden_layer_size, 1)) -1
I don't understand:
Why there is 2* the whole matrix and why is there a -1
If I change 2 to 3 my error becomes greatly lower # Error: 5.616513576418916e-13
I tried changing the 2 to many other numbers along with the change of -1 to many other numbers I get # Error: 2.0 most of the time or the Error is much worst than combination of 3 and -1.
I can't seem to grasp the relationship and the purpose of multiplying the random weights by a number and subracting a number afterwards.
P.S. The idea of the network is to understand a streetlight pattern when people should go and when they should stop depending what combination of the lights in streetlight is on / off.
There is a lot of ways to initialize neural network, and it's a current research subject as it can have a great impact on performance and training time. Some rules of thumb :
avoid having only one value for all weights, as they would all update the same
avoid having too large weights that could make your gradient too high
avoid having too small weights that could make your gradient vanish
In your case, the goal is just to have something between [-1;1] :
np.random.random gives you a float in [0;1]
multiply by 2 gives you something in [0;2]
substract 1 gives you a number in [-1;1]
2*np.random.random((3, 4)) -1 is a way to generated 3*4=12 random number from uniform distribution of half-open interval [-1, +1) i.e including -1 but excluding +1.
This is equivalent to more readable code
np.random.uniform(-1, 1, (3, 4))
Related
I recently learned about Q-Learning with the example of the Gym environment "CartPole-v1".
The predict function of said model always returns a vector that looks like [[ 0.31341377 -0.03776223]]. I created my own little game, where the Ai has to move left or right with ouput 0 and 1. I just show a list [0, 0, 1, 0, 0] to the network, if it ouputs 0 it goes left, if it outputs 1 it goes right. Reach the left 0 and you win, right 0 and you lose. Really easy. When i print my ouput Vector however, i always get something like this:
[[0.01347399 0.04450664]
[0.01347399 0.04450664]
[0.01347399 0.04450664]
[0.1216775 0.38299465]
[0.01347399 0.04450664]]
This messes with the learning function because np.argmax() then returns something like or 5 and the network cannot handle this given the fact that there are only 2 actions to begin with.
This is the init of my model:
def __init__(self, state_shape, num_actions, lr):
super(DQN, self).__init__()
self.state_shape = state_shape # (1,)
self.num_actions = num_actions # 2
self.lr = lr # 1e-3
input_state = Input(shape=state_shape)
x = Dense(20)(input_state)
x = Activation('relu')(x)
x = Dense(20)(x)
x = Activation('relu')(x)
output_pred = Dense(self.num_actions)(x)
self.model = Model(inputs=input_state, outputs=output_pred)
self.model.compile(loss="mse", optimizer=Adam(lr=self.lr))
Full code is available at https://www.mediafire.com/file/rq7ogjxpr990e51/dqn.py/file.
How do i crop the output vector? Or how would i have to change my inputs to get a useful output?
Edit:
I've experimented a little more, and increasing num_actions from currently 2 to for example 4 does increase the vector horizontally, so it looks like this:
[[ 0.00109814 0.01464381 -0.00270887 -0.00422738]
[ 0.00109814 0.01464381 -0.00270887 -0.00422738]
[-0.01450843 0.10628925 -0.06114068 -0.10908635]
[ 0.00109814 0.01464381 -0.00270887 -0.00422738]
[ 0.00109814 0.01464381 -0.00270887 -0.00422738]]
This means num_actions as 2 is not the problem, its rather that it ouputs 5 lines instead of 1.
So after even more experiments, i have found a solution.
The input is still the list [0, 0, 1, 0, 0] which has the len() = 5.
This explains the five rows. If its changed to [[0, 0, 1, 0, 0]] and the state_shape is changed to (5, ) it works and I get a vector with 2 values.
*All the other functions that access the list have to be changed from board[idx] to board[0][idx].
I've been working on a simple neural network.
It takes in a data set with 3 columns, if the first column's value is a 1, then the output should be a 1.
I've provided comments so it is easier to follow.
Code is as follows:
import numpy as np
import random
def sigmoid_derivative(x):
return x * (1 - x)
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def think(weights, inputs):
sum = (weights[0] * inputs[0]) + (weights[1] * inputs[1]) + (weights[2] * inputs[2])
return sigmoid(sum)
if __name__ == "__main__":
# Assign random weights
weights = [-0.165, 0.440, -0.867]
# Training data for the network.
training_data = [
[0, 0, 1],
[1, 1, 1],
[1, 0, 1],
[0, 1, 1]
]
# The answers correspond to the training_data by place,
# so first element of training_answers is the answer to the first element of training_data
# NOTE: The pattern is if there's a 1 in the first place, the result should be a one
training_answers = [0, 1, 1, 0]
# Train the neural network
for iteration in range(50000):
# Pick a random piece of training_data
selected = random.randint(0, 3)
training_output = think(weights, training_data[selected])
# Calculate the error
error = training_output - training_answers[selected]
# Calculate the adjustments that need to be applied to the weights
adjustments = np.dot(training_data[selected], error * sigmoid_derivative(training_output))
# Apply adjustments, maybe something wrong is going here?
weights += adjustments
print("The Neural Network has been trained!")
# Result of print below should be close to 1
print(think(weights, [1, 0, 0]))
The result of the last print should be close to 1, however it is not?
I have a feeling that I'm not adjusting the weights correctly.
I want to use the shape (size) of a tf.where for the bound of a for-loop using Tensorflow in Pycharm. However, when I try to do this, I get the error: 'Tensor' object cannot be interpreted as an integer.
How can I solve this problem?
Development contents.
1. Find the index (idxCut) corresponding to the threshold in the data.
2. Check whether data corresponding to idxCut is TPR.
I want to find the TPR (Turning Point Ratio) about idxCut in the data using a for-loop.
I used a for-loop to obtain the TPR between idx, idx-1 and idx + 1.
I want to find data[idx] is higher than the others data[idx-1, idx+1].
Here's my code:
def funCalculate(data):
### Cut-off Threshold
idxCut = tf.where(data > cutoff)
idxCut = tf.squeeze(idxCut)
### Compute by the size of idxCut
valueCut = []
for ii in range(0, tf.shape(idxCut)):
v1 = tf.where(data[idxCut[ii]] > data[idxCut[ii] - 1], 1, 0)
v2 = tf.where(data[idxCut[ii]] > data[idxCut[ii] + 1], 1, 0)
v3 = tf.where(v1 + v2 > 1, 1, 0)
valueCut.append(v3)
return valueCut
I'm trying to code a neural network with 3 input nodes, a hidden layer with 4 nodes, and 1 output node. (This will change in the final version, so I've turned them into variables.) Even though I've compared my code to the tutorials, and I can't see anything wrong with it, when I try to run it, it gives me a ValueError saying that it can't dot-multiply these arrays.
As this is my first project using NumPy, I'm at a loss as to what I'm supposed to do.
Here's my code:
import numpy as np
import math
# neurons
n_in = 3
n_hidden = 4
n_out = 1
batchsize = 60
def sigmoid(x, deriv=False):
if deriv:
return x*(1-x)
return 1/(1+np.exp(-x))
def error(expected, actual):
rawError = expected - actual
for cell in rawError:
cell = cell * cell
return rawError
# input data
X = np.array([
[0, 0, 1],
[1, 1, 1],
[1, 0, 1],
[0, 1, 1]
])
# answer data
Y = np.array([0, 1, 1, 0]).T
np.random.seed(0)
# synapses
syn0 = 2 * np.random.random((n_in, n_hidden)) - 1
syn1 = 2 * np.random.random((n_hidden, n_out)) - 1
# train
for j in range(60000):
# feed forward to hidden
l1 = sigmoid(np.dot(X, syn0))
# feed forward to out
l2 = sigmoid(np.dot(l1, syn1))
# calculate error in new array
l2_error = error(Y, l2)
if j % 10000 == 9999:
print(np.sum(l2_error))
# gradient descent:
# multiply the error by the input, then the gradient of sigmoid
l2_nudge = l2_error * sigmoid(l2, deriv=True)
l1_nudge = l2_nudge.dot(syn1.T) * sigmoid(l1, deriv=True)
syn1 += l1.T.dot(l2_nudge)
syn0 += l0.T.dot(l1_nudge)
print(l2)
I expected the program to at least run, but it gives me the following error:
Traceback (most recent call last):
File "neural-network.py", line 68, in <module>
l1_nudge = l2_nudge.dot(syn1.T) * sigmoid(l1, deriv=True)
ValueError: shapes (4,4) and (1,4) not aligned: 4 (dim 1) != 1 (dim 0)
I found my issue: I was doing the dot product for l1_error backwards!
l1_error = np.dot(l2_nudge, syn1.T)
I'm doing a robust linear regression on only a constant (a column of 1s) and no exogenous variable. I'm able to calculate the model just fine by inputting a list of 1's equal to the size of the 'xi_list' from the code snippet below.
def sigma_and_miu(gvkey, statevar_dict):
statevar_list = statevar_dict[gvkey]
xi_list = [np.log(statevar_list[i]) - np.log(statevar_list[i-1]) for i in range(1, len(statevar_list))]
x = np.array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
y = np.array(xi_list)
rlm_model = sm.RLM(y, x, M=sm.robust.norms.HuberT())
rlm_results = rlm_model.fit()
sigma = np.std(rlm_results.resid * rlm_results.weights)
miudelta = rlm_results.params[0] + (0.5 * sigma ** 2)
return miudelta, sigma
This function is ran with the following inputs.
dict = {1004:[1796.6, 1938.6, 2085.4, 2009.4, 1906.1, 2002.2, 2164.9, 2478.8, 2357.4, 2662.1, 2911.2, 2400.4, 2535.9, 2812.3, 2873.1, 2775.5, 3374.2, 3345.5, 3466.3, 2409.4]}
key = 1004
miu, sigma = sigma_and_miu(key,dict)
However, I'm looking for a more scalable approach. I was thinking that one solution could be to include a loop that appends as many 1's as the length of the xi_list variable but, this does not seem to be very efficient.
I know there is sm.add_constant() and I tried to add this constant to my 'y' variable and leaving 'x' blank in the sm.RLM() function. This results in not being able to run the model.
So my question is, whether there is a better way to create the list of 1s or should I just go for the loop?
Use basic numpy vectorized computation
e.g.
statevar = np.asarray(statevar_list)
y = np.log(statevar[1:]) - np.log(statevar[:-1])
x = np.ones(len(y))
Aside: The rlm_results should have the robust estimate of the standard deviation that is used in the estimation as a scale attribute.