I'm confused by Jax documentation, here's what I'm trying to do:
def line(m,x,b):
return m*x + b
grad(line)(1,2,3)
And the error:
---------------------------------------------------------------------------
FilteredStackTrace Traceback (most recent call last)
<ipython-input-48-d14b17620b30> in <module>()
3
----> 4 grad(line)(1,2,3)
FilteredStackTrace: TypeError: grad requires real- or complex-valued inputs (input dtype that is a sub-dtype of np.floating or np.complexfloating), but got int32. If you want to use integer-valued inputs, use vjp or set allow_int to True.
The stack trace above excludes JAX-internal frames.
The following is the original exception that occurred, unmodified.
--------------------
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
6 frames
/usr/local/lib/python3.7/dist-packages/jax/api.py in _check_input_dtype_revderiv(name, holomorphic, allow_int, x)
844 elif not allow_int and not (dtypes.issubdtype(aval.dtype, np.floating) or
845 dtypes.issubdtype(aval.dtype, np.complexfloating)):
--> 846 raise TypeError(f"{name} requires real- or complex-valued inputs (input dtype that "
847 "is a sub-dtype of np.floating or np.complexfloating), "
848 f"but got {aval.dtype.name}. If you want to use integer-valued "
TypeError: grad requires real- or complex-valued inputs (input dtype that is a sub-dtype of np.floating or np.complexfloating), but got int32. If you want to use integer-valued inputs, use vjp or set allow_int to True.
I'm referencing the official tutorial code:
import jax.numpy as jnp
from jax import grad, jit, vmap
from jax import random
key = random.PRNGKey(0)
def sigmoid(x):
return 0.5 * (jnp.tanh(x / 2) + 1)
# Outputs probability of a label being true.
def predict(W, b, inputs):
return sigmoid(jnp.dot(inputs, W) + b)
# Build a toy dataset.
inputs = jnp.array([[0.52, 1.12, 0.77],
[0.88, -1.08, 0.15],
[0.52, 0.06, -1.30],
[0.74, -2.49, 1.39]])
targets = jnp.array([True, True, False, True])
# Training loss is the negative log-likelihood of the training examples.
def loss(W, b):
preds = predict(W, b, inputs)
label_probs = preds * targets + (1 - preds) * (1 - targets)
return -jnp.sum(jnp.log(label_probs))
# Initialize random model coefficients
key, W_key, b_key = random.split(key, 3)
W = random.normal(W_key, (3,))
b = random.normal(b_key, ())
W_grad = grad(loss, argnums=0)(W, b)
print('W_grad', W_grad)
And the result:
W_grad [-0.16965576 -0.8774648 -1.4901345 ]
What am I doing wrong here? I gather key is being used in some important way, but I can't figure out why/how it's necessary. To answer this question, please adjust code in the first block as necessary to remove the error.
Jax is telling you it doesn't like integers. grad(line)(1.,2.,3.) (using floats) fixes the problem.
I think the Error here is clear:
TypeError: grad requires real- or complex-valued inputs (input dtype that is a sub-dtype of np.floating or np.complexfloating), but got int32. If you want to use integer-valued inputs, use vjp or set allow_int to True.
To use grad(line)(1,2,3) with Int32, change it to grad(line, allow_int=True)(1,2,3)
Related
I'm trying to use JAX to generate samples from multivariate normal distribution using:
import jax
import jax.numpy as jnp
import numpy as np
key = random.PRNGKey(0)
cov = np.array([[1.2, 0.4], [0.4, 1.0]])
mean = np.array([3,-1])
x1,x2 = jax.random.multivariate_normal(key, mean, cov, 5000).T
However when I run the code I get the following error:
TypeError Traceback (most recent call last)
<ipython-input-25-1397bf923fa4> in <module>()
2 cov = np.array([[1.2, 0.4], [0.4, 1.0]])
3 mean = np.array([3,-1])
----> 4 x1,x2 = jax.random.multivariate_normal(key, mean, cov, 5000).T
1 frames
/usr/local/lib/python3.6/dist-packages/jax/core.py in canonicalize_shape(shape)
1159 "got {}.")
1160 if any(isinstance(x, Tracer) and isinstance(get_aval(x), ShapedArray)
-> 1161 and not isinstance(get_aval(x), ConcreteArray) for x in shape):
1162 msg += ("\nIf using `jit`, try using `static_argnums` or applying `jit` to "
1163 "smaller subfunctions.")
TypeError: 'int' object is not iterable
I'm not sure what the problem is since the same syntax works for the equivalent function in Numpy
In the jax.random module, most shapes must explicitly be tuples. So instead of shape 5000, use (5000,):
x1,x2 = jax.random.multivariate_normal(key, mean, cov, (5000,)).T
I keep getting the error IndexError: tuple index out of range and i am not sure what is happening. My code was working just fine however when i restarted the jupyter notebook i started receiving this error.
this is my code:
X = df.Tweet
y = df.target
from sklearn import linear_model
import pyswarms as ps
# Create an instance of the classifier
classifier = linear_model.LogisticRegression()
# Define objective function
def f_per_particle(m, alpha):
total_features = X.shape[1]
# Get the subset of the features from the binary mask
if np.count_nonzero(m) == 0:
X_subset = X
else:
X_subset = X[:,m==1]
# Perform classification and store performance in P
classifier.fit(X_subset, y)
P = (classifier.predict(X_subset) == y).mean()
# Compute for the objective function
j = (alpha * (1.0 - P)
+ (1.0 - alpha) * (1 - (X_subset.shape[1] / total_features)))
return j
[some more code]
options = {'c1': 0.5, 'c2': 0.5, 'w':0.9, 'k': 30, 'p':2}
# Call instance of PSO
dimensions = X.shape[1] # dimensions should be the number of features
optimizer = ps.discrete.BinaryPSO(n_particles=30, dimensions=dimensions, options=options)
# Perform optimization
cost, pos = optimizer.optimize(f, iters=1000)
i received the following traceback:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-76-bea8cf064cd2> in <module>
2
3 # Call instance of PSO
----> 4 dimensions = X.shape[1] # dimensions should be the number of features
5 optimizer = ps.discrete.BinaryPSO(n_particles=30, dimensions=dimensions, options=options)
6
IndexError: tuple index out of range
It is not absolutely clear, but it seems to me that your df variable might be a Pandas dataframe, and your df.Tweet may be a Pandas series.
In that case, being a series, your X will have only one dimension (so, only the first element of the tuple X.shape, X.shape[0]), instead of two dimensions - reason for the index out of range exception in your code. The two dimensions case occurs only when the variable is a dataframe.
More information: https://www.google.com/amp/s/www.geeksforgeeks.org/python-pandas-series-shape/amp/
I am trying to configure PyMC3 Polynomial kernel with the following hyperpriors;
with pm.Model() as self.model:
EPSILON = 0.1
l = pm.Gamma("l", alpha=2, beta=1)
offset = pm.Gamma("offset", alpha=2, beta=1)
nu = pm.HalfCauchy("nu", beta=1)
d = pm.HalfNormal("d", sd=5)
cov = nu ** 2 * pm.gp.cov.Polynomial(X.shape[1], l, d, offset)
self.gp = pm.gp.Marginal(cov_func=cov)
sigma = pm.HalfCauchy("sigma", beta=1)
y_ = self.gp.marginal_likelihood("y", X=X, y=Y, noise=sigma)
self. map_trace = [pm.find_MAP()]
However, I'm getting Cholesky decomposition failed error as follows;
LinAlgError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/theano/compile/function_module.py in __call__(self, *args, **kwargs)
902 outputs =\
--> 903 self.fn() if output_subset is None else\
904 self.fn(output_subset=output_subset)
24 frames
LinAlgError: 7-th leading minor of the array is not positive definite
During handling of the above exception, another exception occurred:
LinAlgError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/scipy/linalg/decomp_cholesky.py in _cholesky(a, lower, overwrite_a, clean, check_finite)
38 if info > 0:
39 raise LinAlgError("%d-th leading minor of the array is not positive "
---> 40 "definite" % info)
41 if info < 0:
42 raise ValueError('LAPACK reported an illegal value in {}-th argument'
LinAlgError: 7-th leading minor of the array is not positive definite
Apply node that caused the error: Cholesky{lower=True, destructive=False, on_error='raise'}(Elemwise{Composite{((sqr(i0) * i1) + i2 + i3)}}[(0, 0)].0)
Toposort index: 11
Inputs types: [TensorType(float64, matrix)]
Inputs shapes: [(40, 40)]
Inputs strides: [(320, 8)]
Inputs values: ['not shown']
Outputs clients: [[Solve{A_structure='lower_triangular', lower=False, overwrite_A=False, overwrite_b=False}(Cholesky{lower=True, destructive=False, on_error='raise'}.0, TensorConstant{[ 69.79 .. 472.83]}), Solve{A_structure='lower_triangular', lower=False, overwrite_A=False, overwrite_b=False}(Cholesky{lower=True, destructive=False, on_error='raise'}.0, Elemwise{Composite{(sqr(i0) * i1)}}[(0, 0)].0)]]
Changing the hyperpriors seems to change the error like instead of 7th leading minor it will show some other x-th leading minor. But I'm not sure if this is caused by hyperpriors or something else.
Any thoughts are welcome :)
Thanks
import pandas as pd
import numpy as np
class CLF:
Weights = 0
def fit(DF_input, DF_output, eta=0.1, drop=1000):
X, y = DF_input.to_numpy(copy=True), DF_output.to_numpy(copy=True)
N,d = X.shape
m = len(np.unique(y))
self.Weights = np.random.normal(0,1, size=(d,m))
INPUT = pd.read_csv(path_input)
OUTPUT = pd.read_csv(path_output)
clf = CLF()
clf.fit(INPUT, OUTPUT)
I defined a method .fit() for the class I wrote. The first step is convert two dataframes into numpy arrays. However, I got the following error when I tried to use the method, although INPUT.to_numpy(copy=True) and OUTPUT.to_numpy(copy=True) both work fine in their own right. Can somebody help me out here? Why was to_numpy recognized as an attribute rather than a method of dataframes?
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-22-a3d455104534> in <module>
1 clf = CLF()
----> 2 clf.fit(INPUT, OUTPUT)
<ipython-input-16-57babd738b2d> in fit(DF_input, DF_output, eta, drop)
4
5 def fit(DF_input, DF_output, eta=0.1,drop=1000):
----> 6 X, y = DF_input.to_numpy(copy=True), DF_output.to_numpy(copy=True)
7 N,d = X.shape
8 m = len(np.unique(y)) # number of classes
AttributeError: 'CLF' object has no attribute 'to_numpy'
Your problem is that the first input for object method is usually reserved for self. The correct syntax should be:
class CLF:
Weights = 0
# notice the `self`
def fit(self, DF_input, DF_output, eta=0.1, drop=1000):
X, y = DF_input.to_numpy(copy=True), DF_output.to_numpy(copy=True)
N,d = X.shape
m = len(np.unique(y))
self.Weights = np.random.normal(0,1, size=(d,m))
INPUT = pd.read_csv(path_input)
OUTPUT = pd.read_csv(path_output)
clf = CLF()
clf.fit(INPUT, OUTPUT)
An instance method is a type of attribute; this is a more general error message that keys on the . (dot) operator, rather than parsing through to the left parenthesis to discriminate your usage.
The problem is that you defined an instance method fit, but named your instance as DF_input. I think you simply forgot the usual self naming for the implicit instance parameter.
I'm trying to implement my own Bernoulli class with its own fit function in order to fit my train and test lists that contains words (spam detection)
here's my Bernoulli class:
class BernoulliNB(object):
def __init__(self, alpha=1.0):
self.alpha = alpha
def fit(self, X, y):
count_sample = len(X)
separated = [[x for x, t in zip(X, y) if t == c] for c in np.unique(y)]
self.class_log_prior_ = [np.log(len(i) / count_sample) for i in separated]
count = np.array([np.array(i).sum(axis=0) for i in separated]) + self.alpha
smoothing = 2 * self.alpha
n_doc = np.array([len(i) + smoothing for i in separated])
self.feature_prob_ = count / n_doc[np.newaxis].T
return self
def predict_log_proba(self, X):
return [(np.log(self.feature_prob_) * x + \
np.log(1 - self.feature_prob_) * np.abs(x - 1)
).sum(axis=1) + self.class_log_prior_ for x in X]
def predict(self, X):
return np.argmax(self.predict_log_proba(X), axis=1)
And here's my implementation:
nb = BernoulliNB(alpha=1).fit(train_list, test_list)
Expected result:
Been able to fit with my class my train and test lists
But instead I get the following error:
TypeError: cannot perform reduce with flexible type
on the following line:
count = np.array([np.array(i).sum(axis=0) for i in separated]) + self.alpha
I don't know why it fails though, maybe due to the fact that I have lists instead of np? not even sure how to fix it.
Can someone help me or explain to me how to achieve the fitting?
I get this error message by apply sum to a structured array:
In [754]: np.array([(1,.2),(3,.3)], dtype='i,f')
Out[754]: array([(1, 0.2), (3, 0.3)], dtype=[('f0', '<i4'), ('f1', '<f4')])
In [755]: _.sum(axis=0)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-755-69a91062a784> in <module>()
----> 1 _.sum(axis=0)
/usr/local/lib/python3.6/dist-packages/numpy/core/_methods.py in _sum(a, axis, dtype, out, keepdims, initial)
34 def _sum(a, axis=None, dtype=None, out=None, keepdims=False,
35 initial=_NoValue):
---> 36 return umr_sum(a, axis, dtype, out, keepdims, initial)
37
38 def _prod(a, axis=None, dtype=None, out=None, keepdims=False,
TypeError: cannot perform reduce with flexible type
I'm guessing your error occurs in
np.array(i).sum(axis=0)
and that i produces, or is a structured array.
I can't recreate your run just by reading your fit code. You'll need to run it with some diagnostic prints (focus on shape and dtype). A general observation, when running numpy code, never assume that you got things like shape and dtype right. Verify!