How can I avoid value errors when using numpy.random.multinomial?
numerical-stability, numpy, python, random
Solution
I also encountered this problem during my language modelling work.
The root of this problem rises from numpy's implicit data casting: the output of my sorfmax() is in `float32` type, however, `numpy.random.multinomial()` will cast the `pval` into `float64` type IMPLICITLY. This data type casting would cause `pval.sum()` exceed 1.0 sometimes due to numerical rounding.
This issue is recognized and posted here
Problem
When I use this random generator: `numpy.random.multinomial`, I keep getting: ``` ValueError: sum(pvals[:-1]) > 1.0 ``` I am always passing the output of this softmax function: ``` def softmax(w, t = 1.0): e = numpy.exp(numpy.array(w) / t) dist = e / np.sum(e) return dist ``` except now that I am getting this error, I also added this for the parameter (`pvals`): ``` while numpy.sum(pvals) > 1: pvals /= (1+1e-5) ``` but that didn't solve it. What is the right way to make sure I avoid this error? EDIT: here is function that includes this code ``` def get_MDN_prediction(vec): coeffs = vec[::3] means = vec[1::3] stds = np.log(1+np.exp(vec[2::3])) stds = np.maximum(stds, min_std) coe = softmax(coeffs) while np.sum(coe) > 1-1e-9: coe /= (1+1e-5) coeff = unhot(np.random.multinomial(1, coe)) return np.random.normal(means[coeff], stds[coeff]) ```