US Patent 11361213 Using lookup table to represent neural network activation function

Some embodiments provide a neural network inference circuit for implementing a neural network that includes multiple computation nodes at multiple layers. Each of a set of the computation nodes includes (i) a linear function that includes a dot product of input values and weight values and (ii) a non-linear activation function. The neural network inference circuit includes (i) a set of dot product circuits to compute dot products for the plurality of computation nodes and (ii) at least one computation node post-processing circuit to (i) receive a dot product for a computation node computed by the set of dot product circuits, (ii) compute a result of the linear function for the computation node based on the dot product, and (iii) use a lookup table to compute the non-linear activation function of the computation node from the result of the linear function to determine an output of the computation node.