US Patent 11037059 Self-supervised back propagation for deep learning

A computer-implemented method for analyzing a first neural network via a second neural network according to a differentiable function. The method includes adding a derivative node to the first neural network that receives derivatives associated with a node of the first neural network. The derivative node is connected to the second neural network such that the second neural network can receive the derivatives from the derivative node. The method further includes feeding forward activations in the first neural network for a data item, back propagating a selected differentiable function, providing the derivatives from the derivative node to the second neural network as data, feeding forward the derivatives from the derivative node through the second neural network, and then back propagating a secondary objective through both neural networks. In various aspects, the learned parameters of one or both of the neural networks can be updated according to the back propagation calculations.