Patent attributes
Computer systems and computer-implemented methods improve a base neural network. In an initial training, preliminary activations values computed for base network nodes for data in the training data set are stored in memory. After the initial training, a new node set is merged into the base neural network to form an expanded neural network, including directly connecting each of the nodes of the new node set to one or more base network nodes. Then the expanded neural network is trained on the training data set using a network error loss function for the expanded neural network. Training the expanded neural network comprises imposing a node-to-node relationship regularization for at least one base network node in the expanded neural network, where imposing the node-to-node relationship regularization comprises adding, during back-propagation of partial derivatives through the expanded neural network for a datum in the training data set, a regularization cost to the network error loss function for the at least one base network node based on a specified relationship between a stored preliminary activation value for the base network node for the datum and an activation value for the base network node of the expanded neural network for the datum.
