A method, computer program, and computer system is provided for coding video data. Reference samples and magnitudes of transform coefficients corresponding to a current block of video data from an input to a neural network are identified. Sign values associated with the transform coefficients are predicted based on at least the identified reference samples. The video data is encoded/decoded based on the predicted sign values.