US Patent 11372565 Facilitating data reduction using weighted similarity digest

A technique for managing data storage includes generating digests of chunks of data. The digests are arranged to provide similar results for similar chunks but to weight contributions from different regions of chunks differently based on their entropy. Chunks that differ in low-entropy regions tend to produce more similar digests than do chunks that differ in high-entropy regions. The technique further includes identifying similar chunks based on similarity of their digests and preforming data reduction on one chunk by reference to another, storing any differences between the chunks separately.