System and methods for reducing the complexity or area of a non-linear Viterbi detector. In some embodiments, a Viterbi detector calculates branch metrics for a subset of the branches in a trellis diagram. This subset may be selected based on comparing an equalized signal with a signal level table of all the possible branches. These branch metrics may be calculated using high performance branch metric calculation techniques. The remaining branch metrics may be calculated based on the computed branch metrics using a technique that consumes fewer resources. The Viterbi detectors in the present invention may also be used in an iterative decoding scheme, where multiple detectors are cascaded. In these embodiments, a Viterbi detector may select a subset of the branches based on detection results from other Viterbi detectors.