US Patent 10320680 Load balancer that avoids short circuits

Network devices, such as load balancers may be configured to route requests to hosts that are responding in a shorter period of time than other hosts. Sometimes hosts respond in shorter periods of time due to errors (they short-circuit). Such behavior may cause a spike in failed requests and increase the impact of a host malfunction. Disclosed is an enhanced load balancing algorithm that reduces request loads to hosts that are responding to request more quickly than expected or historically observed. A load balancer tracks the hosts' performance. Upon detecting response times shorter than expected from a host, the load balancer will reduce the load on the host. The request routing will go back to normal distribution after the host behaving according to its known performance profile.