Supporting multiple graphics processing units (GPUs) comprises a first path coupled to a north bridge device (or a root complex device) and a first GPU, which may include a portion of the first GPU's total communication lanes. A second communication path may be coupled to the north bridge device and a second GPU and may include a portion of the second GPU's total communication lanes. A third communication path may be coupled between the first and second GPUs directly or through one or more switches that can be configured for single or multiple GPU operations. The third communication path may include some or all of the remaining communication lanes for the first and second GPUs. As a nonlimiting example, the first and second GPUs may each utilize an 8-lane PCI express communication path with the north bridge device and an 8-lane PCI express communication path with each other.