A programmable motion estimation module for processing pixel values from a sequence of digital video images. The module includes a programmable microcontroller, scalar and control register files, arithmetic logic units, a direct memory access unit and a vector array processor. The vector array processor includes a series of processing elements, a memory subsystem for storing pixel values and a crossbar switch for distributing pixel values from the memory subsystem to the processing elements. The module provides a flexible platform that can be programmed to implement a variety of different Motion Estimation (ME) algorithms using an associated Instruction Set Architecture without the need to modify the hardware.