A device that is capable of performing tap detection and other gesture detection using data from an accelerometer. Motion data from the accelerometer is filtered and processed to detect motion. If motion is detected filtered motion data may be processed using a first neural network to detect a tap gesture, which may include one or multiple taps. Motion data may also be processed using a second neutral network to detect other gestures, which may be turned on or off based on a location of the device. Actions may be performed in response to a detected gesture, where the action may be performed by the device or by another device or system.