A composite building panel that includes a fabric reinforcing sheet between one of the metal skins and the core of the panel. The fabric reinforcing sheet, which is preferably made from aramid fibers improves the impact resistance and penetration resistance of the building panel without substantially increasing weight and without adding fuel content to the panel system.