A method for manufacturing a semiconductor device includes forming a semiconductor fin on a substrate. A dummy gate structure is formed crossing the semiconductor fin. The dummy gate structure is replaced with a metal gate structure. An epitaxial structure is formed in the semiconductor fin after replacing the dummy gate structure with the metal gate structure.