A fuel injection device of a diesel engine capable of smoothly starting the diesel engine to match an environment around the engine by saving fuel, reducing the exhaustion of carbon dioxide, and suppressing black smoke in starting the diesel engine. A rack is operated by an actuator from a lowest rack position to a starting increased fuel amount position at a lower speed than that at which the rack is operated by the actuator from the lowest rack position to the starting increased fuel amount position at the maximum operating speed. The operating speed of the rack is set in two stages, and the later operating speed of the rack is reduced less than the operating speed of the rack up to a set intermediate rack target value P2. Then, the operating amount of the rack per hour is reduced after the ignition of the engine is recognized. Also, the operating speed of the rack is controlled according to an atmospheric temperature in starting the engine.