A reinforcement learning algorithm based on Deep Q-Networks (DQN) is used for the path following and heading control of a ship in calm water and waves. The rudder action of the ship is selected based on the developed DQN model. The spatial positions, linear velocities, yaw rate, Heading Error (HE) and Cross Track Error (CTE) represent the state-space, and a set of rudder angles represents the action ...