Quantifying the impact of mergers on the angular momentum of simulated galaxies