The Evolution of AGN Host Galaxies: From Blue to Red and the Influence of Large-Scale Structures