Geometrical frustration in nonlinear photonic lattices