A unified framework for non-Brownian suspension flows and soft amorphous solids