From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication