Infusing invariances in neural representations