One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning