Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning