Rethinking Practical Adversarial Example Attack on Speaker Recognition in Physical Domain

IEEE Transactions on Mobile Computing is a premier journal on mobile computing and communication. IEEE TMC is a CCF-A journal.

Abstract: The integration of deep learning on Speaker Recognition (SR) advances its development and wide deployment but also introduces the emerging threat of adversarial examples. However, only a few existing studies comprehensively investigate its practical threat in real-world applications. In this paper, we explore three primary challenges faced within adversarial example attacks on physical SR systems and introduce a practical threat model. To achieve this threat model, we propose a novel adversarial example attack, PhyTalker++, which enables an adversary to attack speech recognition systems by imperceptibly overlaying perturbations on their live-streaming voice via a portable device. PhyTalker++ generates universal perturbations at sub-phoneme level based on a large-scale dataset and multiple SR models, enabling rapid adaptation to new speeches by leveraging phoneme information extracted or estimated from the speeches. We further study the impact of physical channel in terms of both environmental space and audio device. To address these impacts, we propose two channel enhancement methods, i.e., deterministic-based channel pre-compensation and stochastic-based data augmentation, yielding the enhanced variants PhyTalker++-CP and PhyTalker++-DA. Extensive experiments are conducted in real physical scenarios, and results show that PhyTalker++ achieves an overall Attack Success Rate (ASR) of 98.2% in attacking mainstream SR systems and Mel Cepstral Distortion (MCD) of 2.45 dB in human audibility.

View the full paper