Understanding Adversarial Attacks on Discrete Generative AI
Session Number
CMPS(ai) 33
Advisor(s)
Jiaqi Ma, University of Illinois Urbana Champaign
Discipline
Computer Science
Start Date
17-4-2025 2:30 PM
End Date
17-4-2025 2:45 PM
Abstract
Data attribution methods have been commonly used on generative AI as a way to evaluate the usefulness of data samples. In the future, it may be used to compensate copyright holders for their data. However, these methods have proven to be susceptible to adversarial attacks through various optimization processes. We seek to show that these adversarial attacks can not only be successful, but also understandable to the general public. In particular, we target discrete generative AI in the symbolic music domain, as music has various patterns such as chords and arpeggios that can be easily identified. By highlighting these vulnerabilities, the knowledge can be leveraged to understand other adversarial attacks in different fields, such as text generation and image generation. Our work underscores the need for more robust data attribution methods for the use of generative AI. We attempt various heuristic methods to alter a data sample, including copying influential pieces to another one. We show that simple heuristic methods can also be used to generate adversarial data samples to data attribution.
Understanding Adversarial Attacks on Discrete Generative AI
Data attribution methods have been commonly used on generative AI as a way to evaluate the usefulness of data samples. In the future, it may be used to compensate copyright holders for their data. However, these methods have proven to be susceptible to adversarial attacks through various optimization processes. We seek to show that these adversarial attacks can not only be successful, but also understandable to the general public. In particular, we target discrete generative AI in the symbolic music domain, as music has various patterns such as chords and arpeggios that can be easily identified. By highlighting these vulnerabilities, the knowledge can be leveraged to understand other adversarial attacks in different fields, such as text generation and image generation. Our work underscores the need for more robust data attribution methods for the use of generative AI. We attempt various heuristic methods to alter a data sample, including copying influential pieces to another one. We show that simple heuristic methods can also be used to generate adversarial data samples to data attribution.