distribution inference

We explore the memorization capabilities of Large Language Models (LLMs), categorizing them into six types, and discuss their implications and challenges.

We introduce a technique to add trojans while pre-training models, allowing successful inference of properties of the victim’s downstream training data.

An SoK that presents a game-based framework to systematize the body of knowledge on privacy inference risks in machine learning.

A blog post describing our work ‘Dissecting Distribution Inference’.

We introduce a new attack against distribution inference, use it to evaluate inference risk under realistic assumptions, and develop effective defenses.

We propose a general definition for property inference attacks that supports arbitrary properties, along with a notion of effective dataset size to quantify property inference leakage. Experiments reveal how similar distributions can have starkly different attack success rates, and simple attacks can yield non-trivial accuracy.

We propose a notion of neuron sensitivity in terms of adversarial robustness, along with an attack that works as well as PGD. The notion can be extended as a regularization term, providing adversarial robustness without adversarial training.

A blog post describing our work on Property Inference attacks.

We propose a general definition for property inference attacks that supports arbitrary properties. Experiments reveal how similar distributions can have starkly different attack success rates, and simple attacks can yield non-trivial accuracy.