TensorFlow Privacy: Learning with differential privacy for training data

It can be difficult to distinguish between what the models learned from training and what they have just memorized when evaluating ML model models. This difference can be critical in certain ML tasks, such if ML models are trained with sensitive data. Recent developments have allowed for differentially private training of models in ML, including deep neural network (DNNs) that use modified stochastic gradient descent to offer strong privacy guarantees for training data.

These techniques are practical and easy to use. They do come with their own hyperparameters and can be adjusted in ways that make learning less sensitive for outlier data. This may reduce utility. Ulfar Erlingsson explains the basics of ML privacy. He also introduces differential privacy and its value as a gold standard. He then dives into the principles behind ML privacy and dives into intended memorization and how it differs to generalization.