Disentangled representations

From Deep Convolutional Inverse Graphics Network:

Various work [3, 4, 7] has been done on the theory and practice of representation learning, and from this work a consistent set of desiderata for representations has emerged: invariance, meaningfulness of representations, abstraction, and disentanglement.

In particular, Bengio et al. [3] propose that a disentangled representation is one for which changes in the encoded data are sparse over real-world transformations; that is, changes in only a few latents at a time should be able to represent sequences which are likely to happen in the real world.

Sweet.

For more:

• “Probabilistic Programs Which Make (Common) Sense” by Zenna Tavares (linking to to “Deep Convolutional Inverse Graphics Network” at around 26:30)
• -> Deep Convolutional Inverse Graphics Network
• -> Representation Learning: A Review and New Perspectives