Various work [3, 4, 7] has been done on the theory and practice of representation learning, and from this work a consistent set of desiderata for representations has emerged: invariance, meaningfulness of representations, abstraction, and disentanglement.
In particular, Bengio et al.  propose that a disentangled representation is one for which changes in the encoded data are sparse over real-world transformations; that is, changes in only a few latents at a time should be able to represent sequences which are likely to happen in the real world.
- “Probabilistic Programs Which Make (Common) Sense” by Zenna Tavares (linking to to “Deep Convolutional Inverse Graphics Network” at around 26:30)
- -> Deep Convolutional Inverse Graphics Network
- -> Representation Learning: A Review and New Perspectives