Andrej Karpathy wrote this landmark post about the Unreasonable effectiveness of RNNs.
Generative Adversarial Networks, or GANs for short, also seem more powerful than they should be. This technique feels gimmicky: we create a generator and a discriminator network, and then we have them fool one another… And yet, the results are so great, they are hard to believe.
3 variations of this idea:
Both 3dGAN and stackGAN have been covered by two Minute Papers: