Probabilistic programming refers to the idea of using standard programming constructs for specifying probabilistic models and employing generic inference algorithms for answering various queries on these models. Although this idea itself is not new and was, in fact, explored by several programming-language researchers in the early 2000, it is only in the last few years that probabilistic programming has gained a large amount of attention among researchers in machine learning and programming languages, and that serious probabilistic programming languages (such as Anglican, Church, Infer.net, PyMC, Stan, and Venture) started to appear and have been taken up by a nontrivial number of users.

My talk has two goals. One is to introduce probabilistic programming to the programming-language audience. The other is to explain a few lessons that I learnt about probabilistic programming from my machine learning colleagues in Oxford. They have been developing a high-order call-by-value probabilistic programming language called Anglican, and through many discussions and close collaborations, they taught me why people in machine learning cared about probabilistic programming and pointed out new interesting research directions. These lessons had huge influence on how I think about probabilistic programming. I will try to explain the lessons through their influence on my ongoing projects, which aim at optimising probabilistic programs using techniques from programming languages and providing denotational semantics of high-order probabilistic programs. My talk will be based on joint work with Sam Staton, Jan-Willem van de Meent, Frank Wood (Oxford), Diane Gallois-Wong (ENS), Chris Heunen (Edinburgh), Ohad Kammar (Cambridge) and David Tolpin (Paypal).