Compositional statistical unit testing

How do you unit-test a sampler? Run it a bunch and see whether the histogram looks good—but there’s always some chance that your test will fail randomly even though the sampler is good, or pass randomly despite a bug. And if you try to have more than one test, the chance of random errors goes up. How big of a chance? How much worse does it get? What to do? Read more

Musings on Probprog

People in my circles periodically talk and write about the nature of this emerging new beast that is called probabilistic programming. There’s various talk about how it’s about samplers, or density functions, or Markov chains, or making machine learning or artificial intelligence more accessible, or various other such stuff. Certainly those things hover in the air around probabilistic programming, but I don’t think that gets to the essence of it. I think that probabilistic programming, as opposed to standard programming, is a new scope of problems that can be solved by computing. Read more

On Testing Probabilistic Programs

Testing traditional software strives for an ideal of exact determinacy: If a test fails, there is a bug and the developers must investigate; and if a test passes, the bugs it is testing for are not present, so development can proceed without worrying about them. This is not possible with probabilistic programs—even a correct sampler for a probability distribution always has some chance of producing arbitrarily weird output by bad luck, and an incorrect one can equally well look fine by coincidence. Read more

On Intentionally Random Programs

For a little over two years, I have been professionally dealing with programs whose behavior is intentionally random. Why would one even have intentionally random programs? Read more

On Good Software

Good software is software that admits a simple mental model.

For all that I have observed and participated in plenty of discussions about one or another piece of software as to whether it is or is not good, I am surprised to say that I have never seen Read more

Best Effort vs Strictly Safe

Possibly the greatest tension in the design of programming languages occurs when encountering a user program that doesn’t quite make sense. The two coherent schools of thought on the subject are Read more

How to Compute with a Probability Distribution

What makes a good representation for computing with probability distributions? The two canonical options are samplers and probability density functions. Both are valuable; and the relationship between them turns out to hide two fruitful variations on the idea of a sampler, Read more

Social Function of Module Systems

Observing the state of modern software practice, one might notice that a rough tower of different abstraction mechanisms has emerged. One might then wonder: why do we need so many concepts, if their fundamental job is to give a name to some software, and allow one to use it just by referring to its name? Read more

Stochasticity is a Quantifier

In English, quantifiers are words like “all”, “one”, or “some” that indicate how broadly true the quantified clause is. Formal logic has adopted symbols for such words, namely “all”, “exactly one”, and “some” (in the sense of “at least one”). Probability theory offers us a reason to incorporate another symbol, Read more