Resources for Research Students

Here, I'll collect links to resources for learning various things, some of which will be useful for research students working with me. I'll generally only recommend things I've used or read myself, or that have been recommended to me by people I trust. Some of these are online resources, others are books that may or may not be available at the Auckland uni library or at Auckland libraries in general. I also have a copy of some of these that you can borrow if you're my student. Feel free to search for other resources as well (and let me know if you find any excellent ones).

Bayesian Inference
Nested Sampling
Diffusive Nested Sampling
Computing
Writing

Bayesian inference

Data Analysis: A Bayesian Tutorial by Sivia and Skilling. Good for beginners with a physics background. Has Nested Sampling in it too, but those sections are more challenging.

Bayesian Logical Data Analysis for the Physical Sciences by Phil Gregory. A bit more comprehensive than Sivia, and includes more detail on the Metropolis algorithm. Again, good for people with some physics background.

Practical Bayesian Inference: A Primer for Physical Scientists by Coryn Bailer-Jones. A new addition, and a good introduction with a more computational focus than Sivia, which is useful these days. The programming language used is R.

Kendall's Advanced Theory of Statistics: Volume 2B: Bayesian Inference by O'Hagan and Forster. A nice and fairly comprehensive textbook. The style and notation, and the examples given, are more like what statisticians use, so this is good for people from a statistics background.

STATS 331 Lecture notes by me! A gentle introduction, with some R and JAGS. Aimed mostly at statistics students but should work for anyone. You can also watch all of the 2017 lectures online.

Doing Bayesian Data Analysis by John Kruschke. Probably the best full textbook for beginners with a statistics background.

Information Theory, Inference, and Learning Algorithms by David MacKay (RIP). A very readable and engaging textbook. Covers a wide range of topics apart from Bayes. And there's a free, legal online PDF file of it!

Nested Sampling

Nested Sampling is my favourite Bayesian computation algorithm. I like to use it in combination with the Metropolis algorithm because of the simplicity. My book chapter Bayesian inference and computation: A beginner's guide has a fairly gentle introduction. The corresponding Python code is available here. An R implementation that I use for teaching is available here.

The original Nested Sampling paper, by John Skilling, is very useful, but isn't easy to read. There are a few different versions floating around, but I usually use the one published in the Bayesian Analysis journal. Here it is.

I have an implementation of Nested Sampling written in Julia. With Jared Tobin, I also wrote an implementation in Haskell.

Diffusive Nested Sampling

Diffusive Nested Sampling is an alternative to standard NS, that still uses the Metropolis algorithm to move around. Here is the original paper describing the algorithm, the software, and the paper describing the software. I also made a quick start video for DNest4. There is also now a DNest5.

Computing

It's worth getting good at using the command line of a Unix-like operating system such as Mac OS X or Linux (e.g., Ubuntu). If you only use Windows, that's okay, but it might be a bit more annoying figuring out how to use certain things.

I recommend learning and using a version control system such as git. There probably isn't much point for a 30-point project student, but beyond that, it's pretty indispensible. This tutorial seems like a promising starting point.

My favourite book on C++ is Professional C++ by Marc Gregoire. It assumes you have a reasonable amount of programming experience. If you're just starting out with C++, this tutorial is pretty good. Once you're using C++ confidently, cppreference.com is indispensible.

My favourite book on Haskell is Haskell Programming from First Principles by Chris Allen and Julie Moronuki.

I haven't read any books on Python, but I followed Allen Downey on Twitter for a while so maybe try his one. Use Python 3, and make sure anything you find is teaching it. Forget about Python 2. A good Python distribution with useful packages for scientific stuff is Anaconda.

Many of you will have learned one or two programming languages before, but will have to learn another one in order to work well with me. When I have had to do this in the past, especially for numerical work, I've found various "cheat sheets" useful, which show how to do the same thing in several different languages, side by side. Here are a couple of links which might help:

NumPy for R programmers
Hyperpolyglot

Writing

You're going to have to write a thesis or dissertation. That's a lot of writing, so learn something about writing! Here are a few resources which might help. There are surely heaps more that I haven't read.

The Sense of Style by Steven Pinker.
The Elements of Style by Strunk and White is considered a classic.