I have been formulating a self-study plan for some time now. The plan involves mostly math subjects, as I finally want it to culminate in data analysis/mining/AI/ML - which again are subjects I've always been interested in but never took up seriously. Looking at the experience of people who have done such a thing before, a solid foundation in the underlying math is absolutely essential. So, I've started off with statistics and probability. Having been out of touch for a long time (15 years!) with the basics of either of these, I chose to begin simply - Sheldon M Ross's Introductory Statistics supplemented with Khan Academy's video lectures (These are perfect for brushing the dust from those rusty concepts). After finishing this I'll aim for a book like Bertsekas combined with MIT's OCW lectures.
Have decided to put the solutions to the exercises I'm doing here.