I've been thinking about how to determine what it is I should be doing at any given moment. As Julia Evans puts it:
I'd rather spend time figuring out which work is important than do something that doesn't matter.
Now, I haven't found a solution. I don't even think even close to one, but I have recently been in a situation where it was clear how I could figure out what I should have been doing.
An uncomfortable situation
On Monday 2015-12-14 I had my first exam at ETH: Software verification. I hadn't taken a lot of time to study for this course and the exam was before the holiday rather than after. I had three days to make sure I wouldn't fail this exam. Note that if I failed my exams I would have to pay back my scholarship so the pressure was on!
Figuring out what was important
I knew I wouldn't have time to study all the material the way I would like to have so I had to optimize for time. I was looking for a way to figure out what exactly I should be focusing on. I started by looking at the previous exams that the Professor had published.
On the exams, I found the topics of all the questions:
2006 2007 2008 2009 2010 2011 2012 2013 2014 Axiomatic semantics x , x , x , x , x , x , x , x , x Separation Logic , , x , x , x , x , x , x , x Model Checking , , x , x , x , x , x , x , x Data flow Analysis x , x , x , x , x , x , x , , x Software Model checking , , , x , x , x , , , x Abstract interpretation , x , x , x , , , , x , Real time verification , , , , , , x , x , Termination proofs , , , , x , , , , Component design and testing x , x , , , , , , , Code reuse x , , , , , , , ,
Under the assumption that exams are usually similar to last year's exam, this gave me an indication of which topics would be important. 'Code reuse', for example, seemed like something I could skip on first reading. 'Axiomatic semantics' looked like it was important enough to definitely study.
Then I noticed that the exams also mentioned how much each question would be graded for. That could give me a quantitative indication of importance!
2006 2007 2008 2009 2010 2011 2012 2013 2014 Axiomatic Semantics 20 , 27 , 20 , 12 , 9 , 18 , 18 , 12 , 8 Separation Logic , , 15 , 8 , 13 , 15 , 15 , 10 , 12 Model Checking , , 10 , 14 , 10 , 15 , 15 , 10 , 9 Data flow Analysis 15 , 18 , 15 , 8 , 12 , 10 , 10 , , 9 Software Model checking , , , 14 , 13 , 12 , , , 11 Abstract interpretation , 15 , 10 , 14 , , , , 10 , Real time verification , , , , , , 12 , 10 , Termination proofs , , , , 13 , , , , Component design and testing 15 , 15 , , , , , , , Code reuse 15 , , , , , , , ,
The weighted averages of all the grades could give a prediction of how much each question would be graded for. (Rounded down to integers.)
Constant Axiomatic Semantics 27 Separation Logic 17 Model Checking 16 Data flow Analysis 18 Software Model checking 9 Abstract interpretation 9 Real time verification 4 Termination proofs 2 Component design and testing 5 Code reuse 2
Now, this is a good indication, but older exams shine through way too much. Obviously, neither 'Code reuse' nor 'Component design and testing' would be on the exam.
I started by giving older exams a linearly decreasing importance assumption per year. I also tried giving older exams an exponentially decreasing importance assumption per year. Here are the results: (Rounded down to integers.)
Linear Exponential Axiomatic Semantics 21 19 Separation Logic 19 22 Model Checking 17 18 Data flow Analysis 13 12 Software Model Checking 10 13 Abstract interpretation 7 5 Real time verification 6 6 Termination proofs 2 0 Component design and testing 1 0 Code reuse 0 0
This looked like a better indication. The bottom three topics are not indicated as relevant and 'Separation Logic' was deemed more important by the results on the right because last year it was more important than 'Axiomatic Semantics'. The results on the right are the ones I trusted in the end.
I studied for three days focusing, in that order, on 'Separation Logic', 'Axiomatic Semantics', 'Model checking', 'Software Model Checking' and 'Data Flow Analysis' and went to the exam on Monday.
Reflection
Notice that I all the judgments of 'better indications' are entirely intuitive. In hindsight I should have tried to predict the grades for last exam and judged my models based on the result there. To be fair, I would have made the same decision in that situation, but it would have been more scientific.
This entire endeavor only started because of an unlucky situation. I fully realize that it was risky and that it would have been better to have studied the entire course.
The results
I did well on the exam. My grade was excellent but it's not really relevant here.
Let's have a look at which topics were covered on the exam and how the grades were distributed:
original rescaled Axiomatic Semantics 8 17.4 Separation Logic 10 21.7 Model Checking 9 19.6 Data flow Analysis 9 19.6 Software Model Checking 10 21.7 ------------------------------------------------ total 25 100
To give you some objective numbers, the models had RMSE's of $6.49$, $5.45$ and $4.47$ respectively. These statistics are nice of course, but most importantly, only the topics I focused on were on the exam and I only focused on those.
My little trick had worked!