I think I have been pretty productive on technical tasks lately and the method is (at least to me) interesting. The effect was accidental but I think one can explain it and reproduce it by synthesizing three important observations on human behavior.The three observations are:
1) Jacques Hadamard in “An Essay on the Psychology of Invention in the Mathematical Field” called out the importance of non-voluntary intuitive creative leaps that occur in rest periods between intervals of intense work and preparation.
2) It has been noted again and again that what actually makes people happy (versus what they anticipate would make them happy) are activities and experiences with rising challenges (for example see Daniel Gilbert’s “Stumbling on Happiness”).
3) It is folklore that a number of the greatest computer scientists are also fairly accomplished musicians.
And here is the punch-line: take up a skill building hobby (in my case I am trying to learn how to draw). You definitely enjoy it, but some part of your subconscious also resents being made to work (learning is work, don’t confuse that with repetition). To defend itself your subconscious then starts throwing out more and better technical ideas during periods of repose. Jot these down (without trying to work on them). The effect is even stronger than Hadamard’s effect (where your brain is solving problems for you to end an effort) as it is closer to the classic trick of making progress on one task by procrastinating on another task.
This is similar to the “left brain/right brain” ideas of the 1970s (it assumes the existence of a subconscious) but assumes far less unverified structure of a subconscious. And here is where the “10,000 hours to mastery effect” (Malcolm Gladwell, “Outliers: The Story of Success”) works in your favor- you can use the same source of deliberate practice (remember you have to be learning not puttering around) for a long time.
I think if you are in good health and have enough energy you can pull this trick off at will.
Data Scientist and trainer at Win Vector LLC. One of the authors of Practical Data Science with R.