How often to look up

If you never go the wrong way for an afternoon, you are stopping to check too often.

That sounds like a license to be careless. It isn't. It falls out of a small piece of arithmetic that computers worked out in 1974, for a problem that turns out to be the same as one people have every day.

Two things you can't do at once

You can't be absorbed in something and think about it at the same time. When the writing is going well, you aren't asking whether it's the right piece to write. The moment you ask, the absorption stops. Heidegger had words for this: a hammer in use disappears into the work, and only becomes an object you look at when you stop hammering.

So there's a trade. Stay absorbed and you might drift: solve the wrong problem, polish a section that should be cut, keep going down a road that stopped making sense an hour ago. Stop to reflect and you pay for it: a few minutes of stepping back, and the harder cost of getting back in.

Everyone handles this by feel. Anxious people check constantly. Confident people check rarely and sometimes lose a week.

The computer version

Long-running computer jobs have exactly this problem. A job that runs for days will, sooner or later, hit a failure and lose everything since its last save. A save of the whole state is called a checkpoint. Saving often wastes time on saving. Saving rarely wastes time redoing work after a crash.

In 1974 John Young worked out the balance. If a checkpoint costs time δ, and failures come on average once every M, the best interval between checkpoints is roughly:

interval = square root of (2 × cost of one check × average time between failures)

John Daly later refined it, and Adrian Colyer has a plain-words summary of the refinement (I read the summary, not the paper). The short version: Young's formula holds up well, as long as a check is cheap compared to the time between failures.

Translated

Swap the words. The check is stepping back: rereading the goal, asking if this is still the thing to do, then finding your way back in. The failure is drifting off course. The lost work is whatever you did between drifting and noticing.

One difference first. A computer knows when it has crashed. You don't know when you've drifted; you only find out when you next look up. That sounds like it should break the model, but it doesn't. Either way, the work you lose is the stretch between the failure and the next check, and on average that's half an interval. The arithmetic is the same.

Say stepping back costs you ten minutes, and on a project like this you tend to go off course about once every two working days, call it 16 hours. The formula gives about 2 hours 20 minutes between checks. Not every 20 minutes. Not once a day.

What the square root says

The interesting part is the square root, because intuition is linear and the answer isn't.

Suppose the project gets four times as uncertain: you now drift every four hours instead of every sixteen. The anxious instinct is to check four times as often. The formula says twice as often: about every 70 minutes. Suppose instead you find a way to make checking four times cheaper, a one-line goal taped to the screen instead of a ten-minute reread. Again, check twice as often, not four times.

And at the best interval something exact happens: the time you lose to checking equals the time you lose to drift. In the example, each is about 7% of the day. Neither is zero. A day with no wasted wrong turns isn't a well-run day; it's a day where too much went into checking. Some wrong turns are what the right amount of attention costs.

Where it breaks

The model assumes drift arrives at random, at a steady rate. It doesn't. You're more likely to drift when tired, right after finishing a piece, or when something new comes in. The better move isn't a timer. It's to check where the cost of checking is lowest: at a seam, after finishing a section, when the state you'd have to reload is small. Computers do this too, saving when the data to save is small.

The model also has a limit. Daly's rule says that once a check costs more than half the time you'd go before drifting, stop calculating and check every time you'd expect to fail. In life that's the state where you can't go twenty minutes without losing the thread. That isn't a scheduling problem. It means you don't have a direction yet, and you're exploring, not executing.

The strongest objection

Reflection isn't only overhead. Sometimes stepping back is where the idea comes from. A checkpoint never produces a new insight; a walk does.

True, but then it's not a check. It's work of a different kind, and it should be counted as work. The question "am I still doing the right thing?" is different from "what should I be doing?". The first is a checkpoint, and it should be as cheap and as rare as the arithmetic allows. Mixing them up is how a two-minute check turns into an afternoon of rethinking everything, which is the most expensive way to find out you were on course all along.

The right amount of looking up is the amount that leaves you, now and then, a little lost.