Ouch. I like that way of looking at that scene: he's one minute away from taking the TARDIS out of reality and letting go. Who knows if he'd ever come back to Earth? He didn't seem to have been too keen on carrying on in the timeline where he never met Donna. But he did meet her, and she snags him, makes him say it while he isn't alone.