BOOK REVIEW / NONFICTION : On Letterman, Leno and Others Seeking Late-Night Rule : THE LATE SHIFT: Letterman, Leno & the Network Battle for the Night <i> by Bill Carter</i> ; Hyperion $24.95, 299 pages

The mental image most of us have of David Letterman is that of a buoyant, boyish man with the Alfred E. Newman, devil-may-care grin.

But as we see in this gripping account of the surprisingly heated struggle for the honor of chatting amiably and sipping coffee from “The Tonight Show’s” homey desk, nothing could be further from the truth.

Often self-loathing in these pages, Letterman readily, almost eagerly, catalogues his list of “failed” life decisions. And in the moments when he’s not self-loathing, he’s merely self-critical, as on those frequent occasions when he would hole up in his office to review the tape of a show in which he felt he had stumbled. “The staff,” writes New York Times TV reporter Bill Carter, “would then hear crashing noises coming through the door.”

Like so many great showmen, Letterman’s idol, Johnny Carson, hid much the same fears behind his mask of equanimity: “The most confident, smoothest man on television,” Carter writes, “would sometimes, among his closest associates, blurt out how terrified he was that it might all disappear at any moment.”

It may be disquieting to learn that the people whose whimsical, reassuring perspective on the day’s events has for years helped lull us into a good night’s sleep themselves possess deep fears. But the happy truth is that Carson and Letterman’s fear of failure also has been their secret to success.

Since he has written this book mostly as a business history, Carter never attempts to explain how the two men derive inspiration from insecurity. But their magic clearly has something to do with their hypersensitivity to their moods and to those of their audiences.

Just as Carson seemed to know when a joke was bombing moments before we did, so too does Letterman become acutely sensitive during a show.

In this fly-on-the-wall account, flawed only by Carter’s tendency to lapse into the oversaturated hues of TV writing (e.g. “(Jay) Leno was . . . a prince among the stand up frogs”), we are vividly drawn into Letterman’s hyper-vigilant world.

Here, for instance, is Letterman describing his first appearance on “The Tonight Show,” the one that he says abruptly ended “all those years hanging around the Comedy Store and driving around in your truck and heating up burritos at the 7-Eleven, and drinking warm quarts of beer”:

“I sat down, and Johnny Carson is sitting right there, and you’re just talking and talking and praying to God that it’s over soon, and you’re looking around and you’re seeing stuff that you’ve seen on TV for years. And you can’t let yourself think for a second or, you know, your head would explode. So you’re talking and talking and just praying, ‘Oh please go to a commercial, please go to a goddamn commercial!’ And the next thing you know you’re out of there and it’s just, Holy Christ, I was on ‘The Tonight Show!’ ”

In telling contrast, Leno is portrayed as the less-inspired comedian largely because of his unflappability. To “Tonight Show” staffers who confessed to feeling bothered or blue, Leno would ask, without irony: “What does that mean, stressed? I’ve never been stressed. . . . Down? . . . I’ve never been down.”

If Leno harbors deep emotions, he keeps them under tight rein. Just minutes after witnessing his manager and friend scream at his show’s producer, for example, Leno delivered a monologue as thoroughly hip and laid-back as ever.

In the end, Leno may profit as much from his hard shell as Letterman does from his thin skin. It has enabled him to endure and accept a broad range of criticism, whether in his old Comedy Store nights (he’d hide in a bathroom stall after his gig to hear audience reaction) or in his “Tonight Show” days (he once sneaked into an NBC executive’s closet to eavesdrop on a call in which the network’s executives discussed whether to dump him for Letterman).

Now that Leno has weathered that storm (like Saddam Hussein, he told one reporter, “I think I have a bunker”), the new question is whether Letterman, the man whose entire comic identity seems founded on being the hip, ironic outsider, can hold onto his comic genius now that his 11:30 show on CBS has become the late-night ratings leader.

Will he be able to adapt to the constraints of Establishment television, which for decades he has called “scheming” and “sleazy”? Can this man who delights in steamrollering over Smurf dolls and smashing Energizer Bunnies to smithereens with baseball bats be as receptive as Leno is to the “focus groups” and “demographic master plans” that now rule so much of the mainstream media?

And last but not least, can he conduct kinder, gentler interviews with Cher and the other stars who have damned him with all sorts of four-letter words?

Only time, of course, and Mr. Nielsen’s black box, will tell.