"No tears in the writer, no tears in the reader. No surprise in the writer, no surprise in the reader."
There is, generally speaking, no "correct" way to outline, and no correct amount to outline. But it's a question we all wrestle with, and you’d be wise to formulate a plan, one way or another.
Some authors, like Lee Child, generally have no outline whatsoever. They sit down to write a book without knowing what’s going to happen. At the other end of the spectrum is someone like the mystery writer Elizabeth George, who prepares meticulously with hundreds of pages of notes on characters and the crimes at the story’s center, and the approximate shape of the book.
Hollywood films, likewise, are remarkably tightly structured narratives. This is why biopics are so often rife with exaggerations and distortions, elided details, combined characters, and so on: reality never really conforms neatly to the very precise structure that a Hollywood narrative requires. The first plot point at page 27, the mid-act turning point, etc. Life isn't like that. So the "true" story in Hollywood’s hands is always, it turns, only vaguely true.
As writers of books we, fortunately or not, don't have to obey such strict narrative laws. We get to play and improvise, to follow our story blindly … to some extent.
Like many writers, the thing I love about writing is the joy of writing something unexpected and shockingly alive (shocking to myself, first and foremost). As Lorrie Moore said in her second-person short story "How To Become A Writer": "The only happiness you have is writing something new, in the middle of the night, armpits damp, heart pounding, something no one has yet seen." That’s what keeps me writing, basically.
Or, as Joan Didion put it: “I don't know what I think until I write it down.” The process of writing is a process of discovery.
Often, when writing dialogue, I’m delighted and totally surprised by what the characters are saying. They’re more witty and interesting than I am. This is the only magic I really know in the world — that what happens on the page is almost literally beyond my capacity.
The heat in the story comes from the heat within you, the excitement and the unknown.
When I follow a strict outline, that heat can be impaired. Because I’m filling in blanks, not improvising. And while the structure I imagine in the abstract might be interesting and a good story, somehow it fails to achieve the excitement and energy of that improvisation.
But if I ONLY improvise, and have no plan—then I'm in really big trouble. It’s fine for a short story, perhaps, but if a piece becomes longer than 30 pages I can no longer “see” both sides of the piece at once. I can’t remember the text outside of what I have recently been working on, to be precise.
Writing a book is like creating an enormous painting, 50 feet tall, and equally wide, and you can’t ever get more than two feet away from the canvas. And you have a small paint brush. How are you supposed to work on the overall composition? It’s almost impossible without some kind of map to guide you, and the outline is that map.
There are many templates to choose from. You can imitate the structure of a book you like, or you can go with a more archetypal structure (a Faust story, or a “man comes to town” story, or a “rags to riches” story — more on these structures later). Some people call this approach “scaffolding” (e.g. Zadie Smith), and others call it “structure,” there are a million ways to talk about what we’re talking about, but it’s really the big plan that you’re following.
In any case, all of these templates or scaffoldings or structures are helpful with longer texts like novels, but they don’t guarantee that the book will be good. As a consultant I help authors develop a sensible plan and also accept the limitations of that plan. We’re trying to find a way to be both forward thinking, and alive to the moment, and present with the story in front of us today.