It’s the depth of the darkness that spooks Bobby Western, the haunted man at the heart of Cormac McCarthy’s extraordinary new novel. Western works as a salvage diver in the Mexican Gulf, tending to sunken barges and stricken oil rigs. He’s kicking up clouds in the clay-coloured water and pressing further into the unknown with every weighted step. His colleagues are blase but experience has taught him to take care. He asks: “You ever bump into something down there that you didn’t know what it was?”

Published a full 16 years after the Pulitzer prize-winning The Road, The Passenger is like a submerged ship itself; a gorgeous ruin in the shape of a hardboiled noir thriller. McCarthy’s generational saga covers everything from the atomic bomb to the Kennedy assassination to the principles of quantum mechanics. It’s by turns muscular and maudlin, immersive and indulgent. Every novel, said Iris Murdoch, is the wreck of a perfect idea. This one is enormous. It’s got locked doors and blind turns. It contains skeletons and buried gold.

Some 40 feet below the surface, Western explores a downed charter jet. Inside the fuselage, he picks his way past the floating detritus and the glassy-eyed victims, still buckled in their seats. The plane carried eight passengers but one appears to be missing and the subsequent investigation hints at a government cover-up. Except that this may be a red herring; we’re still in the book’s shallows. Western’s troubles, we realise, are altogether closer to home.

McCarthy began work on The Passenger back in the mid-1980s, before his career-making Border trilogy; building it piecemeal and revisiting it down the years. Small wonder, then, that this family tragedy feels filleted, part of a larger whole and trailing so many loose ends that it requires a self-styled “coda” – a second novel, Stella Maris, published in November – to complete the story. So this is a book without guardrails, an invitation to get lost. We’re constantly bumping into dark objects and wondering what they mean.

Ostensibly the narrative sees Western pinballing around early 80s New Orleans, hobnobbing with the locals, trying to outflank his enemies. But it also casts back through the decades, mining his quasi-incestuous bond with his suicidal sister, Alicia. Along the way it introduces us to her nightmarish hallucinations: “the Thalidomide Kid and the old lady with the roadkill stole and Bathless Grogan and the dwarves and the Minstrel Show”. Alicia likens these demons to a troupe of penny-dreadful entertainers. They materialise at her bedside whenever she skips her meds.

On a prose level, McCarthy – now 89 – continues to fire on all cylinders. His writing is potent, intoxicating, offsetting luxuriant dialogue with spare, vivid descriptions. The bonfire leaning in the sea wind; the burning bits of brush hobbling away up the beach. As a storyteller, though, I suspect that he is deliberately winding down, wrapping up. This novel plays out as a great dying fall.

Western and Alicia, we learn, are children of the bomb. Their father was a noted nuclear physicist who helped split the atom, leading to the destruction of Hiroshima and Nagasaki. Western, in his youth, studied physics himself. He became familiar with protons and quarks, leptons and string theory, but gave up his calling for a life of blue-collar drifting. Quantum mechanics, he feels, can only take us so far. “I don’t know if it actually explains anything,” he says. “You can’t illustrate the unknown.”

McCarthy’s interest in physics has been stoked by his time as a trustee at the Santa Fe Institute, a nonprofit research centre. Since 2014 he’s largely been holed up with the scholars, exploring the limits of science – and presumably of language as well – only to conclude that no system is flawless. High-concept plots take on water; machine-tooled narratives break down. And so it is with The Passenger, which sets out as an existential chase thriller in the mould of No Country for Old Men before collapsing in on itself. Western might outpace his pursuers but he can’t escape his own history. So he heads into the desert, alone, to watch the oil refineries burning in the distance and observe the carpet-coloured vipers coiled in the grass at his feet. “The abyss of the past into which the world is falling,” he thinks. “Everything vanishing as if it had never been.”

What a glorious sunset song of a novel this is. It’s rich and it’s strange, mercurial and melancholic. McCarthy started out as the laureate of American manifest destiny, spinning his hard-bitten accounts of rapacious white men. He ends his journey, perhaps, as the era’s jaundiced undertaker. Come friendly bombs. Come rising oceans. The old world is dying and probably not before time, and The Passenger steals in to turn out all the lights.