The private life of a parent as an individual, not simply a father or mother figure, can be a mysterious thing for a child. But for Saul Wordsworth, the mysteries that surround his late father’s volatile personality and painful secrets have grown and grown. Why were his famed creative gifts never properly harnessed? What tormented his soul, and what was the terrible, hidden incident in his youth? Had he perhaps committed murder?

Some facts were clear: Christopher Wordsworth, born in 1914 in Calcutta, now Kolkata, India, was a renowned, charismatic literary figure: a phrase-maker even, credited with coining the satirical gem “a legend in his own lunchtime”, still frequently deployed today. For several years from the mid-60s, he was a book reviewer and sports reporter for the Observer. And, with writing talents that were hailed on Fleet Street and beyond, he was expected one day to produce a great novel. The blistering essay he once wrote for the Guardian, detailing a period living hand-to-mouth in rural Wales, had set up big hopes among publishers, not to mention confirming Wordsworth’s own belief that he was capable of creating a towering work of fiction.

It didn’t work out that way. “The fanciful novel would remain just that, though that didn’t stop him talking about it constantly, threatening it even,” recalls his son this weekend, on the eve of the release of his own account of uncovering the startling, untold contents of his father’s personal papers, including diary entries that speak of haunting guilt and covert love affairs.

All the way through, though, one overarching ambition survived, “to pen the Great Novel; the Novel To End All Novels; a work of genius that would mark him out as the talent of a generation,” according to Saul. Late in life his father found out he was not, as he had been led to believe, a descendant of the poet, William. It was yet another illusion “stripped” away.

“I based myself when younger on so much that was false, illusory,” he once wrote. “I let myself start on a course that could only end in complete personal disaster.” Yet he had shown early promise, winning a poetry prize at Rugby School and shining under the guidance of the author CS Lewis, his tutor at Oxford.

Answers to several of his son’s questions were in his diaries, inherited after his death on 15 October 1998 – 25 years ago on Sunday. Now Saul is to venture into the rackety past of his father in a podcast, Devil in the Wilderness, released on Apple and other platforms from this weekend.

“Though there was no novel, Dad left behind five diaries that span the years 1953 to 1963, a period after the war when he was divorced, penniless, living in the wilderness of Snowdonia and trying to write his magnum opus,” he says. The journals were stored at the house in Powys his parents had moved to 18 months earlier, but initially looked illegible: “Like so much about my dad, his writing was hard to decipher, the truth somehow hidden, impenetrable.”

Now the work of a specialist transcriber has given Saul access to almost 60,000 unpublished words “each documenting, in shimmering prose, my dad’s thoughts and feelings”. Most dramatic among the revelations is a strong suggestion that Wordsworth was guilty of a violent crime, a conclusion reached by following clues about “that moment in Iraq”, where he served during the second world war. If he really had killed someone, as his son sets out to discover, then he got away with it, only to suffer from a guilty conscience for the rest of his life. “I am faced with the law if I tell the truth…” he wrote, adding elsewhere: “It is that element of calculation that shames me, and makes it murder.”

Research in the British Library and National Archives at Kew fuelled the detective journey, although his son knew “that at the end of it may lie something beyond the pale, hard to bear or hear”. There were indications that a serious misdemeanour had been covered up, probably with the help of Saul’s grandfather, a senior civil servant in India who went on to edit the Statesman.

Interviews with the last of those who knew his father are incorporated into the podcast. And the involvement of a key character, a fellow soldier called Pratt, emerges. Saul has not been able to trace him.

More difficult for Saul, the son of a final marriage to fellow Observer book critic Tamara Salaman, were the necessary forays into his father’s three marriages and love affairs. The diaries portray a long, fiery relationship with the philosopher Bertrand Russell’s daughter in law, Sue Russell, and illicit trysts with the wives of friends. Saul now fears his father’s lost crime may have involved a sex attack, even a rape. There are repeated mentions of averted disgrace.

The biggest issue for Christopher Wordsworth, however, was his lifelong struggle with the act of writing and with the belief he might one day reward readers for what he referred to in the Observer as “the vain search for distinction”.

“Thinking, thinking, thinking about the novel. So that it becomes pointless,” he writes in a diary entry. “Yet I must write or perish. Words are what I have, I have lived with them too long; we torture one another and practise small perversions to salt the darkness, partners in a marriage gone stale.”

His journalistic career had begun late, at 50, with a piece written in 1961 for the Observer journalist Philip Toynbee, father of Polly. He wanted new writing about social grievances for a series called Underdogs. Wordsworth submitted 11 pages titled “The Self-Inflicted Wound”, and they included some powerful lines: “Time that was once the dog at my heels,” he wrote, “is now the wolf at my throat…

“My realities are the weather, the possession or lack of a bottle of paraffin, the impending demise of a shirt. My assets are briefly enumerated: the knack of catching trout in small troubled waters; a handful of friends and enemies on whom I inflict a rasping tongue and a marshmallow heart.”

When Wordsworth died at home in Powys, the latest review copy he was reading for the Observer lay next to him on the floor. An obituary in the Independent compared him to a “Scholar Gipsy” figure, a “distinguished and versatile critic with an aphoristic style”.

His son’s memories are not all so benign: “With my dad, everything was cranked up to 11, 12 even; love, hate, fury. As I explore in the podcast, ours was not an easy relationship.”