The chairman of the BBC Trust, Sir Michael Lyons, has insisted that viewers and listeners would prefer to see licence fee money given back to them, rather than handed to competitors.

In his speech, Lyons also warned that the Conservative party's plan to freeze the annual £3.6bn licence fee would be disastrous, describing it as "a recipe for curbing the editorial independence of the BBC".

In the speech to media executives yesterday, Lyons warned against "top-slicing" the BBC's £3.6bn a year in public funding, arguing that licence fee money should not be used to pay for "things that have nothing to do with the BBC's public purposes".

"People would do well to remember that licence fee payers give us their money in good faith, believing it will be spent on BBC services and content," he said, in a Royal Television Society speech. "We know what the public would like to ­happen to any surplus … They'd like their money back."

Lyons said the BBC had agreed to talk to the government about using any surplus in the digital switchover fund to pay for universal UK broadband access.

"But agreeing to these discussions does not mean we are signing a blank cheque, or agreeing to any more general use of the licence fee to pay for things that don't fall within the BBC's public purposes as set out in the charter," he added.

"To suddenly tell them midway through the settlement that their money is being siphoned off, as some have ­suggested it should be, would be more than an act of bad faith. It would be tantamount to breaking a contract."

Lyons's comments could put the trust on a collision course with the government. The communications minister, Lord Carter, has argued that some of the underspend in licence fee money reserved to subsidise the digital switchover for the disadvantaged in society could pay for a broadband network that would connect every home in the country.

At the government's behest, £650m of licence fee money was earmarked to subsidise the digital switchover between 2007 and 2012 – £130m a year. However, based on lower-than-forecast takeup of the switchover subsidy scheme so far, it is estimated a £250m surplus could be left.

The media regulator, Ofcom, has also said that some of the digital surplus could be used to shore up Channel 4's finances. Last month Ed Richards, the Ofcom chief executive, suggested the digital surplus could provide funding for the independent consortia he envisages will take over ITV regional news provision.

Lyons also addressed Conservative plans for the licence fee. David ­Cameron wants to prevent the fee rising from £139.50 to £142.50 this year, and has forced a Commons vote on the issue. The Conservatives are likely to be defeated, but Cameron has also raised the prospect of carrying out an annual review into the way the BBC is funded, should the party win next year's general election. The BBC's current 10-year charter runs to 2018, while the six-year licence fee deal runs to the end of March 2013.

Lyons said it was very important "the BBC's horizons do not become too closely entwined with the political cycle".

Annual licence fee reviews would inevitably lead to a "shift in the balance of power", he said, "away from an editorially independent BBC and towards the inevitably political agendas of those who would have the final say in these frequent decisions over funding".

Lyons added: "This is not a matter of the BBC defending its own narrow interests. It's about fulfilling the trust's duty to be guardians of the public interest in the BBC."

He also claimed that public affection for the BBC was an at all-time high, despite the controversy over the "Sachsgate" affair last autumn. The Radio 2 broadcast by Jonathan Ross and Russell Brand last year resulted in a record £150,000 fine from media regulator Ofcom.
"The value the public place on the BBC is actually rising," said Lyons, citing research carried out by the BBC Trust earlier this year. Asked if they would miss the BBC, 85% of those surveyed said they would, up from 70% in 2007, when the trust conducted a similar exercise, according to Lyons. The results of the research will be published in July, Lyons revealed.

• To contact the MediaGuardian news desk email editor@mediatheguardian.com or phone 020 3353 3857. For all other inquiries please call the main Guardian switchboard on 020 3353 2000.

• If you are writing a comment for publication, please mark clearly "for publication".