David Cameron and his Liberal Democrat deputy, Nick Clegg, have unveiled emergency surveillance legislation that will shore up government powers to require phone and internet companies to retain and hand over data to the security services.

The emergency legislation – due to be debated on Tuesday and complete all its parliamentary stages by Thursday next week – will also confirm that foreign-based companies should hand over data harvested in the UK, a move that implicitly accepts the revelations by former NSA contractor Edward Snowden may have disclosed surveillance activities that did not have international legal authority.

The government said it was forced to act after a European court of justice ruling on 8 April and because a growing number of foreign-based, predominantly American, phone companies were threatening to stop handing over details of UK customers' data.

The ECJ legal judgment and the new reluctance by companies to cooperate with security services has largely been driven by the civil liberties debate sparked by the Snowden revelations.

Cameron said: "I am simply not prepared to be a prime minister who has to address the people after a terrorist incident and explain that I could have done more to prevent it."

The prime minister has won the support of the Liberal Democrats and Labour in private talks over the past week, warning that a legal cliff-edge faced the government.

In return for the emergency laws, the Liberal Democrats and Labour have won long-sought reforms to the accountability of the security services that the intelligence agencies and the prime minister have repeatedly opposed.

At a hastily called Downing Street press conference Cameron also for the first time embraced the need for a debate about the powers of the security services.

The main measures outlined are:

• A privacy and civil liberties board, based on the US model, will be established to oversee the security services and act as an independent watchdog.

• An annual transparency report will be published on the amount of data intercepted.

• A new flexibility in the length of time data is retained, so it is only held for up to 12 months.

• A two-year expert review of the relevance of the Regulation of Investigatory Powers Act 2000 that will – on the Labour and Liberal Democrat side – look at the process of granting intercept warrants, and – on the Conservative side – look at the possibility of extending the powers of the security services. That review will be completed after the election.

• A reduction in the number of public bodies that will be able to access data so, for instance, local councils will need to ask a third party to look at data.

• A limit on the kind of data that can be accessed by the state.

• An agreement that the legislation will fall in 2016, and require fresh legislation that can be scrutinised by parliament in the normal way.

• A new restriction that data can no longer be gathered purely in the interest of the economic wellbeing of the UK.

• The appointment of a former senior diplomat to negotiate with the US on cooperation over data transfer, including a possible speeding up of the hand-over arrangement under mutual legal assistance agreements.

Advocates of tighter surveillance hope the bill will give the state more uniform powers to sanction phone and internet companies that do not cooperate and lead to a speeding-up in the transfer of data between jurisdictions.

The cabinet was briefed on the bill on Thursday, as were six senior select committee chairs. The bill, entitled the data retention and investigation powers bill, will receive a second reading on Monday.

Privacy campaigners challenged the need to rush new laws through parliament, saying there was no possibility of the government being found in breach of the UK law until late autumn at the earliest.

Explaining the urgency, Cameron said: "Some companies are already saying they can no longer work with us unless UK law is clarified immediately.

"Sometimes in the dangerous world in which we live, we need our security services to listen to someone's phone and read their emails to identify and disrupt a terrorist plot.

"As prime minister I know of examples where doing this has stopped a terrorist attack."

The emergency measures were "about restoring and maintaining capabilities that are vital to our national security", the prime minister said, stressing that access to phone and email metadata were integral to the work of the police and intelligence services.

"Unless we act now, companies will no longer retain the data about who contacted who, where, and when, and we'll no longer be able to use this information to bring criminals to justice and to keep our country safe," he warned.

"This is at the heart of our entire criminal justice system. It is used in 95% of all serious organised crime cases handled by the CPS. It's been used in every major security service counter-terrorism investigation over the past decade."

He added: "We face real and credible threats to our security from serious and organised crime, from the activity of paedophiles, from the collapse of Syria, the growth of Isis in Iraq and al-Shabaab in east Africa.

The deputy prime minister said: "This bill has nothing to do with the so-called 'snooper's charter'. That was a Home Office proposal to store every website you have ever visited for a whole year.

"I blocked that last year and I have blocked every further attempt to bring it back."

He added: "I have only agreed to this emergency legislation because we can use it to kickstart a proper debate about freedom and security in the internet age – debate the Liberal Democrats have been calling for for a long time.

"In the post-Snowden age, people are rightly demanding to know more about what the state does on our behalf. There are fundamental questions to be asked about the scope of existing powers, about whether the Regulation of Investigatory Powers Act has kept pace with technology, about how nation states grapple with the global internet and about how we continue to protect our security, our privacy and our liberties while safeguarding our security."

In the Commons, there was near-unanimous support for the legislation but Labour backbencher Tom Watson, and former Tory party leadership candidate David Davis both accused the front benches of a stitch-up by the elite.

Theresa May, the home secretary, said there was no possibility of responding to the ECJ judgement save by primary legislation. "We can't answer those big questions on the hoof," she said.

Shami Chakrabarti, director of campaign group Liberty, said: "The government says it's only plugging loopholes but its existing blanket surveillance practice has been found unlawful. We are told this is a paedophile and jihadi 'emergency', but the court judgment they seek to ignore was handed down over three months ago and this isn't snooping on suspects but on everyone.

"We are promised greater scrutiny and debate but not until 2016, as it seems that all three party leaders have done a deal in private. No privacy for us and no scrutiny for them. Will Clegg and Cameron's 'debate for the future' really comfort voters and companies today?"

The Conservatives made clear they will not support moves to make the chairman of the intelligence and security committee a member of the opposition in order to underline its independence.