Lord West of Spithead has told MPs the security services are "keeping a very close eye" on thousands of people.
The security minister also told the Commons home affairs committee that he did not support moves to repeal human rights laws in order to make it easier to deal with terror suspects.
He told MPs there are "something like 2,000 people we are keeping a very close eye on who are not under control orders".
Conservative MP David Davies expressed concern that some suspects cannot be deported from the UK because they may be tortured or mistreated in their home countries.
He suggested the home secretary should have the power to deport anyone judged to be security risk.
"Would it not be sensible to tear up the Human Rights Act?" he asked
Lord West said that would be "an outrageous thing to do", adding that he was proud that Britain's stance on human rights marks it out from other countries.
Accepting some risk is the price of a free society, he told Davies.
Conservative MP Patrick Mercer asked if control orders, which have been successfully challenged in the courts, are "the way forward".
He added: "It is desperately expensive and untidy."
Lord West said that placing someone under a control order is cheaper than traditional surveillance by the security services and police, and independent reviewer of terror legislation Lord Carlile of Berriew thinks they are "appropriate".
He pointed out that no-one under such an order has absconded since he became security minister.
In July the House of Lords ruled that "sufficient detail" of the allegations must be disclosed to people subject to control orders to enable them to give effective instructions their legal team.
Lord West said that "some people have come off control orders because the release of more information would harm national security".
He also pointed out that he "would much rather not have to have them", but argued that they are necessary.
"If there was an instant panacea to do this in a different way I would love it," he told the committee.
He said that after the legal ruling all cases are being reviewed, but did not put a figure on how many people are still under control orders.
Lord West said the two men who have had their control orders removes are still under surveillance by the security services.
Labour MP Karen Buck asked about interception warrants, questioning the purpose of interception evidence as it is not admissible in court.
The minister said it is the difference "between intelligence and evidence – if that intelligence shows someone is going to do something criminal you will be able to build a case".
He added that 95 per cent of serious crime investigations use intercepts.
Asked if the job of security minister should be in the cabinet, Lord West replied that while he is happy as a junior minister and gets "all the access I require", consideration should be given to such a move in the future.
Former home secretary John Reid also gave evidence.
Committee chairman Keith Vaz thanked him for coming.
"You are unwell and have left your bed at home to come to speak to us," he said.
Reid said that national security "is above party politics" and added:
"While I am here I am not being investigated in hospital."
He said that Cobra, the government's crisis response committees, "works very well in general terms".
Reid said he was also impressed with the Office for Security and Counter-Terrorism in the Home Office. "It was not unanimously supported at the time in government," he noted.
Unlike Cobra, which is "reactive, temporary and ad-hoc", the office has institutionalised best practices such as the grouping together of intelligence and better co-ordination across Whitehall.
The former home secretary stressed that in any incident the operational control remains with senior police officers, not with ministers.
Reid suggested that the Commons should form a national security committee as the issues "cut across home and foreign affairs".
