Washington (CNN) -- A top senator slammed Pakistan's government Friday, urging more action against terrorists and less complaining about American drone strikes.

"They have gone after some terrorist targets inside Pakistan but the ones they go after are the ones that threaten the Pakistan government," Senate Armed Services Committee Chairman Carl Levin, D-Michigan, said.

Levin's criticism of Pakistan is fresh evidence of U.S. frustration with Pakistan's efforts in the terror fight, nine years after the 9/11 attacks and after billions of dollars of U.S. military and other aid to Pakistan. The United States considers Pakistan's fight against insurgent crucial to the strategy of the war in Afghanistan.

Levin's criticism comes days after Defense Secretary Robert Gates sent to congressional leaders a letter warning of "a very tough fight" in Afghanistan and "more casualties as we increase our operational tempo and take on the insurgents in their strongholds." The letter, sent September 22, was provided to CNN by a congressional source.

Gates said U.S. efforts were hindered by corruption in Afghanistan, admitting the United States may be partly to blame.

"We have acknowledged that we may be part of the problem, through some of our own contracting practices and the amount of money we are injecting into the Afghan system," Gates wrote in the memo to Senate and House Armed Services Committees about his recent trip to Afghanistan.

Levin met Pakistani Foreign Minister Shah Mehmood Qureshi and other officials Thursday. They discussed improved accuracy of U.S. drone attacks on terrorists targets inside Pakistan, Levin told an audience at the Council on Foreign Relations. The United States has intensified attacks on terrorists but while also increasing efforts to limit civilian casualties.

"There is a significant improvement in the accuracy. The minister, the foreign minister of Pakistan, acknowledged this yesterday to me," Levin said. "There are mistakes made but there is a huge improvement in the accuracy and the reduction of mistakes."

Amidst popular outcry in Pakistan against the U.S. attacks, Levin said the United States has the right to target terrorists coming across the Pakistan border to attack U.S. and NATO forces in Afghanistan.

"It is legitimate to target the people who are targeting you," Levin said in a question and answer session the speech.

He said he objected to Pakistani officials making different statements in public and private.

"I had real problems with the Pakistan government publicly attacking us when we accurately hit a target, when it is clear they don't object privately. They don't object," Levin said.

Levin said some Pakistan criticism is understandable.

"They object when we make mistakes," Levin said. "I mean we hit some Pakistani troops by mistake the other day and there is some strong blowback on that," he said. "This is understandable."

But the senator criticized Pakistan for failing to take the fight to the militants crossing their border into Afghanistan and attacking U.S., NATO and Afghan forces.

"Those folks are attacking across the border, they have some responsibility to go after them, and they haven't carried out that responsibility," Levin said.

"It is when a mistake is not made, and a target is hit accurately that I've got problems with the public attack which then creates that huge animosity against us, when number one it is done at least with the acquiescence of the Pakistan government and number two when they are failing to go after those targets." Levin said.

On the Obama administration plan to begin withdrawing troops from Afghanistan in July next year, Levin said sticking to that day is essential to success.

"Standing by that July 2011 date is the key to that progress, the crucial incentive for the Afghans to approach their task with urgency. If the date wobbles, so does the sense of urgency," Levin said. "I am convinced, after talking to President Obama and to administration officials that the president will not waffle in his decision to begin reducing our force levels by July 2011. I'm sure there will be lots of pressure to do so."

Levin echoed that concern about contractors in his speech Friday. "While private contractors are a necessary part of operations in Afghanistan, our contracting practices have often detracted from our mission by empowering warlords and power brokers," Levin said. "The Armed Services Committee will soon release a report addressing just that issue."