The government could terminate the Northern rail franchise if performance does not improve, the transport secretary, Chris Grayling, said, after more than a year of misery for passengers.

Northern has faced intense scrutiny over the past year as travellers have been plagued by delays and cancellations.

Asked by MPs on the transport select committee whether Northern should be stripped of the franchise, Grayling said: “If Northern can’t deliver then of course we would look at all options.”

He added: “I will not resile from taking away a franchise if it is the right thing to do and if I’ve got the contractual basis for doing so. We are watching and we will continue to watch very carefully Northern’s performance in all respects, and I expect them to deliver on what they have committed to deliver. It hasn’t been good enough up to now.”

The government has faced repeated calls to strip the franchise from Northern, which is part of Arriva Group. Arriva’s ultimate owner, German state-owned rail operator Deutsche Bahn, has put the rail and bus business up for sale, with a reported price tag as high as €3.5bn.

In May, the mayors of Greater Manchester and the Liverpool city region called for the franchise to be terminated, saying they had received assurances that services would have improved by then.

Grayling, who was appointed as transport secretary by Theresa May after the EU referendum three years ago, acknowledged: “Fundamentally the infrastructure in the north is not up to handling the needs of a modern commuter railway, and that’s what it needs to be.”

It is unclear whether Grayling will still be serving as transport secretary by the end of the month under a new prime minister. Grayling, who backed the leave campaign, was an early supporter of Boris Johnson, the bookmakers’ favourite to win the vote of Conservative party members.

Grayling has faced repeated calls for his resignation during his tenure, including after a controversy over a £33m out-of-court settlement paid to Eurotunnel. The Department for Transport was sued by Eurotunnel for allegedly breaching public procurement rules over a contract with the ferry leasing company Seaborne Freight and two other ferry firms to provide freight capacity for emergency medical supplies in the event of no Brexit deal.

Grayling, who favours leaving the EU without a deal if there is no trade agreement, said he had been in favour of fighting the action, but that he was overruled by a cabinet subcommittee who worried that the NHS would run out of vital medicines.

“Although we would have liked to fight that case in court, we collectively decided that we could not jeopardise the supply of goods to the NHS, which is why we settled,” he said.

A spokesman for Northern said it was investing £500m in new trains, as well as starting another 2,000 new services a week.

“The north deserves the best possible rail service and we are working hard to improve performance and reliability for customers,” the spokesman said.

“Since last year, we have made a large number of improvements for customers – including better punctuality and further investment in refurbished trains.”