Pep Guardiola said his players “needed to lose” in order to regain their focus, after Manchester City were beaten for the first time in 227 days.

City were poor in the first half then rallied after the break but they failed to become the first English side to be victorious in all six Champions League group matches. Sergio Agüero scored a late penalty after Gabriel Jesus was brought down but it was a small consolation.

Guardiola did not like defeat but having chosen a weakened XI the manager was phlegmatic as his thoughts now turn to Sunday’s Manchester derby.

“Its hurts, it is never nice to lose a game,” he said. “I was [however] feeling we needed to lose – for the players, for the club. In the media people say things when you win and you can forget how to focus.”

The delight of Paulo Fonseca, Shakhtar Donetsk’s manager, at his team also qualifying was enough for him to honour a promise to conduct the post-match media briefing dressed in Zorro regalia of hat, mask and cape. “This is the most joyful press conference I’ve ever had in my life,” the Portuguese said to laughter.

At 17 years and 192 days, City’s Phil Foden became the youngest player to make a Champions League start for an English club since Arsenal’s Cesc Fàbregas in 2004. Guardiola made seven changes from the win against West Ham United on Sunday, with only Ederson, Danilo, Leroy Sané and Eliaquim Mangala surviving as the manager rested key players for the trip to Manchester United.

This suggested City might be vulnerable and they ended the first half two goals behind. There was an early scare when City defender Tosin Adarabioyo’s miskick allowed Facundo Ferreyra to shoot. The contest was occurring in an OSK Metalist Stadium that had plenty of empty seats, which was understandable given Shakhtar’s home is 155 miles way, in what remains a war zone.

From this point Guardiola’s mood darkened. First, he felt Foden had been fouled. Then, while remonstrating with the fourth official, Shakhtar broke and Marlos warmed Ederson’s fingers.

Paulo Fonseca, the Shakhtar Donetsk manager, conducts his post-match media briefing dressed as Zorro to honour a promise he made if his side qualified for the last 16 of the Champions League. Photograph: Stanislas Vedmid/AFP/Getty Images

The goalkeeper could do nothing about Bernard’s opener. Danilo, though, should have. In backing off he allowed the Shakhtar winger time to fashion a sweetly struck curler that beat Ederson to his left. Ukraine’s champions should have doubled the advantage virtually from the kick-off. Once more it was Bernard who had time to aim at goal, though this time Ederson saved to his right.

Fonseca’s men did score a second soon afterwards. Yet again it originated down City’s right as a long ball bounced into space, Ederson mistimed a charge out and Ismaily turned it home.

As might be expected City rallied. Foden mis-kicked one effort, the defender Ivan Ordets blocked an Ilkay Gündogan shot and Jesus ran clear before the goalkeeper Andriy Pyatov intervened. Yet by the interval City had defended two corners and Taison became the latest to unload at Ederson, though the effort went high.

Despite City’s dire first-half showing, Guardiola decided against any changes at the interval. A bright beginning saw Bernardo Silva find Sané, whose first-time effort at last made Pyatov save.

The City manager was soon angered again following yet another Shakhtar attack down their left. Foden was not enjoying the best performance. When City next attacked and Yaya Touré found him, the teenager sliced the ball straight to Pyatov with the Ivorian screaming for a return.

There was, at least, consistent City pressure. Silva stood the ball up for a misdirected header and Guardiola applauded the effort. On the hour he swapped Sané for Brahim Díaz, an 18-year-old making his third City appearance, who added some required zest.

In a bid to protect the unbeaten record, Guardiola next brought on Agüero for Fernandinho. Yet it was Díaz who created an opening for Jesus to hit the post. Agüero slotted in the spot-kick in added time but it was not enough.

Guardiola said Fabian Delph was unable to play owing to illness, while Vincent Kompany “didn’t feel good”.

All eyes now turn to Old Trafford. It will be a major surprise if City are as disjointed there. “Now we prepare for Sunday,” Guardiola said.