Pussy Riot member Nadezhda Tolokonnikova has been freed from prison under an amnesty that allowed for her early release from a two-year sentence for a protest in a church against Vladimir Putin.

Tolokonnikova shouted "Russia without Putin" after she was freed from a prison in the Siberian city of Krasnoyarsk on Monday.

Hours earlier, bandmate Maria Alyokhina was released from a different jail and dismissed the amnesty as a propaganda stunt.

The two women were granted amnesty last week, in what was largely viewed as the Kremlin's attempt to head off criticism of Russia's human rights record ahead of the Winter Olympics in Sochi in February.

Three band members were jailed after being found guilty of hooliganism motivated by religious hatred and sentenced to two years in prison for a performance at Moscow's main cathedral in March 2012. One, Yekaterina Samutsevich, was released on a suspended sentence in October 2012.

Tolokonnikova walked out of a prison gate in the eastern Siberian city of Krasnoyarsk, smiling to reporters and flashing a V sign.

"How do you like our Siberian weather here?" said Tolokonnikova, wearing a down jacket but no hat or scarf in a temperature of -25C . Tolokonnikova said the way prisons are run reflect the way the country is governed.

"I saw this small totalitarian machine from the inside," the 24-year-old said. "Russia indeed works just the way the prison colony. That's why it's so important to change colonies so that Russia would change, too."

Alyokhina was released early on Monday morning by prison officials who drove her from the prison colony outside Nizhny Novgorod, and left her outside the city's railway station, still dressed in her prison uniform.

She told the Guardian she was not allowed to pack her belongings or even say goodbye to fellow inmates.

"This is typical behaviour in our penitentiary system, which is as closed and conservative as jail itself – prison officials' methods are all about secrecy, no information and zero transparency."

Alyokhina told the Dozhd TV channel that she was "too shocked" when she was released from the prison colony to grasp what was going on. She also said she would have stayed behind bars to serve her term if she had been free to choose.

"If I had a chance to turn it down, I would have done it, no doubt about that," she told Dozhd. "This is not an amnesty. This is a hoax and a PR move."

The Russian parliament passed an amnesty bill last week, allowing the release of thousands of inmates. Alyokhina and Tolokonnikova qualified for amnesty because they have small children.

Alyokhina's release came days after Putin pardoned Mikhail Khodorkovsky, a former oil tycoon and once Russia's richest man, who spent a decade in prison after challenging Putin's power. Khodorkovsky flew to Germany after his release and has ruled out entering Russian politics. He has pledged, however, to fight for the release of political prisoners in Russia.

Speaking to the Guardian, Alyokhina said her release from jail felt more like "a secret special operation than an act of humanism".

She described her prison sentence as a time of "endless humiliations", including undergoing forced gynaecological examinations almost every day for three weeks.

She and Tolokonnikova plan to become human rights activists: "We will be creating very special, colourful and powerful programmes to defend other innocent women in Russian prisons, who are being turned into slaves right now."