It is an image that will define Pope Francis's first official visit to the Holy Land. Head bowed in prayer, the leader of the Catholic church pressed his palm against the graffiti-covered concrete of Israel's imposing "separation wall", a Palestinian girl holding a flag by his side. It was, as his aides conceded later, a silent statement against a symbol of division and conflict.

The powerful gesture was made minutes after an appeal to both sides to end a conflict that the pope said was "increasingly unacceptable". The unscheduled, conspicuous stop halfway through his three-day visit to the Holy Land – made en route to an open-air mass in Manger Square, Bethlehem – confirmed Francis's reputation for determined independence.

So too did his invitation to the Palestinian president, Mahmoud Abbas, and Israeli president, Shimon Peres, to join him in Rome to meet and pray together for peace – an unprecedented papal intervention in the stalled peace process.

Francis waves to the crowds at Manger Square. He invited the Israeli and Palestinian presidents to come to the Vatican to pray for peace a month after the collapse of US-backed peace. Photograph: Mohamad Torokman/reuters

Built by Israel as a so-called security fence to protect its citizens from attack after the second intifada, the barrier weaves through the West Bank, cutting through swaths of Palestinian territory and containing Palestinian residents. It has become an emblem of the Israeli occupation.

The pope's scheduled route took him alongside the wall, near Rachel's Tomb outside Bethlehem. His decision to step out of his white, open-sided popemobile and approach it – just days after the Vatican insisted his visit would not be controversial – was a surprise, not least for members of his own entourage.

Surrounded by Palestinian children, Francis's progress towards the concrete barrier was followed carefully by photographers and television cameras, as well as Israeli soldiers revealed in silhouette at the window of a nearby watchtower. "I know all about this," he is reported to have told one Palestinian official.

The Vatican's spokesman, Father Federico Lombardi, said afterwards: "I was not informed [of his plans to stop]. It was planned by him the day before … It was a very significant way to demonstrate his participation in suffering … It was a profound spiritual moment in front of a symbol of division."

Pope Francis touches the wall that divides Israel from the West Bank, on his way to celebrate a mass in Manger Square. Photograph: AP

Despite attempts by the Vatican to insist the visit was "purely religious", it has been loaded with political significance since Francis's arrival in a convoy of Jordanian military helicopters from Amman. While other popes might fly into Tel Aviv and proceed through Israel into Palestinian territory, Francis elected to bypass all Israeli border points.

In a carefully worded statement, delivered with Abbas in Bethlehem on Sunday, Francis referred directly to "the state of Palestine" and called on both sides to summon the courage to forge peace.

"For decades the Middle East has known the tragic consequences of a protracted conflict which has inflicted many wounds so difficult to heal," the pontiff declared. The situation, he said, had become "increasingly unacceptable".

Francis leads an open air mass in Manger Square. Photograph: Thomas Coex/AFP/Getty Images

"Even in the absence of violence, the climate of instability and a lack of mutual understanding have produced insecurity, the violation of rights, isolation and the flight of entire communities, conflicts, shortages and sufferings of every sort."

Francis proceeded from the separation wall to Manger Square in Bethlehem, which was packed with thousands of Palestinian Christians waiting for him to say mass. He entered the square – the reputed site of Christ's birth – to calls of "Viva al-Baba!" – or "Long live the pope!"

The service began with a rendition of the Palestinian song Mawatani – My Homeland – that speaks to the Palestinian desire for independence. The singers' voices echoed across a plaza hung with images linking Christ's suffering to that of the Palestinian people. The altar from which Francis delivered his message showed a baby Jesus wrapped in a keffiyeh, the traditional Arabic scarf that is a symbol of Palestinian nationalism.

Francis ate lunch with five families in a community centre on the edge of Deheishe refugee camp before flying out of Bethlehem into Tel Aviv's Ben Gurion airport, where he was officially welcomed to Israel by Peres.

The helicopter flight meant Francis avoided crossing through the separation wall via a checkpoint as his predecessor, Pope Benedict XVI, had done.

At Ben Gurion, Peres welcomed Francis, saying: "On behalf of the Jewish people and in the name of all the people of Israel, I welcome you with the age-old words from the Book of Psalms: 'Welcome in the name of the Lord.' Welcome at the gates of Jerusalem."

Here, Francis once again diverted from his prepared script. In Tel Aviv, the pope deplored an attack on a Jewish museum in Brussels on Saturday that left four dead, which he described as "this criminal act of antisemitic hatred". He added: "With a deeply pained heart, I think of those who have lost their lives in the cruel attack that occurred yesterday in Brussels."

While in Israel the pope will visit the Holocaust memorial at Yad Vashem, lay a wreath at the grave of the founder of Zionism, Theodor Herzl, and meet the ecumenical patriarch of Constantinople, Bartholomew.

The pontiff visits Israel's separation barrier in Bethlehem. Photograph: Ariel Schalit/AP

Francis will visit the holiest Christian sites in Jerusalem – including the Room of the Last Supper and the Church of the Holy Sepulchre – amid a long-term decline in the population of Palestinian Christians in the Holy Land. A survey conducted by Near East Consulting and released in April found that two-thirds of Palestinian Christians would like to emigrate.

Israeli authorities have imposed tight security measures during his visit, deploying an extra 8,000 police officers. Restrictions on movement throughout the city have prompted some Christians to complain they will have little chance of seeing Francis.

Some of the security has been prompted by the pope's plan to celebrate mass at the Room of the Last Supper – or "Cenacle" – which has angered some Jewish religious hardliners who venerate the site as the tomb of King David.

Twenty-six people were arrested after stones were thrown at police close to the site.