Official relations between the world’s two largest democracies have come a long way since the new Indian prime minister Narendra Modi was once banned from entering the United States because of concerns over his human rights record.

But when Barack Obama touches down in Delhi this weekend to celebrate the two countries’ burgeoning diplomatic love-in with anticipated deals on trade, climate change and defence, he is expected to acknowledge an entirely different factor driving these two giants closer together: the increasingly influential Indian American diaspora.

Though less well-known than older immigrant groups, there are many prominent Indians in American society. The US surgeon general and new ambassador to Delhi; the governors of Louisiana and South Carolina; the chief executives of Microsoft and Pepsi; the deans of Harvard College and Harvard Business School and the frontrunner to become California’s next US senator are all of Indian American heritage.

New Jersey lawyer Seema Singh is typical of these successful professionals encouraging her two home countries to put their many historic differences behind them, and she recalls how much has changed since she once rallied with other expats against Modi’s visa ban in 2005.

“There has been a vast change in attitude; it’s like a total 360-degree turn,” says Singh. “Our community is very involved in politics; they have a lot of clout. It’s a huge vote bank for any politician who is running.”

Not all of the issues that matter to Indian American expats – such as immigration reform and trade liberalisation – are on the US agenda for Sunday’s summit meeting, but the influence of the diaspora cuts both ways, according to those who travel back and forth regularly.

“Almost every upper-middle-class family, and increasingly more middle-class Indian families, has a family member who is a professional in the United States,” says Sujai Shivakumar, a scientist at the National Research Council in Washington.

“Given the relative ease of travel and, in the past decade, the nearly free cost of communication, these family networks have been preserved and extended. This means, among other implications, that a critical mass of educated Indians in India have been exposed to and even understand the American mindset.”

“I think this [summit] very much underlines the role that the Indian diaspora has played in the kind of bilateral relationship,” agrees Waheguru Pal Singh Sidhu, a senior fellow at the Brookings Institution. “Sometimes quite to the surprise of Indians, but I think this is something which is also going to come up and resonate in the course of this particular visit.”

White House officials confirm that the “soft power” relationships binding Delhi and Washington are likely to feature as much if not more than the “hard power” negotiations over arms sales, nuclear energy and regional security that tend to capture most media attention.

“The president will speak to the fact that we’ve often noted this potential for the US-India relationship – two very large economies, the two largest democracies in the world, a large India diaspora that has thrived here in the United States,” says Ben Rhodes, Obama’s deputy national security advisor. “People have long looked at this relationship and seen the fundamentals in place for a really, really close partnership, and yet it’s been a challenge in translating that into outcomes.”

Facebook Twitter Pinterest The war memorial India Gate at Rajpath, the ceremonial boulevard for Republic Day parade in Delhi. Photograph: Altaf Qadri/AP

Exactly what deals are likely to be announced during the visit was still unclear just days before the president was due to arrive, but most observers expect tangible benefits from the growing diplomatic warmth.

“It’s typically Indian to negotiate to the last minute, but there has to be some forward movement, even something small. A president doesn’t come all this way for nothing, after all,” says professor Raja C Mohan, a respected Delhi-based analyst.

Climate change is a major area of discussion with one possibility lying in an agreement for a multibillion-dollar fund to invest in Indian solar and wind power with potentially significant involvement of US businesses. Other deals may include an agreement on exchange of information of security threats – particularly terrorism, movement on a stalled civil nuclear power deal and a package that would ease currently onerous procedures for visas.

For India however, the visit is mainly about something more intangible: a sense that the US respects India as a partner.

Obama is the first US president to visit twice while in power, and this will also be the first time a US president has been present at the Republic Day parade,which celebrates the introduction in 1950 of the country’s own constitution and thus its full independence from Britain. The parade, with its fly-past, tanks, camels and dancing schoolchildren, is a key national ritual celebrating Indian military might and, to a lesser extent, cultural richness.

“The parade began as an anti-colonial gesture, and evolved organically. In the public perception it is seen as a military parade and that sends signals. But it is a very popular event, a carnival, and fun,” said Neelam Deo, director of the Gateway House thinktank in Mumbai and a former diplomat.

Officials have claimed the idea of the invitation was the prime minister’s. Much of the media attention in India this week has focused on security arrangements in Delhi. Several square miles of the centre of the city will be closed to normal traffic and tens of thousands of police and paramilitaries will be deployed. The president and first lady are also expected to travel to Agra to see the Taj Mahal, a journey of around 200km, which can take four or five hours by car or train. The party will go by helicopter. The mausoleum will be closed for the visit and an exclusion zone established around it.

On his previous visit, in November 2010, Obama won over politicians with a pledge to the national assembly to back India’s long-standing bid for a permanent seat on the United Nations security council, and pleased businessmen with the announcement of $14bn worth of deals. Michelle Obama danced, sang and played hopscotch with disadvantaged and disabled children in Mumbai, astonishing and charming a country used to celebrities and leaders obsessed with protocol and status.

In an op-ed in the Wall Street Journal before his trip to New York and Washington, Modi described the US as India’s “natural partner”. This is an optimistic portrayal of a relationship which, though it has improved steadily since the end of the cold war, has often been rocky. In 1998, the US placed India under sanctions following a nuclear test. Modi’s own visa problems stemmed from 1998 US law barring entry to foreigners who have committed “particularly severe violations of religious freedom”.

The decision followed accusations that the former tea-seller and rightwing organiser had stood by during, or even encouraged, sectarian violence in the western state of Gujarat in 2002, when he was chief minister. More than 1,000 people, mostly Muslims, were killed by rioters.

Facebook Twitter Pinterest Indian prime minister Narendra Modi, second left. Photograph: PIB/AFP/Getty Images

Modi has denied all wrongdoing and India’s supreme court ruled that there was insufficient evidence to back the charges against him.

The UK ended its boycott of Modi in 2012. The EU swiftly followed, but the US ambassador only met the then prime ministerial candidate for the first time months before his landslide election win.

Modi inherited a fraught bilateral relationship that had been damaged when the Indian deputy consul-general in the US was arrested for visa fraud in New York, strip-searched and held in police custody in December 2013. The affair prompted a vitriolic reaction in India to what was seen as disrespectful bullying. This in turn prompted US commentators to accuse India of oversensitivity and behaviour unbecoming of an aspirant future power.

Modi won the first outright majority in Indian politics for 30 years in May 2014 after pledging to boost flagging growth in the emerging power where red tape, corruption, poor infrastructure and restrictive laws remain major challenges to business. The 64-year-old from humble origins moved swiftly to repair the damage and accepted an invitation from Obama to visit the US.

The trip, in September, received massive and positive coverage in India and was seen as a turning point. Modi focused much of his speech at a sold-out $1.5m event at Madison Square Garden in New York on the economic potential of India, his desire to reform government and to improve sanitation. The former rightwing organiser appealed directly for the diaspora to help him clean the Ganges, the heavily polluted river sacred to Hindus. Modi received rapturous applause for saying that when people asked him for a “big vision” he replied that he was only a “small man” who started out selling tea.

Indian media highlighted the presence of more than 40 US congressmen who were present to watch the speech, saying it was evidence of India’s emergence as a “great power”.

“Aspirational India, modern India has certainly admired the economic and technological achievements of the US. Their’s is a higher comfort level with the US. This is the government catching up with the people in a sense,” said Deo, the analyst.

But both India and the US, beyond the economic advantages of closer ties, have mutual geopolitical interests too.

India is nervous about Chinese efforts to build influence in the region, particularly in Sri Lanka, though concerns there have been allayed by the defeat of president Mahinda Rajapaksa in the recent election. Washington made no secret of its distaste for Rajapaksa who had swung the island state towards Beijing.

Both India and the US are concerned about China’s ties with Pakistan. When Xi Jinping, the Chinese premier, made his first trip to India last year his announcement of $20bn of investment was seen as a disappointment and was overshadowed by renewed tensions over the contested Himalayan border between the two emerging Asian powers.

There has also been significant tension over past six months between Pakistan and India with long periods of shelling and casualties on both side. India will want the US to toughen its policy towards its hostile neighbour which it blames for much terrorism locally. Delhi will also be pushing for a greater US commitment to Afghanistan as US troops there pull out, fearful of anarchy in the war-torn south Asian state allowing anti-Indian extremists, some based in Pakistan, to destabilise the region.

This will not lead to any friction however, said Gopalapuram Parthasarathy, a former Indian high commissioner to Pakistan and analyst.

“We are in touch and we know each others requirements and limitations,” he said.