Indian spiritual leader Sai Baba, one of the country's most famous gurus, died in hospital on Sunday, triggering an outpouring of grief from devotees around the world.

He was 85 and succumbed to heart problems, his doctors said in a statement, after being in critical condition for more than three weeks.

Thousands of Hindu followers massed at the Institute of Higher Medical Sciences hospital where Sai Baba was being treated in his hometown of Puttaparthi, in the southern state of Andhra Pradesh.

Police used barriers to hold back mourning crowds, and appealed for public calm after the death was announced.

"Sai Baba is no more with us physically. He breathed his last at 7:40 am and died due to cardio-respiratory failure," the hospital statement said. "His body will be kept for public worship for two days on Monday and Tuesday."