Labor leader Bill Shorten refused on three occasions to promise that he will end the indefinite detention of asylum seekers on Nauru and Manus Island, after being confronted over the issue on live television.

In a solo appearance on the ABC's Q&A program on Monday night, Mr Shorten said he would have led Labor to victory if an election was held at any point in the last two years, and he said popularity was not the most important challenge of his job as Opposition Leader.

Mr Shorten's comments come after his one-time leadership rival, Anthony Albanese, for the first time on the weekend ruled out a leadership challenge. Mr Albanese, from Labor's left faction, had previously refused to rule out a challenge since the resignation of several Labor MPs who belatedly admitted they were in breach of the constitution's rules on dual citizenship.

Those byelections have been touted as a test of Mr Shorten's leadership as it approaches its fifth year. The polls will be held on July 28, delaying any showdown within the party over its asylum seeker policy, where parliamentary newcomer Ged Kearney had vowed to lead a change of the party's position.