Indonesia's refusal to accept the return of asylum seeker boats is "very frustrating", Immigration Minister Scott Morrison has said, in what appears to be a swipe at Australia's northern neighbours.

Mr Morrison said in Sydney on Monday there was “no real rhyme or reason” why Indonesian authorities had not taken stricken asylum seekers back to their shores after a three-day impasse late last week.

The stand-off over the Coalition's hardline asylum seeker policies has exacerbated tensions with Jakarta over the past week, with Indonesian officials publicly goading the Abbott government.