A man accused of killing his girlfriend's 12-month-old son dropped the baby on his head five times because he had waited too long for her to get heroin from a Richmond drug dealer, a court has heard.

Paul Ta Vuong allegedly believed his girlfriend, Kate Leithhead, was "f...ing around" or "f...ing" the drug dealer so he decided to play rough with the baby, Silas, the court was told.

Mr Ta Vuong has pleaded not guilty to murdering Silas on May 15 last year at Ms Leithhead's Richmond apartment.

In his opening address to the jury on Friday, Crown prosecutor Andrew Grant said it would be alleged that after his arrest, Mr Ta Vuong was being held on remand when he told two prisoners, to be known as Witness A and Witness B, how he had deliberately dropped Silas on his head five times because he was angry at having to wait so long for Ms Leithhead to return from buying heroin.

A post mortem examination revealed Silas had died from blunt force trauma to the head.