Brain sex differences are established developmentally and generate enduring changes in circuitry and behavior. Steroid-mediated masculinization of the rat amygdala during perinatal development produces higher levels of juvenile rough-and-tumble play by males. This sex difference in social play is highly conserved across mammals, yet the mechanisms by which it is established are unknown. Here, we report that androgen-induced increases in endocannabinoid tone promote microglia phagocytosis during a critical period of amygdala development. Phagocytic microglia engulf more viable newborn cells in males; in females, less phagocytosis allows more astrocytes to survive to the juvenile age. Blocking complement-dependent phagocytosis in males increases astrocyte survival and prevents masculinization of play. Moreover, increased astrocyte density in the juvenile amygdala reduces neuronal excitation during play. These findings highlight novel mechanisms of brain development whereby endocannabinoids induce microglia phagocytosis to regulate newborn astrocyte number and shape the sexual differentiation of social circuitry and behavior.