I feel like a logical answer would be that most of the drama CDs are canon only to the anime world, as a majority of them(I think?) are written by anime staff. For it to be official to manga plot too, it would have to be written by the author himself or based off of the light novels, which are also written by Horikoshi.