I am answering this wrt cows milk, not milk of other animals like goat, camel etc.

As stated here in SB 8.8.11:

Pañca-gavya, the five products received from the cow, namely milk, yogurt, ghee, cow dung and cow urine, are required in all ritualistic ceremonies performed according to the Vedic directions.

So it's an important ingredient to perform vedic rituals. Also cow milk was meant for consumption as cows produce more milk than it needs to feed it's calf but humans are to consume the milk only after the calves had their share.

Quoting from an article posted on bhaktivedantamanor.co.uk here:

A point of consideration is that milk is a food sanctioned and designed by God, by Krishna, for humans and not just for the calf. The cow produces more milk than the calf needs and this is not accidental but by design. To get milk from a cow you need to impregnate the cow (there are numerous examples where some cows gave milk without impregnation but that is another story) and after a pregnancy of nine months a calf will be born and the cow will produce milk. Out of affection for her calf the cow will give as much milk as possible. The first milk is full of colostrum and this will give the best start to the calf. After about five days the milk looks normal and this is then suitable for us to drink. The milk is for the calf and for us.

Read the purpose of Surabhi (cow) as explained here SB 8.8.2:

These cows are the Lord’s pet animals. From the surabhi cows one can take as much milk as one needs, and one may milk these cows as many times as he desires. In other words, the surabhi cow can yield milk unlimitedly. Milk is necessary for the performance of yajña. Sages know how to use milk to elevate human society to the perfection of life.

Citing another quote to support the above statement Satapatha Brahmana Part V (SBE44):