According to the 1961 Convention on the Reduction of Statelessness, Article 3:

For the purpose of assigning nationality, birth on a ship or aircraft shall amount to birth in the territory of the State that gives its flag to that ship or aircraft.

However, only about 40 nations (not including the USA) have ratified this convention - and what territory a birth has occurred on is not necessarily what determines citizenship.

So in reality, it's a huge mess between:

what country the plane is registered in

what country's territory the plane was flying over at the moment of birth

the nationality of the mother and father

whether these countries' laws claim jurisdiction over in-air births

whether these countries' laws are based on the jus soli or the jus sanguinis principle

whether these countries' laws allow dual nationalities

In theory, you may have cases where none of the countries involved would grant the baby citizenship, or where two (or even three) would grant it automatic and exclusive citizenship.

In practice, I suspect that in almost all cases at least one of them would grant citizenship, and it's up to the mother which one she applies for - and up to the bureaucrats how difficult that is.

A separate question is what the "place of birth" on the birth certificate will say - I suspect that's up to the discretion of said bureaucrats, since it will have little importance in most cases.

Update: I recently found an article that has a real-world example. It pretty much agrees with what I wrote, namely the person portrayed got her mother's UK citizenship with a passport note saying "Holder born on an aeroplane 10 miles south of Mayfield, Sussex.", which later had to be changed to "born at sea" to comply with an EU directive. US State Department rules, on the other hand, would list the place of birth as "in the air".