In Lamb's essay, cooking was also an accidental discovery. One day the cottage of a Chinese swineherd burned to the ground, roasting a litter of new-farrowed pigs. The son, Bo-bo, stooped down to feel one of the parched pigs and burned his fingers. He stuck the fingers in his mouth to cool the burn. He liked what he tasted, and from then on it was observed that cottages were burning down every time a sow farrowed. Food-Processing Changes

Dr. Brace's group concluded, in a report in the journal Evolution, that it was not dietary change but a change in food-processing techniques that provided the conditions for the onset of dental reduction seen in the last 100,000 years. The anthropologists also noted that pounding, grinding and milling tools became common in this period in many parts of the world, perhaps contributing to the trend toward smaller teeth.

Soon after the Ice Age ended, about 10,000 years ago, the reduction accelerated to a rate of 1 percent every 1,000 years, the anthropologists said, because of the introduction of pottery, which made it possible to stew food to a liquid consistency. Large teeth became even less essential to survival.

''It is no accident that we discover cemeteries full of people with no teeth in those regions that had pottery and earth ovens,'' Dr. Brace said, noting that the people had apparently survived for years without teeth. ''But in areas without those things, the cemeteries are full of people who still have most of their teeth.''

This theory could explain why the teeth of northern Europeans and their descendants are smaller than those of Australian Aborigines, Dr. Brace said. The glaciers forced the Europeans to develop the cooking technology that led to tooth reduction. Aborigines, however, did not develop the technology until about 10,000 years ago and, consequently, they got a late start in tooth shrinkage. Argument Is Disputed

Dr. Brace disputed the argument that dental reduction could have been caused by an invasion of new people into Europe from the east, replacing the resident Neanderthals. His analysis, he said, showed that fossil teeth from the Middle East at that time were strikingly similar to those of contemporaries in northern Europe.

It was cooking, in the view of Dr. Brace and his colleagues, Mr. Hunt and Karen R. Rosenberg, that ''changed the nature of the selective forces that had once operated to maintain the jaws and teeth.'' The size reduction may thus be the result of mutations alone, making this one instance in which natural selection is not the main agency of evolution.