Though it has only a few dozen employees, Ginnie Mae supervises a $350 billion market, and its troubles have grown quickly because it picks up the losses of mortgage companies that it declares in default. So far this year, there have been seven defaults; there were six in all of 1988 and four in 1987.

As a result of the increased defaults, Ginnie Mae is stuck managing $13 billion in mortgages, up from just $2.8 billion last October, said Jack Flynn, a H.U.D. spokesman. In effect, Ginnie Mae has become an asset manager, a job for which it has little experience, though it does collect more fees for servicing the portfolios, officials say.

Moreover, Ginnie Mae's losses this year already exceed what it set aside as a reserve, and next year's reserve for losses will also be insufficient, Mr. Flynn said.

One top H.U.D. official, who asked not to be identified, said the future consequence of rising Ginnie Mae defaults and inadequate supervision ''is not going to be pretty for the taxpayers'' because the reserves will eventually be insufficient and have to be supplemented with appropriated funds. But no one is saying Ginnie Mae's overall financial reserves of $1.7 billion are not sufficient for the time being.

Guy Wilson, Ginnie Mae's vice president for mortgage-backed securities, acknowledged that the ''entire industry needs stricter oversight'' and that earlier this year Ginnie Mae had put into place a system for more frequent review of mortgage brokers. But he insisted that the agency ''has the highest standards'' and that securities guaranteed by Ginnie Mae had the highest credit ratings.

A recent episode involving mobile homes indicates how lax oversight by the housing commissioner's office, which also oversees the F.H.A., can prove costly for Ginnie Mae. Mobile Homes: Risk Is Shifted An audit report issued last year by H.U.D.'s inspector general and released last month under the Freedom of Information Act showed how a mobile home insurance program designed to limit the exposure of H.U.D. wound up being borne by the department anyway because the risks were transferred to Ginnie Mae.

The report said that what started out as a co-insurance program under the Federal housing commissioner, in which lenders bore some of the insurance risk, became ''a full insurance risk because most lenders place manufactured housing loans in Ginnie Mae pools.''