''The other question you have to ask is, 'What businesses are failing?' '' Mr. Bloomberg added. ''I think businesses that are already in existence -small, medium or large - tend to have greater problems during a recession, as people stop spending or change their buying habits. I'm not sure that has anything to do with new businesses.'' Borrowing Costs Key Factor

Though other sources echoed Mr. Bloomberg's comments, most still say that starting up a business these days is no picnic. Undoubtedly, the primary problem is the cost of borrowing money. For a smallbusiness owner, who is generally forced to borrow at rates far above the prime interest rates charged to a bank's best customers, high capital costs may narrow his choices of business.

''Without a doubt, it's the biggest problem a small-business man will face,'' said Gordon Bizar, president of the International Business Network, a political and economic organization for small businesses. ''On the other hand, if you can get the money, now is the time to be buying a business.''

Most people do it the other way around, Mr. Bizar said. ''They get in when the market is good and pay premium prices,'' he said. ''Those are the businesses that are failing today.''

Of course, not all new businesses are surviving, either. A small company founded in Chicago last year with the goal of buying X-ray negatives from hospitals and selling them for their silver content died when silver prices fell. Other businesses struggle on, hoping to last until consumer spending picks up. Some find their niche right away. 'Nothing to Get Excited About'

Scott Sanders, an actor who co-founded Survival Techniques in his New York loft last December, said sales of his custom-painted decorated pillows, at $22.50 each, had been ''O.K., but nothing to get excited about.''

Mr. Sanders said the recession had definitely hurt his sales, but, because his costs are so low (he and several friends do all the work), the tiny company will probably stay in business.