Jimmy McMillan -- the infamous founder of the "Rent Is Too Damn High" party -- could soon lose his rent-controlled Manhattan apartment ... this according to a NYC judge.

It's all over a dispute with McMillan's landlord, who is now demanding to see financial documents to prove Jimmy's still actually living in the apartment -- because if he's not, it's a big problem.

According to the NY Post, Jimmy admitted after court ... he's temporarily living in another apartment in Brooklyn -- but only to give his son (who lives with him in Manhattan) a chance to put the charms on a lady ... if you know what we mean.

In Jimmy's words ... "I want a grandkid."