Lets go over the parts you are going to need to purchase to build your console.

Raspberry Pi Zero $5

The cheapest, smallest Raspberry Pi yet, perfect for this project. Pick one up on Adafruit, Element 14 or Ebay.

2.4in 240x320 ILI9341SPI TFT Display $5.88

Cheap and easily interfaced with the Raspberry Pi. Good Quality and performance. Buy Here.

Lithium Ion Battery 14500 cell $0.99

These are practically the cheapest rechargeable batteries available. They claim to have a capacity of 1200mah, but that is bogus. My tests show around 400mah. One of these will give you about an hour of playing time. If you have an extra $10 go for one of these. Battery life will increase by a factor of 4. Or, if you have a old cell phone with a 3.7V lion battery in it, use that, and make your console $19. Or, if you are willing to make the console larger, wire a multitude of these batteries in parallel.

Prototyping Board $1

You will need about 7x9cm of perfboard, I got mine on ebay for $1.

Charging Circuit $3

To charge your dirt cheap batteries you will need a charging circuit. Check out my list on Mouser. You will also need a SOT-23 breakout board for the charger IC. If you do not have the skills required to build a board from a schematic or solder a SOT-23 component, you can get a pre-built charger from adafruit.

Micro USB Breakout $1.50

This will be used for charging your console. If you order the Adafruit board this part is not necessary. Buy here.

3V to 5V DC-DC Step Up Boost Converter $1.50

This is used to boost the battery voltage to a voltage usable by the Raspberry Pi. Buy Here.

Total Cost: $19.86

You will also need some wire, basic soldering tools, and the required adapters and dongles required to setup the Pi Zero. Most people already have this stuff so no extra cost.