The Treasury Department, which has already announced plans to put a woman on the $10 bill, is also conducting a review to see if it still makes sense to keep the penny.