Justin D. Barnard makes the most convincing case I've seen against cognitive enhancing drugs:

...while the capacities to procure and to process information are indeed goods of human life, they are neither the highest of human goods nor are they ends in themselves. Yet, the use of cognitive enhancers by the healthy implicitly treats the single good at which the drug aims as though it were the most important or only good of one’s mental life considered as a whole...if merely thinking (very fast!) about lots of information were the most important or only good of the human mental life considered as a whole, why not simply replace us with computers?