Amazon stock hit $2,000 per share for the first time Thursday. Shares rose as much as 1.4 percent to $2,025.57 but ultimately shied away from the record high to close at $2,002.38.

It's a major milestone in the stock's climb to match Apple's $1 trillion market valuation. At Thursday's peak of $2,025, the stock would need to gain just $25 per share in order to reach the 13-digit market cap.

Amazon's record high comes a day after the stock gained 3.2 percent, sparked by a bullish call by Morgan Stanley. Analysts at the bank raised their 12-month price target on Amazon to $2,500 from $1,850. The new target price — the highest on Wall Street — implies a 25.1 percent upside from Wednesday's close.

"We have increasing confidence that Amazon's rapidly growing, increasingly large, high margin revenue streams (advertising, AWS, subscriptions) will drive higher profitability and continued upward estimate revisions," analyst Brian Nowak said in a note to clients Wednesday.

Amazon shares have been on a tear this year, surging 71.2 percent through Thursday's close. The stock has more than doubled over the past 12 months.