Ryen Russillo stops by SportsCenter to talk about Ray Allen's retirement and what he meant to the game of basketball. (1:04)

WALTHAM, Mass. -- How should we describe Ray Allen's legacy with the Boston Celtics? Well, it's complicated.

No one could credibly deny Allen's impact on the success of Boston's most recent Big Three era, especially the team's 2008 title. But while Paul Pierce's No. 34 and Kevin Garnett's No. 5 seem certain to hang in the rafters at TD Garden, the same cannot be said for Allen's No. 20 jersey.

Allen, who announced his retirement after 18 seasons on Tuesday, logged three All-Star appearances in five seasons with Boston. If not for his arrival in a trade on draft night in 2007, the Celtics probably wouldn't have had enough talent to convince Garnett to accept a trade from Minnesota. Without Allen's shooting, the Celtics don't hang Banner 17.

Yet for all of Allen's memorable moments, some fans recall only his struggles. Such as his inconsistencies in the 2010 NBA Finals against the Los Angeles Lakers, which included an 0-for-13 performance in Game 3 (he scored 2 points in 42 minutes) and a 3-for-14 night in Game 7 (13 points in 45 minutes).

When Allen's ankles balked in 2012, he was replaced in Boston's starting lineup by second-year guard Avery Bradley. Allen only got his starting spot back when Bradley's shoulder injuries forced him out of the playoffs, and Allen struggled again in a postseason that saw LeBron James and the Heat rally from a 3-2 deficit in the conference finals. For Boston's Big Three, it would be the team's final run as a true title contender.

And many Celtics fans will simply never forgive what happened next. That summer, little more than a month removed from the sting of that Game 7 loss, Allen signed with the rival Heat, taking the backup role that seemed to bother him so much in Boston.

Boston's Big Three brought a 17th banner to Boston, but Ray Allen's departure left hurt feelings. Fernando Medina/NBAE/Getty Images

"I look back at all our time spent in Boston ... we had a lot of disappointments, but we shared a lot of thrills," Allen said when he was introduced in Miami. "It is sad to me that I'm not going to be with those guys anymore. But I look forward to being here."

Before the Celtics and Heat met on opening night of the 2012-13 season, Garnett said he "lost" Allen's phone number. Then in a national TV game in Miami, Garnett gave Allen a cold shoulder when he tried to dap him up while entering the game.

Outside TD Garden, vendors sold "Judas Shuttlesworth" T-shirts -- a play off of his character of Jesus Shuttleworth in the 1998 Spike Lee film "He Got Game" -- and Allen became a villain for defecting to the enemy rather than trying to help Boston recover. It didn't help that Allen went on to produce the biggest play of the 2013 NBA Finals with his miracle 3-pointer in Game 6 against the San Antonio Spurs, and the Heat won Game 7, giving Allen the second championship ring that has evaded Pierce and the now-retired Garnett.

When Allen noted this fall that he had conversations with the Celtics and Bucks about a possible return, the reception was lukewarm at best from Boston fans. Some were intrigued by Allen serving as a veteran presence to a young team looking to take the next step in its playoff journey, while others wondered if his playing time would be better served for players who had a better chance to impact the Celtics' title hopes moving forward.

No Celtic has worn Allen's No. 20 in Boston since his departure. But early returns on our informal Twitter poll suggests that Celtics fans have no desire to see it go out of circulation.

Asked during a recent appearance on Boston sports radio 98.5 the Sports Hub whether the team would retire Allen's jersey, Celtics president of basketball operations Danny Ainge was noncommittal.

"I don't know. I'm not sure. Time heals, and I think that he was a great player for us," Ainge told the "Toucher and Rich" program. "I love Ray, and I'm grateful for Ray. And I'm grateful for all that he brought to us in the five years he played here. He was a fantastic player. Time will tell on all those."

Ainge, who has not so subtly hinted that Pierce and Garnett's numbers will hang in the rafters, said Celtics co-owners Wyc Grousbeck and Steve Pagliuca and team president Rich Gotham would huddle someday to explore the possibility of Allen's number being retired as well. Pressed on which way he'd vote, Ainge said simply, "I'm going to stay away from that vote right now, kinda like I'm staying away from the presidential election."

Time might very well heal all wounds. We are two years away from Boston celebrating the 10th anniversary of the 2008 title team, and Allen's presence in those moments might help thaw any icy feelings that remain.

For now, things remain a bit awkward for Allen in these parts.