Skip Bayless is concerned with Cavaliers star LeBron James' health moving forward into the playoffs. Stephen A. Smith, on the other hand, is troubled by Kyrie Irving not being happy in Cleveland and how that could negatively affect their locker room. (2:20)

WASHINGTON -- After a 113-99 loss to the Washington Wizards on Sunday, Cleveland Cavaliers guard J.R. Smith said his level of concern is "extremely high" regarding his team, which has lost three of its past four games.

"We can't play basketball like this going down the stretch," Smith said of the Cavs, whose lead over the Toronto Raptors for the top seed in the Eastern Conference sits at 1½ games. "There's 24, 25 games left in the year and you talk about contending, being a championship contender, and get blown out by a team. ... After losing a game to the No. 2 team in the East, then you come out and get thrashed and make it look good at the end.

"We can't do that. If we're serious about who we're supposed to be, then we can't do this."

The Cavs were without LeBron James, whom Cleveland coach Tyronn Lue decided to rest for the day. Washington quickly grabbed control of the game, leading by double digits in the second quarter. When the Wizards' halftime lead grew from nine points to 19 in the third quarter, Lue benched all of the Cavs' regular starters -- Smith, Kyrie Irving, Kevin Love and Tristan Thompson -- with 6:58 remaining in the quarter.

"I wasn't surprised," Smith said of the move. "We weren't playing the way we were supposed to play. We weren't executing our offense. We damn sure weren't playing defense. So I wasn't surprised at all."

Explained Lue: "Stars didn't give the effort we wanted or needed, so we got them out and brought the other guys in."

The Cavs ended up losing by 14 after trailing by as many as 30.

"For me as a competitor, I wanted to stay out there," Irving said. "I came back in two minutes later. I told him face-to-face that I wanted to go back in and still just play hard and play the right way. So, coming back out, it's definitely a challenge personally, but he's the head coach, and I respected what he did right there, but as one of the leaders on our team, I wanted to stay in and I came back in."

The loss comes on the heels of a 99-97 setback in Toronto on Friday in which the Cavs blew a nine-point lead with 5:37 remaining.

"If you lose a game like the other night to a team like Toronto and come out here and play the way we did, and you had a lack of energy, maybe we shouldn't be in this position," Smith said. "I don't know. It's tough. If we're going to play with a lack of energy after losing a game on the road and come out and play the way we did today, then we shouldn't be who we are and be in these uniforms."

Thompson agreed with Smith, calling the loss "embarrassing."

Smith brushed off comparisons between the Cavs' recent rough patch and their late surge last season, when they went on a 32-7 tear from Jan. 15 to April 8.

"Some guys talk about it. Some people don't," Smith said of the contrast in seasons. "I try not to dwell on what we did last year. I try to get better every year, better every game to move in the right direction. I mean, talking about last year is not going to help us right now. We got to figure out what's wrong with us, and hopefully we'll figure it out tomorrow against Indiana."

The Cavs are 0-2 this season and 3-12 over the past two seasons in games that James has missed.

"He gives us everything and he's one of those guys, one of very few that really makes everyone on his team better and makes everybody play at a high level," Love said. "Yeah, we definitely missed him [today]."

Smith said that the Cavs' struggles in James' absence should teach them something about the two-time champion's value to the team.

"It's one of those times you really got to appreciate the person he is and the player he is," Smith said. "So it's just a reality check for some of us."