Kim Kardashian West's younger brother has been stepping away from the spotlight in recent months

Unlike his ever-popular family, Kim Kardashian West‘s younger brother Rob Kardashian prefers to stay out of the spotlight these days.

Be it disagreements with his sisters or reportedly dealing with depression, family and sources say he’s opting for the quiet life.

Get push notifications with news, features and more.

But the 28-year-old made a rare appearance last week. The Keeping Up with the Kardashians star was spotted Wednesday night wearing a black T-shirt, cap and shorts while out to grab dinner at an In-N-Out restaurant in Woodland Hills, California.

“He’s still struggling,” a source tells PEOPLE, adding that Kardashian “just can’t get motivated. Everyone is trying to be encouraging, but for the most part he doesn’t want to hear what he already knows.”

Kardashian’s troubles have long been documented on the family’s reality show. He’s been shown distancing himself from the group and forgoing family appearances and vacations – most notably, he abruptly left Europe ahead of Kim and Kanye West’s wedding.

Putting it frankly, “he’s sad, he’s bitter, he’s single,” a source told PEOPLE back in March. “He is very depressed and has been [for a while]. It’s really hard for him that his sisters are so public and he looks like the shut-in.”

Despite it all, he’s maintained a presence on social media. Amid the media storm surrounding his former stepparent Caitlyn Jenner, Kardashian Tweeted, “You have always been a role model to me and now more than ever, I look up to you. LOVE YOU!!”

As for his public struggle to lose weight, Kardashian’s trainer Gunnar Peterson told PEOPLE last June, “I know he can do it.”

Now, the source tells PEOPLE there’s only one person standing in his way. “He’s got to want to do it on his own.”