"I was pretty annoyed at him at first," Teresa Giudice tells PEOPLE about her brother, Joe Gorga

Blood may run thicker than water, but Teresa Giudice doesn’t want anything coming in between her and her children — including other family members.

When news broke on Oct. 10 that her husband Joe Giudice will be deported to his native Italy — he emigrated to America as a child but never obtained citizenship — at the completion of his prison sentence next year, Teresa wasn’t the one to tell her eldest daughter, Gia.

Get push notifications with news, features and more.

“My daughter [Gia] found out and then I had to tell my other two daughters. It was just — it was not a good day,” Teresa, 46, tells PEOPLE.

Image zoom Joe Gorga/Instagram

Gia, who is a senior in high school, was in class when she was informed by her uncle, Joe Gorga, about her father’s court-ordered deportation.

“I was pretty annoyed at him at first,” Teresa admits about her brother spilling the beans.

“He apologized right away. He said, ‘I’m sorry,’ ” recalls Teresa, who was thinking at the time: “Just wait till you speak to me before calling my child.”

Image zoom Teresa Giudice, Gia Giudice and Joe Giudice Dave Kotinsky/Getty Images

After Gia, 17, learned the news, she immediately called the Real Housewives of New Jersey star.

” ‘Mommy, what happened?’ And then I told her,” Teresa says of their conversation. “And then she left school. She didn’t want to be at school anymore.”

Teresa then informed daughters Gabriella, 14, and Milania, 12, but still has yet to tell Audriana, 9. “She’s too young,” she says of her smallest child.

Image zoom Gia Giudice and Joe Giudice Gia Giudice/Instagram

“The adults can handle anything. Adults can handle it because they’re big. That’s why we’re adults. We can handle it. But who would it affect the most? It’s the children,” Teresa explains. “And that’s what’s sad about it, because the children are affected by it. It’s very sad, because they’re going through all this heartache, meanwhile it’s not even their fault. You know? They’re the ones that hurt the most.”

RELATED VIDEO: Teresa Giudice Splurged on Daughter Gia’s Lavish 17th Birthday Gift: a BMW!

Joe, 46, is currently serving out a 41-month prison sentence for mail, wire and bankruptcy fraud that began in March 2016. He is set to be released in March 2019.

The family is working to appeal the judge’s ruling in the hopes of Joe returning home to New Jersey when he completes his time behind bars.

“I’m taking it day by day. Yeah, whatever we have to do. Whatever the attorneys tell me. Whatever they tell me that’s what I do. You know, we’re going to fight this,” Teresa says.

She adds of her daughters, “They know we’re gonna fight it. They know mommy is doing everything she can.”