A source tells ET exclusively that the former 19 Kids and Counting star and his wife Anna are staying at a farm house in the mountains of Arkansas after it was revealed that Josh had two Ashley Madison accounts and he admitted to being "unfaithful" to his wife.

WATCH: Anna Duggar's Brother Slams Josh Duggar -- 'I Won't Stop Trying to Get That Pig Out of Our Family'

It appears that Anna is continuing to stand by her husband even though her brother Daniel publicly slammed Josh. "I won't stop trying to get that pig out of our family," Daniel wrote on Facebook, adding: "I have been thinking of her and sick to my stomach for her for last few days."

The rest of the Duggars are staying out of the public eye as well. The family has reportedly been keeping behind closed doors at their home in Springdale, Arkansas. However, ET did obtain photos of Josh's mother, Michelle, when she was spotted peeking out the front door of the compound.

The Image Direct

Most of the Duggars even missed a Sunday service at Cross Church, where the regularly attend.

Senior pastor, Ronnie Floyd, delivered a sermon focused on infidelity on Sunday, possibly in reference to the controversy surrounding the church's famous parishioners. The pastor's message was streamed live online, and promoted by Jill Seewald, who still attended the service with her husband Ben. The couple have a deep connection to the church, having gotten married there in June 2014.

See photos of the Duggar couple:



11 PHOTOS Josh Duggar and Anna Duggar See Gallery EXCLUSIVE: Josh and Anna Duggar hide out in mountain home as family avoids public eye NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NEW YORK, NY - MARCH 11: Josh Duggar visits 'Extra' at their New York studios at H&M in Times Square on March 11, 2014 in New York City. (Photo by D Dipasupil/Getty Images for Extra) NEW YORK, NY - MARCH 11: Hilaria Baldwin (R) interviews Josh Duggar and his daughter during their visit with 'Extra' in Times Square on March 11, 2013 in New York City. (Photo by D Dipasupil/Getty Images for Extra) NATIONAL HARBOR, MD - FEBRUARY 28: Josh Duggar speaks during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NEW YORK, NY - MARCH 11: Hilaria Baldwin (R) interviews Josh Duggar and his daughter during their visit with 'Extra' in Times Square on March 11, 2013 in New York City. (Photo by D Dipasupil/Getty Images for Extra) NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) NATIONAL HARBOR, MD - FEBRUARY 28: Anna Duggar and Josh Duggar pose during the 42nd annual Conservative Political Action Conference (CPAC) at the Gaylord National Resort Hotel and Convention Center on February 28, 2015 in National Harbor, Maryland. Conservative activists attended the annual political conference to discuss their agenda. (Photo by Kris Connor/Getty Images) Up Next See Gallery Discover More Like This HIDE CAPTION SHOW CAPTION of SEE ALL BACK TO SLIDE

MORE: Jessa Duggar Posts Bible Passage in Wake of Brother Josh Duggar's Recent Infidelity Confession

According to a source, Josh has not visited his family at their homes in days, and has been keeping a low profile as he and his wife stay sequestered in the mountain house, which was purchased for $55,000.

His sister Jessa Seawald currently resides at one of the homes associated with Josh's Ashley Madison accounts.

TheImageDirect.com

This past week news broke that Josh, 27, had two separate accounts with the Ashley Madison site, whose stated purpose is to help married people find partners to have affairs with. He later admitted to being "the biggest hypocrite ever" to his wife in an open statement to the media.

WATCH: Husbands of Snooki and Real Housewives of NY Star Reportedly Linked to Ashley Madison Accounts

"I have been the biggest hypocrite ever," Josh wrote on the Duggars' official website last Thursday. "While espousing faith and family values, I have secretly over the last several years been viewing pornography on the internet and this became a secret addiction and I became unfaithful to my wife."