"No one ever confronts Jim Bob – no one," a Duggar family source tells PEOPLE

'Duggarville' Exposed: Jim Bob 'Is like a Dictator'; How the Clan Uses a Code Word to Repress Lust

Anna Duggar’s siblings beg her to leave husband Josh after his shocking double life is revealed. Subscribe now for more on the cheating scandal and the Duggars’ ultra-conservative lifestyle, only in PEOPLE!

For a decade on the hit TLC show 19 Kids and Counting, Jim Bob and Michelle Duggar played the roles of “traditional” parents keeping their vast brood in line with warmth, gentle guidance and good humor.

Get push notifications with news, features and more.

But behind the scenes, powerful patriarch Jim Bob, 50, exerts authority over his wife and family, several close sources tell PEOPLE in this week’s issue. (The Duggar family did not respond to requests for comment.)

“Jim Bob’s favorite word is ‘no,’ ” says one Duggar family source. “He’s like a dictator. He once saw a guy kissing a girl before marriage and confronted him about it. And the guy said, ‘Jim Bob, show me that verse in the Bible that says we can’t kiss. Get out your Bible.’ And Jim Bob was like, ‘Uhhhh …’ because no one ever confronts Jim Bob. No one.”

Adds a second source: “Jim Bob calls the shots. Always has and always will.”

In the wake of eldest son Josh‘s cheating scandal, the Duggar family source says Josh’s wife Anna will continue to have little contact with the world beyond the family’s tightly controlled lifestyle.

“The scary thing is, I don’t know who is talking to Anna, outside of the people in Duggarville,” says the source.

RELATED VIDEO: ‘Duggarville’ Exposed: ‘Jim Bob Is like a Dictator’

Key to the family’s belief system, which aligns with a fundamentalist Christian ideology known as a conservative patriarchy, is the belief that sexuality and sex are forbidden topics of discussion in the home.

“They don’t talk about sex. They don’t talk about sexuality,” says the Duggar family source.

Almost entirely cut off from the secular world, the Duggars strictly limited their children’s exposure to popular culture – and the source says that Jim Bob went as far as removing the batteries of any toys that made music “because he didn’t want [his kids] to dance or move their bodies that way.”

In their own 2014 advice book, Growing Up Duggar, four of the Duggar daughters – including Jill, 24, and Jessa, 22 – shared that the family has a code word, “Nike,” to help the men avoid lustful thoughts if an attractive woman crosses their path.

“That’s a signal to the boys, and even to Dad, that they should nonchalantly drop their eyes and look down at their shoes as we walk past her,” they wrote.

Says the Duggar family source: “They say it, and all the boys look straight down to the floor every time. It’s like an automatic military thing. Things like that blow my mind.”

• Reporting by EMILY STROHM