On Tuesday nights this past year, you could find me and my roommates gathered around the TV for our weekly bonding activity: watching TLC’s 19 Kids and Counting with a big bowl of popcorn on the table and a glass of wine in hand. Though intended to be a way to unwind from schoolwork and senior year anxieties, watching Jim Bob and Michelle Duggar and their nineteen kids (plus grandchildren) often led to some interesting discussions.

On the one hand, my roommates and I found many of the family members endearing, and we appreciated their wholesomeness. On the other hand, like many viewers, we criticized the family’s approach to sexual morality, not out of disdain for their high regard for purity, but because the rules they’ve created for romantic relationships distort the true nature of chastity.

In the Duggar family, children who are courting (as opposed to dating, which is forbidden) must be chaperoned whenever they spend time with their significant others, even when texting or talking over Skype. They don’t hold hands until they’re engaged, and all hugs with the opposite sex must be “side hugs” that avoid any frontal contact. They eschew dancing as an occasion of sin. They don’t kiss until their wedding day. The Duggars, who describe themselves as independent Baptists, admit that these rules aren’t necessarily biblical.

There’s something disturbing about the fear that spending time with a boyfriend in a public place such as a park or a restaurant, or Skyping with a girlfriend, will inevitably lead to sin. Chastity, which the Catechism describes as “the successful integration of sexuality within the person and thus the inner unity of man in his bodily and spiritual being” (2337), is a form of temperance, the virtue that orders our appetites and desires. Given human concupiscence and how easy it is for us to misdirect our God-given desire for intimacy, it’s understandable that temperance in romantic relationships and situations entails boundaries and restrictions.

Yet couples can’t spend their whole lives being chaperoned. Yes, they should reasonably expect their friends and families to hold them accountable. But the realities of life will sometimes put them in situations in which the habits they have formed will be crucial to how they respond. What’s going to happen if unforeseen circumstances force an unmarried couple to spend a night together? Or when a spouse is deployed overseas for a year? Can they practice virtue without one of their mothers breathing down their necks?

Let’s take another example. There’s been more than one news story over the past year highlighting the phenomenon of some ultra-Orthodox Jewish men delaying flights because, for fear of being tempted, they refuse to sit next to a woman who is not their wife or an immediate relative. A generous person might sympathize with their perseverance in living out their values; most object to their inconveniencing passengers for what many would consider sexist reasons. I think there’s another reason that people don’t take kindly to these men’s actions: Even in a culture that rejects the existence of sin and embraces sexual liberty as its highest ideal, people implicitly understand that only a depraved man would be tempted by a woman on an airplane who’s minding her own business and isn’t seeking attention through her words, actions, or dress.

To esteem purity while being unable or unwilling to exercise it in day-to-day life is not the sexual integration the Catechism calls chastity. In fact, the Duggars’ and ultra-Orthodox Jews’ rigid rule-based system is a cop-out: It’s much easier to say, “Don’t touch anyone, ever, unless you’re married” than to make prudential judgments about how to be chaste in particular circumstances. The virtues demand effort precisely because they are means between extremes. Chastity allows the soul to govern the body so that a person can give of him or herself to others properly in every context. A sexual ethic based on fear, on the other hand, keeps us away from others. Love and fear are mutually exclusive.

The irony of the Duggars’ and some ultra-Orthodox Jews’ sexual mores is that they end up sexualizing nearly all human touch. Holding hands, rather than being a form of affection, or an expression of comfort, or a method of guidance, is reduced to sexual desire. Bumping elbows with your seatmate on the plane becomes an occasion of sin.

This isn’t to say that there are no set-in-stone rules; premarital sex and actions leading up to it are objectively wrong because they separate the conjugal act from its meaning. But there’s never a clear answer to the classic “How far is too far?” question because individuals and the contexts and cultures they live in vary, requiring them to put some serious thought and effort into living virtuously.

When the Duggars say that almost all human touch between men and women goes “too far,” they miss opportunities for cultivating virtuous relationships with the opposite sex. They’re right that chastity isn’t easy—it’s no wonder the Dominican apostolate dedicated to its cause is the rather militant-sounding Angelic Warfare Confraternity—but it’s worth fighting for the freedom to love that comes with governing the passions.