Atlanta (CNN) Jimmy Carter has seen a lot in his 90 years.

As a son of the South, a governor and president, and as a man who spent decades tackling problems head-on, he knows the world is not perfect. It has too many people without a voice and without hope, too many examples of suffering and injustice.

There are so many wrongs to right. Yet among them, Carter is putting one ahead of all others: Violence and injustice against girls and women.

"This is going to be the highest priority for the rest of my life," he said this week.

He's doing it at the urging of wife, Rosalynn, and for his daughter, three granddaughters and five great-granddaughters. He says he wants them to have the same opportunities and security that men do.

He's doing it for the more than 200 schoolgirls abducted by the Islamist extremist group Boko Haram in Nigeria, for those who have become ISIS militants' sex slaves and for girls everywhere who can't go to school. He's doing it for the estimated 160 million babies aborted or killed at birth in Asia in recent decades because they were not boys. He's doing it for American college co-eds and women in the military who suffer rape and see the men responsible walk free.

Photos: Bring Back Our Girls Photos: Bring Back Our Girls 'Bring Back Our Girls!' – Weeks after the April 14 kidnapping of more than 200 Nigerian girls, worried families and supporters blamed the government for not doing enough to find them. Their cries spread worldwide on social media under the hashtag #BringBackOurGirls. From regular people to celebrities, here are some of the people participating in the movement. Hide Caption 1 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – First Lady Michelle Obama tweeted this picture of herself holding a #BringBackOurGirls sign in support of the schoolgirls abducted by Boko Haram in Nigeria. Hide Caption 2 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – "We hear it like it's not happening in our region so it doesn't concern us but that's not right," says Milliscent Maduagwu from Port Harcourt, Nigeria. "This fight on terrorism is not just the Commander in Chief's and the army, but ours as well!" Hide Caption 3 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – "These girls could have been my sisters or worst still my daughter," says Emmanuel Oleabhiele from Doha, Qatar. "My daughter is 6 months old and I fear for her future as a Nigerian." Hide Caption 4 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – Malala Yousafzai, the world's most famous advocate for girls' right to education, says that "girls in Nigeria are my sisters." This photo was posted to the @MalalaFund Twitter account on May 6. Hide Caption 5 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – Ify Elueze of Bonn, Germany, asks, "How can the world sit and watch?! It is no longer just the responsibility of the Nigerian government, now it is your responsibility and mine!" Hide Caption 6 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – British supermodel Cara Delevingne posted this photo on her Instagram account saying, "Everyone help and raise awareness #regram #repost or make your own!" Hide Caption 7 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – "Government should stop playing politics with our sister[s]," says Nigerian Dauda Kaks Hide Caption 8 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – American singer-songwriter Alicia Keys posted this photo on her Instagram account with this message: "I'm so saddened and enraged that these girls are not back where they belong! Safe at home and at school! Safe with their families! Safe to become the incredible leaders and powerful voices they are yet to be." Hide Caption 9 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – "You are most likely thinking to yourself, 'What's the point in posting another picture or status to speak about this issue?,"' says Uchenna Mildred Udeh from St. John, New Brunswick. "I can tell you this: it will. Do not underestimate the power of your voice. We have to make this personal." Hide Caption 10 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – British singer-songwriter Leona Lewis took a stand to #BringBackOurGirls on her Twitter account on May 7. Hide Caption 11 of 13 Photos: Bring Back Our Girls 'Bring Back Our Girls!' – "I think it's high time we all start praying for Nigeria and stop complaining," says Lotanna Ugwu from Abuja, Nigeria. "It's only God that can touch the hearts of those who kidnapped the young girls." Hide Caption 12 of 13 Photos: Bring Back Our Girls Bring Back Our Girls – Singer Steven Tyler posted this image on both his Twitter and Instagram accounts alonth with the hashtag #RealMenDontBuyGirls. Hide Caption 13 of 13

Carter also is doing it because of women he has met from places like Iraq, Syria and Afghanistan. Some can't go home due to the taboos and dangers facing them in male-dominated societies. Others go anyway, risking their lives.

"They're the heroes," the former President says. "And they inspire me."

Scores of these "heroes" shared stories and brainstormed solutions this week at the Carter Center, the President's namesake humanitarian and advocacy organization. The issues ranged widely, from income inequality to so-called "honor killings." Yet the attendees were united in a belief that women should be on par with men, echoing Carter in his latest book titled, " A Call to Action: Women, Religion, Violence and Power ."

He is making this the fight of his life, urging the conference attendees to be unrelenting.

"Let's not abandon this," he said.

'Death is mercy, compared to the alternatives'

Roughly half the world's population is male, the other half female.

In many ways, that's where the equality stops.

JUST WATCHED Why didn't our girls come home? Replay More Videos ... MUST WATCH Why didn't our girls come home? 02:08

Within families worldwide, women take on leadership roles all the time. But it's a different story if you break down the roster of U.S. or world political leaders, or if you look at the business landscape -- only about 5% of Fortune 500 CEOs are women. And this is not to mention the higher likelihood of females to be sexually abused, turned into sex slaves, subjugated or otherwise mistreated.

"The more (the Carter Center was) involved in 80 different countries around the world, the more we saw clearly the most serious and unaddressed human rights abuse on Earth is among women and girls," Carter told CNN this week. "It's much worse than I ever dreamed it was."

Some women endure physical abuse and personal humiliation because they don't see a way out.

Yet some resist.

JUST WATCHED The girl whose rape changed a country Replay More Videos ... MUST WATCH The girl whose rape changed a country 06:22

Dalia Abd El-Hameed, from the Egyptian Initiative for Personal Rights, said she was inspired to take action during the Arab Spring. She saw Egypt's revolution as "a window to do something different," namely to create a better and more just society.

She learned how activists' dreams don't always become realities, though. Sexual violence against women actually worsened after the fall of longtime President Hosni Mubarak and is now "reaching unprecedented levels," according to El-Hameed.

Yet she's still fighting.

"People are not being silenced," El-Hameed said. "It's a time of social uprising in the whole world. We are not alone in this."

Dalia Abd El-Hameed heads the gender rights program at the Egyptian Initiative for Personal Rights.

Manal Omar, from the United States Institute of Peace, knows about the global push for change from her work in Iraq, Libya, Afghanistan and around the Middle East. Yes, women know they put their health and safety on the line by speaking out, but they do it anyway.

"Death is not the worst thing that can happen," Omar said, comparing it to sexual assault, physical abuse, psychological degradation and the loss of face and hope. "Death is mercy, compared to the alternatives."

Rationalizing through religion

It's one thing to challenge a boyfriend, husband or some other man, but to challenge God?

That's one justification some men cite for their higher standing over women, basing their views on their interpretation of holy books. Groups like ISIS and Boko Haram have used this rationale to horrific ends, claiming women are inferior and effectively property under their strict interpretation of Sharia law.

Often, those who have defied this thinking have paid for it. The most well-known example may be Malala Yousafzai , now living in Britain after being shot by the Taliban for touting girls' education rights from her home in Pakistan's Swat Valley. More than 200 Nigerian girls were kidnapped by Boko Haram because they dared to go to school. And ISIS went so far as to print a pamphlet trying to justify its enslaving and having sex with "unbelieving" women and girls

As senior Carter Center adviser Karin Ryan says, "the extremists are very loud." That doesn't make them right, with more and more people making it their mission to prove such treatment is antithetical to Islam.

JUST WATCHED ISIS justifies abuse of women Replay More Videos ... MUST WATCH ISIS justifies abuse of women 06:07

In Ghana, banners with pictures of Carter and the chief national imam -- who attended the Carter Center conference in Atlanta -- adorn cars, doors and more to get this message across, with the imam committing one Friday sermon each year to promoting women's rights. In Libya, the Noor Campaign cited the Quran in radio ads, billboards and face-to-face conversations about women's security. There is also a push to give women more opportunity to learn about their religion, rather than limiting such study to men.

"We need to ... have knowledge," said Penda Mbow, a professor in Dakar, Senegal. "I have had death threats, but that has never stopped me."

Efforts by Muslims to point out the Quran and the Prophet Mohammad's stance on gender equality are one of "the brightest developments" in recent years, Carter said.

JUST WATCHED Muslim women in the world today Replay More Videos ... MUST WATCH Muslim women in the world today 03:13

He's seen less positive movement among some Christians who cite the Bible when reasoning women are "not as worthy in God's eyes," he said.

Such views can trickle down, from rationalizing violence against women to men dominating church scholarship and leadership. Only a small fraction of leading evangelical theologians and pastors are women, says Mimi Haddad of Christians for Biblical Equality, a nonprofit group that bases its arguments for equal treatment of men and women on the Bible.

And women can't become Roman Catholic priests, though Carter said Pope Francis wrote him that "his opinion was that, in the future years, women's roles within the church would be greatly improved."

"But still, there is a very great reluctance among some Protestants, like Southern Baptists, among others, (to) treat women equally in the eyes of God," said Carter, a longtime Sunday school teacher at a Baptist church in Plains, Georgia. "And that sends a signal that it is OK to discriminate against girls and women."

Carter: U.S. hasn't 'done nearly enough'

What's needed on all fronts, conference attendees said, is leadership. And officials in the United States are not doing a good job providing it, according to the former President.

"The United States is the leading country on Earth," Carter said. "And we haven't done nearly enough."

JUST WATCHED Will sex contracts prevent college rape? Replay More Videos ... MUST WATCH Will sex contracts prevent college rape? 01:14

The issues go beyond money, though.

Some 150 years since the end of the Civil War, an estimated 60,000 people are effectively enslaved in the United States, according to the Global Slavery Index -- with Carter noting they are disproportionately female. There's also domestic violence, recently highlighted in the saga over the NFL's Ray Rice. And, of course, there are alarming statistics about sexual violence -- including a Department of Justice-funded study that found one in five women are sexually assaulted in college, many of them by serial rapists.

JUST WATCHED Rape survivor: Senator betrayed us Replay More Videos ... MUST WATCH Rape survivor: Senator betrayed us 06:04

"Patience is overrated," said Ruth Messinger, president of the American Jewish World Service humanitarian agency and one those pushing U.S. policymakers to act. "We need to all think big."

For all the talk of equal rights, Carter said some men "are very reluctant to give up their power and ... privileged positions in society."

"They say that it's the right thing to do," he said of gender equality. "But when it comes down to (doing things like) voting in the Congress or taking action in their place of employment, they say, 'Well, let's cheat women a bit longer. We can get away with it.'"

The power dynamic

That's because the quest for power -- whether building or holding it -- and quest for equality don't always jive. Men long have been in control in most societies, and many aren't eager to give that up, whether to women or other men.

In part, it's physical: Some men assault women and hold them down because they can. If they swapped body sizes, who's to think that would still hold true?

Plus, men typically lead armed conflicts as national leaders, generals and soldiers. Especially in Africa and the Middle East, there is no shortage of war nowadays.

Alaa Murabit, 24, is a doctor and women's rights activist in Libya.

According to Alaa Murabit, a 24-year-old pediatrician and founder of The Voice of Libyan Women: "Women's rights end when the war begins."

Murabit grew up in Saskatoon, Canada, in a religious family of five brothers and five sisters, each of whom was treated equally in their parent's eyes, she said. She graduated high school at age 15 and moved to the Libyan city of Zawia for medical school when Moammar Gadhafi was still in power, a time when she said "rights as a whole were not respected for anyone."

An armed uprising ended Gadhafi's reign, and since then, Murabit said, "the treatment (of women) has gotten significantly worse" amid the "power vacuum being filled by unsavory groups."

And yet Murabit, an Ashoka Fellow and member of the U.N. Women Civil Society Advisory Group , is still pushing to be heard. A proud "interrupter," she frequently traveled 25 miles to Tripoli to get in front of power brokers, most of whom were men.

"They don't have a lot of women (at) the table," Murabit said. "They only meet with the men with the guns."

JUST WATCHED Brazil President on women in power Replay More Videos ... MUST WATCH Brazil President on women in power 02:09

Still, no one denies men can be part of the solution.

Timothy Njoya is a former Presbyterian minister and head of the Men for the Equality of Men and Women. His mother was the first in her Kenyan village to be educated and not to be circumcised; Njoya's own four daughters, each of whom has a Ph.D., followed their grandmother's lead.

Among other things, his organization has boys do chores like cooking, cleaning and farming. The idea is to make them respect the hard, important work traditionally done by women, and respect women more in the process.

"It's not punitive, but transformative," Njoya told CNN. "I call it human rights masculinity."

For 'your children and your children's children'

While some go through life without really knowing someone of a different skin color, religion or ethnicity, nearly all of them will know someone of the opposite gender. That's why the issue of women's rights can't be avoided. And for many men and women, it's personal.

JUST WATCHED Emma Watson on gender equality Replay More Videos ... MUST WATCH Emma Watson on gender equality 02:01

"Anybody that has a daughter or granddaughter knows how precious that person can be," Carter said. "... And anybody who is interested in the future of (their) country or city that if they deprive half of their own citizens of an equal right to an education or an honest job, then their whole community is going to suffer."

There are signs of progress, like the prominent global role played by German Chancellor Angela Merkel, Christine Lagarde leadership of the IMF and Yousafzai becoming the youngest Nobel Peace Prize winner. But for every two steps forward, it seems, there's at least one step back, as evidenced by recent atrocities against women by Islamic extremists.

"Women cannot wait for change," said Bolanle Makanju, the founder of a faith-based organization in Nigeria. "We've waited for centuries and centuries. We've got to push for it."

It's not just women. Carter is proud to be part of the fight. And, for all his nine decades on Earth, he seems to have the energy, capacity and motivation to do what he can to make a difference.

"That's your children and your children's children," he said, arguing that everyone should be similarly compelled. "And action needs to be taken."