Story highlights "We will succeed as one city," Bill de Blasio says in his inaugural speech

Former President Bill Clinton administers the oath of office outside City Hall

De Blasio was ceremoniously sworn in shortly after midnight outside his Brooklyn home

He replaces billionaire Mayor Michael Bloomberg, who led New York for 12 years

Bill de Blasio, who campaigned on a progressive agenda that he said would narrow the widening gap between the rich and poor in the nation's largest city, was sworn in Wednesday as New York's 109th mayor.

On a chilly first day of 2014, former President Bill Clinton administered the oath of office at a ceremony outside City Hall using a Bible once owned by President Franklin Delano Roosevelt. Clinton's wife, Hillary, a former secretary of state and potential 2016 presidential candidate, also attended.

De Blasio took the oath with his wife, Chirlane McCray, and the couple's children, Chiara and Dante, at his side. His November electoral victory ended a long losing streak for Democrats that began when former Mayor David Dinkins, de Blasio's former boss, lost to Republican Rudy Giuliani in 1993.

In his inaugural speech, de Blasio touched on the key themes of a campaign that portrayed itself as the progressive alternative to the long reign of former Mayor Michael Bloomberg, whose policies de Blasio said favored the rich. He repeatedly used the Dickensian phrase "tale of two cities."

"When I said we would take dead aim at the tale of two cities, I meant it," he said Wednesday. "And we will do it. I will honor the faith and trust you have placed in me. And we will give life to the hope of so many in our city. We will succeed as one city. We know this won't be easy. It will require all that we can muster."

JUST WATCHED NYC welcomes its new mayor Replay More Videos ... MUST WATCH NYC welcomes its new mayor 02:31

On inaugural day, the new mayor outlined a sweeping progressive agenda for his administration, including higher taxes for those earning more than $500,000 a year to fund full day, universal pre-K and after-school programs for middle school students. He also wants to expand paid sick leave for employees of businesses that have 20 or more workers.

JUST WATCHED NYC mayor calls for progressive era Replay More Videos ... MUST WATCH NYC mayor calls for progressive era 01:28

"Big dreams are not a luxury reserved for a privileged few," he said, "but the animating force behind every community, in every borough."

JUST WATCHED Clinton swears in new NYC mayor Replay More Videos ... MUST WATCH Clinton swears in new NYC mayor 02:14

Before the oath, Clinton also talked about the growing wealth gap.

"This inequality problem bedevils the entire country," the former president said. "But it is not just a moral outrage, it is a horrible constraint on economic growth and on giving people the security they need to tackle problems like climate change."

In his speech, de Blasio took a moment to praise Bloomberg, who led New York for 12 years, shepherded the city through the recession, and is credited with making New York safer and more livable.

"To say the least, you led our city through some extremely difficult times," de Blasio said. "And for that, we are all grateful. Your passion on issues such as environmental protection and public health has built a noble legacy."

A virtual unknown nationally, despite 25 years in New York politics, de Blasio defied critics who questioned whether his experience as a city councilman from Brooklyn and, most recently, as a public advocate -- a sort of civic watchdog -- sufficiently prepared him to run the Big Apple. He also ran Hillary Clinton's first U.S. Senate campaign.

His biracial family's visibility in the campaign resonated with residents of a city coping with a 21% poverty rate and increasing racial divisiveness brought on by the controversial stop-and-frisk policy allowing police to search people in high-crime areas.

Stop-and-frisk -- in which police stop, question and frisk people they deem suspicious, even if they've committed no crime -- was one of the most controversial policing techniques in recent time. Law enforcement and other proponents said the practice helped reduce crime. To critics, it's racial profiling.

"We will reform a broken stop-and-frisk policy, both to protect the dignity and rights of young men of color, and to give our brave police officers the partnership they need to continue their success in driving down crime," de Blasio said Wednesday.

And de Blasio inherits a city with the highest level of homelessness since the Great Depression, according to advocates. More than 52,000 people, including 22,000 children, spend each night in city shelters.

"We are called to put an end to economic and social inequalities that threaten to unravel the city we love," the new mayor said during his inaugural. "And so today, we commit to a new progressive direction in New York. And that same progressive impulse has written our city's history. It's in our DNA."

De Blasio was ceremoniously sworn in shortly after midnight on New Year's Day outside his Brooklyn home, with his family next to him -- as they were during much of his campaign.

His son Dante, a sophomore at a public high school in Brooklyn, appeared prominently in campaign ads as de Blasio soared in the polls. Daughter Chiara, a college freshman, danced with the rest of the family at the West Indian Day, and recently spoke publicly about her struggle with depression and substance abuse. McCray, a political activist who once identified herself as lesbian before marrying de Blasio, was a top campaign strategist.

JUST WATCHED Mayor-elect's daughter reveals drug use Replay More Videos ... MUST WATCH Mayor-elect's daughter reveals drug use 02:51

JUST WATCHED Meet New York's new first family Replay More Videos ... MUST WATCH Meet New York's new first family 02:31

JUST WATCHED Dancing with the de Blasios Replay More Videos ... MUST WATCH Dancing with the de Blasios 01:35

"With all due respect to the television show," Clinton said Wednesday, "they're our real modern family."

De Blasio, who has a bachelor's degree from New York University and a master's in international and public affairs from Columbia University, met McCray while working in the administration of Dinkins, who was the city's first black mayor and attended Wednesday's ceremony.

On Wednesday morning, Bloomberg and the de Blasios greeted each other at the City Hall subway station as they arrived for the inauguration.

Also sworn in Wednesday was Letitia James, who as public advocate becomes the first black woman to hold citywide office in New York City.