Some early employees of Alphabet's smart-home company Nest, including co-founder Tony Fadell, are frustrated by how the company's history has played out now that it has been rolled back into Google.

Google bought Nest for $3.2 billion in early 2014, less than two years before it blew up its corporate structure to form the holding company Alphabet. Under Alphabet, Nest became an independent company and was heralded as the model business in the "Other Bets" category, which also includes Alphabet's venture capital arms, its smart city project Sidewalk and other experimental businesses.

But looking back, these early former employees say that the split ended up being a setback for both companies.

"From the outside it looked like Nest was the perfect poster child for Alphabet but, at the same time, separating it was undoing the thing that was most essential for both companies — figuring out how to make them work together," former Nest CEO Tony Fadell tells CNBC via email.

Fadell stepped down as CEO of Nest in mid-2016 after a turbulent year that included product issues and missed revenue targets. Some employees had complaints about his management style as well, as highlighted in an exposé by The Information.

Google reportedly tried to sell the company in 2016, but ended up keeping it.

This week, the company said that bringing Nest back into the fold would help it "supercharge" the company's mission. However, that phrasing felt ironic to another former Nest employee, who requested anonymity because he feared repercussions from his former employer.

This employee worked at Nest since 2011, when it was still independent, and says that when Google CEO Larry Page came to Nest employees before the acquisition, he pledged that Google would accelerate Nest's mission.

"But that didn't last two years," this person says.

Once Nest was turned into a separate company within the Alphabet family, there were no incentives for Google and Nest employees to work together. Nest and Google even worked simultaneously on extremely similar products, including a protocol for smart-home products to talk to each other.

"I think it would have helped had the M&A team had put something in place in the early days to structure and incentivize Google and us in a way that would have led to more cooperation," a third early former employee says.

The real pain point came when Amazon released its smart speaker, Echo, in 2014. As it grew into a hit, Google subsequently increased its own focus on the home. But with Nest as a separate company, employees there were largely kept in the dark about Google's efforts, and vice versa.

"The rise of digital assistants in smart speakers and many other household objects has upended the idea of maintaining a separate company focused on the home," Forrester analyst Frank Gillett tells CNBC, adding that Google now needs to make up for the time it lost by quickly roll out better integration between Nest hardware products and Google artificial intelligence.

To Fadell, that should have already happened.

"Nest was acquired by Google for a specific reason," Fadell says. "I pitched it as 'We are the senses and you are the brain.' For the sake of Nest customers and talent, I hope they follow through on their commitment they made to us 4 years ago instead of trying to sell it off like they did just 2 years ago."

Nest did not immediately respond to a request for comment.

On earnings calls, Google has highlighted Nest as one of the few "Other Bets" that generates considerable revenue, alongside healthcare company Verily and internet service Fiber. Now that Nest is no longer an "Other Bet," revenue in that category could drop in Alphabet's next earnings report.

The re-merger could also result in employee attrition, as Nest workers once again find themselves part of a much larger company.

Nest's other co-founder, Matt Rogers, just announced that he plans to depart the company in the next few months. One of the early employees CNBC spoke to says that they have already received several anxiety-ridden phone calls from people who are still at Nest who are worried about how their roles will change post-merger. Titles and roles are still being ironed out, this person says, although Google has confirmed that current Nest CEO, Marwan Fawaz, will now be reporting to Google hardware boss Rick Osterloh.

Reflecting on his more than five years at Nest, that former employees laughs ruefully: "It was like a soap opera."