The last straw, for both of us, came in February of 2015. Wisconsin Governor Scott Walker had called me into his office ahead of my reconfirmation vote as secretary of the Department of Financial Institutions. He told me he was going to propose merging four of Wisconsin’s state agencies, including mine, into two. This was part of a long-term plan to “shrink government” in Wisconsin, Walker said, and he assured me that the plan would be “well received.”

I knew exactly what he meant. For weeks, there had been talk about Walker throwing his hat into the ring for the Republican presidential primary, and everyone in his Cabinet figured it was just a matter of time before he announced his candidacy. Being able to say that he had “shrunk government” in Wisconsin, he probably thought, would play well with the primary voters in Iowa and especially with big-money donors like Sheldon Adelson and the Koch brothers. He asked me to say that I fully supported the restructuring.

I quickly responded that I did not support the concept. I argued that a merger would hurt our ability to regulate Wisconsin’s financial institutions and would not even save any money, since the same number of employees would still be in place. There would, moreover, be significant up-front costs required to carry out the consolidation. I said I would not be a cheerleader for such a plan. And then I told him to feel free to replace me if my lack of support was a problem. A few days later, he did just that. Right after I was reconfirmed unanimously by the state senate, I was gone. Soon after that, Republican state senators decided that Walker’s consolidation plan was not viable, and persuaded the governor to drop the idea.

Read: How Scott Walker’s hubris killed his campaign.

I joined Walker’s administration at the very beginning, in 2011, and at first I enjoyed my job and respected the governor. I thought he was more of a technocrat than a partisan, driven to improve governance of the state he loved. We’d have regular one-on-one meetings where he showed genuine interest in how we were helping financial institutions pull out of the Great Recession. During my first 90 days in office, I had to close three failed banks, and Walker appreciated the hard work that went into such challenging situations. He held Cabinet meetings every week or two, and encouraged open debate and discussion on the affairs of the state.

Even early on, however, I noticed that not everything was as it should be. At more than one Cabinet meeting, the secretary of the Department of Administration, Mike Huebsch, told us never to send him or the governor any electronic documents of consequence, and to avoid the use of our state-issued cellphones. “If you send me an important report electronically, I won’t open it,” I remember him saying, “and if you call me on your state phone, I won’t answer it.” If we had any important documents, they were to be “walked over” and hand-delivered to the governor’s office. As a result, open-record requests by the media or political opponents would be almost futile. This lack of transparency would be a hallmark of the Walker administration.

Walker’s move to limit the collective-bargaining rights of Wisconsin public employees led to the infamous recall election of 2012 and massive protests at the capitol. Marches were held around the clock, and the capitol rotunda was filled with protesters carrying signs, some pretty clever. My favorite featured a dorky picture of Walker with big, bold letters declaring Dred Scott! During the protests, Walker usually entered and exited the capitol building via a secret underground tunnel.

Later, when Walker titled his 2013 book Unintimidated and bragged about how he “stood up to union thugs and protesters,” I thought back to how he had regularly ducked in and out of the capitol via a tunnel, always escorted by a heavy security detail. Unintimidated is not exactly the word that comes to mind, but Republicans in Washington accepted Walker’s self-image and talked him up as a strong presidential contender.

Read: Even Scott Walker says he’s “at risk” in Wisconsin.

And the more they talked, the worse life in Walker’s Cabinet became. The technocrat I had respected vanished, replaced by a partisan who thought the White House was in reach.

After he won his recall election, Walker rarely attended Cabinet meetings anymore, and radically reduced the number of one-on-ones with Cabinet secretaries. He took more far-right positions, probably because he thought they would play well with the Republican base. Funding for public education and our University of Wisconsin system was cut dramatically. Our infrastructure continued to deteriorate to the point that we ranked 49th in the nation in the quality of our roads and bridges.

In 2014, at the longest Cabinet meeting held during my four-plus years, our transportation secretary, Mark Gottlieb, shared recommendations from a bipartisan blue-ribbon commission that clearly showed the need for additional transportation funding, and also made clear that “kicking the can down the road” would only lead to dramatically higher costs for future administrations to deal with. Still, Walker refused to consider any increase to the state gas tax or to vehicle-registration fees. I believe he was afraid of what talk radio would say about that.

Throughout 2014, Walker was traveling the country, gearing up for a presidential run. He was a regular on Fox News and courting big-money donors. Our marching orders, meanwhile, were to play up each and every administration “success,” and to take care of special interests. Our agencies became politicized.

Wisconsin has some of the laxest laws in the country when it comes to small-dollar lending like payday and auto-title loans. Of the states that allow payday loans, we’re one of just eight with no interest-rate cap of any kind. The average interest rate charged on a payday loan here is 585 percent, and some lenders charge as much as 1,000 percent. But my department was not given room to regulate small-dollar lenders during the Walker administration, a fact I attribute to heavy political contributions to the governor and the Republican-controlled legislature by vested interests.

When a lobbyist for the payday-lending industry asked to meet with me to discuss yet another request for regulatory “relief,” I rebuffed him. Fifteen minutes later, I got a call from the governor’s office directing me to give full consideration to the lobbyist’s requests because he represented big supporters of the administration. I did, but it was another straw on the camel’s back. It seemed to me that, for Walker, political friends and donors came first.

Read: Scott Walker isn’t sorry.

At the Department of Financial Institutions, we were also chartered with the responsibility of promoting financial literacy in the state. As such, I worked with State Superintendent of Schools Tony Evers on a number of projects. Evers is now running for governor against Walker, and as I’ve said elsewhere, he has my full support.

Others who served in Walker’s Cabinet have also soured on their former boss. Ex–Transportation Secretary Gottlieb said in September that Walker “isn’t telling the truth” about Wisconsin’s roads, and blasted him for “taking a high-risk gamble” when it comes to the state’s infrastructure. In August, ex–Corrections Secretary Ed Wall published a book titled Unethical: Life in Scott Walker’s Cabinet and the Dirty Side of Politics, a clear jab at Walker’s book Unintimidated. Earlier this month, ex–Commerce Secretary Paul Jadin resigned from his $208,000-a-year job in economic development in order to publicly share criticism of Walker, declaring that Walker “routinely put his future ahead of the state.”

Walker wanted to do right by Wisconsin when he was first elected governor—or so it seemed to me. But the longer he was in office, and the more public attention came his way, the more he changed his focus from improving the lot of the people of his state to improving his standing with the Republican Party. In so doing, he made decisions that were bad for Wisconsin and—if the latest polls are right—ultimately bad for his political aspirations as well.