BEREA, Ohio -- Johnny Manziel called his weekend of partying during the Cleveland Browns' bye "a lapse of judgment," but coach Mike Pettine said "repercussions would be harsh" if the quarterback has another off-field transgression.

"[It's] hard to put things in a vacuum," Pettine said Wednesday. "I don't want to sit here and say, 'Look at the degree of discipline.' It's hard to say we have a zero-tolerance policy. If something were to occur, I imagine the repercussions would be harsh."

Pettine also conceded that the Browns might not have been aware of how "deep-rooted" Manziel's off-field issues were when they drafted him in the first round in 2014.

"You see the reputation, what was out there," he said. "I don't think we anticipated that his problems, his issues, how deep-rooted they were, the extent of it."

Manziel stressed he's proud of the person he is today compared to a year ago.

"I've by far been way improved from the way I was last year," Manziel said as he prepared to return to the starting lineup for Sunday's game against the San Francisco 49ers. "Now am I perfect by any means? No. But there has been improvement all the way around -- from the way I've conducted myself in the building and for the majority of the time the way I've conducted myself off the field.

"Now is there obviously times where I need to be better and like I said act the way I want to each and every day, every second of every day? Sure. But from where I was last year to the way I am today, I'm still extremely proud of where I am."

Manziel spent 10 weeks in a treatment facility after his rookie season. He started three games this season and was named the starter for the rest of the year prior to the bye week Nov. 22. But a video of him partying in Austin, Texas, surfaced the Monday after the bye. Manziel compounded the problem by lying to the coaching staff about when the video was filmed. He then was demoted from starter to third team, but the team's quarterback situation changed when Josh McCown broke his collarbone and Austin Davis lost to the Cincinnati Bengals 37-3.

"We have to be able to put a team out there and play," Pettine said. "We know that there are some off-the-field issues there with Johnny. It's been very public and debated. Those are things that we're well aware of.

"We're going to spend some time, whether it's now doing what you can and after the season addressing those."

Manziel took six questions about what happened off the field before saying he didn't want to answer more. He did circle back briefly when the notion was brought up that he had lied to the coaches.

"I don't know how certain things got out or what certain people said, but I know what I told Coach Pettine and Flip [offensive coordinator John DeFilippo] and [quarterbacks coach] Kevin [O'Connell] when I was in that room," Manziel said. "That's going to stay family business and I'm not going to speak on it any further."

Manziel let out a long "umm" when asked whether he agreed with the decision Pettine made to demote him.

"In Coach Pettine's mind he thought that that was necessary to get a harsh point across to me," Manziel said. "Did I like it? No. Did I have to sit there with a bad taste in my mouth and accept the consequences for my actions? Sure."

Manziel said he is "not giving up on myself being here" in Cleveland.

"I believe I still have people in this organization that want me to be here," he said. "And I'm still here for a reason."

He added he has not "really done anything" in the NFL, and to see what they have the Browns need to see him on the field. Pettine said Manziel has improved as a quarterback from his rookie season, when in some ways he was at an "elementary level." But the coach emphasized that Manziel has to earn the respect of teammates and the trust of coaches and that there is a bigger responsibility for a quarterback.

"Over the last couple weeks I was forced to learn a hard lesson and I had to learn it the hard way," Manziel said.

What did he learn?

"Obviously there's a way to conduct yourself that 31 others guys in the league do each and every week," he said. "You have to follow that example and realize that this is a very prestigious situation I'm in being a starting quarterback in the NFL. So I have to take pride in that and act accordingly."