ALLEN PARK, Mich. -- It's the thing on the field no player wants to see, yet those yellow flags have been there every game for the Detroit Lions. And throughout the course of this season, penalty flags have been there in bunches.

Penalties have nullified Lions touchdowns and first downs on offense. They've extended drives that have led to opponent points on defense. And overall, they've been one of the primary reasons the Lions are 1-3 a quarter of the way through the season.

The Lions are the third-most penalized team in the NFL this season, racking up 39 accepted penalties of 42 actual flags thrown. They have surrendered the second-most penalty yards in the league with 370. Only the Jaguars, with 392, have been worse.

"That's one of those things that we have to just keep pounding on," Lions coach Jim Caldwell said. "If we have a guy that does it too much, and it's the same character all the time, we've got to take measures to get that corrected.

"Those things that we're involved with, that it's our problem in terms of coaching staff, we've got to get those corrected. We try to make certain that we hold everybody accountable for every single phase of the ball game."

The Lions are averaging over 90 penalty yards per game, a point of frustration for coach Jim Caldwell. Daniel Bartel/Icon Sportswire

The Lions have committed 10 more penalties than their opponents this season and it has definitely affected games. In all, 21 Detroit players have committed at least one penalty through the first four weeks of the season and the Lions have also been charged with two team penalties.

Tight end Eric Ebron has committed the most penalties of any Lions player, with four, followed by cornerback Nevin Lawson and offensive linemen Laken Tomlinson and Taylor Decker with three each.

For perspective, Detroit has accumulated more penalty yards this year (370) than rushing yards (369).

No team is going to be perfect with penalties. Mistakes happen. The game moves fast. But that they happen at such a high frequency level is a concern for Detroit. So why do they keep showing up?

"I think penalties are a symptom of not knowing exactly what to do at certain times and losing your concentration," said guard Larry Warford, who has one penalty this season. "Even I did it, you know. I had a false start and that's because I was thinking about, oh, tags this way if this happens, then this happens. I tricked myself into false starting, you know.

"It's just getting down pat all your assignments to the point where you can just play free and not overthink things. It's, like I said, I feel it's a symptom of not being 100 percent on what you're supposed to do."

That's what confounds the Lions the most. In practice, things go well. Warford said Caldwell tracks every penalty the Lions commit in practices during the week -- and that there haven't been too many when they are on the field in Allen Park.

Caldwell also looks at the penalties to see if they are due to a "technical issue" or if it is a player being too aggressive on a play. Since Caldwell took over in 2014, Detroit has committed 261 accepted penalties -- which is in the bottom-third of the league.

So how can the Lions fix it? They've been in double-digits in penalties twice in four games. The solution will come during practice.

Warford said the way the Lions practice -- using false cadences for the defense and going on the second or third hut for offense -- is designed to help them during games. They also practice at a high speed -- something seen often during training camp -- to get used to the no-huddle look Detroit uses on Sundays.

"We're going to go on false cadences for slowing defensive linemen down so they're not jumping the snap, we're going to do it in a game. And if you're not on the details in practice and you're not practicing it, you get to the game and defenses are jumping the count, you can't go on two or three, then you're screwed," Warford said. "You're just going to have a whole game of defenses jumping the count on you and there's no real way to tempo them down. So that's why you have to do that for the offense.

"On the other side, the defense, they should be reading the ball. They did a good job. Did they have any procedure penalties? They did a great job of taking what we did at practice and applying it to the game. As an offense, especially myself included, we have to live up to what we're doing in practice, too, and not put that stress on our defense also."

Once the Lions do that, one of their biggest issues in the 2016 season will be mostly solved.