The New York Yankee pitcher Jim Bouton, posing in uniform in this undated photo. Photograph by Bettmann / Getty

On a sunny day in mid-June, the New York Yankees celebrated Old-Timers’ Day for the seventieth time. A gaggle of ex-pinstripers, some famous, others less so, returned to the house that Ruth built, that Steinbrenner rebuilt, and that Jeter moved across the street. More than a hundred miles away, in western Massachusetts, on the property of his large, lushly landscaped home, the former Yankee pitcher Jim Bouton threw a bucket of balls against a cinderblock wall that he built in his back yard, aiming each pitch at his hand-painted strike zone and hitting it more often than not. Fastballs and knucklers, mostly.

Bouton, at seventy-seven, is still as lean, gregarious, and witty as he was in his playing days, though he says that the lingering effects of a stroke, suffered four years ago, cause him to get jammed on words from time to time. He had no motor deficits or physical impairment—the damage was limited to his language center. “I don’t have the quickness I used to have,” he explained. “I’ll have a line in my head, but then it’s gone.”

Bouton (Bulldog) was just twenty-three when the Yankees called him up from Double A, in 1962. The following year, Bulldog won twenty-one games, and helped the Yankees win the American League pennant. The next season, he led the team with eighteen wins, and picked up two more in the World Series. But in 1965 he lost his fastball and won only four games. He tried resurrecting his knuckleball, a pitch he’d thrown as a kid in New Jersey, but never matched the success of those early years.

Bouton became famous anyway: in 1970, he published a diary of his 1969 season with the expansion Seattle Pilots, sprinkling it with stories of his career with the Yankees. The book, “Ball Four,” sold millions of copies, thanks in part to the myth-busting stories it told about big-league ballplayers, many of whom came across as overgrown man-children. Here was Mickey Mantle drinking and carousing, and players climbing up to the roof of the Shoreham Hotel, in Washington, D.C., peeping into windows to watch women undress. After it was published, Bouton believes he was no longer welcome at Yankee Stadium, though a team spokesperson, when asked for comment, said that Bouton was never banned. (Mantle, for his part, left Bouton a phone message in 1994, saying that he held no grudge against him and that, contrary to popular belief, he had nothing to do with Bouton’s exclusion from Yankee Old-Timers’ Days. Bouton still has the message on tape.)

Bouton settled into a leather chair in his sun-drenched living room, joined by his wife, Paula Kurman, a retired psychologist and lecturer. Kurman explained that she and Jim don’t usually watch Old-Timers’ Day, or any Yankee games, for that matter. But Bouton’s gaze was fixed on the screen, as the Yankee announcers John Sterling and Michael Kay manned makeshift podiums, set up behind home plate, and began introducing the players. “Where are all the people?” Bouton said, pointing at rows of empty seats. The Stadium did look much less populated than it has on Old-Timers’ Days gone by. Told that he should have been there, Bouton shook his head. He’s not one of the greats, he said. He doesn’t think he’ll be honored at the Stadium again until he dies—unless “something exciting” happens.

He has been back there a couple of times. In 1997, his daughter, Laurie, died in a car accident. The following year, his son, Michael, published an open letter to the Yankees in the Times, asking the Yankee brass to let bygones be bygones. A few weeks later, Bouton was at the Stadium for the first time in thirty years, wearing a uniform with his old number, fifty-six. When he was introduced, the crowd let out a cheer usually reserved for the all-time greats. The standing ovation so weakened Bouton’s knees that he was terrified of stumbling on his way out of the dugout. He still has that uniform tucked away for safekeeping, and a photo of himself standing on the mound, tipping his hat to the crowd.

He lit up when talking about that day. Mel Stottlemyre, an ex-teammate who was the Yankee pitching coach at the time, was on hand for the ceremony. While changing in the locker room, Stottlemyre suggested that Bouton imitate his former self by having his hat fly off during his pitching motion. “Mel said, ‘That’s your trademark,’” Bouton recalled. “You have to have your hat fly off. So just set it on there, and when you throw it will come off. The crowd is expecting that. I said, ‘Mel, that’s a great piece of wisdom. That’s exactly what I’m gonna do.’ So I ran out there, and first pitch, boom, my hat comes off and the place goes nuts.”

The Yankees brought Bouton back the following year, in 1999, and, according to a team spokesperson, Bouton was invited again, more than once, in the early two-thousands. But it’s been a while. Still, Bouton has no regrets about “Ball Four.” “If I hadn’t kept those notes, and put them in the book,” he said, “I would have a big hole in my memory.” On the TV, Sterling introduced Al Downing, and Bouton applauded. He clapped again for another former teammate, Ralph Terry, and then he told a story. On the bus ride from the midtown hotel to Yankee Stadium for Old-Timers’ Day in 1998, Terry sat next to Bouton, and, as if to tell him there were no hard feelings about “Ball Four,” simply said, “It’s the game, baby, the game. This is baseball.” Bouton smiled. “That was a nice thing for Ralph to say.”

The ceremony started up again, and the scoreboard flashed the names of those in the Yankee family who died in the past year. When Bouton spotted Yogi Berra’s name, he practically flew out of his chair. “Here is Old-Timers’ Day,” he said, “and Yogi has just died. Why in God’s name don’t they have Phil Linz come to Old-Timers’ Day, with the whole thing about the harmonica business? You know, this was a big story, it was on the front page of every newspaper in the country.” Bouton launched into the legendary incident, laughing his way through it. It was August, 1964. The team had just lost its fourth game in a row, and Linz was sitting on the team bus, playing “Mary Had a Little Lamb” on the harmonica. Berra, then managing the team, thought Linz was being disrespectful, and told him to knock it off. Linz couldn’t hear Berra, and asked Mantle what he had said. Mantle’s now famous response: “He said to play it louder.” A shouting match ensued.

“This was a terrible incident that turned out to be a wonderful thing, because we ended up going on a hot streak and went on to the World Series,” Bouton said. “And the Hohner Harmonica Company ended up making millions of dollars. They had a six-foot-long harmonica with Phil at one end and Yogi at the other end, and it became the harmonica thing! Come on, why isn’t Phil Linz there? That’s part of the lore!” He shook his head. “They don’t know what they have.”