GREENCASTLE, Pa.—Mitt Romney is known as a man with millions of dollars, but he wants Americans to know that his grandfather wasn't one.

"My dad's dad went broke more than once," the former Massachusetts governor told 200 or so people who had gathered Sunday for the local Lincoln Day dinner in this southern Pennsylvania community. "And my dad learned lessons about the importance of family and of faith and had a great and abiding affection for this country -- lessons he taught me."

The brief biography was a not-so-subtle rejoinder to a claim Democratic President Barack Obama made last week. "I wasn't born with a silver spoon in my mouth," Obama said -- a line his spokesman later said wasn't aimed at his presumptive Republican opponent.

In his Sunday speech, Romney first mentioned his wife, Ann Romney, as "the source of my experience and qualification." Then he added: "And also my parents. I had the extraordinary privilege of being born in America . and I had a mom and a dad and they taught me lasting values."

Romney's father was George Romney, the former Michigan governor and president of American Motors. As his youngest son, Mitt Romney grew up in the tony Detroit suburb of Bloomfield Hills, attended prep school and eventually graduated from the business and law schools at Harvard.

Romney has often brought up his father's upbringing on the campaign trail, discussing his birth in Mexico and explaining that he sold paint out of his car to pay for a honeymoon.

But on Sunday, just a few days after Obama's "silver spoon" comment, the Republican candidate cast those difficulties as life lessons that shaped the son as well as the father.

"My dad, as you might know, was born in Mexico and, ah, of American parents who'd been living there for some time," Romney explained. "There was revolution in Mexico, around the early part of the 20th century, 1910 or so, and so his family came back to the United States and his dad went from place to place. His dad was a contractor, and as you may know contractors have financial difficulty from time to time."

Romney often says his dad grew up poor. But while George Romney's parents did face a series of difficult circumstances, they knew wealth as well as financial hardship.

Romney's grandfather, Gaskell Romney, was a carpenter who led a prosperous life in a Mormon colony in Mexico, according to the book "The Real Romney," written by two Boston Globe reporters. When Romney's father was born there, the family was wealthy -- until the Mexican revolution forced many of the colony's residents to flee back to the United States, when George was about five years old.

The Romney family lost most of its money and moved from state to state. Gaskell eventually found success again as a builder in Utah, but lost much of his money again in the Great Depression.

"He never took out bankruptcy, which he could have done several times," George Romney wrote of his father, according to "The Real Romney."

© Copyright 2012 Associated Press. All rights reserved. This material may not be published, broadcast, rewritten, or redistributed.