Ever since landing my first job post college in 1999, I’ve been determined to build enough passive income in order to not have a job. A future that included getting into work by 5:30am and leaving after 7:30pm each day for decades seemed too brutal to endure.

In 2009, I decided that if I could earn about $80,000 in passive income, I would leave my job permanently and work on Financial Samurai while traveling instead. So I left work in 2012 at the age of 34. Once my wife decided to join me in 2015, also at the age of 34, I decided to shoot for $200,000 in passive income.

With $200,000 a year in passive income, I would have enough income to provide for a family of up to four in San Francisco given we bought a modest home in 2014. Now that we have a son, I’m happy to say that $200,000 indeed does seem like enough, especially if we can win the public school lottery to avoid paying $20,000 – $50,000 a year in private school tuition.

The Keys To Building Enough Passive Income

1) Save until it hurts each month. Most people think they are saving enough through their 401(k) or IRA, but they are not. Developing passive income requires an aggressive after-tax, after-401k/IRA savings amount each month because you can’t draw from your pre-tax retirement accounts before the age of 59.5 without a 10% penalty.

You must sacrifice the pleasures of today for the freedom you will earn tomorrow. In my 20s, I shared a studio with my best friend from high school and drove beater cars worth less than 10% of my annual gross income. I’d stay until after 7:30pm at work in order to eat the free cafeteria food. International vacations were replaced with staycations since work already sent me overseas 2-4X a year. Clothes were bought at thrift shops of course.

2) Focus on income producing assets. Internet growth stocks may be sexy, but they provide no income. To build a large enough passive income stream to survive, you must invest in dividend generating stocks, certificates of deposit, municipal bonds, government treasury bonds, corporate bonds, and real estate. You’re free to invest in non-income producing assets for capital appreciation too. You just want to earn reliable income when the day comes to leave your job.

My favorite type of semi-passive income was rental property because it was a tangible asset that provided reliable income. As I grew older, my interest in rental property waned because I no longer had the patience and time to deal with maintenance issues and tenants. Online real estate became more attractive, along with tax-free municipal bond income once rates started to rise.

3) Start as soon as possible. Building a livable passive income stream takes a tremendously long time largely due to declining interest rates since the late 1980s. Gone are the days of making a 5%+ return on a short-term CD or savings account. For a while, savings interest rates were pitifully low. Thankfully, after a series of Fed rate hikes since end of 2015, you can now earn a healthy online money market rate. Know that every $100 you save can generate at least $2.4 in passive income.

I knew I didn’t want to work 70 hours a week in finance forever. My body was breaking down and I was constantly stressed. As a result, I started saving every other paycheck and 100% of my bonus since my first year out of college in 1999. By the time 2012 rolled around, I was earning enough passive income (~$78,000) to negotiate a severance and be free.

4) Calculate how much passive income you need. It’s important to have a passive income goal, otherwise, it’s very easy to lose motivation. A good goal is to try and generate enough passive income to cover basic living expenses such as food, shelter, transportation, and clothing. If your annual expense number is $30,000, divide that figure by your expected rate of return to see how much capital you need to save.

Unfortunately, you’ve got to then multiply the capital amount by 1.25 – 1.5 to account for taxes. For example, $30,000 / 3% = $1,000,000 in capital needed to generate $30,000 gross. But since you must pay tax on the $30,000 income, you really need closer to $1,250,000 to generate $30,000 in after-tax income at a 3% rate of return.

With the birth of our son in 2017, we’re aiming to generate $250,000 a year in passive income by the time he starts kindergarten in 2022.

5) Make sure you are properly diversified. Capital preservation is underrated. We saw a lost decade for tech stocks between 2000 – 2010 after the first dotcom burst. It actually took 13 years for NASDAQ investors to get back to even. Investors in the Borsa Istanbul Turkey stock market index just gave up 10 years worth of gains after they saw a plunge in their currency, partially due to increased tariffs by the US and no-confidence in the government. Your passive income needs to be properly diversified in order to take the hits.

I currently have 10 main sources of passive income as you’ll see in the chart below, with bonds as my largest source at 30.4% of total. I’ve worked through the 2000 and 2008 meltdowns, and don’t plan to ever lose so much money again.

Passive Income Review

In 2017, I sold my San Francisco rental home which had been generating roughly $60,000 a year in cash flow after expenses, but before taxes. Selling the house brought my passive income down to roughly $150,000 a year, which was a significant 28% step backwards.

Within six months of selling, however, I had reinvested the proceeds from the home sale and brought total passive income for 2018 back up to an estimated $203,724. Without a clear plan for reinvesting the proceeds, I’m not sure I would have sold the house since I’m bullish on the SF housing market long term.

However, because I did have a plan and the challenges of raising a newborn and dealing with rowdy tenants left me feeling a bit stretched, I decided to simplify and sell.

Interest Income ($7,620/year, 3.7% of total)

I’ve got a $185,000 CD generating 3% interest coming due. Although the return is low, it’s guaranteed. The CD gave me the confidence to investment more aggressively in risk over the years. My online interest income has come down since I aggressively deployed some capital at the beginning of the year and again during the February market correction. You’ll see these figures in my quarterly investment income update.

Don’t underestimate the value of your cash and risk-free income, especially during times of uncertainty. The last thing you want to do is be a forced seller in a downturn because panic will be everywhere. Cash allows you to take advantage of corrections, pay for unexpected expenses, and worry less about your risk assets.

Related: How Much Savings Should I Have Accumulated By Age

Stocks & Bonds Income ($103,344/year, 50.7% of total)

In 2017, I ended up deploying roughly $611,000 into stocks and $604,327 into municipal bonds. The stock allocation should boost dividend income by ~$12,500 a year and the municipal bond portion should boost income by ~$18,000 a year after tax ($26,000 pre-tax). Therefore, total passive income gets a ~$38,500 lift, which recovers over half of my $60,000 loss from selling the house.

A good portion of my stock allocation is in growth stocks and structured notes that pay no dividends. The dividend income that comes from stocks is primarily from S&P 500 index ETFs. Although this is a passive income report, as I’m still relatively young, I’m more interested in building a large financial nut through principal appreciation rather than through dividend investing. As an entrepreneur, I can’t help but have a growth mindset.

With interest rates reaching two-year highs, I will be allocating more cash flow to short-term bonds and savings for the remainder of the year, thereby boosting passive income.

Related: The Proper Asset Allocation Of Stocks And Bonds By Age

Real Estate Income ($43,080/year, 21.1% of total)

I’ve now only got a SF rental condo and a Lake Tahoe vacation rental in my real estate rental portfolio. Although I miss my old house, I certainly don’t miss paying $23,000 a year in property taxes, another mortgage, dealing with leaks and managing terrible tenants. I drove by the other day and couldn’t believe how much noisier and busier the street was than where I currently live. I wouldn’t be comfortable raising my son there.

In January 2018, I missed my chance of raising the rent on my new incoming tenants because it didn’t come to mind until very late in the interview process. I didn’t write about my previous tenant’s sudden decision to move out in December 2017 after 1.5 years because they provided a relatively seamless transition by introducing their long time friends to replace them. I didn’t miss a month of rent and didn’t have to do any marketing so I felt I’d just keep the rent the same.

After these tenants move out, I’m thinking of just keeping the rental empty with furniture. It sounds stupid to give up $4,200/month, but I really hate dealing with the HOA, move-in/move-out rules, and maintenance issues. Given the condo doesn’t have a mortgage and I have to pay taxes on some of the rental income, I’m not giving up that much. The condo can be a place for my sister, parents, or in-laws to crash when they want to stay in SF for longer than a week or two.

The Lake Tahoe property continues to be 100% managed by a property management company. It feels amazing not to have to do anything. I can’t wait to bring up my boy this coming winter to play in the snow! I could go up this winter, but I want him to be able to walk and run comfortably before he goes. I’ve been dreaming of this moment for over 10 years now. The income from the property is highly dependent on how much it snows. Summer income is always very strong.

Alternative Income ($49,680, 24.4% of total)

Book sales ($36,000/year): Sales of How To Engineer Your Layoff continue to be steady. I expect book sales to rise once the economy starts to soften and people get more nervous about their jobs. It’s always best to be ahead of the curve when it comes to a layoff by negotiating first. Further, if you are planning to quit your job, then there is no downside in trying to engineer your layoff so you can get WARN Act pay for several months, a severance check, deferred compensation, and health care.

What’s crazy is that my book income is more than my SF rental condo income. Yet, I didn’t have to come up with $1,200,000 of capital (minimum cost to buy my condo today) to create my book. All I needed to create my book was energy, effort, and creativity. I truly believe developing your own online product is one of the best ways to make money.

Venture Debt ($12,240/year): The first venture debt fund has returned almost all my initial capital so I decided to invest $200,000 in the second fund. I took a risk investing $150,000 in my friend’s first fund, so I’m hoping there’s less risk in the second fund given he has four more years of experience on top of his 12+ years experience running a venture debt portfolio for another company.

The whole idea of investing in venture debt is trying to get a mid-to-high teens annual return with less risk than private equity. Venture debt lends money to well-funded private companies with a 1-3 year terms. They go in and out, collect their interest and sometimes gets a warrant. They’re higher on the capital structure as well.

P2P Lending ($1,440/year): I’ve lost interest in P2P lending since returns started coming down. You would think that returns would start going up with a rise in interest rates, but I’m not really seeing this yet. Prosper missed its window for IPO in 2015-16, and LendingClub is just chugging along. I hate it when people default on their debt obligations, which is why I haven’t invested large sums of money in P2P. That said, I’m still earning a respectable 7% a year in P2P, which is much better than the stock market is doing so far in 2018!

Real Estate Crowdfunding ($9,600/year): Once I sold my SF rental, it was natural to reinvest some of the proceeds into real estate crowdfunding to keep sector exposure. I didn’t invest a lot in some of my favorite REITs because I felt a rising interest rate environment would be a stronger headwind for REITs. But if I could be more surgical with my real estate investments by identifying specific investments in stronger employment growth markets, I thought I could do better.

In the summer of 2017, I first reinvested $250,000 into a RealtyShares Domestic Equity Fund. I already had $250,000 invested with them and I liked the projects they were choosing. After spending the rest of the year making low ball offers on SF real estate and losing, I invested another $300,000 in the fund in December 2017. Given 100% of my real estate crowdfunding are equity investments, there is no set monthly dividend. Each of the 12 investments in the fund have different timetables and objectives. I’m simply estimating that I’ll earn $9,600 for the year.

If my Fundrise investments achieves their objective blended return of 15% a year, I could earn a compounded $70,000 – $120,000 a year, which would really boost my passive income returns. However, I don’t expect them or any private investments to achieve their target. Instead, I’m hoping for a solid 8% a year return instead.

Feels Good To Simplify

It was easier recouping the lost $60,000 in rental property income than I expected. For so long, my primary mindset for passive income was rental income. Having $815,000 less mortgage debt, but still generating roughly the same amount of passive income with a much larger cash balance feels great. Further, my passive income portfolio got even more passive, which is good as a stay at home dad to a newborn.

In order to live off $200,000 a year in gross passive income in expensive San Francisco, we own a humble 1,920 square foot, three bedroom, two bathroom home and drive a car worth less than 1/10th our gross income. We never buy any new clothes and we take full advantage of all the free things the city has to offer during the weekdays.

$200,000 a year might sound like a lot to you, but the median home price in San Francisco is roughly $1,400,000, or 7X our annual passive income. For a family of three in 2019, the Department of Housing and Urban Development [HUD] declared income of $105,700 or below as “low income.” Therefore, I consider us firmly in the middle class.

We will continue to save and invest in more passive income generating investments just in case our son doesn’t win the San Francisco public school lottery. That’s right. Even if you pay $20,000 a year in property tax for a decade, your child has no guarantee of going to your neighborhood’s public school.

Although we forewent many luxuries since we graduated from college, there is not a day that goes by where we aren’t thankful for being able to leave our jobs for good at 34. We could have made more money working, but we decided being free was more important. There’s always another dollar to make, but never another second to create.

Regards,

Sam