When I strolled into a Talbots near closing time on a Wednesday night, I wasn't expecting Phipps Plaza in Atlanta's ritzy Buckhead neighborhood to be so dead. Perfect for me. Less so for the store manager. I entered keenly aware of how completely out of place I must have seemed--a heavyset thirtysomething black guy in Walmart dress slacks, trying to look casual while fondling Hil­lary Clinton-esque blouses. If I were on staff, I might have briefly considered the possibility that I had come in only to knock over the place while things were quiet.

And I would have been about right.

I'm a competitive-intelligence researcher. A spy, of sorts. I don't break the law. But I always feel like I'm right on the edge of it, never mind my rigid ethical standards. The information I secure is given freely and obtained legally, and I don't lie to get it, but in the back of my mind I'm always thinking, You probably don't want me to know this.

I found myself on this perverse shopping trip for a smart green pleated skirt because a company had hired me to learn the sales targets and promotional activity from Talbots store managers. I had been recruited to attempt face-to-face social engineering--basically wheedling out the information in person.

At the time, I had only just launched my practice. I had left a decade-long career as a journalist a few years earlier to earn an M.B.A. With typically perfect timing, I entered the job market just as the financial services industry self-destructed. I had long wanted to start my own business; losing my marketing job at a Web start-up in a restructuring in 2010 simply accelerated things. Competitive intelligence seemed a good fit.

As the sales manager and I surfed Talbots's website together, looking for the green mini my wife saw on the website earlier that day, I mentioned offhand that I had just graduated from business school. I talked about how tough it had been to find a "real" job and said I did some business research now, casually identifying the analysts out in California who had hired me. I mentioned that I was really interested in retail stuff--that, heck, I was helping write a report on it for investors, in fact. And wow, isn't the retail world weird these days with the recession and all? Thus began a conversation about the business.

Apparently that store had been having a great year. Best in the region. Hitting its numbers. What numbers? Oh! You must be proud. Any younger folks biting on this new stuff?

I fingered the cell phone in my front shirt pocket, to see if the voice recorder was still working. No, I didn't tell the manager I was recording her. Legally, in Georgia, I didn't have to.

Spies Available at Wholesale Rates

It's difficult to tell exactly how much competitive intelligence is going on in the marketplace. Gartner estimates the business-intelligence market at about $30 billion, of which CI might arguably be considered a small slice. About 80,000 people have the term competitive intelligence in their LinkedIn profiles, although a wise soul in this business might consider using a different term of art to describe himself--or stay off LinkedIn altogether.

CI is usually within the purview of marketing departments. The largest companies with aggressive CI practices in highly competitive industries--Oracle, IBM, Microsoft, and Procter & Gamble come to mind--have entire departments devoted to the mission. Smaller businesses, or companies less focused on competitive behavior, will have one or two analysts, who will rely primarily on industry publications and research reports compiled by outside firms such as Accenture and PwC and supplement that with their own spot research or research from outside contractors.

At smaller companies, CI is an additional task dumped on a midlevel marketing person. It's one of those things that everyone seems to know has to happen--right after all the other things that need to be done. Companies like mine serve to fill the skills gap at small and midsize companies, or to supplement the research of large businesses with some burst capacity.

Sales figures are a fairly typical competitive-intelligence target and, frankly, it's considered grunt-level research. But imagine the value of this information to a competitor. If you know how much revenue a company expects its stores to earn, you can start making projections about how profitable each store might be, how often the managers will try to goose revenue with a sale, the likelihood a given store will stay in its current location given the rent it's probably paying--you can guess at much of the company's strategy.

Industries vary, of course, but revenue numbers are a financial intelligence home run, right up there with gross margin, units sold, inventory figures, customer counts, and cash flow. It's also exactly the sort of thing a business needs to understand about its competitors to effectively price and promote a new product.

I left that Talbots feeling relieved, if not a little...dirty. Third time's the charm, I suppose. This kind of social engineering is like hacking people, seeking a weak point for entry. I was basically laughed out of two other Talbots stores on my first couple of tries. Somehow, word hadn't quite gotten around. I'm consistently surprised by what can be learned from a friendly conversation with the right person. The only thing more surprising has been what I've learned without talking to a soul.

Google Stalking the World

Competitive-intelligence projects usually have three phases. First, figure out what you want to know. Then, conduct secondary intelligence--also called open-source intelligence--by vacuuming up everything you can find in the public record about your targets in online business reports, court records, old advertisements, job postings, keyword analytics, blog entries, and elsewhere. Finish with primary intelligence, which means talking to actual people. Often the goal of secondary research is nothing more than finding the person with the right job title or the right social connection to talk to, and finding enough information about the key intelligence topic to have a clever conversation.

I'll be honest--a lot of this stuff can be done by anyone with a passing familiarity with the Internet, a talent for conversation, and an inquisitive mind. There are two advantages to using someone from outside, though. I can keep the identity of my client confidential. My clients really can't without lying about their identity. And I'm generally faster than an amateur, because it takes experience to know where to look and how to elicit information from sources who might otherwise be unwilling to talk.

Most competitive-intelligence professionals focus on a single industry, and I've done a lot of work in financial services intelligence. But I'm hesitant to specialize in a sector. I'd rather develop expertise in hard-to-get elicitation. It's a difficult skill to duplicate and impossible to offshore.

Speed matters, because sometimes competitive intelligence is a race. In one unusual case, a small, boutique marketing firm focused on pharmaceuticals had discovered a potential partnership opportunity for a pharmaceutical distribution deal, but it appeared to hinge on being the first to communicate with the holder of a patent in Asia. The firm asked me to find the U.S. contact as quickly as possible--like, immediately. Judicious use of Google Translate and rudimentary understanding of international patent filings led to a name, which led to a phone number for a lawyer in New Hampshire a few minutes later.

Most jobs aren't finished quite this fast. I usually spend some time talking through a company's broad strategy to identify the kind of information that would best serve its needs, then build a list of key intelligence topics and questions to address. From there, we work through the list as budget and urgency allow. Typically, a client will commission a competitive-intelligence report looking for answers to broad problems. For example, a soft-drink start-up found capital to expand but didn't fully understand the market in New York City and Philadelphia. It hired me as a subcontractor through an agency to call around to Whole Foods stores and corner bodegas, eliciting information about how various obscure new brands of soda were selling and at what price, to provide a baseline for comparison.

I've found a big difference in attitude toward these questions when I'm asking a sales clerk about individual brands. Ask about the store's overall sales, and the reaction is incredulous and hostile. Ask how many cases of that overpriced kombucha they sell in a week, and the reaction is effusive. Managers care about protecting their own information. The information of their partners? Not so much.

Marketers use the information to make go/no-go decisions when expanding. Would it be worth it to try for New York first? How much could we sell this for in Brooklyn? How many units could we sell? In how many stores? If you have only enough capital to experiment in a couple of places at once, the value of the data resolves into real focus. Once a start-up sodamaker knows how much product a comparable competitor moves in a week, it's managerial math to compare costs between cities and pick one.

Right; Now, Pay Attention, 007

I don't own a spy camera. Yet.

Competitive-intelligence tools are usually a bit more mundane. Access to Dun & Bradstreet reports, court records, and incorporation records serves as a starter. Most of the background secondary research can be conducted with nothing more than an Internet connection and solid research skills.

Unless the target business is small and keeps a low profile. Businesses without a long track record, significant revenue, or many employees will be relatively opaque, even with the most expensive research tools.

A host of crowdsourcing tools has emerged to fill the gap, with wide variations in reliability. Glassdoor, for example, tends to overrepresent disgruntled employees in its commentary. Yelp can give a very broad sense of a company's strengths and weaknesses, but I find it's rare to see entries that aren't either obvious fluffery written by the company's employees and their friends or hit pieces of dubious credibility likely written by competitors. Yelp began filtering its reviews a few years ago, and I find its reasons for hiding reviews to be opaque enough to raise questions about how much influence a business has on what is to be seen.

Sites such as RipoffReport.com and PissedConsumer.com generally refuse to take down a negative posting. Still, the postings on complaint boards themselves are of questionable intelligence value because of the obvious biases. I get more out of making direct contact with people who have posted negative reviews.

I am fond of pulling phone numbers and email addresses from Jigsaw. Jigsaw--recently purchased by Salesforce--allows its members to trade contact information for corporate contacts. Give Jigsaw some business card data, and you will be able to get some for a different contact in return. The more people who use contact data you have entered, the more site credit you receive to access more contacts. It's primarily a sales tool, of course, with sales reps looking for more purchasing managers, but the intelligence value is obvious.

I'm waiting to see what happens with a start-up called InfoArmy, which is attempting to build a team of contributors to write competitive-intelligence reports on small and medium-size businesses. The contributors are paid a little bit to create a report, then get a piece of the revenue any time InfoArmy sells a download of it. So will these folks, whoever they might be, write high-quality reports--or will they write lots of reports? If it's the latter, I don't intend to pay $99 per report, which is the going price.

At any rate, I'm cheap. I always start with Facebook.

In one case, a relatively small information-security-auditing firm with a particularly Web-savvy sales manager asked me to identify as many of a competitor's customers as I could. He had an inkling that the competitor might be screwing up audits under a new standard. Happily enough, I discovered that the target company had liked a bunch of otherwise unrelated companies on its Facebook page. This simplified the research down to calling a representative sample of the companies linked through the Facebook fan page to confirm my hypothesis--that the social-media manager had friended all the company's clients--and weaseling more data from each.

Just the Facts, Ma'am

The job isn't complete until a client has the information necessary to make a decision. I like to produce a report heavy on direct quotes. It's more than color; it's boardroom ammunition. One of the quintessential problems with competitive intelligence is getting anyone to act on the data. A good journalist understands that it takes some emotional resonance to sell a story.

But format can vary greatly, depending on the urgency of the need for the data and the intended audience. When I'm conducting research for a company's internal team, as often as not, it just wants results, raw. When the ultimate audience is corporate investors and analysts, I know I will be contributing to a PowerPoint presentation. I hate PowerPoints; they tend to strip the intelligence of both its emotional impact and the depth of quantitative data needed for informed decision making.

The grand goal of good intelligence gathering has to be more than a pretty deck. There has to be a focus on actionable intelligence, the kind of information that lets you make decisions. Clients reach out for help with a competitive-intelligence problem because they're confused and, perhaps, afraid of making the wrong strategic decision. The idea of sending someone to quietly ask questions at a Talbots or to trawl through Facebook postings looking for the person trashing your business online has a mildly theatrical element to it. That speaks to the emotion underlying the industry--fear. The goal of competitive intelligence, ultimately, is to reduce anxiety about making decisions under uncertainty and with limited information. I suppose that's why I can feel like a con artist at times, even when I'm asking polite, honest, upfront questions. At the end of the day, that's what I'm selling--confidence.

***

Consider It Done

What the author will seek to do or learn on your behalf, the likelihood he will get what he's after, and the time it will take.

Cell-phone number of your competitor's main contact with Walmart:

• 80 percent chance; five minutes.

The names and contact information of the people suing your biggest rival:

• 80 percent chance; 30 minutes.

Stake out your competitor's booth at a trade show to see what people are saying about your product:

• 95 percent chance; four hours.

The market segmentation your competitor is using in its advertising:

• 95 percent chance; one to three days.

Stake out your competitor's manufacturing facility to see how much stuff it's making:

• 95 percent chance; three days to three weeks.

The names of your competitor's top five salespeople and their salaries:

• 80 percent chance; margin of error +/- 10 percent; three days.

Identify which of your competitor's former executives are most likely to come to work for you:

• 80 percent chance; one week.

Profit margin of your privately held competitor:

• 50-50 chance; margin of error +/- 10 percent; five to 10 days.

Playing Offense

Competitive intelligence works best as part of a broader marketing and operations strategy, with a team probing for specific weaknesses to match against its company's core competitive advantages. The trick to finding good answers is asking clever questions.

Build a competitive landscape.

Basics first. Create a list of all the companies that provide a product or service that can get between you and your customers. The best way to start may be to simply ask your customers from whom else they have considered purchasing. Be up front--tell them you're trying to improve your services.

The tough part here is identifying indirect competitive threats, the sort of products and services that lie outside of the marketing-defined idea of competing products. (The classic example from business school: Coke competes with Pepsi. It also competes with water.) Examine the capabilities of potential competitors as much as the actual products in play.

Know thyself.

You want to target your intelligence gathering to expose competitors' weaknesses against your strengths. But what is your company supposed to be good at? Do you know your core competitive advantage? What's your strategy? The questions you will want to answer stem directly from the strategy you are pursuing and the strategies competitors are pursuing.

Exploit the lousy job market.

A job posting will result in a flood of resumés. Take note of how many of the resumés you receive come from current or former employees of your competitors. The hiring process can be an opportunity to conduct some competitive research. This is an ugly and callous sentiment, but it's up to the job candidate to protect most kinds of competitive information. However, do not mess with a nondisclosure agreement. Google "tortious interference."

While you're at it, Google "industrial espionage." Don't do that.

Quietly "friend" your enemies.

I find some value in watching the comings and goings of a company's personnel through LinkedIn. It's imperfect intelligence, of course--not everyone is on LinkedIn. People usually take their time updating LinkedIn when they get canned, but when they quit--especially when they quit for a new job--they want people to know right now.

Don't just eat your own dog food. Eat cat food and bird food and chicken feed.

The best kind of competitive product intelligence comes from intimate familiarity with your competitors' products and services. If you own a salon, get a haircut from someone else. If you run a restaurant, order from neighboring places. Don't rely on secondhand impressions if firsthand information isn't hard to get.

Playing Defense

The good news is that small and medium-size businesses rarely serve as competitive-intelligence targets of serious professionals. So you probably won't be defending yourself against people like me, but rather people like you--amateurs who just want to see what some probing will get them. The bad news is that this probing, if well directed, works. Here's how to guard against it.

Work on your corporate culture.

Competitive-intelligence defense starts with the relationship between your company and its employees. Human factors are always the weakest link in a security system--and that's not true only of computer security. People who feel mistreated by their employer tend to be more willing to discuss that employer's shortcomings, both on the phone and online. Nondisclosure agreements can help--I recommend boilerplate NDA language in every employment contract--but they're difficult to enforce on the rank-and-file employee or former employee. And there's this little irony: A corporate culture designed to promote openness of communication may create resistance to practices that improve information security.

Beware the invisible phone number and the un-Google-able identity.

When I'm cold calling a company's executives, I'll generally use the *67 function on my phone, which blocks caller ID. Although I will always identify myself by name and company to the folks I'm talking to, I generally ID myself as a business researcher--which is true--and try to move on deftly to another phase of conversation. Though I have a corporate e-mail address, I also use a Gmail address and other personal e-mail that is substantially less Google-able.

Clean up your own house.

Please tell me your own website is not a source of competitive-intelligence leaks. Look deep into your site. Clean it up. Then consider having your Webmaster add a file called robots.txt at the top level of your domain, so that Archive.org will wipe out the archived version of the site.