Firewall audit products are maturing, but the product class is still a relatively young, small market, defined by compliance requirements. You have a fairly limited choice of vendors, including Tufin Software Technologies, AlgoSec, Secure Passage and Athena Security, which all come with firewall audit pedigrees, and RedSeal Systems and Skybox Security, which are primarily vendors of risk-mitigation tools, and so go beyond firewall audit to feature sophisticated risk-assessment and risk-management capabilities.

Take the time to define your requirements, narrow down your choices and put candidates to the test.

See the companion article Firewall audit tools: features and functions on CSOonline.com

DO look at platform and device coverage. These products generally support all the major firewall vendors and some others, as well as major network devices, so you should be covered. Take both present and future needs into account. For example, you may run a single platform across the organization now, but future acquisitions may run on other vendors' infrastructures. These tools should be able to help whether you plan to migrate onto a single platform or continue to manage several while still realizing the efficiencies they promise. See if the vendor has a software development kit that can allow it to integrate with unsupported platforms.

Check that coverage for network devices is included. There are a couple of considerations here. First, it may be important to you to clean up and optimize access control lists on your routers, and second, routers are increasingly featuring more built-in security capabilities.

DON'T overlook scalability. Those vendors that focus largely on enterprise deployments claim they can scale up to thousands of devices. Determine what that actually means in terms of management and the ability to perform under stress.

"In addition, the magnitude of environment brings huge demands on technology and methods that can be used," says the telecommunications company security officer. "What in a smaller company can be rock solid may not be applicable in a big environment. You need be cautious about the limitation of technology."

Choose with growth in mind. Even if a product scales to your current requirements, how well-suited is it to meet greater demands as the business grows, services are added, acquisitions are integrated and traffic increases?

DON'T buy more than you need. Some of these products are aimed at complex, heterogeneous environments with hundreds of firewalls and network devices. Measure the tool's capabilities and cost against your environment. If your firewall environment is relatively simple and static and your traffic is fairly predictable, choose a less-expensive product that you can apply initially for your optimization project and periodically to keep your firewalls under control.

DO put these products to the test once you narrow your choices to those that claim to meet most of your requirements.

"Pick two or three of your favorites and bake them off in real-world situations," says John Kindervag, senior analyst at Forrester Research. "The nice thing about firewall-auditing products is that you can test them on a live production environment because they are passive tools."

Kindervag recommends testing how well they do at finding unused rules, optimizing configurations and so on, then comparing reports.

"Run the results by your firewall guru or bring in one who can say, 'Yes, that's a good rule change,'" he says.

You can also determine whether they actually scale and deliver analysis at the speeds they claim and what kind of hardware they'd require.

DO determine your reporting requirements and evaluate the products' capabilities accordingly. Audit reports should come first and foremost for most organizations. Evaluate the quality of summary reports—are they sufficient to prove that your control policies are, in fact, carried out?

Also, make sure that you can produce satisfactory reports on demand in response to specific auditor queries. Some products offer regulation-specific reports, usually for PCI DSS, which may be useful.

Since these are management tools, you'll want to see useful operational reporting that quickly lets you see what has been done and what needs to be addressed. Make sure the reports deliver the information you want at the level of detail you need. For example, rule usage can change over time. A rule that was optimally placed at first may become a bottleneck as it's hit with more and more traffic, and may need to be moved up in the hierarchy.

Finally, high-level reports can demonstrate overall improvements in efficiency and security, as well as highlight which business units may be lax in properly managing their networks.

DO consider workflow integration. Most vendors offer complimentary workflow products to integrate their core capabilities with change-management workflow tools, such as ticketing systems. This may not be important if your organization has a well-defined process and supporting tools, either homegrown or commercial. But some companies find this capability useful in automating their change-management programs.

DON'T give short shrift to hardware, especially if you are running one of these products in a virtual environment in which resource-sharing may be an issue.

Make sure you have enough CPU and memory muscle to support the product under live conditions, and make provisions for growth as traffic increases.

Alternatively, you could go with one of the three appliance-based solutions Tufin offers in addition to its software.

DO review and refine your policies and procedures before buying and deploying a firewall audit product.

Enterprise IT governance and information security is built on well-defined policies and processes. Technological tools reduce error, improve efficiency and automate analysis that frustrates manual efforts, but you won't get their full benefit if you are simply throwing technology at a problem. Every organization is different, but here are some basic guidelines: