We've talked a lot here about the edge-of-the-net problem -- the real issue for performance on the web being in the proverbial last mile, not the much-hyped allegedly-choking core.

That reality means that figuring out how well your applications and services are performing for users is not as simple as testing your stuff back at the home office. Gomez, Inc. conducts tests that model conditions in the field, and the results are often surprising, says the company's chief technology officer, Imad Mouline.

"I've seen jaws dropped to the floor. They say, 'This is not what my people are telling me, all the lights are green in the data center.' They think they have 99.99% availability when it may be more like 60%. You have to measure from the customer's point of view, not just behind the firewall, or at a key spot on the net."

To monitor the last mile, Gomez maintains a network of about 45,000 points of presence around the world. Different services and methods of delivery are tested, e.g., Comcast users in Boston, or low-bandwith users in Beijing, and regular reports delivered via a dashboard product. "We create a lab-like environment that takes into account all the vagaries the net delivers," says Mouline.

The differences in user experience between different US cities, or between various web browsers, can be startling. Yet, "some people are completely oblivious to their application's performance on different browsers." A consistent experience is important to customers, but many companies don't even account for variables that affect performance or track them after launch.

Mouline says this is a CIO-level issue. "You need to understand the impact of performance on your overall organizations. Keep in mind that the application is no longer what your developers are putting together, or what's running behind the firewall, the application is what the end users see. You need to develop it and test it with that in mind."