|
|
This table identifies alternative technology approaches to achieve robust testing of WebSites. It describes the pros and cons of the four primary technical approaches.
See also Comparative Capability Summary for an indication of how these factors affect testing of particular features of WebSites. See also this summary of eValid Advantages.
| WebSite Feature | Windows Desktop Based | HTTP Protocol Emulation | Browser Proxy Based | eValid InBrowser Technology |
| Short Technical Description | Monitor Windows event loop, drill-down using MFCs, record and play. | Wrapper around HTTP implementation to grap outbound URL data, save selected inbound files. | Program that takes over the HTTP protocol and processes URL's outbound and data inbound. | nBrowser technology solution: functionality entirely inside the browser. |
| Availability and Status | Many commercial products | Some commercial products, some freeware | Some commercial products, some freeware | eValid is a unique patented commercial product |
| Capability for WebSite, Sub-WebSite Dynamic Analysis | Access to information about what to do next is not directly available. | Guiding the search requires detailed parsing of each page as it is downloaded. | Guiding the search requires detailed parsing of each page as it is downloaded. | Superior. Recursive-descent search with limits is built in, as are certain fixed filters. Additional user-supplied filters are possible. |
| Capability for Functional Testing and Validation | Usually very good. Depends on the methods used to intercept and analyze the events. | Very poor. Tests that require detailed access to page properties generally require parsing the downloaded HTML and then GETing additional pieces. | Very poor. (Same story as with HTTP GETs.) | Superior. The approach is powerful flexible and general. |
| Capability for WebSite Timing, Page Tuning | Limited to Windows timing accuracy, which may be difficult to use reliably. | It is possible to time downloads, but rendering entire pages is very difficult or impossible.
Very difficult to handle secure sessions. See performance measurement warning below. | Timings possible with an agent outside the proxy.
See performance measurement warning below. | Superior. 1 msec internal timer make it possible to measure nearly everything about a website. |
| Capability for WebSite Loading | There's only one event loop, so load simulations do not have the opportunity for expansion. Instead, one has to create some kind of engine that uses recorded parameters to create load. | It is easy to generate a lot of activity, but very difficult
to have that activity be coherent, e.g. preserve state.
See performance measurement warning below. | Similar to HTTP GETs:
Easy to drive around and download URL's individually.
See performance measurement warning below. | Superior. Sessions detail selectable are FULL or TEXT or URL level. Very accurate timings. Very realistic behavior. |
Performance Measurement Warning: Caution is advised when using timing and loading data for WebServer sizing because of the significant difference between retrieval of a single URL (a basepage) and retrieval of a complete pages as viewed by a browser.
Typically the basepage for a WebSite, specified by a single URL, e.g. http://www.mysite.com, consists of a body of text plus references to style sheets, JavaScript files, and images. What varies is the total amount of data downloaded to satisfy a request. URL-only retrievals obtain only the single named URL, whereas a browser automatically downloads not only the single named URL but all of the associated files. Browsers do this automatically by making a series of subsidiary URL requests through the HTTP protocol.
| Type of Download | Percentage of FULL | Margin of Error Range | Average Margin of Error |
| FULL | 100% | None | None |
| TEXT | 20%-40% | 5:1 to 3.3:1 | 3:1 |
| URL | 0.5%-20% | 5:1 to 200:1 | 100:1 |
The margin of error is important in this way: if by using URL-only measurements you conclude your WebSite servers can handle 1000 users, you may in fact have a capacity of 1000 / 100 = 10 users.
