Software enables easy Web surveys

Information is power. And while e-mail and the World Wide Web have made it possible to distribute information around the globe faster than ever, a new type of software is turning the Internet into a channel for collecting information.

Computer-based survey applications have been available for a long time. However, until recently these applications focused on generating hard-copy surveys that required manual data entry when the surveys were returned. The latest Web-enabled packages have changed all of that.

Need to find out which contractors have a specific capability? Generate a survey, send it via Internet e-mail and wait for the responses to come back. When they do, you can feed the replies into a database, crank out an easy-to-digest report and pop it into your supervisor's inbox.

Alternatively, you can publish your survey to your department's Web site and invite visitors to complete it. Just as with e-mail surveys, the responses can be processed automatically.

In this product comparison, we evaluated three offerings that promise to make it easy for non-experts to design, distribute and process surveys on Web sites and via e-mail: Raosoft Inc.'s EZSurvey, EZReport and SurveyWin; Saja Software Inc.'s Survey Select; and Perseus Development Corp.'s SurveySolutions for the Web.

Aimed largely at savvy computer users, these programs come with guidance for constructing effective surveys, and they provide automated wizards or similar helpers to ease the process. They also offer easy analytical and reporting tools.

These programs let you prepare questions, provide response fields and collect responses. However, there are a few design features that separate the top programs from the merely competent. Some enable the creation of branching question paths, where one answer to a question sends the respondent to certain

follow-up questions. Also, some packages provide entry-validation tools for working with databases. Other welcome features are the ability to add a departmental or agency logo, to section the form with a colorful dividing line and to insert navigation buttons. Most of these issues could be addressed with some simple Hypertext Markup Language coding, a Java applet or a scripting language, but it is far easier for users if these features are built into the survey software.

None of the products we examined does it all. Our testing, which weighted questionnaire substance over appearance and stressed ready-to-use analytical and reporting capabilities, gave the highest overall marks to Raosoft's trio of EZSurvey, EZReport and SurveyWin. If ease of use matters more to your department, Saja's Survey Select or Perseus' SurveySolutions for the Web would be a good choice. If you want to add HTML touches to your questionnaire without resorting to another editor, SurveySolutions for the Web is your only choice—and costs almost one-fifth the price of the competing packages.

***

Raosoft EZSurvey, EZReport and SurveyWin

Raosoft's stable of products offers a modular approach to Web surveys for a range of users. SurveyWin was the strongest of the products we reviewed when it came to analysis and reporting, especially when coupled with the company's EZReport module. But this duo doesn't provide automated Web and e-mail capabilities. To add that you need EZSurvey, which lets you throw together a quick and easy Web-based survey, though the included analytical tools are modest.

EZSurvey, despite its name, is a bit more difficult to learn than the other applications in this comparison. That's because it uses a non-standard (for Windows) HTML online help system and virtually no wizards.

However, there is a lot of power and flexibility in the package. And while it may not be as easy to learn as the competition, it's not all that daunting. For starters, creating your own questions from scratch and selecting from numerous types of response fields is easy enough within the utilitarian question-editing module. What's more, EZSurvey is the only package we reviewed that enables you to reduce the rate of bad responses by setting options for limited data entry validation and assistance. These options include range checking, which is setting minimum and maximum values for numeric responses; required responses, which are for write-in text, date, time and numeric responses; conversion to full capitalization, which is for write-in text; and response-text selection, which enables text to be selected in a write-in response field when the cursor moves to that question.

EZSurvey supports two kinds of conditional branching: A follow-up question can be displayed for any question if a specific response is given, or a user can be sent to another point in the questionnaire, depending upon his response to a question.

EZSurvey's editor has limitations. If you want to dress up your form with lines, images, navigation buttons, formatting or color, you'll have to open the output file in another editor before you post it to your server. But once you start to look a little further under the hood, EZSurvey's design seems more impressive. The application integrates with Microsoft Corp.'s Exchange and Mail; Lotus Development Corp.'s Notes, cc:Mail or BeyondMail; or Internet mail via Post Office Protocol, including clients such as Netscape Communications Corp.'s Messenger and Qualcomm Inc.'s

Eudora. EZSurvey saves incoming responses to a separate directory for each form you use. We set the program's options to process these files immediately to a database, which worked seamlessly.

Other options include recognizing specific subject headings, in case you don't have an e-mail account dedicated to your project, as well as automatic receipt acknowledgments and rejection notices for cases in which the response cannot be processed. Moreover, you can set the system to accept or reject submissions from specified users and domains, enabling both narrowly targeted and slightly restricted surveys over the Internet.

EZSurvey's analysis and reporting features are easy to employ, although the output is slightly more limited than what you will find in the competition. Basic reports are easy to generate. Oddly, though, we couldn't print charts or tables directly. We had to output pages as a bitmap or HTML file for viewing and printing from a browser.

Files can be read into the SurveyWin/EZReport duo for heavy-duty analysis and extra reporting features. This option provides compound and extended queries, cross-tabulation and advanced statistical analysis methods such as variance and chi-square. It even will determine your minimum sample size for a specified margin of error and confidence level. (Response databases also can be read into Microsoft's Access or Excel or any other DBF-capable application, or they can be exported to an ASCII-delimited or SPSS format.)

Overall, EZSurvey's branching and validation capabilities in form design, coupled with the powerful analysis and reporting available with SurveyWin and EZReport, gave it the edge in our comparison. But you won't be able to perform HTML editing of your questionnaires in EZSurvey as you can in SurveySolutions for the Web, and you don't get Survey Select's huge question library and integrated interface.

***

Saja Survey Select

Saja's Survey Select is a strong package, combining ease of use with well-selected analytical tools. Not only is it almost as easy to create effective survey forms here as in SurveySolutions for the Web, its automated analysis beat that found in EZSurvey, although not that provided by SurveyWin.

What we liked best was Survey Select's well-engineered user interface. It delivers an integrated rapid-deployment environment for surveys, with its design, administration and analysis modules joined in a button-led navigational structure. The Web and e-mail capabilities come as a separate module, which adds a button to the navigational structure.

Creating, administering and analyzing a Survey Select survey is largely a matter of clicking your way down a stack of buttons on the screen, with each successive button calling up a window for accomplishing the next step in the process. The program makes it simple to select response types, though the available formats are relatively limited because Survey Select does not offer the drop-down list option provided by the other programs. And conditional branching is available only when using the runtime distribution program, Survey Taker, which lets you distribute surveys via disk, as an e-mail attachment or as an application over your network.

Survey Select does, however, offer a unique response type: dual-scale/gap analysis. With this type of question, the respondent is asked a question in two parts, with an answer scale for each part. A typical question, for example, might ask the respondent how much importance he attaches to a service your agency offers. The second half of the question asks him to rate the agency's performance. The survey designer then can have the program correlate the results of the two answers. (EZReport lets you achieve a similar effect at the analytical stage, but it is a bit more complicated to set up and use.)

Survey Select's page-formatting tools are limited, with background color being one of the few options available. You cannot insert graphics, other text or HTML tags from within the program.

We also were disappointed to find that Survey Select depends solely on Microsoft Exchange or Windows Messaging for e-mail service, so if you don't have one of these platforms, you can't use e-mail with Survey Select.

If you have such a system running, it will be invoked in the background only for the transmission of messages. You have to create and maintain your list of addresses and groups with Survey Select's e-mail list tool. This may seem to be an extra step compared with the other applications, especially if your survey is posted to the Web and doesn't need to be mailed to recipients. But it enables the program to keep tabs on who has been sent a survey and who has returned it, and you can import address lists from Exchange or ASCII files.

Although Survey Select's chart types are varied, the options governing their appearance are crude, and reports consist of simple tables for each question in the selection. But if you want to slice or graph data in a variety of ways, you can do it more easily here than in SurveySolutions for the Web or in EZSurvey.

One outstanding feature of this program is its context-sensitive Guide window, which is available at all times unless you toggle it off. Here you'll find the steps to all the actions you need to perform, with links to explanations of concepts and terms. We found this unusual helper to be more useful than wizards.

Survey Select's best assets are its integrated interface and capable data-slicing tools. However, EZSurvey's branching and data validation in questionnaires are superior, as are SurveySolutions for the

Web's formatting and HTML insertion capabilities.

***

Perseus SurveySolutions for the Web

Perseus, a longtime player in the computer-assisted interviewing applications business, conducted a market survey that produced one surprising result: More than 75 percent of respondents prepared their questionnaires in a word processor before building them in a survey application. Perseus' response to this news was to design its Internet survey application to feel like a word processor and to be able to import word processor files in rich text format. It uses a few simple elements to designate response fields, sections and so forth for automatic recognition and formatting. SurveySolutions for the Web also provides some of the strongest design tools, includes convenient wizards for most procedures and has an easy-to-use interface that will seem familiar to users of Microsoft Office applications.

SurveySolutions is not only the easiest program to use, its automatic HTML generation produces the most attractive forms. In addition to the standard radio buttons and data entry fields, SurveySolutions offers features designed specifically for electronic surveys, such as drop-down list boxes and repeated-scale questions.

There is, however, a small price to pay for some of this convenience. Unlike EZSurvey, which enables users to set display options for each response field, Survey-Solutions requires that response fields of one type be treated as a group.

We encountered surprising oversights in SurveySolutions. We thought it was strange that a Web-centric program would provide formatting options that do not convert to HTML, especially without tagging them or otherwise warning users. We also were disappointed to find that conditional branching of questions was not an option.

SurveySolutions includes Perseus' MAPI Mail Manager to manage outgoing e-mail questionnaires and incoming responses. If your department uses a different protocol, you'll have to download the Perseus Universal Mail Manager. Those who depend on America Online or CompuServe for e-mail will have to save response messages as text files and manually enter them into the database.

Publishing our newly designed survey to the Web was generally easy. The Web Publishing Wizard made it a snap to choose options and set parameters for layout and submission of completed questionnaires. You can choose to e-mail responses from the user's browser, which is a more secure approach. Or you can use your Web server via a common gateway interface script to automatically send an acknowledgment of receipt.

Wizards are provided to make it easier to generate presentations and reports. However, reporting data is not Perseus' forte, and heavy number-crunchers will want to take advantage of the program's ability to export to SPSS format. Choosing the Auto Presentation Wizard generates a set of basic chart and text slides from your answer set, though it is easy to change the chart type and display parameters. (You also can change data slices, but the data querying tools to do the job are not documented.) The Auto Report Wizard does a similar job with text tables of data for hard-copy output. It will give you standard statistics (average, standard deviation, count, sum, minimum and maximum) for each question that has a numeric response, and it offers counts and percentages for single and multiple entries.

The embedded database is Microsoft Access 2.0, and powerful tools for custom analysis let you set filters and build queries in that environment, with special emphasis on cross-tabulations.

The biggest knock on SurveySolutions is its documentation, which we found inadequate. The user guide is organized in a peculiar manner, starting with three versions of its table of contents and ending with an unhelpful and inaccurate index. And in the electronic version of the guide, which serves as online help, a linked table of contents appears in a very large type size, but the entries are barely readable.

SurveySolutions for the Web excels at producing good-looking Web forms almost automatically, and it is the only product in this comparison that enables insertion of text and HTML features directly into the questionnaire form.

But expert users may not like the lack of more sophisticated statistical functions, and non-experts may not be able to slice data adequately. If you're going to be using a separate statistical analysis and reporting package, however, these objections wither away.

—Marshall is a free-lance writer who has been reviewing computer software for the past 10 years.

***

Survey software leaders plan upgrades

Other strong survey software packages to consider are The Survey System from Creative Research Systems and the combination of SPSS Inc.'s SPSS Data Entry and TextSmart. Both firms plan to ship upgrades this year that handle Web-based surveys.

SPSS provides a modular product line. SPSS Data Entry 1.0 is split into two programs—SPSS Data Entry Builder and SPSS Data Entry Station—that are sold separately. Data Entry Builder's Rule Wizard offers a powerful set of tools for form design, including data validation, logical-condition checking and skip-and-fill rules for multiple-part questions. Data Entry Station is the data-entry module. A third package, TextSmart 1.1, provides automated assistance in closed-coding information from verbatim text responses so that the data can be used in high-powered analytical applications. The program makes it a snap to categorize, subcategorize and graph data. If you want a summary report, however, you will need to buy one of SPSS' flagship statistical analysis products.

Using Data Entry Builder to set up forms is just like using Microsoft Corp.'s Access: You create, format and place each element on the page and specify its relationship to other elements and to data functions. This is a tedious process, but it provides a lot of flexibility.

TextSmart, on the other hand, is quick and easy. Simply specify a tab-delimited survey file, and the program reads questions with text responses into the question window, lists each word with its frequency of occurrence for inclusion or exclusion and automatically generates categories with summary and count statistics.

Data Entry Builder is available on the General Services Administration schedule for $493, SPSS Data Entry Station costs $244, and TextSmart costs $1,008. SPSS can be reached at (800) 525-4980.

The Survey System 6.0 from Creative Research Systems is a do-it-all product with advanced features such as optional modules. The program provides extensive analytical and reporting capabilities in its core package, and it has a forms designer capable of incorporating limited data validation and skip rules. The program also handles double-entry verification.

But The Survey System has some obvious limitations. Response types are limited to single-digit answers, or up to 10 multiple responses per question. A separate module is needed to provide an open-ended text response to questions. Another module is available for up to 500 multiple choices per question, and still others are available for real-number responses, advanced statistical analysis and various types of survey distribution and collection methods, including e-mail.

The Survey System is available direct for $499 (an evaluation version costs $49), with most extra modules costing $200, except for the Real Number Module, which costs $100. Creative Research Systems can be reached at (707) 765-1001.

***

How We Tested Web Survey Software

We installed each product onto a Windows 95 system, then used whatever tutorials, wizards, templates, and question libraries were available to construct our questionnaire. After posting the resultant survey questionnaires to an Apache Web server on a test intranet, we ran e-mail responses through Microsoft Exchange (which we also used to test the products' e-mail distribution) and whatever facility each program used to incorporate responses into its database. From there we ran the numbers through the available analytical, reporting and presentation tools.

Installation & Configuration: For our installation and configuration score, we looked at how effortlessly and flexibly we could set the program up for use. To award a score of satisfactory, we had to get the application up and running without extensive manual configuration. Extra points were awarded for the capacity to work with multiple e-mail systems and for configuration options to help manage survey traffic. Points were subtracted when it was necessary to download additional components.

Questionnaire Design:Questionnaire design was scored on a three-part basis: ease of use, including content assistance; structural features; and HTML formatting options. To receive a score of satisfactory, the program had to provide tools to generate an acceptable Web form. An exceptionally useful interface, large question library, data-validation options, question branching capabilities, and the ability to add HTML features and external files to the Web pages all raised the scores.

Analysis & Reporting:We rated each program's analysis and reporting capabilities by examining the tools provided for calculating survey responses and generating reports of those responses. To earn a score of satisfactory, a program had to provide quick means to view useful summaries of responses. Extra points were awarded for data-slicing capabilities and for the capacity to do more sophisticated analyses, especially if these are automated. Extra presentation capabilities-such as slide show presentation-also earned extra points.

Documentation: At a minimum, documentation had to tell us how to install the program and make use of the program's features. Comprehensive, well-organized and well-written manuals received higher scores. We lowered the score if the manual was poorly organized, lacked a table of contents and index, did not include information or contained factual errors in the text.

Technical Support: We based technical support scores on the quality of service we received during multiple anonymous support calls. Busy signals, voice mail-only service and excessive resolution times all resulted in lower scores.

Support Policies: To receive a score of satisfactory, the company must provide telephone technical support. We awarded bonus points for unconditional money-back guarantees, extended support hours, bulletin board support (such as CompuServe) and a toll-free number. We subtracted points for no technical support or a limited support period.

Prices: Programs that cost less scored higher in this category. We rated program prices for departmental-level project managers according to the following ranges:$0 to 200: Excellent$201 to 500: Very Good$501 to 1,000: Good$1,001 to 2,000: Satisfactory