Study Methodology

(View this document as a PDF.)

This study focuses on the 17 westernmost states of the continental U.S., where water is a particularly important topic due to rapid population growth and generally less abundant water supplies.  Data were collected in two stages: Focus groups were conducted in selected regions of the study area to identify water issues of high priority and assist in the design of the survey that would follow.  An Internet survey was then used to uncover Western households’ perceptions and preferences regarding water use and management, and their willingness to pay a fee in support of a number of water conservation and reallocation programs. 

Focus Groups

There were two technical focus groups—in Denver, CO and Reno, NV—and one non-technical focus group—also held in Denver.  Participants in the technical focus groups were water stakeholders in agricultural water issues (e.g., representatives of the agricultural industry and policymakers related to agriculture and water).  Participants of the non-technical focus groups were members of the general public. 

Internet Survey

The second stage of the study involved an Internet questionnaire of western households.  The questionnaire was developed by an interdisciplinary group of researchers at Colorado State University and drew from the Colorado Institute of Public Policy report entitled, “Water in 2025: Beliefs and Values as a Means for Cooperation” (2006), which used a Q-survey methodology to uncover the full range of beliefs and values for Colorado water stakeholders. 

The survey was administered by Survey Sampling Inc., a private firm specializing in sampling and administering surveys.  E-mail invitations containing a link to the survey were sent to a random sample of households throughout the study area from May 2008 to June 2008.  A total of 203,750 e-mail invitations were sent; of the 6,883 who opened the e-mail, 6,250 completed the survey, for a response rate of 90.8 percent. To test for non-response bias, an e-mail containing a small sub-set of the original survey questions was sent to non-respondents.  The results did not reveal any significant differences between members of sample who completed the questionnaire and those who did not; thus, weighting for non-response bias was not warranted.

The survey contained a map of the United States, with the 17-state study area highlighted and defined as “the West” for the purpose of the survey.  In addition to collecting demographic information about respondents, the survey measured Western households’ familiarity with a number of water terms; perceptions regarding water use, scarcity, and policy; preferences regarding water use, conservation, and leasing, attitudes toward rural communities; and willingness to pay a fee in support of a number of water conservation and reallocation programs outlined in the survey.  The next section describes the question categories and formats.  The full survey can be obtained from the authors upon request.

In a number of questions, respondents were asked how they might prioritize among a list of options.  For example, one question asked respondents to rank the 3 most important water uses during times of short-term scarcity from a list of 8 potential uses.  This ranking question provides insights into the relative tradeoffs that survey respondents are willing to make among water uses in times of scarcity.  In cases where respondents are asked to rank only a subset of all possible options, the mean rank can yield conflicting results (Leuschner et al., 1988).  Thus, Leuschner et al.’s (1988) Relative Importance statistic is adapted to the survey responses as the appropriate measure of the relative rank of each item.  This Relative Preference (RP) statistic is defined as:     
                

where wijk  = weight for rank i assigned to item j by respondent k
= 0 if item j is unranked
= I – i + 1 if item j has rank i.

RPj is the proportion of total weights that the category received.  The range of RPj lies between zero and 100, and the sum of all RPj’s for a given question will equal 100.  In the present case, I = 3, so a respondent’s first choice was given a weight of 3, while the second choice was given a weight of 2 and the third a weight of 1.  The remaining unranked categories received a weight of 0.  The weights given by all respondents to a particular category were then summed and divided by the sum of the weights from all categories.  The resulting percentage is the RP statistic for that category; it represents the proportion of total weights that the category received.  The sum of all RPj’s for a given question will equal 100 but, as with mean rank values, statements cannot be made about the statistical significance of differences between RPj values. 

Other questions asked respondents for their level of agreement with a number of statements.  These attitudinal questions were scored using a Likert scale which measured the extent of agreement with a statement as indicated by selection of one of five responses: strongly disagree, disagree, neither agree nor disagree, agree, or strongly agree.  If a respondent strongly agreed with the statement, the response was given a value of 5, whereas agreed, neutral, disagreed, and strongly disagreed responses were given values of 4, 3, 2, and 1, respectively.  For ease of interpretation, responses of 4 and 5 were grouped together into one “agree” category; responses of 1 and 2 were similarly grouped into one “disagree” category.  Then, for each attitudinal question, the percentage of those who agreed with the statement was calculated along with the percentage that disagreed with the statement and the percentage of those who were neutral.  The average level of agreement among survey respondents was also tabulated for each question.

The dichotomous choice (DC) question format was used to assess households’ willingness-to-pay (WTP) a fee on their summer water bill to fund one or more of eight water programs.  Compared open-ended questions, DC questions generate a scenario that is more similar to that encountered in day-to-day market transactions: respondents are simply asked whether they would pay a given dollar amount.  This technique is also less stressful for the respondent than requiring a specific value to be named, thus lowering item non-response.  Extensive evaluation of DC surveys in experimental settings suggests that their implications can be quite reliable, despite a variety of early concerns about the potential for biased responses (Cameron, 1988). 

Respondents were given a list of eight programs related to water conservation and reallocation and then were asked whether or not they would be willing to pay a fee on their water bill during the summer months to support the programs.  The proposed fee amount ranged from $5 to $25 in five-dollar increments.  The WTP question was worded as follows:
“Water providers might consider increasing water rates in order to find new sources of water, to pay for water conservation programs, or to help with problems that may arise as water is shifted to cities from other areas. Would you pay an additional $X per month on your water bill during the summer months if the fee was divided among the following programs?”

The eight programs were listed as:

1. To implement programs and technology to reduce household water consumption.
2. To construct a reservoir for water storage.
3. To create a system to reuse household waste water for watering public landscapes.
4. To set aside water for wildlife habitat in and around nearby streams.
5. To help keep irrigated farms in production.
6. To make infrastructure improvements in rural communities as compensation for water being transferred to cities.
7. To set aside water for public water-based recreation.
8. To provide subsidies on water-efficient appliances.

Respondents were then asked, regardless of whether they were willing to pay the fee, what proportion of the fee, should such a fee be instated, they would like to be allocated to each of the listed programs.

Survey Instrument

Study Methodology

Search our site: