As the newsletter theme attests, there is strong current interest in monitoring the effects of management on watershed resources, providing an opportunity to review a basic principle of data collection. I hope to convince you that sampling directly affects the quality and reliability of the information collected.

The English scientist Lord Kelvin said,"...when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind..." I would add this caveat; "if you express something in numbers, take care how you collect those numbers or your knowledge may still be of a `meagre and unsatisfactory kind.' Worse, it could be misleading."

Properties of estimates made from data (e.g., means, confidence intervals, etc.) depend on the method of "selecting" population items for measurement. Since sampling applies to any situation where part of a population is measured, it applies to monitoring. Little attention is usually given to sampling, but the results always depend on how it is done.

Estimates calculated from sample data vary since population units comprising the sample vary. Sample composition depends on the scheme used to "select" sample units so any sampling/ estimating scheme has a "distribution" of possible outcomes. A sampling scheme is evaluated and different schemes are compared by their distributions. The quality of a single estimate can be judged only by the properties of its distribution of possible outcomes.

Only "random samples" (i.e., those with calculable probabilities of sample selection) have predictable properties. Schemes depending on judgement or on "haphazard" selection are not random; random schemes require some formal selection process based on random numbers and requirements of the estimator.

One way to study sampling schemes is to simulate many samples from a known population using different schemes and compare the resulting distributions. This was done for a seven-day population of 1008 10-minute suspended sediment loads from the North Fork of the Mad River near Korbel, California [Figure 1].

Histograms for 500 sample estimates of load are shown in Figure 2 for seven sampling schemes . The scales are identical and the vertical line through all plots gives the known true load of 3058 metric tonnes. Statistics for each scheme are given in the table.

In the first nonrandom scheme the station is visited at 1100 each day. A weekly total of 21 concentration specimens were collected with daily sample sizes proportional to stage (constant for all samples). Specimens were taken at uniformly distributed "waiting times" from one to eight periods after 1100 or the previous specimen. (This element of randomness does not constitute random sampling due to fixed and infrequent station arrival times.) The plot of this scheme shows samples concentrated tightly around their mean but to the right of the true load. Therefore, this scheme has small variance but is "biased" because the estimates are centered 1204 (4262-3058) tonnes above the true load.

Scheme 2 is identical to scheme 1 except the arrival time is 0600. The difference is striking. Variance and bias increased substantially. Apparently, this innocent change in the sampling program has major unanticipated effects on the distribution of estimates.

Scheme 3 is the same as 2 except the sample size was doubled to 42, by collecting twice as many specimens each day, and choosing waiting times from one to four. The variance dropped, but the bias changed little. For schemes 2 and 3, the probability that a given estimate is "close" to the true load is essentially zero.

Scheme 4 is the most basic random scheme. "Simple random sampling" selects 21 distinct specimens at random throughout the week. It is "unbiased" as indicated by the mean estimate of load being close to the true load. However, the variance is large and the estimates are distributed asymmetrically around their mean. The level of asymmetry depends on population form, sample size, and the random scheme.

Doubling the simple random sampling size to 42 gives scheme 5. This scheme is still unbiased, is more symmetrical, and has lower variance. Increasing the sample size has obvious and predictable benefits.

Variance can also be reduced by using sampling schemes that capitalize on knowledge of population structure. One such scheme is stratified random sampling which takes simple random samples from each of two or more relatively homogeneous strata. Scheme 6 is stratified random using daily strata with 3 specimens in each day. This scheme, using 21 specimens, does almost as well as scheme 5 with 42. The distribution is somewhat asymmetrical, but is unbiased.

Improving allocation of sampling resources in stratified sampling further reduces variance. Scheme 7 uses the first daily stage to set stratum size. Scheme 7 has the second lowest variance and lowest mean squared error (bias2+variance) of the seven schemes, is unbiased, and is nearly symmetrical.

Performance of nonrandom schemes is hard to assess. We saw how these particular schemes behaved and that arbitrary alterations in method can change performance in unpredictable ways. This is generally true. Nonrandom schemes can perform well, but to know for sure, they must be tried with particular populations. And good performance with one population is no assurance of similar performance with others.

Random schemes, alternatively, perform as indicated by theory. Unbiased estimators are unbiased for any population and variance is reduced by increasing sample size or using knowledge of population structure. If an appropriate random sampling scheme is rigorously applied, the results will have the expected properties; it is the only way to obtain estimates with known and reliable properties.

Formulas to calculate variance from samples are available for random but not for nonrandom schemes. Variance estimates for random schemes have little or no bias, while using the usual formulas with nonrandom schemes gives invalid results. This is illustrated in the simulation by comparing "true" variances calculated directly from each set of 500 load estimates to the mean of the variances estimated for each simulated sample. The mean sample estimates greatly exceed the corresponding "true" variance estimates for the nonrandom schemes; similar comparisons for the random schemes show near agreement.

Bob can be reached at (707) 822-3691