Step 2.06.01 , Experimental Design
Overview of experimental designs
-
Impact-only design
An activity has already been implemented (no before-activity data exist) and there are no spatial controls or reference sites, watersheds or groups being used for comparison; monitoring is limited to the site, watershed, or group where activities have been implemented. These responses can be evaluated against specific predictions based on the conceptual model. Causal links between temporal changes in responses are difficult if not impossible to determine because the changes might have occurred without the activities being implemented.
-
Reference-Impact design
A modification of (1) above, where there are no before-activity implementation data but the same parameters are monitored through time in a reference and impact site, watershed, or group, which represents the desired direction of change for the impact. This design provides slightly better evidence for a causal link between temporal changes in response, because natural changes through time can be measured at reference sites as well. It is also possible to assess whether the trend of change at the impact location is towards the reference condition, if that is desired.
-
Control-Impact design
Similar to (2) above except that comparison is with a control site, watershed, or group. This design provides stronger inference about causality because comparison with the spatial control reduces the likelihood that effects from activities are statistically confounded with natural change.
-
Control-Reference-Impact design
This is a combination of (2) and (3) above. Statistical analyses test for divergence in temporal trends between the impact and the control, and for convergence in temporal trends between the impact and the reference site, watershed, or group. This design provides causal strength similar to (3), with the added advantage of assessing whether the trends are moving toward reference conditions, if that is desired.
-
Before-After-Impact design
This is a standard “impact analysis” design comparing parameter values before versus after activities have been implemented. The “before” data provide baseline or temporal control conditions. Evidence for causal links is limited by lack of spatial controls, therefore it is unclear whether or not the change would have occurred independently of the activities being implemented. This design is also difficult to use if activities are implemented gradually, if there is a long lag-time for impacts to occur, or if the difference that occurs is not large.
-
Before–After, Reference–Impact (BARI) design
This is similar to (5) but with a spatial component –a reference site, watershed or group that provides some measure of whether natural changes coincide with changes seen in the impact site. This design also allows assessment of whether the trend of a response is towards the reference condition. The test of interest is whether any before-after difference at the impact location is the same as at the reference location. The causal inference associated with this design is limited because the reference and impact sites, watersheds, or groups have different conditions prior to activity implementation. This makes it difficult to rule out a response to other factors coinciding with the start of the implementation of the activity.
-
Before–After Control–Impact (BACI) design.
Similar to (6), but using a spatial control instead of a reference. This design provides strong inference about causality because comparisons with spatial and temporal controls reduce the likelihood of confounding effects with natural spatial and temporal changes.
-
Before–After Control–Reference–Impact (BACRI) design.
A combination of (6) and (7) that provides strong evidence for causal links between activity and response, and also measures whether the change is towards reference condition, if that is desired.