Experimental design with two groups, pre-post measurement, and where everyone is treated

Question

I am designing a study in which a large number of people are being treated. I cannot influence the treatment (e.g., a delayed treatment). I am able to make pre and post treatment measures on the subjects. The treatment has many effects on the subjects. I have a screener that I believe will identify subjects that will predict individuals that will improve on measure A.

The proposed experimental design is for two groups to be identified based on a screening test (the screening test can be administered pre or post treatment). Both groups will be subjected to the same treatment. Prior to and following treatment both groups will be tested on measures A and B. The hypothesis is that the treatment will effect the groups differently on measure A but not on measure B. Is there a term for this design? Is this a valid design?

Answer 1

I would call it a pre-post quasi-experiment, albeit where you are assessing the intervention in two different pre-existing groups.

In terms of assessing the effect of the intervention, there are more threats to causal inference, when you don't have a control group (i.e., one where you have pre and post measures but with no intervention or a control intervention). I.e., In addition to the effect of the treatment, there are many other explanations for any changes observed pre-post. For example, learning, maturation, fatigue, and so on. Background knowledge may be able to assist you in appraising which if any of these are likely to be significant.

You then have two observed groups. This aspect is more like an observational study. You are assessing the interaction between group and the treatment. Because group is an observational variable, you would want to take care in attributing causal explanations to any differences between groups in treatment effect to group membership. Nonetheless, often in this context, the interest is more about understanding how well a treatment generalises to different populations.

So in summary, It would be better if there was a control group. Ideally, you would have a fully crossed design. I.e., group (A and B) by control (treatment and control) where participants were randomly allocated to treatment or control. But if that's not possible, the data would still be interesting; you'd need to think carefully about carry-over effects.