STUDY DESIGN

The essence of scientific study is generally some numerical determination of a result. It is this numerical analysis that we approach with our statistical calculations. However, without a careful design of the study in the way that the numerical data are generated, formal statistical analysis of the numbers will yield scant benefits and may even be very misleading. In the typical study we are usually attempting to either characterize some variable (e.g., mean or trend) or make a choice to accept or reject among various hypotheses. No matter what the question is in our minds, we must always be careful that the results genuinely answer the question and that we are not misleading ourselves with inappropriate conclusions.

Scientific studies may be observational studies in which we gather data without performing any specific intervention affecting the assignment of groups or effects on group members. Because in an observational study we have to take what we find as data, we are always very subject to other unrecognized factors that may distort our interpretation or limit our ability to ask questions about our data.

In an interventional study, we pick the group members, determine what is being done to them, and attempt to look out for and eliminate potential confusion before the data are collected. A clinical trial is a typical interventional study in which different therapies are compared.

In the design of a scientific study one major fear is a statistical bias, which is a systematic effect in the study that produces an error in our interpretation of the results. When considering the various possible study designs, we need to beware of the potential biases that may creep in. A selection bias occurs when we are comparing groups with respect to some variable, but do not realize that the groups are different in other (but important) ways. Because groups of subjects may differ in multiple ways, our assignment of subjects into groups may result, either from misdirected choice or just random "bad luck," in groups that are not fairly comparable. So although we think that our results are determined by the variable under study, it may really be that some other factor misleads us in our conclusion. As an example, consider a study that is comparing surgical and medical treatment of some illness and ends up with a grouping in which a higher proportion of men than women are undergoing surgical therapy. Any conclusion from this study about surgery versus medical therapy is biased and might be truly just reflecting a difference in the results of therapy on men versus women.

A confounding bias occurs when multiple variables are intimately intertwined so that although we may assume that the variable under study is important, the truth is that the confounding variable is more important. As an example, consider a study attempting to determine the effects of obesity on longevity. Because diabetes mellitus is very closely correlated to obesity, a result that purported to be about obesity might easily be more accurate about diabetes. When variables do not correlate with each other so that changes in one do not go along with changes in the other, we call them (in a rough nonmathematical sense) independent variables. Variables that are correlated with each other are not independent.

A measurement bias occurs if the methods used for making measurements when comparing different groups have different scales or sensitivities. As an example, consider attempting to get a history of chest pain in groups of patients with and without known coronary artery disease. Patients who know that they have heart disease might be imagined to remember brief pain more thoroughly than healthy patients do, or perhaps they might deny pain more in their wish to believe that they are healthy—here the art of the experimenter is to predict and avoid such biases.

In a blinded study (more properly called a masked study), measurement bias may be avoided if the person performing the measurement does not know which group is being measured so that subtle bias in approaching or determining the data can be avoided. As an example, consider an active drug being compared with an inactive placebo. A researcher who knew which patient was taking the active drug might be more diligent in pursuing benefits or side effects. In a double-blind clinical study, neither the patients nor the experimenters obtaining the data should know which subjects are in which group to avoid subtle measurement bias.

Observational studies may be categorized as case, case-control, or cohort studies. In a simple case study, an individual case or group of cases is reported. Such reports may demonstrate the existence of some observation or effect, provide the presumably typical character of the observation, and suggest a therapy or natural history. These types of studies do not have the nature of a proof because it is always a possibility that some other hidden effect is producing the observation or that the observed characteristics are atypical in some fashion. Nevertheless, often "seeing is believing," so much of what is reported in medicine is in the form of case observations.

887

The distinction between cohort and case-control studies is important, but often misinterpreted. In both categories, groups of subjects are compared, usually in regard to the effects of some intervention. In a case-control study, the factor separating the groups is determined after the intervention. Typically, a comparison is made of some outcome in one group, the case group, versus a similar group that does not have the outcome and functions as the control group. Because the groups are separated after the intervention, selection biases or unappreciated confounding variables can mislead the investigators. As an example, consider a study investigating the effects of hypertension on surgical mortality. If the groups are divided into patients with and without perioperative cardiac events, it may well be that one group will be different from the other. However, such a result may be misleading; perhaps it is really the renal disease that is associated with the hypertension that is significant (a confounding bias). Alternatively, perhaps patients with hypertension are sent to surgery by their physicians only when they have worse surgical problems, so the groups are not fairly compared (a selection bias).

In a cohort study, the study group is monitored before the intervention occurs. The group is assembled to be as similar as possible and then monitored forward in time. Such studies are useful for describing the natural history of disease and may be helpful in suggesting etiologies. However, as with the other observational studies, selection and confounding bias may occur and lead to misleading results.

A case-control study is sometimes termed a retrospective study because the analysis can only be done after the subjects complete the study (so that it can be determined to which group they belong). In this sense, a cohort study is a prospective study inasmuch as the data must be gathered before the intervention. Unfortunately, these terms may be misused because obviously, a case-control study can be planned in advance ("prospectively"). In addition, no matter how truly prospective a study is, once the data are gathered, the analyses are performed after the fact ("retrospectively"). These terms are best avoided.

After considering the nature of observational trials we can see the strength of the interventional clinical trial, where the investigators determine the membership of the groups to be compared in advance and attempt to make the groups as alike as possible. In a randomized clinical trial, the subjects are assigned by chance to the groups so that if there are some fluctuations in the subjects or some subtle selection bias, the effect will be the same in each group (and presumably cancel out in the result). Such a study is necessarily prospective in nature.

Randomized clinical trials are the idealized standard in medical research because they give the best chance to minimize biases. However, the effort and expense in setting up a clinical trial may be very considerable. Recognize that such a trial requires patients to be enrolled before the medical intervention is performed. Patients give up choice of the therapy that they receive and allow their therapy to be randomly selected among various options. Such trials (even if they involve standard therapies) require full ethical board clearance and explicit patient consent.

Even with the best clinical trial methodology, potential difficulties include dealing with subtle selection biases regarding who is entered into the trial. Studies that are actively watched by investigators and their highly motivated associates may result in atypical practices that cannot be generalized into standard clinical practice. The patients in clinical trials are themselves mostly motivated by the search for personal benefit and cannot be depended on to follow protocols strictly or to stay with a program that does not appear to be successful to them.