Contrast Variable Potentially Providing a Consistent Interpretation to Effect Sizes

Traditional contrast analysis tackles the question of whether a linear combination of group means is exactly zero using significance testing. However, in reality, the key question of interest is usually how far a linear combination of values in multiple groups involved in a comparison is away from zero in a distribution level. Therefore, to effectively compare groups, we need additional analysis to incorporate information in a distribution level. In addition, the value of a traditional contrast has issues in capturing data variability. The p-value from classical t-test of testing a traditional contrast can capture data variability; however, it is affected by both sample size and the strength of a comparison. This issues with significance testing of mean difference are reflected in Tukey’s comments [33] “It is foolish to ask ‘Are the effects of A and B different?’ They are always different --for some decimal place.” All the problems in traditional contrast analysis attribute to the calls for a critical re examination of the common use of p-value and significance testing in medical research [9,15,16,23,30,36].


Introduction
Traditional contrast analysis tackles the question of whether a linear combination of group means is exactly zero using significance testing. However, in reality, the key question of interest is usually how far a linear combination of values in multiple groups involved in a comparison is away from zero in a distribution level. Therefore, to effectively compare groups, we need additional analysis to incorporate information in a distribution level. In addition, the value of a traditional contrast has issues in capturing data variability. The p-value from classical t-test of testing a traditional contrast can capture data variability; however, it is affected by both sample size and the strength of a comparison. This issues with significance testing of mean difference are reflected in Tukey's comments [33] "It is foolish to ask 'Are the effects of A and B different?' They are always different ---for some decimal place." All the problems in traditional contrast analysis attribute to the calls for a critical re examination of the common use of p-value and significance testing in medical research [9,15,16,23,30,36].
To address the issues of traditional contrast analysis, various effect sizes have been proposed [8,13,20,34]. Many of them may fall into two categories: probabilistic indices for comparing groups in a distribution level [1,4,6,12,14,19,21,25,28,31] and metrics for capturing both mean and variability, which includes various effect sizes similar to standardized mean differences [4,7,10,14,21,25,2 6,28,29,31,32,35,36]. However, different effect size measures are suitable for different types of data, and the interpretations of effect sizes are generally arbitrary and remain problematic [13,24,26]. Therefore, clear and consistent interpretations for the strength of group comparison are important.
In this article, I explore how to use recently proposed concept of contrast variable [39,40] to integrate the two categories of effect sizes. Meanwhile, based on the concepts of contrast variable and standardized mean of contrast variable (SMCV), I explore a rule for classifying the strength of comparison. The thresholds in this rule have meanings from both strength and probability perspectives; hence this rule may provide a clear and consistent interpretation to the strength of group comparisons. The concept of contrast variable can be applied not only to measure the strength of any contrast including difference between two groups in general but also to assess the size of main effects, interaction effects and any other contrasts (such as linear or quadratic effects) in Analysis of Variance (ANOVA). Hence, the use of contrast variable and associated classifying rule may have the potential to establish a consistent standard for assessing the strength of comparisons and the size of various effects in ANOVA.

Contrast variable, SMCV and c + -probability
Contrast analysis is a widely used statistical method to compare the values in multiple groups or treatment levels. Traditionally, a contrast is defined as a linear combination of means in the groups or treatment levels where the coefficients sum to zero. Traditional contrast analysis focuses on testing whether an average effect is exactly zero, above zero, or below zero. However, in many cases, scientists are interested in knowing the magnitude of effect beyond the question of average effect being exactly zero and the direction of a contrast. As a potential solution to these issues, contrast variable, SMCV and c + -probability have been proposed for addressing the strength of a comparison [39,40].
Briefly, a contrast variable is defined as a linear combination of random variables representing the values in the groups (or treatment levels) themselves, not just a linear combination of their means, where the coefficients sum to zero. The mean of a contrast variable Volume 1 • Issue 3 • 1000108 equals a traditional contrast. SMCV is the ratio of mean and standard deviation of a contrast variable representing a comparison or relationship of interest. Consequently, an effect size (i.e., SMCV) and a traditional contrast (i.e., contrast mean) are now two characteristics of the same random variable; thus a contrast variable integrates both an effect size and a traditional contrast. c + -probability is the probability that a contrast variable obtains a positive value. In other words, when one draws one value from each group involved in a comparison and calculate a linear combination of these drawn values based on the contrast's coefficients, c + -probability is the proportion that the calculated value is greater than zero when the drawing process is repeated infinite times. c + -probability is a probabilistic index accounting for distributions of compared groups whereas SMCV is an extended variant of standardized mean difference (such as Cohen's d, Glass's ∆ and Hedge's g) incorporating both mean and variance of groups. There is a link between SMCV and c + -probability [39]. Thus, standardized mean difference and probabilistic index are now integrated to effectively assess the strength of a comparison. In addition, the concepts of SMCV and c + -probability are applicable to not only the comparison of two groups but also the comparison of more than two groups.
The concepts in contrast analysis can be formalized mathematically as follows. Suppose the random values in t groups represented by random variables G 1 , G 2 ,...,G t have means µ 1, µ 2, …, µ t and variances where σ ij is the covariance of G 1 and G t. When G 1 , G 2 ,...,G t are independent, The c + -probability for a contrast variable V is the probability that the contrast variable btains a positive value, i.e., Pr (V > 0). There is a strong relationship between c + -probability and SMCV of a contrast variable V as follows [39]: Note: CL denotes the common language effect size indicator in the 2-group case proposed by [21].

A classifying rule for interpreting the strength of comparisons
Clear and consistent interpretations for the strength of group comparison are important and in urgent needs, which is reflected in Rosenthal, Rosnow and Rubin [26]'s comment, "Despite the growing awareness of the importance of estimating sizes of effects along with obtaining levels of significance, problems of interpretation remain." as well as in [13]'s comments, "The interpretation of the index value magnitude is, perhaps, the biggest limitation of the use of effect size." Based on contrast variable, SMCV and c + -probability, we can provide a clear and consistent interpretation to the strength of a comparison.
With contrast variable, we can assess the strength of a contrast from two aspects: one is based on the integration of both mean and variability of a contrast, represented by SMCV and the other is based on distributions in multiple conditions, represented by c + -probability. Moreover, because of the relationship between SMCV and c +probability, we can classify the strength of contrast based on SMCV which simultaneously contains information from c + -probability. Based on SMCV, some key values of interest are 0.25, 0.5 and 1, which means that the average value of a contrast is one quarter, one half and one time of the standard deviation of the contrast variable. If one treats the mean of a variable as signal and standard deviation of a variable as noise, then the SMCV values of 0.25, 0.5 and 1 for a contrast variable means that the signal-to-noise ratio for the contrast variable representing a comparison is 25 percent, 50 percent and 100 percent respectively. Under normality, SMCV = 0.25, 0.5 and 1 indicates c + -probability = 0.60, 0.7 and 0.84 respectively. If based on c + -probability alone, one key value of interest for SMCV is 1.645. This is because SMCV = 1.645 corresponds to c + -probability = 0.95 under normality. In addition, 1.645 equals approximately 8 3 , an important value in the relationship between SMCV and c +probability for non-normal distributions. SMCV = 1.645 indicate that the corresponding c + -probability is at l east 0.918 when the contrast variable has a symmetric unimodal distribution, and is at least 0.836 when the contrast variable has a unimodal distribution. Therefore, based on both SMCV and c + -probability, some key values of interest for SMCV are 0, 0.25, 0.5, 1 and 1.645.
Based on the key values of SMCV and their corresponding values of c + -probabilities, it is reasonable to construct SMCV-based criteria for assessing the strength of a comparison. For example, when the mean of a contrast variable is zero (i.e., SMCV=0), c + -probability is 0.5. When SMCV is between 0 and 0.25, the signal-to-noise ratio is between 0 and 25 percent and the c + -probability is between 0.50 and 0.60, which is slightly above 0.50; thus the strength of the comparison (or effect size) is small. When SMCV is between 0.25 and 1, the signal-tonoise ratio is between 25 percent and 100 percent and the c + -probability is between 0.60 and 0.84; thus the strength of the comparison is medium. When SMCV is between 1 and 1.645, the signal-to-noise ratio is between 100 percent and 164.5 percent and the c + -probability is between 0.84 and 0.95; thus the strength of the comparison is large. Finally, when SMCV is greater than or equal to 1.645, the signal-to-noise ratio is at least 164.5 percent and the c + -probability is at least 0.95; thus the strength of comparison is extra large. This is for the positive direction. Similarly, we can classify the strength of comparison in the negative direction. The resulting SMCV-based classifying rule is shown in Panel A of Table 1.
The criteria in Panel A of Table 1 can be applied to any combination of random variables in which the interest is how far the combination is away from zero although here we focus on the linear combination of random variables. The concepts of SMCV and c + -probability accommodate various situations with or without independence and with or without variance homogeneity. Thus, if one uses SMCV only, one can use the criterion in the second column of Panel A in Table 1. This criterion should be applicable in a group comparison context with or without independence and with or without variance homogeneity and is consistent in various experiments in terms of signal-to-noise ratio. On the other hand, if one prefers to use c +probability, one can use the criterion in the third column of Panel A in Table 1. Regardless of data distributions, this c + -probabilitybased criterion is consistent in various experiments in terms of the probability of a contrast variable being greater than zero. However, The correspondence between SMCV-based and c + -probabilitybased criteria listed in Table 1 relies on the normality assumption. Consequently, if one uses the c + -probability-based criterion in Table  1, corresponding values of SMCV may be adjusted according to the underlying nonnormal distributions. Similarly, if one uses the SMCVbased criterion in Table 1, corresponding values of c + -probability may be adjusted according to the underlying nonnormal distributions.

Two-group comparisons
For two-group comparisons, the difference of values between two groups is a contrast variable with coefficients (1, -1). Thus, SMCV becomes strictly standardized mean difference (SSMD), i.e., the ratio of mean to standard deviation of a difference, and c + -probability becomes d + -probability, i.e., the probability that a difference is positive [35][36][37]41]. The d + -probability is equivalent to the wellknown common language effect size indicator (CL) in the 2-group case [21] or P(X >Y) [1,28,42]. By replacing SMCV and c + -probability with SSMD and d + -probability, we can obtain the relationships between SSMD and d + -probability described in [38].
Huberty [13] pointed out, it appears that the only cutoffs of effect size to which applied researchers have paid attention are those standards initiated by Cohen [5]: Cohen's d being 0.20 for small effects, 0.50 for medium effects, and 0.80 for large effects. Later, McLean [22] suggested the following criteria: Cohen's d being 0.50 for small effects, between 0.50 and .00 for moderate effects, and above 1.  Table 1). Hence Cohen's small, medium and large effects have a signal-to-noise ratio of 14.14, 35.36 and 56.56 percent respectively and a d + -probability of 0.556, 0.638 and 0.714 respectively under normality. In other words, if one randomly draws one value from each of two groups, Cohen's small, medium and large effects means that the chance of the value from the first group is greater than the second group is 0.556, 0.  Table  1). That is, Mclean's small, moderate and large effects have a signalto-noise ratio between 0 and 35.36 percent, between 35.36 and 70.71 percent, and greater than 70.71 percent respectively. Based on the d + -probability, if one randomly draws one value from each of two groups, Mclean's small, moderate and large effects means that the chance of the value from the first group is greater than the second group is between 0.5 and 0.64, between 0.64 and 0.76, and greater than 0.714 respectively in the positive direction. Therefore, SSMD criterion gives interpretations to Cohen's and Mclean's criteria from both strength and probability perspectives. From Table 1, we can see that the small, medium and large effects based on Cohen's criterion are respectively small, medium and medium large effects based on SSMD. Mclean's small and medium effects are roughly SSMD's small and medium effects respectively. Mclean's large effects contain SSMD's medium large, large and extra large effects.

Concepts:
The contrast analysis in one-factor experiments aims at the comparison of multiple levels in one factor. Consider one-way ANOVA, in which t groups (or treatment levels) are used in ontrast analysis, and the random variable representing the i th group G i has population mean µ i and variance 2 e σ . Then a traditional contrast is Clearly, the mean of V equals its corresponding traditional contrast. The SMCV (λ) of V is the ratio of mean and standard deviation of contrast variable V.
In the situation where n i observations Y ij ′ S are obtained from the N -t is the degree of freedom, namely and N is the total number of observations. Then the estimate of traditional contrast L is A simple estimate of SMCV λ is the method-of-moment (MM) estimate which is A better estimate is the uniformly minimal variance unbiased estimate (UMVUE) [39], which is It can be shown that This non-central t-distribution can be used to construct the CI of SMCV [39,40].
The above method of contrast variable and SMCV can be applied to any contrast in one-way ANOVA although different contrasts may have different coefficients. After constructing contrast variable, we can apply the same classifying rule in Table 1 to assess the strength of any contrast. For example, the effect in the i th group is commonly  (11,12,13,14,15). The means are indicated by "+" and are connected using linear segments in each panel. The linear relationship between putative performance score and age is investigated using a contrast and its effect sizes.
. Thus the size of α i effect for the i th group can be estimated using formula 2 or 3. The average magnitude of effects in all t levels is estimated to be the average value of ˆ' Correspondingly, the average absolute value of effects in all t levels is estimated as The confidence interval of SMCV can be obtained using a non-central t-distribution [39].
Currently, standardized effect size (SES) is used to assess the size of treatment effects. SES at the i th level is defined as Rosenthal, Rosnow and Rubin [24] for more details about these four effect size correlations.

A case study and simulation
An example: A study was conducted to investigate psychomotor skills against ages [26]. In this study, the sample means of performance scores of ten children at each of five age levels (11,12,13,14,15) were 25, 30, 40, 50 and 55 respectively, and the MSE was 1575. The contrast for level effect at age 11 has coefficients 4 1 1 1 1 ,-,-,-,- . The value of SMCV for this contrast is estimated to be -0.42 and corresponding c + -probability is 0.34, which suggests that the effect at the level of age 11 is medium and negative. The c + -probability of 11 0.34 indicates that, if one value is drawn from each age group, the chance that the value at age 11 is larger than the average value of the 5 draws at the remaining ages is 34%. Using the noncentral t-distribution described above, the 95% confidence interval (CI) of SMCV of this contrast is (-1.049, 0.209). Similarly we can investigate the effect at other age levels. We also calculated SES and its 95% CI.
The results are displayed in Table 2. In this example, the values of SES are all very close to their corresponding SMCV values. This is because the square root of the sum of squared coefficients for each treatment level is close to 1 in this example.
A simulation study: to see how strong the linear relationship between performance score and age is, four scenarios A, B, C and D were simulated using normal distributions and transformation based on the psychomotor example ( Figure 1). The age groups have the same sample means (25,30,40,50,55) in each scenario whereas the MSE and sample size vary (the first row of Table 3). Although all four scenarios have the same set of group means, the simulated data displayed in Figure 1 clearly show that the linear relationship between the putative performance score and age is not large in Scenarios A and B but extra large in Scenarios C and D. This is because the simulated studies have large data variability (i.e., data variance being 1575) in Scenarios A and B but much smaller data variability (i.e., data variance being 16) in Scenarios C and D.
The contrast variable representing a linear relationship between performance score and age has coefficients (-2, -1, 0, 1, 2). For this linear relationship, in each scenario, we calculated contrast mean (i.e., tradition contrast L), SES, SMCV and their 95% confidence intervals as well as γ alerting , γ contrast , γ effectsize and γ BESD that are proposed by Rosenthal, Rosnow and Rubin [26]. The SMCV ˆL inear λ of this contrast variable is estimated to be 0.6375 in scenarios A and B, which indicates that the contrast for the linear relationship has a medium large positive effect; in other words, the study shows that the performance score Table 3: Contrast mean (i.e., traditional contrast L), SMCV, SES and their confidence intervals (CI), correlational effect sizes for the contrast variable representing a linear relationship between performance score and age as well as average magnitude of effects at treatment levels in a simulation.  Table 3).
The traditional contrast L has the same value of 80 in all four scenarios; thus it cannot indicate the strength of the linear relationship. The 95% confidence interval of L becomes narrower each step from Scenarios A to D, but can hardly represent the strength of the linear relationship. If using p-value of testing L=0, we would conclude marginally significance in Scenario A and strong significance in the remaining three scenarios, which hardly offers useful information about the strength of the linear relationship in the data. Therefore, traditional contrast cannot provide useful information about the strength of the linear relationship. The results in Table 3 also show that (1) using γ alerting , the linear relationship is the same for all four scenarios; (2) using γ BESD , the linear relationship is weak in Scenarios A and B, strong in Scenario C, and in between in Scenario D; (3) using γ contrast or γ effectsize , the linear relationship is weak in Scenarios A and B but very strong in Scenarios C and D; (4) using SMCV, the linear relationship is medium large in Scenarios A and B but extra large in Scenarios C and D. The results obtained using γ alerting and γ BESD do not match with the observation and intuition about the strength of linear relationship displayed in Figure 1. SMCV, γ contrast and γ effectsize give similar sensible judgment about the strength of the linear relationship.
The SES values in Scenarios A and B (both being 2.0158) is much smaller than those in Scenarios C and D (both being 20). Thus SES represents the relative strength of the linear relationship effectively. However, the values of SES are affected by the coefficients of contrasts; thus they are not comparable for contrasts with different coefficients. Consequently, there are not any consistent SES-based criteria for classifying the strength of contrasts with different coefficients. The SMCV values in Scenarios A and B (both being 0.6375) is smaller than those in Scenarios C and D (both being 6.325). The confidence intervals of SMCV in the scenarios with 10 per group are much wider than those in the corresponding scenarios with 200 per group. Thus, like SES, the sample size has the right impact on SMCV estimation: it affects the precision but not mean of SMCV estimation. Unlike SES, the value of SMCV is comparable to various contrast variables with different coefficients and has the same probabilistic meanings. Moreover, the values of corresponding c + -probability provide additional information about the linear relationship: if one draws five values (y 1, y 2, y 3, y 4, y 5 ) each from one of the five age groups, the probability that a contrast variable representing the linear relationship (i.e., -2y 1 -y 2 +y 4 +2y 5 ) is greater than 0 (i.e., no positive linear relationship) is 0.738 in Scenarios A and B and nearly 1 in Scenarios C and D.

Contrast analysis in multi-factor experiments
Concepts: In multifactor experiments with multiple levels in each factor, we use a random variable P k to represent the random values in the l th ( l =1,...n l ) level of the k th (k = 1 ,..., K ) factor. Suppose a set of coefficients represent a comparison for the levels in the k th factor. A contrast variable based on this set of coefficients is defined as where G l is a random variable whose mean (and variance) equals the weighted mixture of means (and variances) of m combinations of factor levels containing the l th level of the k th factor.
Consider a two-factor experiment in which the two factors have I and J levels respectively as shown in Table 4. Let us use a random P ij variable to represent the random values in the combination of the i th level of factor 1 and the j th level of factor 2. Assume P ij has mean µ ij and variance 2 ij σ . A contrast variable for a comparison in the levels of Factor 1 is The corresponding c + -probability is the probability that the contrast variable V based on P i• ′ s is greater than 0. .
As shown in Formula 4, the definition of a contrast variable for a comparison in levels of a factor is based on multiple random variables The standardized mean of the linear combination U is Therefore, 1 . .
Because U directly consists of factor levels and is easier to handle than contrast variable V, we can work on U to get statistical inference on parameters from V, based on formulas 7 -9. For convenience, we use a term "contrast core" to refer to the comparison (represented by c i• ′S) embraced in a linear combination U. Meanwhile, we use another term "core number" to refer to the number of combinations with equal weights (i.e., m) [40].
In a two-way ANOVA, we usually assume P i• ′S that are independent with equal variance 2 Based on the method of contrast variable and SMCV presented above, we can apply the same classifying rule in Table 1 to assess the strength of any contrasts in two-way ANOVA including main effect and interaction effect although different contrasts may have different coefficients. The treatment effect, main effect and interaction effect can be explored using traditional contrasts [2]. Now, each of them can be expressed as the mean of a contrast variable or a linear combination of random variables For main effect τ •j = µ •j − µ •• in the j th level of factor 2, the core number is I and the coefficients are Using contrast variable, SMCV and the coefficients in (11)-(14), we can use the classifying rule in Table 1 to provide a consistent interpretation to any effects that are commonly used in two-way ANOVA. And the same method can readily be extended to ANOVA with more than two factors.

Estimation and Inference of SMCV
Then the estimate of traditional contrast L is F λ ⋅ be the cumulative distribution function of noncentral τ(V,bλ) and T obs be the observed value of T. Then we can find λ L and λ U such that ( , ) The estimates of SMCV λ and its distribution can be derived as follows. Based on the above three properties for the independent, homoscedastic and normal model in two-way ANOVA, where 11 12 , ,..., 2 IJ n n n ≥ . In deriving the above equations, I used  is a proportional noncentral t-distribution with ν degrees of freedom, non-central parameter c and proportion a. The proportional noncentral t-distribution is defined as follows: if T ~ noncentral t(ν,c) then aT ~ pnc t(ν,c, a) [37].
In ANOVA, we usually assume that the values in each combination of factor levels independently have a normal distribution with equal variance. Consequently, any contrast variable also has a normal distribution. Therefore, we can plug the estimate of SMCV into the relationship c + -probability = Φ(λ)to obtain an estimate of c + probability for contrast variable V.

An illustrative example
Here I construct an illustrative example based on Johnson et al's safety assessment study [18] to demonstrate how the method works. In this example, TUNEL staining fold increases were measured to assess apoptosis in rat myotube cultures treated with two drugs A and B respectively at 4, 10 and 16 hours. The data is listed in Table 5 and displayed in Figure 2.
This is an example of ANOVA with two factors, drug and time. The factor drug has two levels, drugs A and B, and the factor time has three levels, hours 6, 9 and 12. That is, I=2 and J=3. Let random variables P 11 , P 12 , P 13 , P 21 , P 22 , P 23 represent the fold increases by drug A at hours 4, 10, 16 and by drug B at hours 4, 10, 16, respectively. Then the main effect for the drug can be assessed using a contrast variable 1 1 2 where P 1• and P 2• are random variables constructed as P i• in Table 4. Thus, P 1• is a random variable for the drug A whose mean (and variance) equals the equally weighted mixture of means (and variances) of 3 combinations of levels, i.e., drug A at hours 4, 10, 16, and P 2• is a similar random variable for drug B. The mean and SMCV of V 1• can be obtained using the linear combination 1  11  12  13  21  22  23   1  1  1  1  1  1  , , , Note: P ij has mean µ ij and variance σ µ µ  Table 5: Data in an illustrative safety assessment study for two drugs at three time points.
, the contrast core is 1 1 , and the core number is 3. Thus, the SMCV of V is estimated to be 0.442 and the confidence interval of λ 1• can be obtained using 1 1~n oncentral ( , ) 1 2 where ν e =52 and b = 5.3785. Similarly, we applied the method of contrast variable to investigate other main effects at levels of either factor and to explore linear and quadratic relationships. The results are listed in Table 6.
The results in Table 6 show that the SMCV of the contrast variable for the main effect of Drug A is estimated to be 0.442 and its corresponding c + -probability is 0.671. Thus the main effect of Drug A is medium; so is the main effect of Drug B. The average magnitude of drug effects is 0.442 which is a medium effect. In other words, the drug factor has a medium effect or the overall difference between Drugs A and B is medium.  Table 6). The average magnitude of time effects is 1.979. Thus, the time factor has an extra large effect.
The SMCV is estimated to 2.834 for the contrast variable representing the linear relationship between time and fold increase, which indicates that this linear relationship is extra large. From the coefficients and core numbers, the contrast variable representing the linear relationship also denotes the difference of fold increases between Hours 16 and 4. The SMCV is estimated to 1.028 for the contrast variable representing the quadratic relationship between time and fold increase, which indicates that this quadratic relationship is large. From the coefficients, the contrast variable representing the quadratic relationship also denotes the main effect at Hour 10, namely the difference between Hour 10 and the average of Hours 4 and 16. Judged by observing the data displayed in Figure 2, all the above conclusions about the strength of main effects and other contrasts obtained using SMCV are reasonable.
If we use the values of contrast means (namely traditional contrasts) and associated pvalues, we would conclude that all the contrast means listed in Table 7 are significant and we can hardly obtain useful information about the strength of comparison. One more case is that we know the contrasts with coefficients (-1, 2, -1, -1, 2, -1) and   for contrasts D and G (Table 7). This is one advantage of these correlational effect sizes. As shown in the simulation in one-way ANOVA shown in Table 3, alerting γ cannot take into account data variability. In that simulation study, contrast γ approximately equals to effect size γ in each situation. However, the results of contrasts in Table  7 shows that, contrast It seems that effect size γ is the best among these four correlational effect sizes. However, it seems that the consistent meaning of effect size γ has yet to be discovered whereas SMCV has consistent meanings.
As demonstrated by the comparison of contrasts D and G in Tables 6 and 7, one issue of SES is that multiplying a constant to the coefficients of a contrast will change the SES value. For example, To address this issue, one argument for adopting SES is that before calculating SES, one should convert the coefficients ' c ij s into  (Table 7). However, the use of USES incurs another issue as demonstrated in the following example.
In this example, assume population means are known as shown in Table 8; MSE is 100 (or equivalently σ = 10 ) and sample size is 10 in each cell. It is trivial to derive that the main effect of Drug C is 1.9, corresponding USES is 0.4654. Now, if we combine the 3 days, we would have cell means shown in the last column of Table 8 and we would still have σ = 10 because there is no column effect. Then USES becomes 0.2687. Thus, USES produces anomalous behavior. By contrast, SMCV is 0.2687 (SES is 0.19) for the effect of Drug C either in the factorial experiment or in the combined days. Therefore, USES is problematic and cannot help the adoption of SES.

Conclusions and Discussions
For 2-group comparisons, the most commonly used probabilistic index for effect sizes may be McGraw and Wong's CL and the most commonly used ratios of mean difference to variability for effect sizes are Cohen's d, Glass's ∆ and Hedge's g. With the recently proposed concept of contrast variable and associated terms of SMCV and c + -probability, we can link all of them together. When the concept of contrast variable is applied to 2-group comparison, the contrast variable represents the difference between two groups. Subsequently, c + -probability becomes d + -probability and SMCV becomes SSMD. d + -probability is equivalent to CL [21] or P(X > Y) [1,17,28,42]. Meanwhile, Cohen's d = 2 SSMD in situations where there is no correlation between two groups. SMCV and c + -probability are two characteristics of a contrast variable and there is a strong link between SMCV and c + -probability. Therefore, a contrast variable and its two characteristics can link the commonly used effect sizes together and give interpretations to the strength of comparison from two perspectives: a probabilistic index and a signal-to-noise ratio.
Based on SMCV and c + -probability, we have a rule for classifying the strength of a comparison in a contrast framework. The SMCVbased classifying rule can be consistently applied to any comparison in which the interest is in how far the comparison is away from zero. The common comparison is a contrast where coefficients sum to zero. It works effectively for either relationship or group comparison in either independent or correlated situations and in either two or more than 2 groups. As explored in this article, treatment effect, main effect, interaction effect, linear relationship, quadratic relationship and many other effects and relationships can all be addressed consistently using contrast variables and contrast-based classifying rule. The examples analyzed in the article also show that the results reached using contrast variable and SMCV-based classifying rule are sensible and matched with observations and intuitions from the data. The SMCV-based classifying rule also gives an interpretation to both Cohen's criterion and McLean's criterion. Based on these interpretations, if one prefers the use of Cohen's criterion or McLean's criterion, one can readily extend these criteria to any contrasts involving more than 2 groups by transforming Cohen's d to d + -probability and then to SMCV under normality.
As described above, SMCV and c + -probability have strong connections with commonly used effect sizes. However, SMCV and c + -probability have their own features that other effect sizes do not have. For example, although d + -probability is equivalent to CL in 2-groupcomparison, it is easier for one to guess its meaning from the word " d + -probability": the probability of a difference being positive. Moreover, it is much easier to extend d + -probability to n-group comparison with a consistent content than CL. When d + -probability is extended to n-group comparison, it becomes c + -probability which is still the probability of a linear combination of random variables (with coefficients summing to zero) being greater than zero. Now we only need to extend the number of groups from 2 to n. By contrast, when CL is extended to n-group comparison, it has a different meaning: the probability of a value from one group being greater than a value from the other groups [21].
In contrast analysis for factorial experiments, currently the most commonly used effect sizes may be SES's explored by Steiger [29] and correlational effect sizes proposed by Rosenthal et al [26,27] in addition to those described by Huberty [13] and Vacha-Haase and Thompson [34]. As shown in this article, SES effectively takes into account data variability and the accuracy of its estimation is robust to sample size. The value of SES for the contrast with the same coefficients is comparable across different experiments. The drawback is that the value of SES is not comparable for contrasts with different coefficients and timing a constant to a contrast may change the value of SES. The SES after transforming the coefficients may solve this drawback but generates new problems with the merging of factor levels. As to the correlational effect sizes, like SMCV, both contrast γ and effect size γ have good properties that they capture data variability effectively and they are robust to sample size and coefficients. alerting γ does not represent the size of relationship effectively due to lack of ability in capturing data variability and BESD γ does not work effectively because it is not robust to sample size (Table 3). have a drawback that all variance-accounted-for effect sizes have. As Hedges and Olkin [11] pointed out, variance-accounted-for effect sizes are intuitively appealing but are not well suited for combination across studies because these indices are inherently nondirectional, and can have the same value even though the research studies exhibit substantively different results.  In summary, the concepts of contrast variable and associated terms of SMCV and c + -probability can link together the most commonly used probabilistic index for effect sizes such as McGraw and Wong's CL [21] and the most commonly used ratios of mean difference to variability such as Cohen's d [5]. A contrast variable can provide both a probabilistic meaning and an index of signal-tonoise ratio to interpret the strength of a comparison, which offers us a strong base to classify the strength of a comparison as shown in Panel A of Table 1. SMCV and c + -probability also give interpretations to both Cohen's and McLean's criteria [5,22]. The contrast variable, SMCV and c + -probability works effectively and consistently for either relationship or group comparison in either independent or correlated situations and in either two or more than 2 groups. Treatment effect, main effect, interaction effect, linear relationship, quadratic relationship and any other contrasts can all be addressed consistently using contrast variables. The examples in the article show that the results reached using contrast variables and the classifying rule are sensible and matched with observations and intuitions from the data. Therefore, contrast variable, SMCV, c + -probability and associated classifying rules may have the potential to offer a consistent interpretation to effect sizes.