DNA microarray technique has the potential to provide a more accurate and objective cancer diagnosis than traditional histopathological approach with its high throughput capability of simultaneously measuring relative expression level of tens of thousands of genes. The success, however, greatly depends upon the supervised learning algorithm selected to classify gene expression data.

Many well-established methods are available for gene expression profile classification. According to Lee et al (2005), they can be classified into four categories: (1) classical methods, such as Fisher’s linear discriminant analysis, logistic regression, K-nearest neighbor, and generalized partial least square, (2) classification trees and aggregation methods, such as CART, random forest, bagging and boosting, (3) machine learning methods, such as neural network and support vector machines (SVMs), and (4) generalized methods, such as flexible discriminant analysis, mixture discriminant analysis, and shrunken centroid method. The performance of many methods, however, relies upon careful choice of model parameters, which can be done via model selection procedure such as cross validation. For example, the model parameters for SVMs include kernel parameters and the penalty parameter C. A recent controversy regarding the performance comparison between SVM and random forest just exemplifies the importance of model selection. The study by Diaz-Uriarte et al. (2006) concludes that random forest outperforms SVM, and the conclusion in paper (Stanikov et al, 2008) is totally opposite. The main difference between these two studies is that model selection is carefully designed in the latter study but not in the former study. The incident also shows that model selection may be the obstacle of the extensive application of SVM in classification of gene expression profile. Since classification performance is a nonconvex function of model parameters, it is usually difficult to find optimal model parameters by model selection.

Least square for classification, on the other hand, has no need of model selection. Consider a general classification problem with N classes. A linear model is built for each class k

Y_k = W^T_k X + W_k0, k = 1,2,..., N.      (1)

The N equations can be grouped into

y=Wx˜         (2)

where y = [y₁, y₂,...,y_N]^T, W is a matrix whose kth row is [ w^T_k, w_k0 ], and x˜ =[x^T,1]^T. For a training dataset {(x_i , t_i ),i= 1, 2,....,n}, where t_i is 1-of-N binary coding vector of the label of the ith feature x_i , i.e., a vector containing zeros everywhere except 1 in the kth position, if x_i belongs to category k. Denote by X the feature matrix whose kth row is [x^T_k ,1] , and T the target matrix whose kth row is t^T_i.The linear regression model in (2) can be fitted simultaneously to each of columns of T, and the solution is in the form

Ŵ = (X^TX)^-1X^TT      (3)

• Calculate the fitted output ŷ = Ŵ[x^T,1]^T(an N-dimensional vector);

• Label = argmax_k ŷ(k), k = 1,2,.. .,N. More details can be found in literature (Bishop, 2006; Hastie et al., 2001).

The above approach, however, is very sensitive to outliers, especially for multicategory classification (N ≥ 3). Furthermore, when least square for classification is applied to gene expression data, problems can become more severe due to the curse of dimensionality caused by the great number of genes in each sample.

Inspired by the recent progress in sparse signal recovery via l₁ – norm minimization (Candès et al., 2006, Candès and Tao, 2006; Donoho, 2006), we propose a new approach to overcome the major drawback of least square for classification by casting the linear regression problem as a constrained l₁ – norm minimization problem. The obtained sparse solution is much less sensitive to both outliers and curse of dimensionality. In addition, multicategory classification is realized via one-versus-rest (OVR) and one-versus- one (OVO) approaches which decompose the original multi-category problem into a series of binary problems. The new method is validated by comparing caner diagnosis performance with SVMs.

The success of l₁ least square may lie in its sparse linear model coefficient vector obtained from l₁ – norm minimization. Figure 1 shows the model coefficient vector w which is the solution of l₁ least square for classifying binary dataset DLBCL. The sparsity suggests that those genes with greater absolute coefficients could have played more important roles in classification. As a result, the classification performance does not depend on all the genes, especially those with very small absolute coefficients. The sparsity has the potential to greatly alleviate curse of dimensionality and increase the robustness to outliers.

Another implication of sparsity is that those genes with larger absolute coefficients may correspond to biological markers. Hence, sparsity could be also used for gene selection. We did a small experiment to verify this possibility. The binary dataset DLBCL is used to fit l₁ least square model. Gene selection is done by choosing M genes with M largest absolute coefficients. Binary SVM is used to classify the gene-selected data. The results are compared with KW and BW methods for gene selection. Figure 2 shows the performance of the three gene selection methods for M =10, 20, 30, 40, and 50, respectively. The new method significantly outperforms both KW and BW methods when a small number of genes are selected.

The above gene selection approach is in spirit similar to lasso (Tibshirani, 1996) formulated as follows

min || Xw - y ||²₂ subject to || w ||₁ ≤ t      (17)

where X, w, and y follow the definitions given in section 2.1 for binary l₁ least square, and t is the model parameter for lasso. In addition, lasso can also be used in classification by replacing (7) with (17) for binary case, (10) with

min || Xw_k - y_k ||²₂ subject to || w_k ||₁ ≤ t_k      (18)

for multi-category OVR approach, and (14) with

min || Xw_i,j - y_i,j ||²₂ subject to || w_i,j ||₁ ≤ t_i,j     (19)

for multi-category OVO approach.

Similarly, we can also replace l₁ least square regression with Dantzig selector (Candès and Tao, 2007), which is given below for binary classification

min || w || subject to || X^T(y - Xw) ||_∞ ≤ ( 1 + t^-1) √(2logd.σ)      (20)

where t is model parameter, ands is the noise standard deviation. Dantzig selector for multicategory classification can be similarly defined.

Both lasso and Dantzig selector for classification, however, still need to select optimized model parameters by model selection procedure, such as cross validation.

In this paper, we have described a specialized regression method for cancer diagnosis using expression data. The new approach, called l₁ least square, casts linear regression as a constrained l₁-norm minimization problem to overcome the major drawback of least square for classification: lack of robustness to outliers. Besides binary classifier, multicategory l₁ least square including OVO and OVR approaches are also proposed.

Numerical experiment shows that OVR- l₁ least square can match the best performance achieved by SVMs with careful model selection. The main advantage of l₁ least square over other methods including SVMs is that it has no need of model selection. As a result, the method based on l₁ least square is totally automatic. l₁ least square also has the potential to be used for gene selection.

The l₁ least square classifier may become a promising automatic cancer diagnosis tool by consistently distinguishing gene profile classes. Those genes with great absolute regression coefficients may serve as biological marker candidates for further investigation.