LambdaMART
LambdaMART is a state-of-the-art Learning to Rank (LTR) algorithm that represents the evolution of ranking models developed at Microsoft Research. It is the boosted tree version of LambdaRank, which in turn is based on RankNet. The name is a combination of 'Lambda' (from LambdaRank) and 'MART' (Multiple Additive Regression Trees), which is another name for Gradient Boosted Decision Trees.
Technical Overview
The primary challenge in Information Retrieval is that standard evaluation metrics like NDCG (Normalized Discounted Cumulative Gain) and Mean Average Precision (MAP) are either flat or discontinuous with respect to model parameters, making them difficult to optimize using Gradient Descent. LambdaMART solves this by bypasssing the requirement of a differentiable loss function. Instead, it defines 'lambdas'—gradients that specify the direction and magnitude of change required for each item in a ranked list to improve the overall list score.
Implementation and Performance
The algorithm was popularized after winning the Yahoo Learning to Rank Challenge in 2010. It is widely implemented in major machine learning libraries such as LightGBM and XGBoost. By utilizing an ensemble of Decision Trees, LambdaMART can capture non-linear relationships between features, making it highly effective for complex search tasks in Bing and other modern search engines.
For further reading, see the original research paper From RankNet to LambdaRank to LambdaMART: An Overview by Christopher Burges, or the Wikipedia entry on ranking models.