In year 2004 Microsoft took the initiative to improve usage of web search results. The system used that time was called Flying Dutchman. The Flying Dutchman took several days and a cluster to produce a model (to rank web search results). Performance optimization leads to the origination of a new neural net ranker system called RankNet which took an hour to produce a ranking model using one machine. Enhancements leads to finding of a family of models called Boosted Decision Trees (BDTs) that easily solved different kinds of predictive problems like:
1. Ranking Problems for example, the most relevant web search results at the top of the list
2. Classification Problems for example, for determining if a particular email is spam or not
3. Regression Problems for example, for predicting what price your house might sell
** Note: Flavours of ensembles of decision trees, and boosting is used almost to every type to add additional efficiency to prediction
Theory applies to how Bing work. When a query is issued to Bing before applying ML it scan the documents in created index, A large number of candidate documents are weeded out by applying some very fast filters (e.g. skip the documents that have no words in common in respect to the applied query). Preparation of the ranking model includes preparation of this list of features and map it to a single score that encodes the relevance of that document for that query. Several metrices (for example: NDCG) are used to measure the quality.
Findings showcase a long journey of research, experience and learnings. BDT for preparing ranking model is applied in engineering of Bing.