展示廣告點(diǎn)擊率預(yù)估方法研究

1、本文格式為Word版，下載可任意編輯 展示廣告點(diǎn)擊率預(yù)估方法研究 Abstract With the rapid development of the Internet, online advertising acts a pivotal part in the Internet in our daily life and it has become the most popular approach to do brand promotion and product marketing for the advertiser. Accurate click-through rate (CTR)

2、 prediction is the most important part of online advertising. Improving the accuracy of the ads CTR estimation can not only benefit to advertisers, but also improve user experience. Many traditional click through rate prediction methods, such as logistic regression, have been applied to advertising

3、click rate prediction system and achieved good results. Furthermore, it has been large-scale deployed in the industry. Recently, the deep learning technology has achieved great success in multiply fields of Natural Language Processing and Computer Vision, such as Textual Entailment, Text Summarizati

4、on, Image Generation and so on. Meanwhile, a number of deep learning models right now have been used in personalized recommender system and CTR prediction and their model structures are similar. Both of them reduce the dimension of the feature by vectorization, then utilize nonlinear operation to ex

5、tract the feature combination, and calculate nonlinear relationship between the features and the click rate through by neural network. The content of this paper of the following three main aspects: (1) Ensemble Learning by multiple traditional machine learning models based CTR model. We first do fea

6、ture engineering on two large-scale real-world display advertising datasets manually and extract high-order combination feature by GBDT. Then we calculate CTR by mature machine learning models such as logistic regression and factorization machine. Then we utilize ensemble learning base on multiple s

7、ingle models. Finally, we calculate the result of ensemble learning method. (2) Advance deep learning model based CTR model. We use deep neural network and recurrent neural network to do click-through rate prediction. We try to combine the features extracted from feature engineering and get the inpu

8、t of deep neural network through feature hashing and feature connection. Finally, we calculate the result of advance deep learning model. (3) Multi-Embedding deep model based CTR model. We propose a novel CTR predicting model, Multi-Embedding Deep Model. We implement deep neural network based and co

9、nvolutional neural network based traditional multi-embedding deep model, and also implement deep neural network based and convolutional neural network based bilinear multi-embedding deep model. which - II - we utilize bilinear matrix to do feature interactions instead of factorization machines. We d

10、esign a system to address the cold-start problem for static data set by combining clustering method and marking rare embedding vectors method. We evaluate the proposed model on IPinYou and Avazu datasets, two large-scale real-world display advertising datasets. Experimental results show that the mod

11、el can improve the estimation performance of ads click-through rate effectively. Keywords:online advertising, click-through rate, deep learning, convolutional neural network, bilinear - III - 目錄 摘要 I ABSTRACT II 第1章緒論 . (1) 1.1課題的來源及研究的目的和意義 (1) 1.1.1 課題的來源 (1) 1.1.2 課題的研究目的和意義 (1) 1.2國內(nèi)外研究現(xiàn)狀 (2) 1.

12、2.1 基于機(jī)器學(xué)習(xí)的點(diǎn)擊率預(yù)估模型研究現(xiàn)狀 (2) 1.2.2 基于深度學(xué)習(xí)的點(diǎn)擊率預(yù)估模型研究現(xiàn)狀 (4) 1.3數(shù)據(jù)集與問題定義 (5) 1.3.1 數(shù)據(jù)集描述 (5) 1.3.2 點(diǎn)擊率預(yù)估的問題定義 (9) 1.3.3 點(diǎn)擊率預(yù)估的評價指標(biāo) (9) 1.3.4 基線系統(tǒng)選擇 (11) 1.4本文的主要研究內(nèi)容 (13) 1.5本文內(nèi)容安排 (14) 第2章基于模型融合的點(diǎn)擊率預(yù)估研究 (15) 2.1引言 (15) 2.2單模型點(diǎn)擊率預(yù)估 (15) 2.2.1 GBDT高階特征組合模型 (15) 2.2.2 FM點(diǎn)擊率預(yù)估模型 (18) 2.2.3 FFM點(diǎn)擊率預(yù)估模型 (19) 2

13、.3集成學(xué)習(xí)點(diǎn)擊率預(yù)估 (20) 2.3.1 強(qiáng)模型融合 (20) 2.3.2 機(jī)器學(xué)習(xí)元算法 (21) 2.4基于模型融合的點(diǎn)擊率預(yù)估模型 (23) 2.5試驗(yàn)結(jié)果與分析 (24) 2.5.1 模型參數(shù)設(shè)置 (24) 2.5.2 試驗(yàn)結(jié)果比較分析 (25) 2.6本章小結(jié) (27) - IV - 第3章基于深度學(xué)習(xí)的點(diǎn)擊率預(yù)估研究 (29) 3.1引言 (29) 3.2基于傳統(tǒng)深度模型的點(diǎn)擊率預(yù)估研究 (29) 3.2.1 激活函數(shù) (29) 3.2.2 Dropout (30) 3.2.3 Batch Normalization (31) 3.2.4 反向傳播算法 (33) 3.2.5 基

14、于傳統(tǒng)深度神經(jīng)網(wǎng)絡(luò)的點(diǎn)擊率預(yù)估模型 (34) 3.3基于循環(huán)神經(jīng)網(wǎng)絡(luò)的點(diǎn)擊率預(yù)估研究 (34) 3.3.1 循環(huán)神經(jīng)網(wǎng)絡(luò) (35) 3.3.2 長短期記憶網(wǎng)絡(luò) (36) 3.3.3 門控循環(huán)單元 (37) 3.3.4 雙向循環(huán)神經(jīng)網(wǎng)絡(luò) (38) 3.3.5 基于時間的反向傳播算法 (39) 3.3.6 基于循環(huán)神經(jīng)網(wǎng)絡(luò)的點(diǎn)擊率預(yù)估模型 (40) 3.4淺層特征與深層特征結(jié)合的點(diǎn)擊率預(yù)估模型 (41) 3.5試驗(yàn)結(jié)果與分析 (41) 3.5.1 模型參數(shù)設(shè)置 (42) 3.5.2 試驗(yàn)結(jié)果比較分析 (42) 3.6本章小結(jié) (46) 第4章基于MULTI-EMBEDDING的點(diǎn)擊率預(yù)估研究 (48) 4.1引言 (48) 4.2卷積神經(jīng)網(wǎng)絡(luò)相關(guān)技術(shù)研究 (48) 4.2.1 卷積層 (48) 4.2.2 池化層 (50) 4.3雙線性特征組合 (51) 4.4冷啟動問題模型 (52) 4.5基于傳統(tǒng)M ULTI-E MBEDDING的點(diǎn)擊率預(yù)估模型 (53) 4.5.1 基于深度神經(jīng)網(wǎng)絡(luò)的傳統(tǒng)Multi-Embedding點(diǎn)擊率預(yù)估模型 (53) 4.5.2 基于卷積神經(jīng)網(wǎng)絡(luò)的傳統(tǒng)Multi-Em