[阅读笔记]Introduction to Boosted Trees

概念(要学什么)

回归树(CART)

回归树emsemble

模型

$$y_i = \sum_{k=1}^{K}f_k(x_i) , f \in F$ $

参数

GB(如何学习)

additve training(boosting)

L(t)=∑i=1nl(yi,ŷ (t−1)i+ft(xi))+Ω(ft)

L(t)≈∑i=1n[l(yi,ŷ (t−1)i)+gif(xi)+12h2if2(xi)]+Ω(ft)

很关键的一点

gi=∂ŷ (t−1)il(yi,ŷ (t−1)i)

hi=∂2ŷ (t−1)il(yi,ŷ (t−1)i)

L̃ (t)=∑i=1n[gif(xi)+12h2if2(xi)]+γT+12λ∑j=1Tw2j

Rabit文档

https://github.com/dmlc/rabit/blob/master/doc/guide.md

allreduce和ps的对比

http://hunch.net/?p=151364

陈天奇关于rabit的说明

http://weibo.com/p/1001603801281637563132?pids=Pl_Official_CardMixFeedv6__4&feed_filter=2