您的位置:首页 > 移动开发

Thinking about a paper "A Refinement Approach to Handling Model Misfit in Text Categorization"

2006-11-30 19:01 447 查看
in this paper, i think the most interesting thing is why there is no overfitting? overfitting is inevitable when training on training examples too much. because decision line(surface) fits specified property owned exclusively by these training examples, generality has been lost. in this paper, the authors use a refinement on classifier(Simon Haykin's book <<Neural Networks: A Comprehensive Foundation>> sec 7.5 shows a similar example: boosting by filtering. but difference should be noticed: this paper's method don't vote, it is not a boosting) to improving classifier performance. in the view of decision line, whatever refinement it is, the effect must be adjusted decision line to fitting training examples better. surprising experiments results shows there is no overfitting, authors just show this fact without explanation.
how could this be? i have noticed that this method generates decision line is different with traditonal line: you can imagine this "line": root classifier correspond to main line, its children classifers correspond to two smaller lines near the main line and so on... it is similar to fractal!! i guess these decision lines correctly express the STRUCTURE of target(real) decision, because i believe fractal is the nature of nature. but the problem still remained, ok, structure, so what? i guess:
1 structure is more general than line, however this structure is obtained by training examples but it still has more generality than line. so overfitting is alleviated.
2 to see gobal from local, this is property of fractal, similarity. training examples decision structure is similar to all examples. however regions contain arbitrary +,-.
3 this method classifier don't vote, the classification is only determined by leaves nodes, intermidate nodes only do the dispatch job: translate input to a leaves node. this behavior contradict intuition and hard to explaining. leaves nodes should be specified version of intermidate nodes, but it replaces its parents judgements without too much error.
may be each input is specified input, so correct classification in intermidate nodes can't be viewed as a "perfect classification", it just a "estimate on imperfection".
further work is how to take advantage of this character of "decision line" if i guess it right.
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: 
相关文章推荐