中文分词资料

1，ictcas 包括Java，LinuxC， WindowsC 的版本均在 http://w http:// ww.ictclas.org/index.html 有下载。

2，imdict-chinese-analyzer 是 imdict智能词典 的智能中文分词模块，作者高小平，算法基于隐马尔科夫模型(Hidden
Markov Model, HMM)，是中国科学院计算技术研究所的ictclas中文分词程序的重新实现（基于Java），可以直接为lucene搜索引擎提供中文分词支持。 也可以在 http://www.ictclas.org/index.html 下载。

3，LingPipe is a suite of Java libraries for the linguistic analysis of human language. http://alias-i.com/lingpipe/index.html。 这个工具中的分词部分中，可以通过学习形成模型，或者从网站上下载模型。

4，MMSEG: A Word Identification System for Mandarin Chinese Text Based on Two Variants of the Maximum Matching Algorithm

5，Lucene 中文分词

6，开源中国社区中文分词
7，Microsoft Research S-MSRSeg

2012

Fast Online Training with Frequency-Adaptive Learning Rates for Chinese Word Segmentation and New Word Detection
Sun, Xu and Wang, Houfeng and Li, Wenjie
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics {(Volume} 1: Long Papers)

Reducing Approximation and Estimation Errors for Chinese Lexical Processing with Heterogeneous Annotations
Sun, Weiwei and Wan, Xiaojun
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics {(Volume} 1: Long Papers)

Capturing Paradigmatic and Syntagmatic Lexical Relations: Towards Accurate Chinese Part-of-Speech Tagging
Sun, Weiwei and Uszkoreit, Hans
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics {(Volume} 1: Long Papers)

Joint Chinese Word Segmentation, {POS} Tagging and Parsing
Qian, Xian and Liu, Yang
Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

Unified Dependency Parsing of Chinese Morphological and Syntactic Structures
Li, Zhongguo and Zhou, Guodong
Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

Iterative Annotation Transformation with Predict-Self Reestimation for Chinese Word Segmentation
Jiang, Wenbin and Meng, Fandong and Liu, Qun and Lü, Yajuan
Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

Incremental Joint Approach to Word Segmentation, {POS} Tagging, and Dependency Parsing in Chinese
Hatori, Jun and Matsuzaki, Takuya and Miyao, Yusuke and Tsujii, Jun'ichi
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics {(Volume} 1: Long Papers)

2011

Improving Chinese Word Segmentation and {POS} Tagging with Semi-supervised Methods Using Large Auto-Analyzed Data
Wang, Yiou and Kazama, Jun'ichi and Tsuruoka, Yoshimasa and Chen, Wenliang and Zhang, Yujie and Torisawa, Kentaro
Proceedings of 5th International Joint Conference on Natural Language Processing

Enhancing Chinese Word Segmentation Using Unlabeled Data
Sun, Weiwei and Xu, Jia
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

A Stacked Sub-Word Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging
Sun, Weiwei
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

Parsing the Internal Structure of Words: A New Paradigm for Chinese Word Segmentation
Li, Zhongguo
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

Syntactic Processing using the Generalized Perceptron and Beam Search
Zhang, Y. and Clark, S.
Computational Linguistics

A New Unsupervised Approach to Word Segmentation
Wang, H. and Zhu, J. and Tang, S. and Fan, X.
Computational Linguistics

2010

A Fast Decoder for Joint Word Segmentation and {POS-Tagging} Using a Single Discriminative Model
Zhang, Yue and Clark, Stephen
Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing

Joint Tokenization and Translation
Xiao, Xinyan and Liu, Yang and Hwang, {YoungSook} and Liu, Qun and Lin, Shouxun
Proceedings of the 23rd International Conference on Computational Linguistics {(Coling} 2010)

A Character-Based Joint Model for Chinese Word Segmentation
Wang, Kun and Zong, Chengqing and Su, Keh-Yih
Proceedings of the 23rd International Conference on Computational Linguistics {(Coling} 2010)

A Local Generative Model for Chinese Word Segmentation
Zhang, K. and Sun, M. and Xue, P.
Information Retrieval Technology

Joint training and decoding using virtual nodes for cascaded segmentation and tagging tasks
Qian, X. and Zhang, Q. and Zhou, Y. and Huang, X. and Wu, L.
Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing

2009

Automatic Adaptation of Annotation Standards: Chinese Word Segmentation and {POS} Tagging – A Case Study
Jiang, Wenbin and Huang, Liang and Liu, Qun
Proceedings of the 47th {ACL}

Character-Level Dependencies in Chinese: Usefulness and Learning
Zhao, Hai
Proceedings of the 12th Conference of the European Chapter of the {ACL} {(EACL} 2009)

基于字依存树的中文词法-句法一体化分析
赵, 海 and 揭, 春雨 and 宋, 彦
中国计算机语言学研究前沿进展 (2007-2009)

基于 {CRFs} 的中文分词和短文本分类技术
滕, 少华

A Simple and Efficient Model Pruning Method for Conditional Random Fields
Zhao, H. and Kit, C.

Chinese text segmentation: A hybrid approach using transductive learning and statistical association measures
Tsai, R. T. H.
Expert Systems with Applications

Bayesian Unsupervised Word Segmentation with Nested Pitman-Yor Language Modeling
Mochihashi, Daichi and Yamada, Takeshi and Ueda, Naonori
Proceedings of the Joint Conference of the 47th Annual Meeting of the {ACL} and the 4th International Joint Conference on Natural Language Processing of the {AFNLP}

Punctuation as Implicit Annotations for Chinese Word Segmentation
Li, Zhongguo and Sun, Maosong
Computational Linguistics

An Error-Driven Word-Character Hybrid Model for Joint Chinese Word Segmentation and {POS} Tagging
Kruengkrai, Canasai and Uchimoto, Kiyotaka and Kazama, Jun'ichi and Wang, Yiou and Torisawa, Kentaro and Isahara, Hitoshi
Proc. of {ACL-IJCNLP} 2009

2008

Word Lattice Reranking for Chinese Word Segmentation and Part-of-Speech Tagging
Jiang, Wenbin and Mi, Haitao and Liu, Qun
Proceedings of the 22nd International Conference on Computational Linguistics {(Coling} 2008)

Joint Word Segmentation and {POS} Tagging Using a Single Perceptron
Zhang, Yue and Clark, Stephen
Proceedings of {ACL-08:} {HLT}

A Cascaded Linear Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging
Jiang, Wenbin and Huang, Liang and Liu, Qun and Lü, Yajuan
Proceedings of {ACL-08:} {HLT}

Unsupervised segmentation helps supervised learning of character tagging for word segmentation and named entity recognition
Zhao, Hai and Kit, Chunyu
The Sixth {SIGHAN} Workshop on Chinese Language Processing

An Empirical Comparison of Goodness Measures for Unsupervised Chinese Word Segmentation with a Unified Framework
Zhao, Hai and Kit, Chunyu
The Third International Joint Conference on Natural Language Processing {(IJCNLP-2008)}, Hyderabad, India

Bayesian semi-supervised chinese word segmentation for statistical machine translation
Xu, J. and Gao, J. and Toutanova, K. and Ney, H.
Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1

Statistical Properties of Overlapping Ambiguities in Chinese Word Segmentation and a Strategy for Their Disambiguation
Qiao, W. and Sun, M. and Menzel, W.
Text, Speech and Dialogue

Information retrieval oriented word segmentation based on character associative strength ranking
Liu, Y. and Wang, B. and Ding, F. and Xu, S.
Proceedings of the Conference on Empirical Methods in Natural Language Processing

2007

Chinese Segmentation with a Word-Based Perceptron Algorithm
Zhang, Yue and Clark, Stephen

基于有效子串标注的中文分词
赵, 海 and 揭, 春雨
中文信息学报

中文分词十年回顾
黄, 昌宁 and 赵, 海
中文信息学报

A dual-layer {CRFs} based joint decoding method for cascaded segmentation and labeling tasks
Shi, Y. and Wang, M.
Proceedings of {IJCAI}

A hybrid approach to word segmentation and pos tagging
Nakagawa, Tetsuji and Uchimoto, Kiyotaka
{ANNUAL} {MEETING-ASSOCIATION} {FOR} {COMPUTATIONAL} {LINGUISTICS}

Rethinking Chinese word segmentation: tokenization, character classification, or wordbreak identification
Huang, Chu-Ren and Simon, Petr and Hsieh, Shu-Kai and Prévot, L.
Proceedings of the 45th Annual Meeting of the {ACL} on Interactive Poster and Demonstration Sessions

2006

Subword-Based Tagging for Confidence-Dependent Chinese Word Segmentation
Zhang, Ruiqiang and Kikui, Genichiro and Sumita, Eiichiro
Proceedings of the {COLING/ACL} 2006 Main Conference Poster Sessions

汉语词典的快速查询算法研究
李, 江波 and 周, 强 and 陈, 祖舜
中文信息学报

An improved Chinese word segmentation system with conditional random field
Zhao, H. and Huang, C. N. and Li, M.
Proceedings of the Fifth {SIGHAN} Workshop on Chinese Language Processing

Discriminative pruning of language models for Chinese word segmentation
Li, J. and Wang, H. and Ren, D. and Li, G.
Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics

Contextual Dependencies in Unsupervised Word Segmentation
Goldwater, Sharon and Griffiths, Thomas L. and Johnson, Mark
Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics

2005

Chinese Word Segmentation and Named Entity Recognition: A Pragmatic Approach
Gao, Jianfeng and Li, Mu and Huang, Chang-Ning and Wu, Andi
Computational Linguistics

A conditional random field word segmenter for sighan bakeoff 2005
Tseng, H. and Chang, P. and Andrew, G. and Jurafsky, D. and Manning, C.
Proceedings of the Fourth {SIGHAN} Workshop on Chinese Language Processing

Perceptron Learning for Chinese Word Segmentation
Li, Y. and Miao, C. and Bontcheva, K. and Cunningham, H.
Proceedings of Fourth {SIGHAN} Workshop on Chinese Language processing {(Sighan-05)}

The second international chinese word segmentation bakeoff
Emerson, Thomas
Proceedings of the Fourth {SIGHAN} Workshop on Chinese Language Processing

A Statistic Study of Three-character Unknown Words in Chinese
Duan, {ZWXZH}
Journal of Chinese Language and Computing

2004

Chinese Segmentation and New Word Detection using Conditional Random Fields
Peng, Fuchun and Feng, Fangfang and {McCallum}, Andrew
Proceedings of Coling 2004

Chinese Part-of-Speech Tagging: One-at-a-Time or All-at-Once? Word-Based or Character-Based?
Ng, Hwee Tou and Low, Jin Kiat
Proceedings of {EMNLP} 2004

基于无指导学习策略的无词表条件下的汉语自动分词
孙, 茂松 and 肖, 明 and 邹, 嘉彦
计算机学报

Applying conditional random fields to Japanese morphological analysis
Kudo, T. and Yamamoto, K. and Matsumoto, Y.
Proc. of {EMNLP}

Adaptive Chinese word segmentation
Gao, J. and Wu, A. and Li, M. and Huang, C. N. and Li, H. and Xia, X. and Qin, H.
Proceedings of {ACL-2004}

Unsupervised segmentation of Chinese corpus using accessor variety
Feng, Haodi and Chen, Kang and Kit, Chunyu and Deng, Xiaotie
Natural Language Processing {IJCNLP} 2004

Accessor variety criteria for Chinese word extraction
Feng, Haodi and Chen, Kang and Deng, Xiaotie and Zheng, Weimin
Computational Linguistics

2003

{HHMM-based} Chinese lexical analyzer {ICTCLAS}
Zhang, H. P. and Yu, H. K. and Xiong, D. Y. and Liu, Q.
Proceedings of the second {SIGHAN} workshop on Chinese language processing-Volume 17

Chinese lexical analysis using hierarchical hidden markov model
Zhang, H. P. and Liu, Q. and Cheng, X. Q. and Zhang, H. and Yu, H. K.
Proceedings of the second {SIGHAN} workshop on Chinese language processing-Volume 17

Chinese Word Segmentation as {LMR} Tagging
Xue, Nianwen and Shen, Libin
Proceedings of the second {SIGHAN} workshop on Chinese language processing-Volume 17

Chinese Word Segmentation as Character Tagging
Xue, Nianwen
Computational Linguistics and Chinese Language Processing

The first international Chinese word segmentation bakeoff
Sproat, R. and Emerson, T.
Proceedings of the second {SIGHAN} workshop on Chinese language processing

A maximum entropy Chinese character-based parser
Luo, X.

Improved source-channel models for Chinese word segmentation
Gao, J. and Li, M. and Huang, C. N.
Proceedings of the 41st Annual Meeting on Association for Computational Linguistics

Chinese word segmentation using minimal linguistic knowledge
Chen, A.
Proceedings of the second {SIGHAN} workshop on Chinese language processing

Combining segmenter and chunker for Chinese word segmentation
Asahara, M. and Goh, C. L. and Wang, X. and Matsumoto, Y.
Proceedings of the 2nd {SIGHAN} Workshop on Chinese Language Processing

2002

Combining classifiers for Chinese word segmentation
Xue, Nianwen and Converse, Susan
Proceedings of the 1st {SIGHAN} Workshop on Chinese Language Processing

Corpus-based methods in Chinese morphology
Sproat, R. and Shih, C.
Tutorial at the 19th {COLING}

Corpus-based methods in Chinese morphology and phonology
Sproat, R. and Shih, C.
{COOLING} 2002

2001

汉语自动分词研究评述
孙, 茂松 and 邹, 嘉彦
当代语言学

Defining and automatically identifying words in Chinese
Xue, Nianwen

Self-supervised Chinese word segmentation
Peng, F. and Schuurmans, D.
Advances in Intelligent Data Analysis

2000

A compression-based algorithm for Chinese word segmentation
Teahan, W. J. and {McNab}, Rodger and Wen, Yingying and Witten, Ian H.
Comput. Linguist.

1999

Discovering Chinese words from unsegmented text (poster abstract)
Ge, X. and Pratt, W. and Smyth, P.
Proceedings of the 22nd annual international {ACM} {SIGIR} conference on Research and development in information retrieval

1998

串频统计和词形匹配相结合的汉语自动分词系统
刘, 挺 and 吴, 岩
中文信息学报

Chinese Word Segmentation without Using Lexicon and Hand-crafted Training Data
Sun, Maosong and Shen, Dayang and Tsou, Benjamin K
Proceedings of the 17th international conference on Computational linguistics-Volume 2

A hybrid approach to word segmentation
Kazakov, D. and Manandhar, S.
Lecture notes in computer science

1997

中文信息处理中的分词问题
黄, 昌宁
Applied Linguistics

An unsupervised iterative method for Chinese new lexicon extraction
Chang, J. S and Su, K. Y
International Journal of Computational Linguistics \& Chinese Language Processing

1996

A stochastic finite-state word-segmentation algorithm for Chinese
Sproat, R. and Gale, W. and Shih, C. and Chang, N.
Computational Linguistics

Useg: A retargetable word segmentation procedure for information retrieval
Ponte, J. M. and Croft, W. B.
Symposium on Document Analysis and Information Retrieval

1992

An efficient implementation of trie structures
Aoe, {Jun‐Ichi} and Morimoto, Katsushi and Sato, Takashi
Software: Practice and Experience

Word identification for Mandarin Chinese sentences
Chen, K. J and Liu, S. H
Proceedings of the 14th conference on Computational linguistics-Volume 1