机器学习常用工具

Support Vector Machine

SVMlight

An implementation of Vapnik's Support Vector Machine

LIBSVM

A Library for Support Vector Machines

Decision Tree

C4.5

The "classic" decision-tree tool, developed by J. R. Quinlan Tutorial

Maximum Entropy

YASMET

Yet Another Small MaxEnt Toolkit

Conditional Random Field

CRF++

A simple, customizable, and open source implementation of Conditional Random Fields (CRFs) for segmenting/labeling sequential data

自然语言处理

综合

OpenNLP

An organizational center for open source projects related to natural language processing

CMU Statistical Language
Modeling Toolkit

A suite of UNIX software tools to facilitate the construction and testing of statistical language models

The Dragon ToolKit

A Java-based development package for academic use in information retrieval (IR) and text mining. Include many NLP tools

LingPipe

A suite of Java libraries for the linguistic analysis of human language, including

track mentions of entities (e.g. people or proteins);
link entity mentions to database entries;
uncover relations between entities and actions;
classify text passages by language, character encoding, genre, topic, or sentiment;
correct spelling with respect to a text collection;
cluster documents by implicit topic and discover significant trends over time; and
provide part-of-speech tagging and phrase chunking.

Natural Language Toolkit

Open source Python modules, linguistic data and documentation for research and development in natural language processing and text analytics, with distributions for Windows, Mac OSX and Linux.

Antelope

Advanced Natural Lange Object-oriented Processing Environment.包括一系列工具(特别c#的stanford parser)

分词

ICTCLAS

中科院的中文分词系统

Stanford Chinese Word
Segmenter

A Java implementation of a CRF-based Chinese Word Segmenter

词性标注

Brill tagger

A error-driven transformation-based tagger implemented by Eric Brill

Stanford POS Tagger

A Java implementation of the log-linear part-of-speech taggers descriped by Kristina Toutanova, et.al.

MBT:Memory-based Tagger
TreeTagger

A decision tree based tagger from the University of Stuttgart.

SVMTool , a POS Tagger based on SVMs
QTAG
Part of speech tagger

An HMM-based Java POS tagger from Birmingham U.

命名实体识别

Stanford Named Entity Recognizer

A Java implementation of a Conditional Random Field sequence model, together with well-engineered features for Named Entity Recognition

LingPipe

Tools include statistical named-entity recognition, a heuristic sentence boundary detector, and a heuristic within-document coreference resolution engine. Java. GPL. By Bob Carpenter, Breck Baldwin and co.

YamCha

SVM-based NP-chunker, also usable for POS tagging, NER, etc. C/C++ open source. Won CoNLL 2000 shared task. (Less automatic than a specialized POS tagger for an end user.)

Stemming

Porter Stemming

A process for removing the commoner morphological and inflexional endings from words in English byMartin
Porter

Snowball

A small string processing language designed for creating stemming algorithms for use in Information Retrieval.

句法分析

Stanford Parser

Java implementations of probabilistic natural language parsers, both highly optimized PCFG and dependency parsers, and a lexicalized PCFG parser.

Berkeley Parser

文本挖掘

摘要

Rouge Rouge在Windows下的配置

其他

加密

OpenSSL

包括众多加密算法，RSA、DES、MD5、SHA等 Win32安装版

压缩

zlib

A Massively Spiffy Yet Delicately Unobtrusive Compression Library

日志

Apache Logging Services

Creates and maintains open-source software related to the logging of application behavior and released at no charge to the public, including

log4j for Java,
log4cxx for C++, and
log4net for MS .Net framework.

注: log4cxx官方版本有内存泄漏问题

Unicode

ICU

A mature, widely used set of C/C++ and Java libraries providing Unicode and Globalization support for software applications

XML

Xerces

A validating XML parser, including C and Java edition

多字符串匹配

AC in C# : Aho-Corasick string
matching in C#

HTML Parser

Html Agility Pack , an agile HTML parser
that builds a read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files.
Majestic-12 ,
an open source high-performance .NET C# module that was created to parse HTML for links, indexing and other purposes. 速度快，但不生成dom树

外部联接

An annotated list of resources by
Stanford NLP Group
KDnuggets 有一些与KDD相关的软件等