模式识别科学发展与现状(1.介绍)

The Science of Pattern Recognition
Achievements and Perspectives
 
Robert P.W. Duin1 and El˙zbieta P_ ekalska2
1 ICT group, Faculty of Electr. Eng., Mathematics and Computer Science
Delft University of Technology, The Netherlands
r.duin@ieee.org
2 School of Computer Science, University of Manchester, United Kingdom
pekalska@cs.man.ac.uk
 
Summary. Automatic pattern recognition is usually considered as an engineering area which focusses on the development and evaluation of systems that imitate or assist humans in their ability of recognizing patterns. It may, however, also be considered as a science that studies the faculty of human beings (and possibly other biological systems) to discover, distinguish, characterize patterns in their environment and accordingly identify new observations. The engineering approach to pattern recognition is in this view an attempt to build systems that simulate this phenomenon. By doing that, scientific understanding is gained of what is needed in order to recognize patterns, in general.
自动模式识别通常被认为是这样的一个工程领域：专注于开发和评价模仿或辅助人类识别模式能力的系统，但是也可能被认为是这样的一门科学：学习人类(或其它生物系统)在所处环境中发现、区别和找出特征从而标识出观察结果的本领。模式识别中工程的观点是试图建立模拟生物识别能力的系统，通过工程中的实践，总的来说，科学上的理解在模式识别中的技术需求方面得到了发展。
 
Like in any science understanding can be built from different, sometimes even opposite viewpoints. We will therefore introduce the main approaches to the science of pattern recognition as two dichotomies of complementary scenarios. They give rise to four different schools, roughly defined under the terms of expert systems, neural networks, structural pattern recognition and statistical pattern recognition.
象任何科学一样，对模式识别的理解能够从不同方向来建立，有时甚至是相反的观点。我们将介绍模式识别科学中的主要方法，即两种不同方向且各有两个不同种类的技术，这些技术产生了四个不同学派，粗略地可以定义为：专家系统，神经网络，结构模式识别和统计模式识别。
 
We will briefly describe what has been achieved by these schools, what is common and what is specific, which limitations are encountered and which perspectives arise for the future. Finally, we will focus on the challenges facing pattern recognition in the decennia to come. They mainly deal with weaker assumptions of the models to make the corresponding procedures for learning and recognition wider applicable. In addition, new formalisms need to be developed.
我们将简要地描述这四个学派的发展成果，它们之间的相同点及不同点，它们各自碰到的局限性及未来发展的展望。最后，我们再来看模式识别在未来几十年所面临的挑战,这个挑战主要是解决在学习和识别更大范围适用性时所碰到的为建立相应处理的模型的脆弱问题。再有就是需要发展新的模式识别形式。
 
1 Introduction
1 介绍
 
We are very familiar with the human ability of pattern recognition. Since our early years we have been able to recognize voices, faces, animals, fruits or inanimate objects. Before the speaking faculty is developed, an object like a ball is recognized, even if it barely resembles the balls seen before. So, except for the memory, the skills of abstraction and generalization are essential to find our way in the world. In later years we are able to deal with much more complex patterns that may not directly be based on sensorial observations.
对于人类的识别能力我们是非常熟悉的。因为我们在早些年就已经会开发识别声音、脸、动物、水果或简单不动的东西的技术了。在开发出说话技术之前，一个象球的东西，甚至看上去只是象个球，就已经可以被识别出来了。所以除了记忆，抽象和推广能力是推进模式识别技术的关键技术。最近几年我们已可以处理更复杂的模式，这种模式可能不是直接基于通过感知器观察出来的。
 
For example, we can observe the underlying theme in a discussion or subtle patterns in human relations. The latter may become apparent, e.g. only by listening to somebody’s complaints about his personal problems at work that again occur in a completely new job. Without a direct participation in the
events, we are able to see both analogy and similarity in examples as complex as social interaction between people. Here, we learn to distinguish the pattern from just two examples.
例如，我们能够观察发现某个讨论会的中心议题或人与人之间关系的微妙的模式。后面一种模式是可能可以被明显观察到，例如倾听某人在新的工作中因人际关系问题而产生的抱怨，我们不用切身其中就能够发现这种相似和相同的例子，其复杂性莫过于人与人之间的社会相互影响。这里我们要学会区分只是从两个例子中得到的模式。
 
The pattern recognition ability may also be found in other biological systems:the cat knows the way home, the dog recognizes his boss from the footsteps or the bee finds the delicious flower. In these examples a direct connection can be made to sensory experiences. Memory alone is insufficient; an important role is that of generalization from observations which are similar,although not identical to the previous ones. A scientific challenge is to find out how this may work.
模式识别的能力也可以在其它生物中被发现到：猫可以知道回家的路，狗能够识别主人的脚印，蜜蜂会发现它要采蜜的花。这些例子中每一个直接联结都是通过感观来实现的。不只是记忆方面，推广能力是重要的一方面，从观察到的相似事物中，虽然前后不一样，也能够进行识别，发现动物是怎么做到这一点是一个科学挑战。
 
Scientific questions may be approached by building models and, more explicitly, by creating simulators, i.e. artificial systems that roughly exhibit the same phenomenon as the object under study. Understanding will be gained while constructing such a system and evaluating it with respect to the real object. Such systems may be used to replace the original ones and may even improve some of their properties. On the other hand, they may also perform worse in other aspects. For instance, planes fly faster than birds but are far from being autonomous. We should realize, however, that what is studied in this case may not be the bird itself, but more importantly, the ability to fly.
科学问题可以通过建立模型来解决，更确切的说是建立模拟器，例如人工系统通过学习来粗略地展示具有相同功能的东西，在建立这个系统和取得真实对象相关参数的过程中获得得了对这个事物的理解，这样的系统可以替换原来的对象，甚至可以提高原来的性能，但在其它方面可能是更差。例如，飞机可以飞得比鸟快，但在智能方面却远远不如鸟，然而，我们的研究不是为了达到跟鸟全部一样，更重要的是飞行能力。
 
Much can be learned about flying in an attempt to imitate the bird, but also when differentiating from its exact behavior or appearance. By constructing fixed wings instead of freely movable ones, the insight in how to fly grows.
通过模仿鸟的飞行可以学到很多飞行方面的技术，但无法学到其精确的分辨能力。通过建立固定不动的翅膀，而不是自由扇动的翅膀，我们知道了怎么飞行。
 
Finally, there are engineering aspects that may gradually deviate from the original scientific question. These are concerned with how to fly for a long time, with heavy loads, or by making less noise, and slowly shift the point of attention to other domains of knowledge.
最后，存在希望逐渐从原来的科学问题中引申出来的工程技术，如在重载下怎么飞得更长时间，怎么减少噪音，慢慢地把注意点转移到其它的知识领域。
 
The above shows that a distinction can be made between the scientific study of pattern recognition as the ability to abstract and generalize from observations and the applied technical area of the design of artificial pattern recognition devices without neglecting the fact that they may highly profit from each other. Note that patterns can be distinguished on many levels,starting from simple characteristics of structural elements like strokes, through features of an individual towards a set of qualities in a group of individuals,to a composite of traits of concepts and their possible generalizations. A pattern may also denote a single individual as a representative for its population, model or concept. Pattern recognition deals, therefore, with patterns, regularities,characteristics or qualities that can be discussed on a low level of sensory measurements (such as pixels in an image) as well as on a high level of the derived and meaningful concepts (such as faces in images). In this work, we will focus on the scientific aspects, i.e. what we know about the way pattern recognition works and, especially, what can be learned from our attempts to build artificial recognition devices.
上面表明，模式识别(源于观察的抽象和归纳能力)科学研究和应用技术领域中的人工智能模式识别设备设计存在差别，后者不会放过任何相互间互利的因素。注意这里所说的模式在很多层次上是有区分的，就如结构元素的简单特征（如笔画），体现了从在一组个体中表示某一个性质集的个体特征，到综合概念和归纳的特征。一个模式可能表示成一个单独个体，如某个总体、模型或概念的表示。结合模式、规律、特征或性质，模式识别所做的事可以说是在感观测定的低层次上(如图像的象素)，也可以说是在推理和有意义概念的高层层次上(如图像中的人脸)。这里，我们注重在科学研究方面，如模式识别的实现途径是什么，特别是我们在建立人工识别设备需要具备什么技术。
 
A number of authors have already discussed the science of pattern recognition based on their simulation and modeling attempts. One of the first, in the beginning of the sixties, was Sayre [64], who presented a philosophical study on perception, pattern recognition and classification. He made clear that classification is a task that can be fulfilled with some success, but recognition either happens or not. We can stimulate the recognition by focussing on some aspects of the question. Although we cannot set out to fully recognize an individual, we can at least start to classify objects on demand. The way Sayre distinguishes between recognition and classification is related to the two subfields discussed in traditional texts on pattern recognition, namely unsupervised and supervised learning. They fulfill two complementary tasks. They act as automatic tools in the hand of a scientist who sets out to find the regularities in nature.
已经有些人在讨论基于模拟和建模尝试的模式识别科学了。在开始的六十年里，其中有个叫Sayre的人做了关于感知器、模式识别和分类的哲学研究，他断言分类方法在某些程度上可以被成功实现，但或许也会失败。根据问题的一些情况我们可以进行模拟识别。虽然我们不能完全识别某个个体，但是我们至少可以根据需要把对象分类出来。识别和分类的Sayre区分方法跟模式识别的两个传统的学习方法有关：无监督学习和有监督学习，这个两个方法可以实现识别和分类方法，科学家利用这个自动化工具来发现自然界中的规律。
 
Unsupervised learning (also related to exploratory analysis or cluster analysis) gives the scientist an automatic system to indicate the presence of yet unspecified patterns (regularities) in the observations. They have to be confirmed (verified) by him. Here, in the terms of Sayre, a pattern is recognized.
无监督学习(也称为试探性分析或聚类分析)：这个方法给研究者一种在观察中自动表示未确定模式(规律)方法，通过这种方法模式种类被确定(检验)了下来，依此，根据Sayre观点，一个模式就可以被被识别出来了。
 
Supervised learning is an automatic system that verifies (confirms)the patterns described by the scientist based on a representation defined by him. This is done by an automatic classification followed by an evaluation.
有监督学习：是这样的一个自动系统，检验(确定)已被研究者通过一种表示方法定义好了的模式，这就是通过评估来实现的自动分类方法。
 
In spite of Sayre’s discussion, the concepts of pattern recognition and classification are still frequently mixed up. In our discussion, classification is a significant component of the pattern recognition system, but unsupervised learning may also play a role there. Typically, such a system is first presented with a set of known objects, the training set, in some convenient representation. Learning relies on finding the data descriptions such that the system can correctly characterize, identify or classify novel examples. After appropriate preprocessing and adaptations, various mechanisms are employed to train the entire system well. Numerous models and techniques are used and their performances are evaluated and compared by suitable criteria. If the final goal is prediction, the findings are validated by applying the best model to unseen data. If the final goal is characterization, the findings may be validated by complexity of organization (relations between objects) as well as by interpretability of the results.
尽管Sayre已做了相关论述，但是模式识别和分类的概念还是经常被混起来。我们认为，分类是模式识别系统的一个重要组成部分，但是无监督学习也可能可以实现一样的功能。典型的如：一个最初以已知对象集(训练集)得到的智能系统，这些对象以某种方便的方式来表示，学习过程依赖于发现对系统的数据描述，使该系统可以正确地表达、标识或分类出不同的例子。经过适当的预处理和适应性修改后，各种训练方法就可被很好地用到训练整个系统上，有许多的模型和技术也可以被用上，且它们的性能有相应的标准来进行评估和比较，如果最后的目标是可以预测的，则最后得到的系统可以通过把最佳模型应用到新数据来检验，如果最后的目标是可以被描述的，则最后得到系统可以通过综合检验，就象对结果进行解释说明一样。
 
 


Fig. 1 shows the three main stages of pattern recognition systems: Representation, Generalization and Evaluation, and an intermediate stage of Adaptation[20]. The system is trained and evaluated by a set of examples, the Design Set. The components are:
图1显示了模式识别系统的三个主要阶段：表示、推广和评估，还有一个中间过程是适配。这个系统通过一个设计样本集(Design Set)来训练和评估。每个组成部分分别描述如下：
 
• Design Set. It is used both for training and validating the system. Given the background knowledge, this set has to be chosen such that it is representative for the set of objects to be recognized by the trained system.There are various approaches how to split it into suitable subsets for training,validation and testing. See e.g. [22, 32, 62, 77] for details.
设计样本集：用于训练和检验识别系统。用于训练的样本被选择时必须是典型的对象。有各种不同的方法可以把样本集分成合适的子集以用于训练、检验和测试，可以看附录[22,32,62,77]中的详细介绍。
 
• Representation. Real world objects have to be represented in a formal way in order to be analyzed and compared by mechanical means such as a computer. Moreover, the observations derived from the sensors or other formal representations have to be integrated with the existing, explicitly formulated knowledge either on the objects themselves or on the class they may belong to. The issue of representation is an essential aspect of pattern recognition and is different from classification. It largely influences the success of the stages to come.
表示：真实世界中的对象得用一种合适的方法来表示，以利于被象计算机这样的机器工具来分析和比较。此外，不管是用于识别对象本身还是所从属的类别，从感应器或其它形式化表示方法中提取出来的观察结果也得和现存的形式化的知识相结合。表示的问题是模式识别的要点，且不同于分类，它会大大影响识别的成功率。
 
• Adaptation. It is an intermediate stage between Representation and Generalization,in which representations, learning methodology or problem statement are adapted or extended in order to enhance the final recognition.This step may be neglected as being transparent, but its role is essential.It may reduce or simplify the representation, or it may enrich it by emphasizing particular aspects, e.g. by a nonlinear transformation of features that simplifies the next stage. Background knowledge may appropriately be (re)formulated and incorporated into a representation. If needed, additional representations may be considered to reflect other aspects of the problem. Exploratory data analysis (unsupervised learning) may be used to guide the choice of suitable learning strategies.
适配：这是个中间阶段，介于表示和推广之间，在表示方法中，学习方法或问题表示形式被适应性地修改或扩展以提高最后的识别能力。这个阶段也可以被忽略，当作是透明的，但它的地位是重要的，它可以减少或简化表示方法，或通过特定方法使得表示方式更灵活，例如通过非线性变换来简化下个阶段的处理过程。背景知识可以适当地被形式化和组合成一种表示方法。如果需要，可以考虑加入其它的表示方法来反映其它问题形式。实验数据分析(无监督学习)可以被用来指导选择合适的学习策略。
 
• Generalization or Inference. In this stage we learn a concept from a training set, the set of known and appropriately represented examples, in such a way that predictions can be made on some unknown properties of new examples. We either generalize towards a concept or infer a set of general rules that describe the qualities of the training data. The most common property is the class or pattern it belongs to, which is the above mentioned classification task.
推广或推断：在这个阶段我们从一个训练集(已知的、以某种表示形式表示的对象集)中学会一个概念，据此就可以用来预测新对象的未知属性。我们既可以从一个概念进行推广也可以从一组描述训练数据性质的一般性规则中进行推断。找出属性最为相似的类别或模式，这个类别或模式便是所要的结果，这就是上面所提到的分类方法。
 
• Evaluation. In this stage we estimate how our system performs on known training and validation data while training the entire system. If the results are unsatisfactory, then the previous steps have to be reconsidered.
评估：这个阶段我们通过已知的训练和检验数据来评估训练出来的系统性能。如果评估结果不令人满意，则前面的步骤就得重新考虑设计或调整。
 
Different disciplines emphasize or just exclusively study different parts of this system. For instance, perception and computer vision deal mainly with the representation aspects [21], while books on artificial neural networks [62],machine learning [4, 53] and pattern classification [15] are usually restricted to generalization. It should be noted that these and other studies with the words “pattern” and “recognition” in the title often almost entirely neglect the issue of representation. We think, however, that the main goal of the field of pattern recognition is to study generalization in relation to representation[20].
识别系统的不同部分分别应用到不同的方法技术，如对于感知器和计算机视觉技术主要是应用于表示部分，而人工神经网络、机器视觉、模式分类则与推广技术紧密相关。要注意的是在这里和其它以“模式”和“识别”为题的学术中经常把表示的问题忽略掉，然而，我们认为：模式识别领域的主要目标是研究与表示方法相联系的推广能力。
 
In the context of representations, and especially images, generalization has been thoroughly studied by Grenander [36]. What is very specific and worthwhile is that he deals with infinite representations (say, unsampled images),thereby avoiding the frequently returning discussions on dimensionality and directly focussing on a high, abstract level of pattern learning. We like to mention two other scientists that present very general discussions on the pattern recognition system: Watanabe [75] and Goldfarb [31, 32]. They both emphasize the structural approach to pattern recognition that we will discuss later on. Here objects are represented in a form that focusses on their structure.A generalization over such structural representations is very difficult if one aims to learn the concept, i.e. the underlying, often implicit definition of a pattern class that is able to generate possible realizations. Goldfarb argues that traditionally used numeric representations are inadequate and that an entirely new, structural representation is necessary. We judge his research program as very ambitious, as he wants to learn the (generalized) structure of the concept from the structures of the examples. He thereby aims to make explicit what usually stays implicit. We admit that a way like his has to be followed if one ever wishes to reach more in concept learning than the ability to name the right class with a high probability, without having built a proper understanding.
在表示上下文上，特别是图像方面，推广能力已经被Granander充分研究透了，特别值得一提的是他解决了表示范围不限(如未样本化图像)的处理问题，因此避免了经常由此而产生的维度问题，从而可以直接专注于模式学习的高层次和抽象层次。对模式识别系统进行归纳性的讨论另两位科学家是：Watanabe和Goldfarb，他们都侧重于结构模式识别方法，我们在后面会提到这个方法，他们都强调要把识别对象进行结构化表示，如果抛开结构化表示，从学习概念入手进行推广是十分困难的，例如那些可能可以被实现的模式分类，但模式的定义却是无法明确表达的。Goldfarb提出传统上使用数字表示方法是不够的，采用一个全新的结构表示方法是必要的，我们觉得他要从样本结构中学习概念的结构(具有推广性的结构)是非常困难的，Goldfarb因此把目标转向把模糊的东西清晰化，我们承认这种做法在这种的情况下是需要的：不去建立一个合适的理解模型,但却想在概念学习上比通过概率来正确分类达到更好的效果。
 
In the next section we will discuss and relate well-known general scientific approaches to the specific field of pattern recognition. In particular, we like to point out how these approaches differ due to fundamental differences in the scientific points of view from which they arise. As a consequence, they are often studied in different traditions based on different paradigms. We will try to clarify the underlying cause for the pattern recognition field. In the following sections we sketch some perspectives for pattern recognition and define a number of specific challenges.
下一节我们将讨论和叙述在模式识别领域中众所周知的科学方法，从各自被提出的不同的基本科学观点上，我们将详细地指出这些方法的区别，正因如此，它们常是基于不同模式上在不同的传统领域里被研究，我们将尝试分清模式识别领域中的重要依据。在下一节我们将勾画出模式识别前景和一些具体要克服的问题。