GO FOR A WALK AND ARRIVE AT THE ANSWER: REASONING OVER PATHS IN KNOWLEDGE BASES USING REINFORCEMENT
go for a walk and arrive at the answer: reason over paths in knowledge bases using reinforcement learning
来源
2018 ICLR MINERVA模型
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer
Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, Andrew McCallum
frajarshi, sdhuliawala, luke, akshay, mccallumg@cs.umass.edu
manzil@cmu.edu, ishand@cs.utexas.edu, alex@smola.org
University of Massachusetts, Amherst, Carnegie Mellon University
University of Texas at Austin, Amazon Web Services
背景
自动推理长久以来一直是人工智能的长期目标,自动推理能力指的是计算机具有从目前的证据推断出新的证据的能力。本文主要专注于知识图谱上的推理,知识图谱通常是不完整的,很多事实都没有在知识图谱中并且通常是可以根据图谱中的事实推断出来。本文的目标是机器自动化学习这样的路径,将这个问题建模为查询问答,问题的形式是(es,rq,?)(e_s, r_q, ?)(es,rq,?)。
在自动推理的研究工作中,主要分为三类:早期基于符号表示的推理,这类方法的泛化能力较差,被第二种基于向量表示的方法替代,一些使用张量分解或者神经网络的方法,这些方法可解释性差,并且不能够捕捉知识图谱中路径表示的链式推理。神经网络多跳模型通过在知识库路径(embedding)上解决上面的链式推理问题,但是这些模型都需要通过在知识图谱上进行随机游走获取路径作为输入。本文利用强化学习方法高效的在知识图谱上进行路径搜索,不需要预先计算路径。
模型
state: S=(et,e1q,rq,e2q)S=(e_t, e_{1q}, r_q, e_{2q})S=(et,e1q,rq,e2q)
observation: 环境完整的状态不是完全可见的,智能体知道当前所在位置ete_tet, 查询(e1q,rq)(e_1q, r_q)(e1q,rq), 但是答案是不知道的e2qe_{2q}e2q
action:As={(et,r,v)∈E}⋃{(s,∅,s)}A_{s} = \{(e_t,r,v) \in E\} \bigcup \{(s, \emptyset, s)\}As={(et,r,v)∈E}⋃{(s,∅,s)}
transition: δ(S,A)=(v,e1q,rq,e2q)\delta(S,A) = (v, e_{1q},r_q,e_{2q})δ(S,A)=(v,e1q,rq,e2q)
rewards: 如果agent 当前的节点是正确答案,则返回奖励1,否则是0
策略网络:策略网络:策略网络:
at−1a_{t-1}at−1是t−1t-1t−1时刻的action, oto_tot是ttt时刻的observation
训练:
##代码
代码
- Using UTF-8 as the internal representation for strings in C and C++ with Visual Studio
- Along with all the above benefits, you cannot overlook the space efficiency and performance gains in using DataFrames and Dataset APIs for two reasons.
- Tracing SQL Queries in Real Time for MySQL Databases using WinDbg and Basic Assembler Knowledge
- Using MERGE in SQL Server to insert, update and delete at the same time
- how to config the tftpd-hpa server in the ubuntu using for remote tftp client '-p' and '-r' command
- Get and display the size of file and directory in Linux system using du command 获取和现实linux文件大小(三)
- Using 'dlopen' in statically linked applications requires at runtime the shared libraries
- How do I iterate over a Scala List (or more generally, a sequence) using theforeach method or for loop?
- Embedding Documents in Word 2007 by Using the Open XML SDK 2.0 for Microsoft Office
- Using the WebLogic 10.3.4 Maven Plug-In for Deployment
- VS代码段扩展Snippet Designer is a Visual Studio plug in which allows you to create and search for snippets inside the IDE
- warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries f
- No plugin found for prefix 'war' in the current project and in the plugin groups
- Security and Privacy for Cloud-Based Data Management in the Health Network Service Chain: A Microser
- A meta-model for the analysis and design of organizations in multi-agent systems
- No plugin found for prefix 'jetty' in the current project and in the plugin groups
- 错误:Mixed Content: The page at ‘https://XXX’ was loaded over HTTPS, but requested an insecure........
- 错误整理:No plugin found for prefix 'jetty' in the current project and in the plugin groups
- Controls over the EditText and the IME in Android
- Using JPA and JAXB Annotations in the Same Object