⾃然语⾔理解和⾃然语⾔处理_4种⾃然语⾔处理和理解的⽅法⾃然语⾔理解和⾃然语⾔处理
by Mariya Yao
姚iya(Mariya Yao)
4种⾃然语⾔处理和理解的⽅法 (4 Approaches To Natural Language Processing & Understanding)
In 1971, Terry Winograd wrote the program while completing his PhD at MIT.
1971年，Terry Winograd在⿇省理⼯学院攻读博⼠学位时编写了程序。
SHRDLU features a world of toy blocks where the computer translates human commands into physical actions, such as “move the red pyramid next to the blue cube.”
SHRDLU具有玩具积⽊世界，其中计算机将⼈⼯命令转换为实际动作，例如“将红⾊⾦字塔移到蓝⾊⽴⽅体旁边”。
To succeed at such tasks, the computer must build up mantic knowledge iteratively, a process Winograd discovered as brittle and limited.
为了成功完成这些任务，计算机必须迭代地建⽴语义知识，⽽Winograd发现该过程脆弱且受限制。
The ri of chatbots and voice activated technologies has renewed fervor in natural language processing (NLP) and natural language understanding (NLU) techniques that can produce satisfying human-computer dialogs.
聊天机器⼈和语⾳激活技术的兴起重新激发了⼈们对⾃然语⾔处理(NLP)和⾃然语⾔理解(NLU)技术的热情，这些技术可以产⽣令⼈满意的⼈机对话。
Unfortunately, academic breakthroughs have not yet translated into improved ur experience. Gizmodo writer Darren Orf declared Mesnger chatbots “” and Facebook admitted a for their highly anticipated conversational assistant, “M.”
不幸的是，学术上的突破尚未转化为改善的⽤户体验。 Gizmodo作家Darren Orf宣布Mesnger聊天机器⼈“ ”，⽽Facebook承认其备受期待的对话助⼿“ M”的 。
Nevertheless, rearchers forge ahead with new plans of attack, occasionally revisiting the same tactics and principles Winograd tried in the 70s.
尽管如此，研究⼈员还是提出了新的进攻计划，偶尔会重温Winograd在70年代尝试过的相同战术和原
则。
OpenAI recently leveraged reinforcement learning to by “dropping them into a t of simple worlds, giving them the ability to communicate, and then giving them goals that can be best achieved by communicating with other agents.” The agents independently developed a simple “grounded” language.
OpenAI最近利⽤强化学习来是“将他们放到⼀组简单的世界中，赋予他们交流的能⼒，然后赋予他们可以与其他代理⼈进⾏交流的最佳⽬标。” 代理商独⽴开发了⼀种简单的“扎根”语⾔。
汪口
MIT Media Lab this satisfying clarification on what “grounded” means in the context of language:
⿇省理⼯学院媒体实验室就语⾔中“扎根”的含义令⼈满意的澄清：
“Language is grounded in experience. Unlike dictionaries which define words in terms of other words, humans
understand many basic words in terms of associations with nsory-motor experiences. People must interact
physically with their world to grasp the esnce of words like “red,” “heavy,” and “above.” Abstract words are acquired only in relation to more concretely grounded terms. Grounding is thus a fundamental aspect of spoken language, which enables humans to acquire and to u words and ntences in context.”
语⾔基于经验。与⽤其他词来定义词的词典不同，⼈类根据与感觉运动体验的关联来理解许多基本词。⼈们必须与⾃⼰的世界进⾏互动，以掌握“红⾊”，“沉重”和“上⽅”等词语的本质。仅与更具体的基础术语相关地获取抽象词。因此，扎根是⼝头语⾔的基本⽅⾯，它使⼈类能够在上下⽂中获取和使⽤单词和句⼦。”
幼儿园课程内容
The antithesis of grounded language is inferred language. Inferred language derives meaning from words themlves rather than what they reprent.
基本语⾔的对⽴是推断语⾔。 推断语⾔是从单词本⾝⽽不是它们所代表的含义中获得含义的。
When trained only on large corpus of text — but not on real-world reprentations — statistical methods for NLP and NLU lack true understanding of what words mean.
如果只接受⼤型⽂本语料库的训练，⽽不能接受真实世界的表⽰法的训练，那么NLP和NLU的统计⽅法就⽆法真正理解单词的含义。
OpenAI points out that such approaches share the weakness revealed by John Searle’s famous thought experiment. Equipped with a universal dictionary to map all possible Chine input ntences to Chine output ntences, anyone can perform a brute force lookup and produce conversationally acceptable answers without understanding what they’re actually saying.
OpenAI指出，这种⽅法具有约翰·塞尔(John Searle)著名的思想实验所揭⽰的缺点。 配备了通⽤字典，可以将所有可能的中⽂输⼊句⼦映射到中⽂输出句⼦，任何⼈都可以执⾏暴⼒查询并产⽣对话可接受的答案，⽽⽆需了解他们的实际意思。
语⾔为何如此复杂？ (Why Is Language So Complex?)
Percy Liang, a Stanford CS professor and NLP expert, into four distinct categories:
斯坦福⼤学CS教授和NLP专家Percy Liang 为四个不同的类别：
1. Distributional
分配式
2. Frame-bad
基于框架
3. Model-theoretical
模型理论
心理罪电影4. Interactive learning
互动学习
First, a brief linguistics lesson before we continue on to define and describe tho categories.
⾸先，在我们继续定义和描述这些类别之前，简要讲授语⾔学课程。
There are three levels of linguistic analysis:
语⾔分析分为三个级别：孝心故事
1. Syntax — what is grammatically correct?
语义学–是什么意思？
3. Pragmatics — what is the purpo or goal?
语⽤学–⽬的或⽬标是什么？
Drawing upon a programming analogy, Liang likens successful syntax to “no compiler errors,” mantics to “no implementation bugs,” and pragmatics to “implemented the right algorithm.”
根据编程的类⽐，梁将成功的语法⽐喻为“没有编译器错误”，语义⽐喻为“没有实现错误”，⽽语⽤⽐喻为“实现了正确的算法”。
He that ntences can have the same mantics, yet different syntax, such as “3+2” versus “2+3”. Similarly, they can have identical syntax yet different syntax, for example 3/2 is interpreted differently in Python 2.7 vs Python 3.
他 ，句⼦可以具有相同的语义，但可以具有不同的语法，例如“ 3 + 2”和“ 2 + 3”。 同样，它们可以具有相同的语法，但是可以具有不同的语法，例如3/2在Python 2.7与Python 3中的解释不同。
Ultimately, pragmatics is key, since language is created from the need to motivate an action in the world. If you implement a complex neural network to model a simple coin flip, you have excellent mantics but poor pragmatics since there are a plethora of easier and more efficient approaches to
solve the same problem.
归根结底，语⽤是关键，因为语⾔是出于激发世界⾏动的需要⽽创建的。 如果您使⽤复杂的神经⽹络对简单的硬币翻转进⾏建模，则您将拥有出⾊的语义，但实⽤主义却很差，因为存在许多解决同⼀问题的更简便，更有效的⽅法。
Plenty of other linguistics terms exist which demonstrate the complexity of language. Words take on different meanings when combined with other words, such as “light” versus “light bulb” (that is, multi-word expressions), or ud in various ntences such as “I stepped into the light” and “the suitca was light” (polymy).罗丹简介
存在许多其他语⾔学术语，这些语⾔论证了语⾔的复杂性。 单词与其他单词结合使⽤时，具有不同的含义，例如“ light”和“ light bulb”(即多词表达)，或者在各种句⼦中使⽤，例如“ Isteping the light”和“⼿提箱是轻”(多义)。
Hyponymy shows how a specific instance is related to a general term (a cat is a mammal) and meronymy denotes that one term is a part of another (a cat has a tail). Such relationships must be understood to perform the task of textual entailment, recognizing when one ntence is logically entailed in another. “You’re reading this article” entails the ntence “you can read.”
副词表⽰特定实例与⼀般术语(猫是哺乳动物)之间的关系，⽽副词则表⽰⼀个术语是另⼀术语的⼀部分(猫有尾巴)。 必须理解这种关系以执⾏⽂本包含的任务，认识到⼀个句⼦在逻辑上包含在另⼀个句⼦中。 “您正在阅读本⽂”包含句⼦“您可以阅读”。
Aside from complex lexical relationships, your ntences also involve beliefs, conversational implicatures, and presuppositions. Liang provides excellent examples of each. Superman and Clark Kent are the same person, but Lois Lane believes Superman is a hero while Clark Kent is not.
除了复杂的词汇关系外，您的句⼦还涉及信念，会话含义和预设。 梁提供了很好的例⼦。 超⼈和克拉克·肯特是同⼀个⼈，但路易斯·莱恩(Lois Lane)认为超⼈是英雄，⽽克拉克·肯特不是。
If you say “Where is the roast beef?” and your conversation partner replies “Well, the dog looks happy”, the conversational implicature is the dog ate the roast beef.
如果您说“烤⽜⾁在哪⾥？” 并且您的对话伙伴回答“好吧，狗看起来很⾼兴”，对话的含义是狗吃了烤⽜⾁。
Presuppositions are background assumptions that are true regardless of the truth value of a ntence. “I have stopped eating meat” has the presupposition “I once ate meat” even if you inverted the ntence to “I have not stopped eating meat.”
预设是与句⼦的真值⽆关的真实背景假设。 即使您将句⼦改为“我没有停⽌吃⾁”，“我已经停⽌吃⾁”的前提还是“我曾经吃过⾁”。
Adding to the complexity are vagueness, ambiguity, and uncertainty. Uncertainty is when you e a word you don’t know and must guess at the meaning.
If you’re stalking a crush on Facebook and their relationship status says “It’s Complicated”, you already understand vagueness. Richard Socher, Chief Scientist at Salesforce, gave an excellent example of ambiguity at a recent AI conference:“The question ‘can I cut you?’ means very different things if I’m standing next to you in line or if I am holding a knife.”
如果您对Facebook情有独钟，并且他们的关系状态显⽰“很复杂”，那么您已经了解了模糊性。 Salesforce的⾸席科学家Richard Socher在最近的AI会议上给出了⼀个模棱两可的很好的例⼦：“我能切你吗？” 如果我排在你旁边或者我拿着⼑，那意味着完全不同的事情。”
Now that you’re more enlightened about the myriad challenges of language, let’s return to Liang’s four categories of approaches to mantic analysis in NLP and NLU.
既然您对语⾔的⽆数挑战有了更多的了解，那么让我们回到Liang在NLP和NLU中进⾏语义分析的四类⽅法。
1：分配⽅法 (1: Distributional Approaches)
Distributional approaches include the large-scale statistical tactics of machine learning and deep learning. The methods typically turn content into word vectors for mathematical analysis and perform quite well at tasks such as part-of-speech tagging (is this a noun or a verb?), dependency parsing (does this part of a ntence modify another part?), and mantic relatedness (are the different words ud in similar ways?). The NLP tasks don’t rely on understanding the meaning of words, but rather on the relationship between words themlves.
分布⽅法包括机器学习和深度学习的⼤规模统计策略。 这些⽅法通常将内容转换为⽤于数学分析的单词向量，并且在诸如词性标注(这是名词还是动词？)，依存关系分析(句⼦的这⼀部分是否修改了另⼀部分？)之类的任务上表现出⾊。以及语义相关性(这些不同的词是否以类似的⽅式使⽤？)。 这些NLP任务不依赖于理解单词的含义，⽽是依赖于单词本⾝之间的关系。
Such systems are broad, flexible, and scalable. They can be applied widely to different types of text without the need for hand-engineered features or expert-encoded domain knowledge. The downside is that they lack true understanding of real-world mantics and pragmatics. Comparing words to other words, or words to ntences, or ntences to ntences can
all result in different outcomes.
这样的系统是⼴泛的，灵活的和可扩展的。 它们可以⼴泛地应⽤于不同类型的⽂本，⽽⽆需⼿⼯设计的功能或专家编码的领域知识。 缺点是他们对真实世界的语义和语⽤缺乏真正的了解。 将单词与其他单词进⾏⽐较，或者将单词与句⼦进⾏⽐较，或者将句⼦与句⼦进⾏⽐较，都可能导致不同的结果。
Semantic similarity, for example, does not mean synonymy. A nearest neighbor calculation may even deem antonyms as related:
例如，语义相似性并不意味着同义词。 最近邻居计算甚⾄可以将反义词视为相关：
Advanced modern neural network models, such as the pioneered by Facebook or the invented by Salesforce can handle simple question and answering tasks, but are still in early pilot stages for consumer and enterpri u cas.
先进的现代神经⽹络模型，例如Facebook倡导或Salesforce发明的可以处理简单的问答任务，但仍处于消费者和企业使⽤的早期试验阶段案件。
Thus far, Facebook has only that a neural network trained on an absurdly simplified version of The L
ord of The Rings can figure out where the elusive One Ring is located.
到⽬前为⽌，Facebook仅 ，在荒谬的简化版《指环王》上训练的神经⽹络可以找出难以捉摸的“⼀环”的位置。
Although distributional methods achieve breadth, they cannot handle depth. Complex and nuanced questions that rely linguistic sophistication and contextual world knowledge have yet to be answered satisfactorily.
尽管分布⽅法可达到⼴度，但它们⽆法处理深度。 依赖于语⾔复杂性和上下⽂世界知识的复杂细微问题尚未得到令⼈满意的回答。
2：基于框架的⽅法 (2: Frame-Bad Approach)
“A frame is a data-structure for reprenting a stereotyped situation,” explains Marvin Minsky in his called “A
Framework For Reprenting Knowledge.” Think of frames as a canonical reprentation for which specifics can be
Maven·明斯基(Marvin Minsky)在 “代表知识的框架”中解释说：“框架是代表刻板印象的情况的数据结构。” 可以将框架视为可以互换细节的规范表⽰。
Liang provides the example of a commercial transaction as a frame. In such situations, you typically have a ller, a buyers, goods being exchanged, and an exchange price.
梁以商业交易为例提供了框架。 在这种情况下，您通常会有⼀个卖⽅，⼀个买⽅，正在交换的商品以及⼀个交换价格。
Sentences that are syntactically different but mantically identical — such as “Cynthia sold Bob the bike for $200” and “Bob bought the bike for $200 from Cynthia” — can be fit into the same frame. Parsing then entails first identifying the frame being ud, then populating the specific frame parameters — i.e. Cynthia, $200.
在句法上不同但在语义上相同的句⼦(例如“ Cynthia以200美元的价格卖给Bob的⾃⾏车”和“ Bob以200美元的价格从Cynthia买来的⾃⾏车”)可以安装在同⼀框架中。 然后进⾏解析需要⾸先识别正在使⽤的帧，然后填充特定的帧参数，即Cynthia，$ 200。
The obvious downside of frames is that they require supervision. In some domains, an expert must c
reate them, which limits the scope of frame-bad approaches. Frames are also necessarily incomplete. Sentences such as “Cynthia visited the
bike shop yesterday” and “Cynthia bought the cheapest bike” cannot be adequately analyzed with the frame we defined above.
框架的明显缺点是它们需要监督。 在某些领域，专家必须创建它们，这限制了基于框架的⽅法的范围。 框架也⼀定是不完整的。 上⾯定义的框架⽆法充分分析“ Cynthia昨天去过⾃⾏车商店”和“ Cynthia购买了最便宜的⾃⾏车”之类的句⼦。
3：模型理论⽅法 (3: Model-Theoretical Approach)
The third category of mantic analysis falls under the model-theoretical approach. To understand this approach, we’ll introduce two important linguistic concepts: “model theory” and “compositionality”.
第三类语义分析属于模型理论⽅法。 为了理解这种⽅法，我们将介绍两个重要的语⾔概念：“模型理论”和“组合性”。
Model theory refers to the idea that ntences refer to the world, as in the ca with grounded langu
age (i.e. the block is blue). In compositionality, meanings of the parts of a ntence can be combined to deduce the whole meaning.
模型理论指的是句⼦指的是世界，就像扎根的语⾔⼀样(例如，⽅框是蓝⾊的)。 在构词性上，可以将句⼦各部分的含义组合起来以推断出整个含义。
Liang compares this approach to turning language into computer programs. To determine the answer to the query “what is the largest city in Europe by population”, you first have to identify the concepts of “city” and “Europe” and funnel down your arch space to cities contained in Europe. Then you would need to sort the population numbers for each city you’ve shortlisted so far and return the maximum of this value.
工资转移介绍信梁⽐较了将语⾔转换为计算机程序的⽅法。 要确定“⼈⼝最多的欧洲城市是多少”这⼀查询的答案，您⾸先必须确定“城市”和“欧
洲”的概念，然后将搜索范围集中到欧洲所包含的城市。 然后，您需要对到⽬前为⽌已⼊围的每个城市的⼈⼝数量进⾏排序，并返回该值的最⼤值。
To execute the ntence “Remind me to buy milk after my last meeting on Monday” requires similar composition breakdown and recombination.
牛肉披萨的做法
要执⾏“在周⼀的上次会议后提醒我购买⽜奶”的句⼦，需要类似的成分分解和重组。
Models vary from needing heavy-handed supervision by experts to light supervision from average humans on Mechanical Turk. The advantages of model-bad methods include full-world reprentation, rich mantics, and end-to-end processing, which enable such approaches to answer difficult and nuanced arch queries.
模式从需要专家的严格监督到从普通⼈在Mechanical Turk上的轻度监督，不⼀⽽⾜。 基于模型的⽅法的优点包括全域表⽰，丰富的语义以及端到端处理，这使此类⽅法能够回答困难⽽细微的搜索查询。
The major con is that the applications are heavily limited in scope due to the need for hand-engineered features.
穿井
Applications of model-theoretic approaches to NLU generally start from the easiest, most contained u cas and advance
