natural language processing artificial intelligence

Module 5Natural language Processing Two of the most difficult tasks that facing AI researchers are - developing programs that understand Natural language & -comprehend Visual scenes Developing Programs that can understand Natural Language is very difficult. Why? Natural languages are large They contain a number of different sentences. New sentences can always be produced. There is ambiguity in a natural language Many words have several meanings and sentences can have several meanings in different contexts English sentences are incomplete descriptions of the information. -some dogs are outside. The same expression means different things in different contexts: Where is the water ? Advt: communicate about an infinite world using a finite number of symbols. No natural language pgm can be complete because new words, expressions and meanings can be generated quite freely. There are lots of ways to say the same thing. Mary was born on October11. Mary’s birthday is on October 11. Semantic knowledge 5. Morphemes.Pragmatic knowledge 6.World knowledge .Syntactic knowledge 4.Overview of Linguistics Linguistics – study of language Levels of knowledge used in Natural language understanding 1. Morphological knowledge . Phonological knowledge -knowledge which relates sounds to the words Phoneme –smallest unit of sound 2.lexical knowledge related to word constructions from basic units called morphemes.smallest unit of meaning 3. ” Pragmatic analysis “ Do you know what time it is ” .Morphological Analysis -punctuations are seperated from words Syntactic analysis Eg: Boy the go the to store Semantic analysis “Colorless green ideas sleep furiously” “I dropped my dimond” Discourse Integration “ John wanted it. Vt. nouns or verbs in English NonTerminals symbols – can be decomposed further or expanded by rules. Sentences are constructed using a set of rules called grammer. eg: adjectives .Grammers and languages Language – can be considered as a set of strings of finite or infinite length String – constructed by concatenating symbols( alphabets) Alphabets – symbols of the language. Language generated by grammer G – L(G) Grammer G can be defined as G = (Vn. eg: Noun phrases or Verb phrases . p) Terminal symbols – symbols which cannot be decomposed further. s. following sentence can generated: The boy ate a popsicle The frog kissed a boy A boy ate the frog .Most common way to represent grammers is as a set of production rules S NP VP NP ART N NP N VP N V ART V NP boy | popsicle | frog ate | kissed the | a |flew With this G. only that they are structurally correct. A grammer does not gurantee the generation of meaningful sentences.S NP VP ART N VP the N VP the boy VP the boy V NP the boy ate NP the boy ate ART N the boy ate a N the boy ate a popsicle. “The Popsicle flew a frog” . Structural Representations Sentences can be represented as a tree or graph to expose the structure of the constituent parts. S NP VP ART N the boy V ate Phase marker or syntactic tree NP ART a N popsicle . The structure of a sentence can be represented with a syntactic tree or a list .Basic Parsing Techniques The process of determining the syntactical structure of a sentence is known as parsing. The process of analyzing a sentence by taking it apart word-by word and determining its structure from its constituent parts and sub parts. it is necessary to find a way in which that sentence could have been generated from the start symbol.Begin with the sentence to be parsed and apply the grammar rules backward until a single tree whose terminals are the words of the sentence and whose top node is the start symbol has been produced. This can be done in two ways: Top-Down Parsing – Begin with the start symbol and apply the grammar rules forward until the symbols at the terminals of the tree correspond to the components of the sentence being parsed Bottom-up Parsing .To parse a sentence. . Parsing an input to create an output structure Input string Parser Lexicon Output representation structure . Kathy jumped the horse S NP VP N VP Kathy VP Kathy V NP Kathy jumped NP Kathy jumped ART N Kathy jumped the N Kathy jumped the house Top down Parsing Kathy jumped the horse N jumped the horse N V the horse N V ART horse NP V ART N NP V NP NP VP S Bottom up Parsing . Access to the words may be facilitated by indexing.The Lexicon A dictionary of words. or combinations of these methods. . with binary searches. where each word contains some syntactic. semantic and possibly some pragmatic information Usually made up of variable length data structures such as lists or records arranged in alphabetical order. hashing. 2s.2p.3s.1p. past participle : : Orange adjective Noun {3s} To preposition .3p} Carried verb form: past.Typical entries in a lexicon Word Type Features a Determiner {3s} beVerb Trans:intransitive Boy Noun {3s} Can Noun {1s. Comparing and matching the input to real world situations. . Syntactic and semantic directed analysis. Of these second approach is the most popular one. General approaches to natural language Understanding The use of keyword and pattern matching.Understanding written text is easier than understanding speech. Transformational Grammars Provide a mechanism to produce single representations for sentences having the same meanings through a series of transformations Generative Grammers -produce different structures for sentences having different syntactical forms even though they may have the same semantic content. Consider the following sentences . S S NP VP V Susan printed NP NP VP ART N The file ART N the file V was PP printed by susan . (printed (agent Susan) (object File) Mother baked for three hours (baked (agent Mother) (timeperiod 3-hours) (baked (Object Pie) (timeperiod 3-hours) .Case Grammars Grammer rules are written to describe syntactic rather than semantic regularities. Cause of the event or object used in causing the event(inanimate) (D) Dative. (O) Object – Entity that is acted upon or that changes.Place of the event (S) Source – Place from which something moves (G) Goal – Place to which something moves (T) Time – Time at which the event occurred. Describe relationships between verbs and their arguments.(animate) (L) Locative.Different Cases are used by Case grammer are (A) Agent – Instigator of the action(animate) Instrument . .Entity affected by the action. The process of parsing into a case representation is heavily directed by the lexical entries associated with each verb open [ _ _ O (I) (A)] The door opened John opened the door John open the door with a chisel. Die [ _ _ D] John died Kill [ _ _ D (I) A] Bill killed John Bill killed John with a knife. . Parsing using a case grammer is expectation-driven . .Transition networks Another popular method used to represent formal and natural language structures Based on the application of directed graphs(digraphs) and finite state automata. Consists of a number of nodes and labeled arcs. or performing a language translation. The context relates to previous expressions. desires and intentions of the speakers. . -includes object descriptions. such as retrieving information from a data base. the setting and time of the utterances . The domain refers to the knowledge that is part of the world model the system knows about. The task is part of the service the system offers. relationships and other relevant concepts.Semantic Analysis and Representation structures Semantic interpretation is the most difficult stage in the transformation process. and the beliefs. providing expert advice. there is usually a semantic action associated with each grammar rule.Lexical semantics Approaches 1.OBJECT and TENSE (INGEST (ACTOR nil) (OBJECT nil) (TENSE past) .a context free grammar in which the choice of nonterm inals and production rules is governed by semantics as well as syntactic function. .uses conceptual dependency theory. Eg: Primitive action INGEST with unfilled slots ACTOR. Semantic Grammar . based on Semantic grammars 2. (INGEST (ACTOR nil) (OBJECT nil) (TENSE past) The boy drank a soda (INGEST (ACTOR (PP NAME boy)(CLASS PHY-OBJ) (TYPE ANIMATE)(REF DEF))) (OBJECT (PP(NAME soda)(CLASS PHY-OBJ) (TYPE INANIMATE)(REF INDEF))) (TENSE past) . the semantic interpreter would produce the following predicate clause (CONTAIN sample24 silicon) .The target knowledge structures constructed in this approach are typically logic expressions such as the formulas of FOPL. .Compositional semantics Approaches The meaning of an expression is derived from the meanings of the parts of the expression. Eg: NL statement - Sample24 contains silicon Result of parsing (S DCL (NP (N Sample 24))) (AUX (TENSE(PRESENT))) (VP (V contain)) (NP (N (silicon)))) Using this structure. and how the utterances should be stated which form is better(active or passive) which words and structures best express the intent when to say what. More difficult than understanding.Natural language Generation Exact inverse of language undestanding.because the system must decide . .what to say. Text planning Process of organizing the content to be communicated so as to achieve the goals of the speaker. a question or argument in order to convey the meanings set forth by the goals of the speaker. and 3) achieving a realization of desired utterances. Content determination Concerned with what details to include in an explanation. Realization – the process of mapping the organized content to actual text. a request.The study of language generation falls naturally into three areas: 1) the determination of content 2) formulating and developing a text utterance plan. . developed by Gary Hendrix The SHRDLU System .developed by Terry Winograd .Language Interface facility with Ellipsis and recursion .Natural Language Systems A few of the more successful natural language understanding systems are The LUNAR system The LIFER System . Pattern recognition Systems are used to identify or classify objects on the basis of their attribute and attribute-relation values.a process whereby computer programs are used to recognize various forms of input stimuli such as visual or acoustic(speech) patterns. Recognition is the process of establishing a close match between some new stimulus and previously stored stimulus patterns. .Pattern Recognition Computer pattern recognition . learning. and many other cognitive acts.• Object classification is closely related to recognition.essential for decision making. • The ability to classify or group objects according to some commonly shared features is a form of class recognition.Depends on the ability to discover common patterns among objects. . . • Classification is . The more prominent attributes( such as size. color. shape.The recognition and classification process Step 1 . . The values of these attributes and their relations are used to characterize an object in the form of a pattern vector X . and texture) produce the strongest stimuli.The range of characteristic attribute values is known as the measurement space M Step 2 A subset of attributes whose values provide object grouping or clustering are selected.stimuli produced by objects are perceived by sensory devices. The range of the subset of attribute values is known as the feature space F. or decision functions. The range of the decision function values or classification rules is known as the decision space D. . object or class characterization models are learned by forming generalized prototype descriptions. Step 4 Recognition of familiar objects is achieved through application of the rules learned in Step 3 by comparison and matching of object features with the stored models.Step 3 Using the selected attribute values. classification rules. The pattern recognition process Classification Stimuli Sensors Feature selection Matching Classification rules Learning . • There are two basic approaches to the recognition problem 1)The decision theoretic approach 2)The syntactic approach . a kind of grammar is defined for object descriptions. .Decision Theoretic classification • Based on the use of decision functions to classify objects.vocabulay is based or shape primitives. • A decision function maps pattern vectors X into decision regions of D. . . Syntactic Classification -The syntactic recognition approach is based on the uniqueness of syntactic “structure” among the object classes. grammars or other rules can be performed in either of the two ways. it must posses knowledge of the characteristics features for those objects • Learning decision functions.Learning Classification Patterns • Before a system can recognize objects. through • Supervised learning • Unsupervised learning . .Supervised Learning - accomplished by presenting training examples to a learning unit. This knowledge is used to adjust parameters in decision functions or grammer rewrite rules. The attribute values and object labels are used by the learning component to extract and determine pattern criteria for each class. The examples are labelled beforehand with their correct identities or class. Unsupervised Learning Labled training examples are not available and little is known beforehand regarding the object population. find common pattern among them. the system must be able to perceive and extract relevant properties from the unknown objects. and formulate descriptions or discrimination criteria consistent with the goals of the recognition process. - .In such cases. What clustering algorithms can best meet the criteria . and what weights should be given to each? 2. What representation scheme should be used to describe the cluster groupings or classifications? 4. The clustering problem gives rise to several subproblems 1.a discovery learning process in which similar patterns are found among a group of objects. What representation formalism should be used to characterize the objects? 3. What clustering criteria is most consistent with and effective in achieving the objectives relative to the context or domain? 5. . What set of attributes and relations are most relevant.Learning through Clustering - Clustering is the process of grouping or classifying objects on the basis of a close association or shared characteristics. users are free to perform other tasks concurrently with the computer interchange. • With speech as communication medium. • Untrained personnel would be able to use computers in a variety of applications .Recognizing and Understanding Speech • The ability to communicate directly with programs offers several advantages. • It eliminates the need for keyboard entries • Speeds up the interchange of information between the user and system. The program then determines what the user was probably saying and either outputs it as text or issues a computer command . To do this . The system filters the digitized sound to remove unwanted noise.a representation of the sounds we make and put together to form meaningful expressions.screen text . The analog-to-digital converter(ADC) translates this analog wave into digital data that the computer can understand. The program examines phonemes in the context of other phonemes around them. When we speak we create vibrations in the air.How speech recognition works ? • To covert speech to on. A phoneme is the smallest element of a language. The software language model compares the phonemes to words in its built-in dictionary. a computer has to go through several complex steps. and sometimes to separate it into different bands of frequency. it digitizes the sound by taking precise measurements of the wave at frequent intervals. Next the signal is divided into small segments and the program then matches these segments to known phonemes in appropriate language. . Expert System Architectures .  Application Domains Law aerospace Chemistry military operations Biology finance Engineering banking Medicine geology manufacturing .have proven to be effective in a number of problem domains which require the kind of intelligence possessed by a human expert.a kind of knowledge based systems . Expert Systems .a recent product of AI . but to assist them. .Definition A set of programs designed to act as an expert in a particular domain.  Not meant for replacing experts in that domain. Characteristic features of Expert systems  Use knowledge rather than data  Knowledge is encoded and maintained separately.  Capable of explaining how a particular conclusion was reached  Use symbolic representations for knowledge  Can reason with meta knowledge Importance of Expert Systems . rules. Each rule represents a small piece of knowledge relating to the given domain of expertise...then. If .Expert System Architectures 1) Rule based System or Production Systems -use knowledge encoded in the form of production rules ie .... .. Components of an Expert System Explanation Module INPUT I/O Interface OUTPUT Editor Inference Engine Knowledge base Learning Module Case history file Working memory .  Eg: IF : The patient has a chronic disorder. Knowledge Base Contains facts and rules about some domain. and the patient shows condition A.and the age of the patient is less than 30. and test B reveals biochemistry condition C THEN: Conclude the patient's diagnosis is autoimmunechronic-hepatitis . lessthan(patient.value_A). The Inference Process .accepts user input queries and responses to questions through the I/O interface and uses this dynamic information together with the static knowledge(the rules and facts) stored in the knowledge base.30) same(patient.disorder.age. .chronic). same(patient.symptom_A.autoimmune-chronic-hepatitis):same(patient.value_C).In PROLOG conclude(patient.biochemistry.diagnosis. The inferring process is carried out recursively in three stages: 1) match 2) select 3) execute . The Production system Inference cycle Knowledge base Working Memory match Conflict Set Select execute . Building a Knowledge base   An editor is used by developers to create new rules for addition to the knowledge base.or to modify existing rules in some way. to delete outdated rules . This should be done without adding redundant or unnecessary rules.  Eg: of an intelligent editor – TEIRESIAS (developed to work with systems like MYCIN) . Most difficult task in creating and maintaining production systems is -building and maintaining of a consistent but complete set of rules. . Eg: MYCIN has a vocabulary of some 2000 words. The system must have special prompts or a specialized vocabulary which encompasses the terminology of the given domain of expertise.I/O Interface    Permits the user to communicate with the system in a more natural way. Not common components of expert systems .Used to assist in building and refining the knowledge base .Learning module and history file . Instead of rules.less common expert system architecture. .Non Production system Architectures . . these systems employ more structured representation schemes like  Associative or semantic networks  Frame structures  Decision trees  Specialized networks like neural networks. useful in representing hierarchical knowledge structures.Associative or Semantic Network Architectures . . . Eg: Expert system based on the use of an associative network representation – CASNET CASNET – Causal Associational Network -used to diagnose and recommend treatment for Glaucoma . where property inheritance is common.Not a popular form of representation for standard expert systems.can be used in natural language systems or computer vision systems also. fly CAN A-KIND-OF tweety bird HAS PARTS COLOR wings Fragment of an associative network yellow . ISA Bob MARRIED TO Sandy Professor OWNS House DRIVES Bike . . Decision Tree Architectures Knowledge for expert systems may be stored in the form of a decision tree when the knowledge can be structured in a top-to-bottom manner.Frame Architectures Eg: for a frame based expert system .PIP system PIP – Present Illness Program Medical knowledge in PIP is organized in frame structures. Knowledge base can be constructed with a special treebuilding editor or with a learning module. A segment of decision tree structure attribute1 Burn test orange yes _______ red yes no _______ _______ Compound-38 blue no yes _______ no Solubility test _______ Compound-39 . . called a blackboard  Control Information .Differs from pure forward or pure backward chaining .Either direction may be chosen dynamically at each stage in the problem solution process. . .Blackboard System Architecture .Blackboard systems are composed of  a number of knowledge sources  a globally accessible database structure.a special type of knowledge-based system which uses a form of opportunistic reasoning. plastic. or wood that form a picture when fitted together. . Also called picture puzzle.jigsaw puzzle A puzzle consisting of a mass of irregularly shaped pieces of cardboard. Components of blackboard systems Blackboard Knowledge sources Control Information . alternatives. .Knowledge sources .Contain current problem state and information needed by the knowledge sources such as input data. Each knowledge source may be thought of as a specialist in some limited area needed to solve a given subset of problems Blackboard . final solutions .may contain knowledge in the form of procedures.Communication and interaction between the knowledge sources takes place solely through the black board.Knowledge sources make changes to the blackboard data.separate and independent sets of coded knowledge . control data. rules. or other schemes. partial solutions. . on the black board.One of the application of Blackboard System Architecture was in the HEARSAY family of projects(speech understanding systems) . . . or possibly in a separate module.Control Information .Monitors the changes to the blackboard and determines what the immediate focus of attention should be in solving the problem.May be contained within the sources. The inference mechanism must be able to extend known situations or solutions to fit the current problem and verify that the extended solution is reasonable.solve new problems like humans. .Analogical Reasoning Architectures . .Will require a large knowledge base having numerous solutions and other previously encountered situations or episodes. . by finding a similar problem solution that is known and applying the known solution to the new problem. possibly with some modifications. Neural Network Architectures Artificial Neural networks Artificial Neural networks ANN are mathematical inventions inspired by observations made in the study of biological system. Loosely based on the actual Biology Can be described as mapping an input space to an output space. Consists of artificial neurons composed of weights and connections. Neurons • Neurons are connected to one another • A simplified model of the neuron Modeling Neurons A simplified model of the neuron I N OUTPUT P U T S Articial neuron can be thought of as a small computing engine that takes in input. . process them and then transmit an output. 3 Z=f ∑Wi Xi i =0 X3 X2 X1 W3 W2 W1 W0 X0 Z . . No output is produced if the total input is less than T .Neural Network Architecture Neural networks Large networks of simple processing elements or nodes which process information dynamically in response to external inputs The nodes are simplified models of neurons.If the sum of all the inputs to a node exceeds some threshold value T. the node executes and produces an output which is passed on to other nodes or is used to produce some output response. The link weights serve to enhance or inhibit the input stimuli values which are then added together at the nodes. The knowledge in a neural network is distributed throughout the network in the form of internode connections and weighted links which form the inputs to the nodes. . General system characteristics and support available. 3. When evaluating building tools for expert system development. 4. 2. Knowledge representation methods available. the developer should consider the following features and capabilities: 1. Inference and control methods available. User interface characteristics.Knowledge System Building Tools - these tools range from high level programming languages to intelligent editors.     Personal Consultant Plus Radian Rule master KEE(Knowledge Engineering Environment) OPS5 System . Documents Similar To natural language processing artificial intelligenceSkip carouselcarousel previouscarousel nextLexical Analyser ParserUT Dallas Syllabus for cs6353.501.07f taught by I-ling Yen (ilyen)Concurrent Context Free Framework for Conceptual Similarity Problem Using Reverse DictionaryPrologAndWatson1-1Syllabuslect6Abney_1983.pdfsan1slide4Stevey's Blog Rants_ Rich Programmer FoodSystem ProgrammingM.Tech. in Computer Science & EngineeringManaging Uncertanity in Text-to-Sketch Tracking ProblemHirarki of LanguagesExploiting Learners Tendencies for Detecting English Determiner Errors Ryo Nagata and Atsuo KawaiMeasuring Semantic Relatedness Between Two Wikipedia ArticlesSystem ProgrammingA Survey on Relation ExtractionMC0073 SEM3 SMU 2011Ficha de Avaliação/Trabalho de InglêsClassification Review IJRS 2007Minor ClasesMaking DidFail SucceedFull SyntaxQuiz1.wi11JavaCC IntroBirds View on Human LanguageIndex.pdfDHSch2part3A Survey and Analysis on Classification and Regression Data Mining Techniques for Diseases Outbreak Prediction in DatasetsMore From Lince SebastianSkip carouselcarousel previouscarousel nextAbstract Windowing ToolkitExport Tables by Command SQLExport Tables by Command :-SQL New Text DocumentError Correction and DetectionMSQL Export Tables by CommandType ConversionHardware Support for Mutual ExclusionContinuous TestingMind Reading Cell PhonesSelection sort in C data structure(CsLince)IP SpoofingCompiler - IntrodFooter MenuBack To TopAboutAbout ScribdPressOur blogJoin our team!Contact UsJoin todayInvite FriendsGiftsLegalTermsPrivacyCopyrightSupportHelp / FAQAccessibilityPurchase helpAdChoicesPublishersSocial MediaCopyright © 2018 Scribd Inc. .Browse Books.Site Directory.Site Language: English中文EspañolالعربيةPortuguês日本語DeutschFrançaisTurkceРусский языкTiếng việtJęzyk polskiBahasa indonesiaSign up to vote on this titleUsefulNot usefulYou're Reading a Free PreviewDownloadClose DialogAre you sure?This action might not be possible to undo. Are you sure you want to continue?CANCELOK