Technical Support Chatbot

FACULTY OF AUTOMATION AND COMPUTER SCIENCECOMPUTER SCIENCE DEPARTMENT TECHNICAL SUPPORT CHATBOT LICENSE THESIS Graduate: Sorin Nicu MARIAN Supervisor: S. l. Dr. Info. Camelia CHIRA 2017 FACULTY OF AUTOMATION AND COMPUTER SCIENCE COMPUTER SCIENCE DEPARTMENT DEAN, HEAD OF DEPARTMENT, Prof. dr. eng. Liviu MICLEA Prof. dr. eng. Rodica POTOLEA Graduate: Sorin Nicu MARIAN TECHNICAL SUPPORT CHATBOT 1. Project proposal: The present diploma thesis presents the development of an application which aims to replace the first level technical support with a chatbot that responds to simple technical issues and maintain its responses through sentiment analysis reports. 2. Project contents: Presentation page, Introduction, Theoretical approach, Analysis and design, Implementation, Testing and validation, Conclusions, References and Appendix 3. Place of documentation: Technical University of Cluj-Napoca, Computer Science Department 4. Consultants: S. l. Dr. Info. Camelia CHIRA 5. Date of issue of the proposal: November 1, 2016 6. Date of delivery: June 14th, 2017 Graduate: ____________________________ Supervisor: ____________________________ FACULTY OF AUTOMATION AND COMPUTER SCIENCE COMPUTER SCIENCE DEPARTMENT Declaraţie pe proprie răspundere privind autenticitatea lucrării de licenţă Subsemnatul(a) Sorin Nicu MARIAN, legitimat cu CI seria SM nr. 506835 CNP 1940410303941, autorul lucrării: Bot Conversațional pentru Suport Tehnic, elaborată în vederea susţinerii examenului de finalizare a studiilor de licență la Facultatea de Automatică și Calculatoare, Specializarea Calculatoare din cadrul Universităţii Tehnice din Cluj-Napoca, sesiunea Iulie 2017 a anului universitar 2016 – 2017 declar pe proprie răspundere, că această lucrare este rezultatul propriei activităţi intelectuale, pe baza cercetărilor mele şi pe baza informaţiilor obţinute din surse care au fost citate, în textul lucrării, şi în bibliografie. Declar, că această lucrare nu conţine porţiuni plagiate, iar sursele bibliografice au fost folosite cu respectarea legislaţiei române şi a convenţiilor internaţionale privind drepturile de autor. Declar, de asemenea, că această lucrare nu a mai fost prezentată în faţa unei alte comisii de examen de licenţă. In cazul constatării ulterioare a unor declaraţii false, voi suporta sancţiunile administrative, respectiv, anularea examenului de licenţă. Data Sorin Nicu MARIAN 14.07.2017 _______________________________ Semnătura ........ Motivation ..2...........2.... Primary objectives ......................................... Background .......... Project context ............................................................... 7 2.....................................1.3.....Table of Contents Chapter 1........................ Analysis and Theoretical Foundation ............................................ Project motivation ...............5...... 1 1.......................... 11 3.......................................................... 19 Chapter 4.......................................... 3 Chapter 2.......... Detailed objectives............. 3 1...................................... 21 4.... Input text classification ..........................1...............1.............................................................................. 27 1 ......1........................................ Key concepts..3.................. IBM Watson .............. Bot structure.................. 10 3...... 18 3...................... 9 3...................................4... 26 4...................... 13 3.................................................................1..................... 21 4............................................... Current status ............................................................................................ 2 1............................... Related Work ...........1..... 16 3............................................ Technical Analysis............................1..........................3..........................3........................2..............................................2......2........................................................3...... Project proposal .........2...............................2.................................... 9 3............................................... Introduction .....1..... 11 3...... 9 3....................... 23 4...........................1............................................ 1 1.........................................................3....................2.....................................4................................. 26 4..................................... Natural Language Processing ..................................................................................4..............................................1...............2.. Project Objectives .................................................... Bot Framework ...................................................................................... Intent detector models ......... 15 3.......1......................................................3....................................... Microsoft Bot Framework ....... 21 4...............................1...2...1...................................................................... Functional requirements ...............2...............................2... 7 2..................1.............................................................. Chatbots ............................. 8 Chapter 3........................... Microsoft LUIS..............1.........................................1..... Sentiment Analysis ......... Chatbots overview .................. Technical Support ............. Functional Analysis .................................................. 1 1............................1... 2 1........... Bibliographic Research .................................. 14 3............................................1............ 26 4..........................................3..................... Customer Service ........1............. 1 1................................... Main use cases .............. User Summary ...............................3................................ 14 3........................................... 4. Deployment Diagram ................................................... 35 5...................................3...........1......................................... Main technologies: an implementation perspective ............. Sentiment Analysis ..................... Contextualization ........ 42 5................ High-level testing..........3................................ Layered Architecture ......................................................................................... 57 8................................. 51 Chapter 7...... Conclusions.......................................................2.........................2.. Bot Application Architecture..............................4. Common Data Access Layer .......1..... User Manual......... Sentiment analysis use of scores ........ 57 8......3......... Achieved results............1........................... LUIS application implementation ....2..........1...... Low-level testing .. Business Layer common implementation detail .... Natural language understanding improvement ......................1........................ Testing and Validation .......3...... 43 5.......1...................2....... 60 2 .................................................................................................3............ 57 8.......................................... User’s manual ................... 58 8..............................4....................................... 49 6.................. 31 Chapter 5....... Contributions ..... 54 7.....1........................................................ LUIS ................................................3.................................... 59 Bibliography ............................................. 40 5............ 46 5......... 4.................. 49 6...............................................................3..........................2.. 44 5........................ 53 7......................................................................2.............................. 36 5..................2.................... 41 5.........2..1........ V-Model ......................................................................................3...............................................4.............. Publishing the application ...................................................................................... Installation Manual ............................................... 29 4............ 58 8.... 40 5.........................................................................1.................. Future work ............. 44 5................................................................2.........................1........................................1.4.................................... 53 7............................................1..................... Detailed Design and Implementation ....................... Real-time decision making ...................................................................................... 53 7..............1.......................3....2.........................................3.. Architecture overview .................. 57 8..... 56 Chapter 8.............................................. 49 6.. Database modelling .........................................3............................................. Reports Web Application Architecture ...... 35 5............................. 47 Chapter 6................................ Bot framework dialog management ..................4. .............................. 15 Figure 3...... 17 Figure 3................2 Flow of events diagram of the „Input requirement” use case .....................2 ...........6 Utterances of a built-in intent „Calendar................................... 41 Figure 5... 38 Figure 5......2 24/7 People that responded to a survey concerning the 24/7 availability of the technical support of companies [6] ............... 13 Figure 3.. 31 Figure 4..............4 IBM Watson Workspace example [12] .................................... 39 Figure 5................................................................................1 A sample dialog with ELIZA [9]............3 ..................... 9 Figure 3............................. 19 Figure 4..................................................... 10 Figure 3............................ 28 Figure 4....................................................... 42 Figure 5...1 Email trends of customers based on age group [6]....................... 16 Figure 3...................................................3 Bot Framework Architecture [11] ..............................2 Overall sentiment score function of all conversations ...................................................................................10 Implementation detail of getting a Subscription Model in the form of a TV Subscription Model....................................8 Basic example of results from the Microsoft Text Analytics Service [19] ........................ 35 Figure 5.4 Final form of the Entities Database model ............9 Manually plotted chart representing the user reply sentiment analysis of conversation from Table 4................ Figure 1................................ 11 Figure 3...3 Example of an open incident created in „ServiceNow” ticketing system [16] ................... including external services ... 40 Figure 5........................8 ICE (Interactive Classification and Extraction): Tool at Microsoft Research for interactive learning [13]...........................3 Intermediate form of the Entities Database model .............5 Data Access Layer Conceptual Architecture .... 7 Figure 2..........................7 BaseDomainService abstract class simple implementation ................... 14 Figure 3.......................................................6 Communication between the Data Access Layer and Business Layer 40 Figure 5......................... 41 Figure 5.................................................. 4 Figure 1...........................................7 Prebuilt domains that can be added to a LUIS application [18] . 34 Figure 5. 37 Figure 5...................................1 Main Use Case Diagram ..........9 Actual conversation with „Poncho” the weather expert chatbot from Facebook ................. 25 Figure 4..................... 30 Figure 4...........................................5 Mathematical representation of an Intent i [13] ...... 23 Figure 4..............................8 Highlight of the UnitOfWork object inherited from the BaseDomainService Class .. 5 Figure 1.......... 27 Figure 4................................................................... 8 Figure 3................................................................Add” [18] .............. 29 Figure 4...2 Initial Entities Database model .............................................................................1 Overall architecture of the application......10 Manually plotted chart representing the user reply sentiment analysis of conversation from Table 4..........5 Simple example of dialog hierarchy with IScorable [17] ...........................9 Architecture of the Bot Application module ...............................3 Unsatisfied customers regarding on-hold calls [7] .... 32 Figure 4.................................1 Example of communication of the user with the Technical Support Chatbot ............................................. 43 3 ...............................................................7 IBM Watson’s process to derive a response from a question [14] ........................ 24 Figure 4......... 6 Figure 2...................................................................2 Relationships between classes of software-based dialog systems.................................... [10] ...4 Simple example of dialog hierarchy without IScorable [17]....................................................6 Processing pipeline used for personal assistant dialog systems [13] ..... ............. 44 Figure 5......... 50 Figure 6..........2 Real-time request information given by the Bot Framework Channel Emulator........................................................................4 Overview of the present channels on which the Technical Support Chatbot is published [21] .......3 Homepage of the Bot Application endpoint URL [21] .................... 49 Figure 6............. 55 4 ...12 Class Diagram of the Dialogs folder within the Presentation Layer ...............................................5 Sentiment Score function of specific intent conversations.................. 53 Figure 7......... 55 Figure 7................................................14 LUIS application list intents with the number of utterances for each intent .............................1 V-Model Validation Testing Model .3 Chart representing the number of requests per day made to the LUIS application....................................1 Deployment Diagram .......... 47 Figure 5...............15 Example of Luis intent identification in the context of ScorableMakePayment Class ....... shown in the LUIS Web interface .............................................. 43 Figure 5.................................... 48 Figure 6.......................13 An implementation of the Task GotChannel() method that represents the process of the ScorableTVIssueDialog ..................11 Conceptual architecture of the Reports Web Application ............................................. 52 Figure 7....... 46 Figure 5................................................................4 Sentiment Score function of all conversations ......................... 51 Figure 6...........................................................................................2 All resources of the Azure Portal Server [22] ................ 54 Figure 7........................16 Scaling conversations to 100 replies (left) and to 20 replies (right)....................................... Figure 5.. 45 Figure 5............. 51 Figure 6........................................ The companies that offer this type of technical support are known as managed service providers. Customer Service Customer service is the supply of service to customers before. different ticketing platforms that may include live support or directly on a website. Technical Support By definition. technical support constitutes a plethora of services which certain companies offer as assistance to users of different technology products (software or hardware) which are supplied by the respective companies. mostly represented by a discussion board for users. From that view. phone. The perception of attainment of such actions is dependent on employees who can fit themselves to the personality of the client. Chapter 1 Chapter 1. customization or other. this support covers products that the mentioned companies have sold. but he also pays a technical person based on a pre-negotiated rate when he encounters a problem. In the present. Customer service plays a critical role in a system's capacity to produce profit and revenue. either for free or for a certain fee.1. Crowdsourced technical support is a low-cost solution. Customer service holds the high status a company assigns to customer service relative to units such as piece of work innovation and pricing. An example of such a discussion board constitutes a forum website on which the users of the companies’ products to interact. Also. Project context 1. The upcoming sub chapters will present the categories of technical support by means of coverage [1]. Usually. Call in is the most common form of technical support in the industry. 1. Block hours is another form of technical support which may constitute paying a minimum fee for providing a service or purchase a number of hours upfront based on an agreed price. customer service should be involved as part of a general approach to regular improvement [2]. As a general rule.1. there are few limitations to the channels over which technical support can be delivered. a form of technical support which is not provided by companies is the help of experienced users over the Internet. the customer pays for the technical products (software or hardware).2. during and after an acquisition.1. the provided services may address specific problems rather than the provision of training. 1 . The managed services team receives a list of well-defined requirements that they must fulfil on an ongoing basis for a fixed fee. Introduction 1. Managed services is a type of technical support that includes help desk and 24/7 monitoring of servers or different services. Last but not least. it can be delivered via e-mail. in general.1. In this type of support. 1.4. as well as French. multilingual and cross- language information retrieval (CLIR). Natural Language Processing Natural Language Processing (NLP) is an area of research and application that explores how computers understand and manipulate natural language text or speech in order to achieve meaningful results. The base of NLP relies on a number of disciplines. Together with the high processing power and the high amount of available data. namely. The generalization concept behind it was simple and founded on keyword matching. computer and information sciences. an NLP algorithm can provide a much faster alternative than certain commands from physical input devices. artificial intelligence and robotics. The idea of chatbot systems originated in the Massachusetts Institute of Technology (Weizenbaum 1966. Chapter 1 Customer Support is a particular type of customer service provides the user the information to make cost effective and correct use of the product. Other automated means constitute touch-tone phone with a main menu using the keypads as options (i. different input method exists with the use of speech to text conversion. Chatbot technology integrates a language model and computational algorithms to emulate informal chat communication between a human user and a computer using natural language. nowadays different automated means are implemented. and psychology. Transcribed dialogue corpus has been used in software to machine-learn conversational patterns to create a range of chatbots speaking various languages including varieties of English. maintenance. training. Chinese and many other.1. and expert systems. A primary example would be an Internet site. 1967). but in the present. The input was expected from a keyword in the past. Because of the need of automation and the possibility to solve it. speech recognition. Chatbots A chatbot is a machine conversationial system which interacts with humans via natural conversational language. artificial intelligence. It may include the following: installation. which has the advantage of 24-hours a day service and is much more cost efficient than to have human customer service stay in 24-hours a day.1. A huge advance in speed is obtained when combining with speech to text recognition [3]. The fields on which NLP can be applied on. Automated Customer Service constitutes a form of customer service that functions without the help of humans. 1. mathematics. 2 . include a number of fields of study. researchers aim to gather knowledge on how people understand. where Weizenbaum implemented the ELIZA chatbot to emulate a psychotherapist. process and make use of language. NLP constitutes an alternative to the classic mouse and keyboard (or touchscreen nowadays). „For English press 1”). user interfaces.3. natural language text processing and summarization.e. This is done in such a way that appropriate tools and techniques can be developed to prepare computer systems understand and manipulate natural languages to perform desired tasks. such as machine translation. electrical and electronic engineering. In the majority of the cases. Through natural language processing. customer service is provided by humans. linguistics. although this domain intersects with Technical Support. Project proposal This thesis focuses on a development of a Technical Support Chatbot that replaces the first layer of technical support. in order to create a chatbot that is as contextual as possible and has a great problem-solving rate. Moreover. This concept is enhanced by the external Language Understanding Intelligent Service. detailing attributes based on their criteria. Because of the amount of the existent data. In the domain of Technical Support. 1. For example. the Technical Support Chatbot would take part in the Managed Services. it can provide meaning extraction from natural language input. Many of the Technical Support Chatbot’s responses are automated.3. but it is the same base principle. the sentence is mapped according to a rule associated with the keyword. It provides fast responses based on the matching keyword generalization concept. Chapter 1 Based on the mentioned concept. The last and most specific domain in which the Technical Support Chatbot takes part in is obviously the domain of chatbots. great contextual enhancements can be made within the first stages of usage. if the statement contains the keyword “mother”. A single second delay in a website loading time can result in a 7% loss in conversion.2. so a therapist should encourage the patient to open up about their family. if a keyword is matched. 1. It just matches the keyword and answers with a standard response [4]. placement in each sub category will be highlighted. people are used to instant results over everything. The great enhancement is brought by an external Language Understanding Intelligent Service. a connected free remark or an earlier transformation (under certain conditions) is retrieved. It is a 24/7 service available on different channels and it requires a very well specified list of requirements that it needs to satisfy. In the domain of Customer Service. By integrating it in an application. it belongs to the Automated Customer Service. ELIZA can reply “Tell me more about your family”. Furthermore 40% of web users will abandon a website if it takes longer than 3 seconds to load [5]. 3 . the results have to be extracted more rapidly than ever in order to increase customer satisfaction. If there is no keyword found. The difference between ELIZA and a therapist is that the program does not really ‘understand’ this psychological strategy. The reason behind is obvious. This rule is based on the theory that a mother and her family are central to psychological problems. no matter what the value of that result is. In order to place the Technical Support Chatbot in the contexts mentioned above. Project motivation In the century of speed. The Technical Support Chatbot’s stand-alone Natural Language Processing is at a primary level. it’s almost 2 times less. they must try to satisfy this need on constant availability over different instant messaging applications. Chapter 1 Figure 1. These statistics prove that there is an ascendant trend of the messaging applications which will further increase in the future.5 times less frequently than other means of communication. together with greater internet coverage.1. This means there are now 2. In order for companies to assure that they understand the customer needs.1 billion messaging application users. Another key argument sustaining this point is that smartphone functionality has been greatly increased in the last years. consumers under 24 use email 3. For 25 – 44-year olds. 4 .1 Email trends of customers based on age group [6] Based on a study depicted in Figure 1. 2 24/7 People that responded to a survey concerning the 24/7 availability of the technical support of companies [6] Furthermore. the same study.2. Chapter 1 Figure 1. There is a customer expectation for the instantaneity and convenience the consumer experiences when talking with peers to also happen with the support provided by companies. 5 . 7 days a week. analyzed the answer of customers when they were asked if a business should respond to them 24 hours a day. 50. As shown in Figure 1.6% agreed (on different levels) that a business should ensure this type of support. 3. giving a polarity result (0% -negative response. numerous costly trainings are needed. This includes crowdsourced technical support or even more precarious ways in which results can be extracted from data. a comparison of the quality of the service that can be obtained from a Technical Support Chatbot with other types of human technical support can be drawn. By using this. In order to achieve this for human technical support. A solution for this problem is to constrain that domain even further to a level that is still relevant but minimizes training time and maximizes accuracy. they compensate each other. 100% . all can be standardized by means required from the company. Due to current resources and machine learning algorithms. reply by reply. the efficiency can be verified and adjustments of the current algorithm can be made in order to achieve a greater performance. The latter provides an end result that can prove the company that may use this product that it satisfies the customer needs. human technical support is cost efficient support with ensured 24/7 availability.positive response). based on the statistics mentioned above. Although chat bots and emotional analysis are solutions to 2 different problems. The findings in this report suggest that clients want to use texting for customer service activities and are moving their business to companies that satisfy this need. the results can be put to an emotional analysis which classifies the given text. In conclusion. An advantage over the crowdsourced technical support. from another study show that most customers get frustrated being connected to a phone or computer while waiting for customer service help.3 Unsatisfied customers regarding on-hold calls [7] Other important statistics which is depicted in Figure 1. by the use of natural language processing. like the user mainly searching and filtering the information he needs. After the bot-human conversations. Another notable advantage is the standardization of the messages. 6 . chatbots require high training time to develop a decent amount of accuracy on a relatively small domain. This frustration is certainly understandable. Throughout the responses of the Technical Support Chatbot. The biggest advantage over the other category. is that the user can state his information needs in a more accessible way. Chapter 1 Figure 1. Chapter 2 Chapter 2. Figure 2. the emotional analysis done behind.1.1. Project Objectives 2. Also. It can respond to simple problems or requests and forward to the second level support if it is not capable of solving them.1 Example of communication of the user with the Technical Support Chatbot The first and principal is the Bot Application which has the main objective as interaction with the user. it has the ability to improve the solution in the context of the problem area. by analyzing the Technical Support Chatbot by means of objectives. This includes the actual communication with the user. persisting information into the database and interaction with the first layer of the human technical support team. 7 . Furthermore. with an example illustrated in Figure 2. as mentioned before. we can split the application in two. This can be achieved by analyzing the results of the reports and charts that represent the bot chat performance in some problem categories and improving the classification done by the Language Understanding Intelligent Service. Primary objectives The primary objective of this thesis is to design a Technical Support Chatbot that can act as a first layer of support in a team of technical support and can be greatly enhanced within the first months of use. during use (in production). Correctly identify the user .2.Prove correct ticket forwarding process to the human technical support team 8 .2. as depicted in Figure2.2 Overall sentiment score function of all conversations The second main objective is the ability to improve the Technical Support Chatbot.Procure an accurate display of the overall sentiment scores chart . Here the administrator can view the sentiment score charts.Assignable – provide the responsible .Prove Multi-channel support integration .Publish the Reports Web Application on a publicly addressable server . These objectives were chosen conforming to the SMART criteria [8].Ensure the security of the Reports Web Application through authentication with an administrator account .Procure an accurate display of the specific sentiment scores chart based on intent classification .Ensure security of the database behind the bot application . Chapter 2 Figure 2. the administrator can adjust the Language Understanding Intelligent Service to better classify the replies of the users by adding more utterances in the areas which the administrator identifies lack of performance.Prove the positive impact of the Technical Support Chatbot by the objective analysis of the overall sentiment scores chart and identifying the ascendant slope . Detailed objectives In the upcoming paragraph. Moreover.Realistic – tailor to realistically achievement . These charts contain the average sentiment scores function of all the replies in all the conversations.Measurable – quantify a progression indicator . a detailed list of all the objectives of the Technical Support Chatbot will be described. In this way.Specific – target a specific area for development . these charts can be filtered by means of intent. This part is done with restricted web applications called "Reports". SMART is a mnemonic acronym that stands for the following criteria: .Correctly identify the start and end of a conversation .Publish the Technical Support Chatbot in the Skype Directory . below can be found the detailed list of SMART objectives: . 2.Time-related – specify time bounds of the achievement With respect to the outlined SMART criteria. In the first subchapters. more explicitly: Microsoft Cognitive Services and IBM Watson. Chapter 3 Chapter 3. As depicted in Figure 3. Bibliographic Research The current chapter focuses on placing the Technical Support Chatbot in a projected timeline of the Chatbots domain. highlighting evolution and impact. ELIZA lacked such capabilities. It used keyword matching and minimum context identification and it could not pass the Turing test.1. Afterwards. Moreover. the best findings in technologies are discussed. Original systems like ELIZA [8]. Chatbots overview 3.1. were built as a proof of concept to pass the Turing Test (the computer system is put to a test by a human interrogator.1. Even 9 .1 A sample dialog with ELIZA [9] Radziwill [10] highlights the history of chatbot technologies which is very vast and dates back to 1960. the human interrogator has to mark it as “intelligent” as a human). the current status of the Chatbots domain is described. 3.1. Background Figure 3. brief background information is given in order to get a feel of the starting points of chatbots. in order for the computer system to pass the Turing test. not for its capabilities. but for providing a foundation for the Artificial Intelligence Markup Language (AIML) which is used to declare rules for pattern-matching. Chapter 3 though it was very primitive. humanoid robots). animals. Current status With the help of open-source code and Software as a Service (SaaS). ELIZA constituted the base of the following chatbot systems and was a great step in this domain. An example of IVR is typically used for Telephone Service Providers: „For English Press 1”. as Radziwill points out in [10].2. In 1980 ALICE was developed [10].1.) or imaginary (example: avatars. They mislead humans into thinking they are real in many contexts and sustained conversations. [10] As depicted in Figure 3. up to a fifth of the comments and responses on Twitter were driven by fully or partially 10 . This chatbot marked another great step. chatbots are a sub-category of Dialog Systems and more specifically of Conversational Agents. IVR systems constitute also Dialog Systems. 3.2 Relationships between classes of software-based dialog systems. chatbot systems have greatly improved in the present time. AIML is based on eXtensible Markup Language (XML) and it supports the majority of chatbot platforms used today. Furthermore. chatbots were participating in a third to a half of all online interactions”. cited from [10]. Embodied Conversational Agents are virtual forms of entities that can exist in real life (example: persons. A key argument sustaining this point is. „Between 2007 and 2015. etc. Furthermore.2. but they are not used as Conversational Agents in the majority of the use cases. This high percentage of the presence of chatbot systems in the online medium is possible due to their increased performance. “In the 2016 US Presidential election. Figure 3. Also. together with easy guides to publish the chatbots. a direct line chat can be inserted in a page of a certain web application.1. with Microsoft Bot Framework having many advantages over the bot structure of IBM Watson. In the following sub-chapters. namely Direct Line. Through an HTML Embed code snippet. eventually. Microsoft Bot Framework Figure 3. contained in the Input text classification part is that IBM Watson does not have a separate service for this need. The basic definition for Dialog is the following: Dialog is an entity that helps to model a conversation and to manage a conversation flow. alternative flows of the same use cases should be handled in order to have an organized and correct structure which encourages reusability. This feature is lacking in the IBM Watson. Channels have the same meaning of concept in both technologies. Bot structure Both technologies offer support for multiple channels.3 Bot Framework Architecture [11] 11 . A dialog can be composed with other dialogs to maximized reuse. etc. An example. those technologies included. Further details are different. the analysis will be done with respect to a concrete structure. alternative technologies for building a Technical Support Chatbot will be discussed. is the Dialog. Within a Dialog. “Skype”.2. resulting in the increase of certain social statuses which had a great impact on the entire globe.” This was unfortunately intended to artificially increase the number of followers. 3. Examples of channels are: “Facebook Messenger”. Although they have different structures. The respective technologies are Microsoft Cognitive Services and IBM Watson. Chapter 3 autonomous Twitter accounts. All is done within a single API.2. the principal component of the bot structures in general. There is another advantage for the Microsoft Bot framework regarding this point of view. 3. .” when the „Typing” activity from the context of the bot occurs). like “Text” or “From”. The start of a dialog in a bot application is managed by the Message Controller.. Within the controller certain actions can be implemented if an activity occurs (example: Type „. Chapter 3 In Figure 3. managing Dialogs from a primary level perspective is done in the main Dialog. which can be used to show graphical output so that the user is informed if the bot is “thinking”. a Scorable class can be implemented for each Dialog class which grants the score. Dialog flow is a concept for managing dialogs within a chatbot application. it classifies this information and sends output data back to the Bot API. 12 . The score means the probability of a Dialog to be the right choice for a given input from the user. dialogs are kept in the Dialog Stack. Moreover. regardless of the actor that generated it (user or bot) and has metadata which can be used to identify brief contextual information. A medium complexity approach constitutes having scores that are integers (either 1 or 0). the Microsoft perspective of an architecture of a bot can be observed. By viewing activity from this perspective. essential information can be extracted. A very primary example would constitute the “IsTyping” activity. LUIS) takes as input data the messages received from the user through Microsoft Bot framework. Activity represents the concept of an event. Microsoft Language Intelligent Understanding Service (short. Integration with the Language Understanding Intelligent and multi-channel support are highlighted. With the IScorable Interface. Another example that would be higher in the Activity hierarchy is the “Reply” Activity. A more complex approach to manage Dialogs dynamically is with IScorable. brief descriptions of the particular concepts used in the Microsoft Bot Framework are mentioned. Here there are treated all the input text classification that may lead to additionally creating new Dialogs. namely Root Dialog. the current Dialog is added to the top of the Dialog Stack.3. However. Whenever a new dialog is created and a message is processed within that Dialog. A high complexity approach has float scores varying from 0 to 1. Furthermore. namely Input text classification. a comprehensive platform for conversation management can be built. classification is done by searching in the decision tree for a node that contains the condition that is satisfied. the Workspace can be accessed from a web interface with many suggestions from IBM on how to model the present entities. Input text classification The current subchapter focuses on finding the similarities and differences of the two mentioned technologies with respect to input text classification. while Microsoft goes with a more SaaS (Software as a Service) approach. 3. Through it.1.4.4 IBM Watson Workspace example [12] Although both technologies have a framework. Dialog Flow represents the management of a conversation with the help of decision trees.3. Chapter 3 3. As depicted in Figure 3. The “brain” of an IBM Watson chatbot is within the same place as the “body”. Both use “Intent detector 13 . the main difference is that IBM Watson does not have very strict borders to separate the bot structure from the input classification. but that will be later covered in the following sub-chapter. What is done in the Microsoft Bot Framework Dialog. IBM covers in the Dialog Flow.2. Further complexity can be built with the help of the decision tree algorithm. Workspace . Dialog – this concept is viewed as a service.1. IBM Watson Bot structure Figure 3.as mentioned in the previous paragraph. all the information is placed here. Based on the user input. Contextual conditions are implemented in nodes. If y = 1 the formula indicates that the intent is present in the utterance and if y = 0 indicates it is not. “GetAllAlarms” are possible intents. The model can be estimated in many ways. By using this type of classifiers. the key concepts have to be explained. A set of named intents may be that corresponds to actions that are specific to the application may be defined.5: Intent i is a model of Pi(y|x) structure. the “SetAlarm”. Chapter 3 models” as classifiers. the (i = ∅). Intent detector models In both mentioned technologies. but this implementation detail is not made publicly available for either of the mentioned technologies. It may be a sentence: “Set an alarm at 8:00 AM” or a fragment: “8:00 AM alarm”. a more mathematical description of an intent can be found in Figure 3. An Entity can be viewed as the noun in a sentence but in a context of an Intent. sequences of words can be mapped to sets of pre-defined or contextualized intents.3. they are not always well-formed and there can be vast variations for a particular intent. and y stands for the binary variable. For example. for instance by using machine learning algorithms such as Support Vector Machine [13] and Deep Neural Networks [13]. in an Alarm application. in the Utterance “Set an alarm at 8:00 AM” we can identify the “Time” Entity. but for further understandings. more specifically represented by “8:00 AM”. 3. 14 .5 Mathematical representation of an Intent i [13] According to Williams [13] . “Intent detector models” classifiers are used as the main input text classification algorithm. Figure 3. It represents an instance of an object relevant to the user intent. the formula becomes: P∅(y = 1|x). If the input text is not present in any domain utterances of all intents. Key concepts An Intent represents the purpose of an action a user wants to make expressed as in a user input. Due to usual user input behavior. where x stands for the words in the utterance.2.3.1. For example. 3. An Utterance is the textual input from the user that the application has to interpret and it represents an example of an Intent. identifying. After extraction. relationally and structurally. “Identity detector models”. Built-in intents can be observed (example “ReadFromCalendar”) which infer the user intent. IBM Watson An enhancement that IBM Watson brings.6 the process of classification of a user input (text or speech) on a concrete example can be found. Chapter 3 Figure 3. When it comes to text Watson doesn't just look for key word matches or synonyms.3.6 presents a case study for digital assistants’ scheduler functionality. mostly entities such as the date “today” or “noon”. in this case. by breaking down a sentence grammatically. In that latter function call response management is handled. like a search engine. besides the algorithm mentioned in the previous chapter. 15 . 3. It shows how Intent detection is separated from the Entity extraction (parallel) and how a function call can be made after classification. entity extraction from the utterance substring is done.3. the entities are mapped to canonical forms such as: “2017-03-22”. through entity resolution. This is very different than simple speech recognition which is how a computer translates human speech into a set of words. this technology has a great capability of understanding text of a higher size order. Moreover. Watson tries to understand the real intent of the user’s language and uses that understanding to possibly extract logical responses and draw inferences to potential answers. In parallel. It actually reads and interprets text like a person does. Figure 3.6 Processing pipeline used for personal assistant dialog systems [13] In Figure 3. discerning meaning from the semantics of the written material. in this case. By going through the content to into a form that is easier to work with. Any application that initiates conversations with users. This corpus corresponds to all categories of unstructured knowledge. • It develops a range of hypotheses by searching across the corpus for sections that have a potentially valuable response. The steps of the process mentioned in Figure 3. • Watson repeats this process for each of the candidate answers until it can discover responses that surface as being stronger candidates than the others. which is a very challenging step. • Each resulting score is then weighted against a statistical model which can be used to summarize a degree of confidence that Watson has about the candidate answer. it analyzes to extract the major features of the question.7 IBM Watson’s process to derive a response from a question [14] Knowledge corpus is very important for the operation of Watson mentioned in Figure 3. Microsoft LUIS Microsoft Language Understanding Intelligent Service (LUIS) provides increased performance to understand human language and react accordingly to user requests. • Each reasoning algorithm yields one or more scores.3. can pass the user input to the LUIS application and receive the results that provide natural language understanding. Watson assimilates the corpus. Williams [13] mentions a technique called “Interactive Learning” which is used in Microsoft LUIS to solve the problem of rapidly adding intents.7 by High [13]. chatbots.7 can be detailed as follows: • When a question is presented to Watson.4. indicating the depth to which a likely response is inferred. Chapter 3 Figure 3. • It executes a deep examination of the language of the question and the language of each potential response by using various reasoning algorithms. 3. which is one of the main 16 . where classifier labeling. In the top center. Interactive Learning (IL) is a way of efficiently making classification models. and evaluation are done by a single developer. Utterances are shown in the central panel. It requires a broad database of unlabeled data instances. Similar to active learning. such as text of utterances. This allows intent detectors to be built in advance of practical implementation. Figure 3. IL is fit when unlabeled data is abundant but labeling is expensive. model building. This technique is executed on the “Intent detector models” previously mentioned.8 the ICE Tool that does the active learning behind the Microsoft Language Understanding Intelligent Service is presented. The red 17 . a search term or a score from which to sample can be entered. with their scores under the current model shown to the left of each utterance.8 ICE (Interactive Classification and Extraction): Tool at Microsoft Research for interactive learning [13] In Figure 3. The database must contain positive instances. Chapter 3 bottlenecks to scaling applications like this. at least for intents expressed at the first turn. IL integrates active learning but extends it substantially. After each label is entered. and area under the curve (AUC) as a function of how many labels have been entered. Emotional Analysis (widely known as Emotion recognition) is a higher form of sentiment analysis that further categorizes the input text from a different perspective. including confusion matrices.4. the instant replies should be chosen. Second. Sentiment Analysis Sentiment Analysis identifies. Labels can be positive. and returns a score between 0 and 1. In the domain of Chatbots. precision-recall curves. all of the instances labeled so far can be displayed graphically. among many others. it is applied to all of the unlabeled data. Microsoft Text Analytics has 3 key features: Sentiment analysis. In this subchapter research will be solely focused on Sentiment analysis. a model is built on the training set which is used in 3 ways. features generated from part-of-speech tags. and embedded words. For a more accurate. The input features to the classifier include n-grams. Its objective is to find the attitude of a human actor which has provided the input text with respect to the overall contextual polarity or emotional reaction to a document. for example: “Happiness”. The procedure starts by searching for data instances using textual search terms based on their domain knowledge. Features can be added and edited on the left. “Surprise”. but a label. when searching for new instances. “Sadness”. that shows how the input text is placed on a scale from negative (0) to positive (100). which can be changed by clicking. Sentiment score is produced by applying classification methods. Afterwards. or a special “don’t know” label. Third. The searches yield results are then labeled. or event. showing the distribution of scores. negatives. real-time analysis which can be a source for decision making algorithms. interaction. 18 . These emotions are understood to be cross-culturally and universally communicated. sentiment analysis can be applied either on replies or conversation. “Fear”. regardless of size. the model is used to propose labels. giving an overview of performance. practical result is not a number. This is usually measured based on precision and recall over the two target categories of negative and positive texts. each time the model is rebuilt. all the labels are randomly divided between a training and test set. Key phrase extraction and Topic detection. These labels can be. First. extracts and quantifies the affective states of subjective information. how well it aligns with human assessments. without having the user to input feedback explicitly. This type of analysis can be a method for evaluation of products. “Anger” “Disgust”. This speeds up labeling and also gives an indication of the overall performance of the model on unseen data. The accuracy of a sentiment analysis is. The final. which allows drawing samples of unlabeled instances at a particular score. Chapter 3 (False) and green (True) bars indicate their suggested label by the model. The result of sentiment analysis is a percentage ranging from 0 to 100. The performance of the model is shown in the bottom panel. 3. available on Facebook Messenger. As comparison subject.9 Actual conversation with „Poncho” the weather expert chatbot from Facebook 19 . but also by the multi-channel support. Comparisons to chatbots that have publicly available algorithms would not be objective nor relevant to the case. but not on the “Technical Support” domain only. namely “Poncho” [15]. because of the strict business environment which limits to preview only features in order to protect their products. the most popular chatbot from the most popular social media channel was chosen. as most of them are outdated. Related Work This subchapter focuses on a comparative analysis of relevant existing chatbots. Chapter 3 3. It is designed to be a helpful weather expert. Figure 3.5. The Technical Support Chatbot would not be better only by means of natural language processing capabilities. It can send alerts up to twice a day (if the user opts in) with the weather conditions and can answer questions like “Should I wear a jacket today?”. Different responses to users within the same location can be only acquired if the user sends some particular context information. As overview. through the help of “Cards”. meaning that it is a pro-active bot. All the responses to users from the same area are modelled exactly the same. this can have a great impact on the overall performance of the chatbot. Through cards. for example. but the latter offers the great advantage of contextualization and may have increased performance within the specific domain of technical support. Cards are substituent for buttons in the environment of an instant messaging application. because the user has to provide the exact TV Program that does not behave correctly. meaning whenever the chatbot is added to the channel of a user (note: the user has to be a customer of the company that provides Technical Support through the respective chatbot). “Poncho” provides rich user interface and a much faster approach than writing text. are available for the Technical Support Chatbot to work on and can constitute a source of decision making. only capabilities of “Poncho” and the Technical Support Chatbot will be compared. but it does not know the user’s identity. The Technical Support Chatbot does not implement this type of input because of contextualization. pick one of the options represented on the cards rather than writing text. its responses are modeled only by location. More explicitly. In this way. like a response. for example “TV Program is not working”. it will give weather recommendations based on the user location. the Weather Expert Chatbot provides rich user experience and may have greater natural language processing power over the Technical Support Chatbot. Chapter 3 As previously mentioned. It does not make sense to use a default card. In conclusion. for example. A huge benefit of this usage is that it can correct spelling mistakes. A feature that “Poncho” lacks is the contextualization.9. “Poncho” influences the behavior of the user in such a way that he will. As depicted in Figure 3. it gets all the information of the user from the database. It does not wait for user input in order to post messages and it can be seen as a content generator. Although it has a scarcely idea of context. The approach of the Technical Support Chatbot differs. most likely. A main feature “Poncho” provides is the daily recommendations. due to privacy policies. 20 . all subscriptions that the user has enrolled to. It is a different approach from Technical Support Chatbot which always waits for user input to provide a message. Use the application in an Consumer have paid for a educated manner. service/subscription without spam or other or bought a product intrusive ways and would complain about their issues to the Technical Support Chatbot Technical The persons that the . they must be enrolled as customers.Ensure application Technical have access to the maintenance Support Team reports generated by Lead . 4. In order to have the ability to add the Technical Support Chatbot in a channel of their choice. the architecture of the application that integrates all of the above will be presented in the next chapter.1. Check conversation logs Chatbot . “Detailed Design and Implementation”. Analysis and Theoretical Foundation The current chapter focuses on presenting the requirements analysis.Improve performance Table 4.1.Check background Technical Support Team users will be information of the user Support Member redirected to if the profile given by the Second Level problems are not Technical Support solved by the Chatbot Technical Support . The Normal User constitutes the application user with the lowest level of access.1 summarizes all the users of the Technical Support Chatbot application and provides a brief overview of their responsibilities and stakeholder placement. Try to find a solution Administrator The person that will . This has to be done without their explicit 21 .1. Identify the user problem . User Summary Table 4.Analyze the charts the Technical Support Chatbot . Functional Analysis 4. Chapter 4 Chapter 4. Following that.1 User Summary Name Description Responsibilities Stakeholder Normal User The persons that will . technologies and algorithms used in the process of developing the Technical Support Chatbot. This type of user has the higher level of access. together with the conversation logs. nor via email. by adding new utterances to the intents that lack performance. Chapter 4 knowledge. a new ticket within the specific platform should be created. This can be done after the analysis of the charts. The “ticket” is presented as an email. 22 . The Technical Support Team Member is the application user which handles the problem that cannot be solved by the Technical Support Chatbot. The Administrator is the application user that has the responsibility of ensuring that everything works in a correct manner. A ticketing platform should be configured in such a way that whenever an email is sent to the Technical Support Team email. The responsibility of the Normal Users is to use the application with the specific purpose of solving a problem they have with a product or service they are paying for. including the server on which the application is deployed. When the chatbot does not have the necessary knowledge to find a solution to a problem given by a Normal User. in such a way that the chatbot is not publicly available in the specific channel directory. it forwards “a ticket” to an email address that represents the whole Technical Support Team. The subject of the email contains the id of the conversation and the body would contain a short message. An important responsibility of the administrator within the first months of the application usage is to improve performance. The interaction of the Technical Support Team Member with the application is not done via the instant messaging channel. 1 Main Use Case Diagram Figure 4. 23 . Main use cases Figure 4.1 depicts the main use cases of the Technical Support Chatbot application.1.2. highlighting the separation of the 3 actors within each different system. Chapter 4 4. 2 Flow of events diagram of the „Input requirement” use case 24 . Chapter 4 Figure 4. Within this system. Furthermore. He has the option to write as text input a problem he has with a paid service or product. Based on this decision. he has to take the ticket forwarded from the chatbot. the chatbot provides a possible solution that has to be confirmed by the user. Figure 4. The administrator uses the application through the Reports Web interface. The action of forwarding the problem to human technical support is included in the event “A solution is provided” in order to set a clear separation between the “Input problem” use case and the “Investigate forwarded ticket” use case. the chatbot can provide other similar solutions or end the communication. Did the provided solution work? Please answer yes or no?”). interaction between the Technical Support Team Member and the Technical Support Chatbot is done via an email that opens a new ticket.3 Example of an open incident created in „ServiceNow” ticketing system [16] In Figure 4.2 an external Ticketing System environment is depicted. Within this system. which constitutes a replacement for the first layer of technical support in a specified amount of time. he can view the overall performance of the application or chose to view 25 . He is the only actor that interacts with the Chatbot application through the specific channel interface. after the user specifies the problem.2 provides a detailed overview of the single use case performed by the Normal User. Chapter 4 Figure 4. He must provide objective information and try to respect the requests provided as responses from the Technical Support Chatbot (example: „I did not understand that. Depending on the policies and service level agreements of the specific company that the Technical Support Team Member works for. and the analysis of the studies documented in chapter 3. Another use case which can be done by the administrator is improving performance.1. while IBM Watson goes a step higher to extract possible answers directly from the processed input text. to an increased overall performance. They define the function of all the internal and external components included in the Technical Support Chatbot application.1. Microsoft Cognitive Services focuses on a more basic level of natural language understanding that lets the developer formulate the answers. After analyzing the performance charts. Functional requirements The current subchapter provides a list of the functional requirements of the Technical Support Chatbot application based on the analysis provided in the previous subchapters. The administrator should be able to add utterances in order to improve performance 4. Based on the functional analysis illustrated in the previous subchapter. The Normal User shall be able to input a specific requirement via text in the instant messaging channel and he shall receive a response from the Technical Support Chatbot that contains a solution . This results in increased chances of recognizing more forms of user input text regarding the specified intent and. Motivation This subchapter focuses on the motivation behind the decision of choosing a set of technologies for developing the application. the administrator can add new utterances. The Normal User shall be able to add the Technical Support Chatbot in the messaging channel only if he is a customer . in the end. given correct metadata and conversation logs . Chapter 4 specific performance of certain intents (example: “view performance of the InternetIssue intent”). “Bibliographic Research”. 4. The administrator of the application should be able to authenticate given he provides correct credentials . The application administrator should be able to view specific intent performance charts .2. The Technical Support Team Member should be able to investigate a ticket forwarded from the chatbot. The Technical Support Chatbot Team Member should be able to view all the necessary information about a user profile from the external Ticketing Platform System .3.2. the following conclusions can be drawn. Technical Analysis 4. By accessing a specific intent. 26 . The Technical Support Chatbot administrator should be able to view overall performance charts . The functional requirements of the system are: . Sentiment Analysis from the Text Analytics Service In the following subchapters. 4. for example. all the chosen technologies from Microsoft Cognitive Services will be described.2. highlighting the best approach to use them. Microsoft Cognitive Services were chosen as a base layer of technologies.Language Understanding Intelligent Service . The score is given by a matching pattern. In the field of Technical Support. which in this case is handled by LUIS. In line with the conclusions drawn above. but offers great support for a complex and even dynamic structure of responses. because in this way. the approach taken by Microsoft is recommended. which include the following: .Bot Framework . It is appropriate for more complex problem-solving scenarios that can be found in the fields of medicine. based on the best score. The ability to dynamically interrupt a dialog by a certain user input that gives the best score from another dialog is possible with the help of the IScorable interface. IBM Watson does not provide a great skeleton for a complex structure of responses. Chapter 4 The set of technologies included in Microsoft Cognitive Services is more appropriate for chatbots with a medium natural language processing. specific responses can be created and the full control over the response management is granted.2. Bot Framework This technology provides great support for dynamically switching between dialogs at run-time.4 Simple example of dialog hierarchy without IScorable [17] 27 . considering the most important problems that may be encountered. Figure 4. Pretty [16] illustrates a simple dialog hierarchy in the domain of a bank chatbot application. Figure 4. the current dialog is switched and the user can execute the make payment action without going through the whole process. if the user asks to check balance and changes his mind and wants to make a payment instead. 28 . When an interrupt is initiated by the user input text.4. an obvious improvement in the loose coupling design principle can be identified. Only afterwards the user is able to issue an action of making a payment.5 Simple example of dialog hierarchy with IScorable [17] In Figure 4. A key ability that Microsoft Bot Framework lacks is the poor support for easily integrating a dependency injection container. from Figure 4. Without the implementation of the IScorable interface. This is done with the help of the Dialog stack. provided he specifies the destination account and the amount. With the IScorable approach. he would have to continue to the normal process which constitutes checking the current balance and then returning to the “Root Dialog”. if the user issues a make payment action in the middle of another action of checking balance. At return. When returning from the “MakePayment” dialog. the user can check the balance of his current account or of a savings account.3. the dialog is switched back to the “CheckBalance” dialog. the user can opt to make a payment. Within the “BalanceDialog”. the current dialog “CheckBalance” is switched to “MakePayment” dialog. Chapter 4 In Figure 4. In the “MakePayment” dialog. meaning it goes to the top of the Dialog stack. but with the IScorable interface implemented. Pretty [16] the same simple dialog hierarchy is depicted. the second dialog “CheckBalance” comes back to the top of the Dialog stack. The problem is that the dialog classes are serializable and they cannot depend on instances of services that are not serializable. In comparison with the first approach.3. 6 Utterances of a built-in intent „Calendar.5 depicts a simple example of the interface of the LUIS application. 4. More explicitly. Even though it is built-in. it illustrates the “Calendar. it manages to successfully solve the problem.2. the intent can be specifically modelled for the context of a particular application. 29 . an utterance “Meet Mike at 9:00 AM tomorrow” can be added. In this way. Chapter 4 The solution is to use a static dependency injection container that provides static instances of the services in the serializable Bot Framework dialogs.3.Add” [18] Figure 4. even though the referenced services are not serializable. For example. LUIS Figure 4.Add” built-in intent which is provided by LUIS among many others. Cancel” which provide a greater performance rather than implementing these intents manually. In the current domain of Technical Support Chatbots. Chapter 4 Figure 4. “Utilities. LUIS makes use of interactive learning which is a form of active learning. together with utterances in order to set up an application rapidly. the “Utilities” prebuilt domain proved to be useful.7 Prebuilt domains that can be added to a LUIS application [18] As depicted in Figure 4. but it should be scheduled. LUIS offers the possibility to add an entire set of intents. This results in a short down-time of the application and it should not be done while the application is in production. “Bibliographic research”.6. for example once a month is a best practice recommendation. After adding new intents or utterances. the LUIS application requires training. 30 . It provides intents as “Utilities. A very important aspect of the LUIS application constitutes the training. As mentioned in chapter 3.Confirm”. communication with the user within the technical support domain was analyzed.8 the results of the Text Analytics Service are depicted on a default example. It is used to extract information from an input text of any size. The analysis included frequency. Of the provided results. Chapter 4 4. Sentiment Analysis Figure 4. The text analysis includes language detection. key phrases extraction and sentiment detection. size and total number of input requests forms from a user.4. only the sentiment score is relevant to the Technical Support Chatbot application. In order to achieve a high accuracy sentiment detection of the user input. such as replies and conversations.8 Basic example of results from the Microsoft Text Analytics Service [19] In Figure 4. 31 .2. Figure 4. Chapter 4 A clear advantage of extracting a sentiment score from each reply individually over extracting the sentiment score from an entire conversation is the ability to monitor real- time the emotional impact the chatbot responses has over the user.2 User reply sentiment analysis of a simple conversation in the technical support domain Sentiment score User reply text 98 Hello 31 I have a problem with the internet 2 My internet is not working 81 ok 58 I put it 48 now what? 81 ok 73 I'll wait 100 great 99 thank you Table 4.9 Manually plotted chart representing the user reply sentiment analysis of conversation from Table 4. A simple conversation was chosen in order to provide a base for constructing a function of the sentiment score of a conversation.2 32 . but it comes with a cost.2 depicts a simulation of a real-life scenario within the technical support domain. Making a request to the Text Analytics Service each time the user sends a reply to the Technical Support Chatbot may slow down the application. Another advantage of this approach is the accuracy. Table 4. In the first part of the conversation. thank you.3 a conversation with 2 different problems is depicted. Further verifications have to be made in order to exclude them. This represents the transition of the user’s „Hello” greeting intent to the specific problem he has. Chapter 4 In Figure 4. the chatbot provides a solution which has a positive impact on the user. It's solved 70 I have another problem 35 HBO channel shows a black screen since Monday 98 Ok.through the sentiment analysis of replies. If the conversation does not end in an ascendant slope. up to point 3 we can identify a descendent slope. After the problem has been specified. The user’s problem is represented by the minimum point of the function. Table 4. Problem-solving ratio . If the sentiment score function of a conversation has ends with an ascendant slope. A problem consists of wrongfully classifying this type of conversations as successfully solved because of the ascendant slope given by the user satisfaction if he. This can be calculated as following: from point 3 to point 9 the scarcely ascendant slope can be identified. namely problem-solving ratio of the Technical Support Chatbot.3 User reply sentiment analysis of a conversation in the technical support domain with 2 problems Sentiment score User reply text 97 Hi 26 I have an internet problem 8 INTERNET IS DOWN 80 ok 100 Great. for example thanks the Technical Support Chatbot for forwarding to human support. This can be depicted from a scarcely ascendant slope from approximately point 3 to point 9. it can be concluded that the problem of the user has been solved. I forgot that. In Table 4. a more concrete performance metric of the application. 33 . The problem-solving ratio should exclude the conversations which had problems that were out of the Technical Support Chatbot knowledge base and forwarded to the human technical support. it can be concluded that the user’s problem has not been solved. a problem of a possible intent “InternetIssue” and another one with “TVIssue”. This type of conversations may provide inaccurate results for the construction of a sentiment score function of a conversation.9. a manual plotting of the conversation sentiment score function is depicted. thus his increase of sentiment polarity. More explicitly. it should be transferred to the second level support. the 2 problems can be clearly identified. the decision of splitting conversations like the one depicted in Table 4. can be identified a second time in a conversation. then it results the user is getting angry. Chapter 4 Figure 4. If an intent from the „Issues” sub-domain.10 in 2 has been made. This is done with the help of LUIS and Bot Framework. such as „InternetIssue” or „TVIssue”. and as a result.10 Manually plotted chart representing the user reply sentiment analysis of conversation from Table 4.3 and Figure 4.10 the sentiment score function of the conversation from Table 4. For example. Identifying angry replies – through the real-time sentiment analysis of replies. like the problem-solving ratio. which results eventually in a performance metric of the application. the Technical Support Chatbot creates a new conversation in the background. finding out when the user gets angry can be made possible.3 is depicted. In order to provide a better accuracy of the sentiment analysis of replies. without the user knowledge. 34 . By analyzing points 3 and 7.3 In Figure 4. if 3 consecutive replies have a sentiment score below 30. The key modules of the application are represented by the following: .Reports Web Application . Chapter 5 Chapter 5.1 the Technical Support Chatbot architecture is depicted. including external services In Figure 5.1 Overall architecture of the application. database modelling and critical design decisions taken during the development process of the Technical Support Chatbot application. main components descriptions.Data Access Layer Both the Bot Application and the Reports Web Application are based on the Layered Architecture. having a common Data Access Layer to the Entities Database. The Reports Web Application’s focus is getting the conversations data from the Entities Database. processing and showing the overall sentiment score functions of the conversations through a Web interface. Architecture overview Figure 5. 5. Detailed Design and Implementation The present chapter provides the detailed architecture of the system. 35 . references to external services. highlighting the main components and the relationships between them. The Bot Application module focuses primarily on the interaction with the user and storing data that comes from this communication.1.Bot Application . Subscription – represents the subscription the user is enrolled to. In the following subchapters. but not before presenting the database model of the main entities. Chapter 5 The Data Access Layer focus is obviously to separate the logic that retrieves the data from the Entities Database and maps it to the entity model from the business logic through Entity Framework.Equipment – models the equipment that is given when a user enrolls for a new subscription . Internet Subscription . can be TV Subscription. the database suffered two major changes which are detailed below. 36 . 5. each principal module will be further detailed.User – represents a unique instance of the Normal User that interacts with the Bot Application .Conversation – is the model of an actual conversation between the Normal User and the Technical Support Chatbot.Reply – models the activity of type reply and contains information such as Sentiment Score that is accessed by the external services . together with the common architecture of the Bot Application and the Reports Web Application. including the database diagrams.Other tables that were solely used for mapping many to many relationships In the process of development. it is mapped 1 to 1 to the Bot Framework Conversation Class .TV Channel – represents the entity of all the TV channels available in the TV Subscriptions (example: “Digisport”) .2. Phone Subscription.Payment – models an invoice after the user makes a payment . Database modelling The Entities Database has the following main models: . 37 .2 Initial Entities Database model In Figure 5. Chapter 5 Figure 5. When accessing the data in the Business Layer without a transitional mapping from the Entity Framework models to Data Transfer Object (DTO) models and afterwards Business Models. The actual implementation of DTO models and separate Business Models and mapping between them would have also been a difficult approach. it is very difficult to use them in the Business Services. TV Subscription. A key aspect in the initial database form was the decision of modelling a table for each type of subclass subscription (internet.2 the initial form of the database is depicted. which had separate tables for Subscription. but did not have the Payment table. Internet Subscription and Phone Subscription. tv or phone). TV Subscription and Phone Subscription tables. The Payment Table exists in the final form and makes an important subject in the „Input requirement” use case. 38 . Chapter 5 Figure 5. A second approach for solving the Subscription Table problem was found.3 Intermediate form of the Entities Database model In Figure 5. more specifically in the concrete cases of making a payment or getting the latest invoice. that constituted of eliminating the Subscription Table completely and adding the details contained in this table (Name. Description and Price) to the Internet Subscription. The main disadvantage and reason for not implementing this approach is that the many-to-many relationship between the Subscription Table and the Equipment Table would have been very difficult to maintain.3 the addition of the Payment table is illustrated. This is the only difference between the initial Entities Database Model and the intermediate form. Chapter 5 Figure 5. which represents the type of subscription (Internet. Furthermore.4 the final form of the Entities Database model is depicted. in the Business Layer. Internet Subscription and Phone Subscription Tables and moving all the fields from those tables to a single Subscription Table. in the Business Layer. This implementation detail will be further discussed in the following subchapters.4 Final form of the Entities Database model with a single table for the Subscription entity In Figure 5. By having all the information in one place. 39 . all the information from the Subscription Class can be accessed. a „Type” field was added. TV or Phone). The final solution for the Subscription Table problem was solved by dropping the TV Subscription. In this way. a Subscription class can be accessed as a TV Subscription class with the help of an interface. without having to manually map from two separate models in Entity Framework to a single one in the Business Layer (example: number of minutes and messages from the Phone Subscription Table and the price of this phone subscription taken from the Subscription table). it made sense to externalize it as a Class Library so it can be referenced from both main application without any differences.1. Common Data Access Layer Figure 5. with slight differences and similarities. Also. the modelling of the Entities Database is done with the help of Entity Framework.6 Communication between the Data Access Layer and Business Layer 40 .5 Data Access Layer Conceptual Architecture From the conceptual architecture illustrated in Figure 5.Business Logic Layer is further modeled in Services which can be categorized as Common Services and Core Services. Layered Architecture The Layered Architecture or Multilayered Architecture provides a model for creating applications that advocate flexibility and reusability. Figure 5.Data Access Layer – in this particular context.3. In the context of the Technical Support Chatbot Application.Data Layer – consists of the Entities Database and the Administrator Database As previously mentioned. . which will be highlighted in the following subchapters. . the concrete model of 4-tier Architecture is implemented. 5. Chapter 5 5. The implemented 4-tier Architecture has the following layers: . the 4-tier Architecture is implemented for both the Bot Application and the Reports Web Application. . the use of Unit of Work and Repository design patterns can be identified.3.Presentation Layer which can provide a rich interface and heavy client side in the case of the Reports Web Application and a minimal graphical interface and heavy server side in the case of the Bot Application.2. By implementing the domain services in this way. but conceptually. Figure 5. is an abstract class which has a UnitOfWork object from the Data Access Layer. More specifically. Chapter 5 The mapping from the Entity Framework models to Business Domain Models was implemented with the help of Domain Mapper. 5. In Figure 5.8 Highlight of the UnitOfWork object inherited from the BaseDomainService Class All the Domain Services extend the BaseDomainServices. Figure 5.3. initializing a UnitOfWork object is avoided. a concrete example within the ConversationService is depicted. it stays between the limit of the Data Access Layer and the Business Layer. each time a different service is used. 41 .7. Business Layer common implementation detail This subchapter focuses on concrete implementation detail (not structural) that are common to the Bot Application and the Reports Web Application.7 BaseDomainService abstract class simple implementation BaseDomainService illustrated in Figure 5.2. the implementation of the BaseDomainService class. This design pattern is practically implemented as a service and located in the Business Layer.8. This UnitOfWork object is a protected member so that all the Domain Services that inherit this abstract class are able to use it. 7. Bot Application Architecture Figure 5. which already has a certain structure. further enhancement of structuring classes was achieved by applying the 4-tier Architecture.3. Chapter 5 5. as illustrated in Figure 5. 42 .9 Architecture of the Bot Application module Although the Bot Application is built with the help of Microsoft Bot Framework.3. Phone Subscription. which gets its reference from the Subscription Repository through the Unit Of Work. even though all of the fields exist in the Subscription Model and do not have values from the database. the referenced TV Subscription Model has only the specific fields of the TV Subscription. as illustrated in Figure 5. an object of type ITVSubscriptionDomainModel is declared but it references a SubscriptionDomainModel object.8: An ITVSubscriptionDomainModel interface has the needed fields for the TV subscription only (id. they are protected from modification. For example. Reports Web Application Architecture Figure 5.10 Implementation detail of getting a Subscription Model in the form of a TV Subscription Model An important implementation detail of referencing a Subscription Model object from Domain Models in the Domain Services is through the implementation of interfaces that represent the TV Subscription.3. When a TV Subscription Model object is needed in a Domain Service. Chapter 5 Figure 5. type and channels).4. 5.11 Conceptual architecture of the Reports Web Application 43 . in the Domain Service. In this way. which are implemented by the Subscription Model Class. Internet Subscription. ScorableTVIssueDialog. Chapter 5 As illustrated in Figure 5.1. it is possible for the users within the process of a certain Dialog.8. the Reports Web Application has obviously a very different approach from the Chatbot.12 Class Diagram of the Dialogs folder within the Presentation Layer As depicted in Figure 5. This is achieved with the use of the ASP.NET MVC 5. mentioned in Chapter 4. 4 separate dialog classes are implemented: ScorableGetPaymentDialog. and ScorableInternetIssueDialog.4. ScorableMakePaymentDialog.0 Framework. to interrupt and dynamically switch to another Dialog without having to iterate through the whole process. Figure 5. Main technologies: an implementation perspective 5. exposing the charts in a web rich user interface. the Reports Web application architecture is very similar to the Bot Application architecture. Bot framework dialog management Dialog management within the Bot Application is done with the concept of Scorable Classes. with the help of ScorableBase abstract class. Each Scorable Dialog Class has a well-defined process that is implemented through Task methods. 44 . mostly following the same structure within the Business Layer.12. Besides the RootDialog Class which only has the function to initiate the conversation.4. In the Presentation Layer though. 5. the method GotChannel() is called.13. the following: the contract may have been expired and reverted to the standard package.13 An implementation of the Task GotChannel() method that represents the process of the ScorableTVIssueDialog In Figure 5. for example. In this method. The case when the TV Channel is not included in the user’s subscription is. the process of a TV Issue problem can be identified. 45 . After another exchange of replies between the user and the Technical Support Chatbot in which the TV Channel name is provided. Also. the user is presented the possibility to make a call. The StartAsync() method within this Dialog is called when the TVIssue intent is identified by the LUIS service. Chapter 5 Figure 5. resulting in such a case that the user does not have the mentioned TV in his subscription anymore). the user is acknowledged through a message that his subscription does not include the mentioned TV Channel and also the list of included TV Channels. If this case is met. in order to re-new his subscription. a call is made to the TV Subscription Service in order to check if the user specified TV Channel that is not working is included in his actual TV Subscription. The intents were selected mostly with respect to the single use case possible for the Normal User. Chapter 5 5. LUIS application implementation Figure 5.2. MakePayment and GetPayment. TVIssue.14 all the intents used in the Technical Support Chatbot application are illustrated.14 LUIS application list intents with the number of utterances for each intent In Figure 5. 46 . with the most important of them being: InternetIssue.4. within the Web interface of the LUIS application. Chapter 5 Figure 5.15 Example of Luis intent identification in the context of ScorableMakePayment Class Figure 5.15 depicts an example of how the integration of the LUIS intent identification and Bot framework dialog management is implemented. All the other important intents mentioned above are integrated in the same manner. 5.4.3. Sentiment analysis use of scores In order to be able to contain the functions of the sentiment score of all conversations into an average function, the conversations need to be scaled to the same number of replies. Methods for scaling a single conversation and multiple conversations are implemented in the Math Service of the Reports Web Application’s Business Logic Layer. For the graphical representation of the function, a Line Chart was implemented with the help of Google Charts JavaScript library. Unfortunately, this library has a great disadvantage in this particular context. For example, given a conversation with 4 replies, when trying to scale the conversation to 20 replies, the new position of each reply has to be calculated. This is done by calculating a step with the following formula: step = (scale limit) / number of replies – 1. In this particular case, the step is 6.67. Given that, the position of the first reply is the integer part of the newly calculated index, that is: index * step = 6.67 * 0 = 0. The next positions of the replies are: second reply – [6.67 * 1] = 7, third reply – [6.67 * 2] = 13 and the last one – [6.67 * 3] = 20. By scaling the conversations, a much more accurate analysis can be done over them. Another problem regarding scaling is filling up the empty spaces left between the replies that have been scaled upwards. This first step of implementing this is by calculating the number of empty spaces between a reply 1 and a reply 2. The next step is to calculate the average values starting from reply 1 to reply 2 with the help of the mean function. For 47 Chapter 5 example, given 3 empty spaces between 2 replies with sentiment scores of 20 and 60, the 5 empty spaces will be filled with: 30, 40 and 50. Figure 5.16 Scaling conversations to 100 replies (left) and to 20 replies (right) In Figure 5.16, a comparison of scaling to different number of replies is illustrated. The readability of the right chart is clearly higher and that is due to 2 reasons. The first one is regarding the limitations of the Google Charts library, which has obvious problems when smoothing a curved function between 2 very close points. The second reason is that the majority of the conversations within the Technical Support Chatbot context do not have more than 20 replies. Therefore, scaling to 20 replies has been chosen. All the above- mentioned steps are considered when scaling a single conversation. The Sentiment Score function of all conversations uses the scaling of a single conversation, for each conversation from the database, with the help of the weighted average (weight = total number of conversations). The result is very similar to a function representing a single conversation. The first step of having the possibility to calculate the Sentiment Score function of specific intent conversations, as presented in the Chapter 4. Analysis and Theoretical Foundation, is to set a main intent from the most important ones (InternetIssue, TVIssue, MakePayment and GetPayment) for each conversation and store it in the database. This classification is done whenever a Scorable Class is reached. If the user opts, for example to input a new requirement in the context of the same conversation and a new main intent is identified, a new conversation is created in the back-end of the application, without the user knowledge, in order to provide a better accuracy for the problem-solving ratio (1 problem solved per conversation). 48 Chapter 6 Chapter 6. Testing and Validation 6.1. V-Model The current chapter focuses on the process of evaluating the finite software product of the Technical Support Chatbot in order to determine if the specified requirements are satisfied. Figure 6.1 V-Model Validation Testing Model As depicted in Figure 6.1, the V-Model is a form of development process, more specifically a particular extension of the Waterfall model and it includes a structured process for the testing phase. Based on the V-Model test process, a similar approach to test and validate the Technical Support Application was made, starting with low-level tests and finalizing with high-level tests. 6.2. Low-level testing At a low level, different integrated technologies were tested, as it can be seen in the following paragraphs. For testing the SQL database interaction, an SQL script was used to test if a reply that was given by the user would be correctly inserted into the database. Verification was done by checking the id given by the Microsoft Bot Framework context to the id of the reply inserted in the database, resulting in a success. The SQL script consisted of a simple SELECT based on a known id of a reply. Testing other technologies was mostly done manually during implementation. No errors worth mentioning were encountered during testing. 49 2. the connection to the LUIS application and Text Analytics Service were tested successfully. Chapter 6 Figure 6.2 Real-time request information given by the Bot Framework Channel Emulator As illustrated in Figure 6. a great help for testing purposes was the Bot Framework Channel Emulator. 50 . Within this context. The information provided by this emulator is given in real-time when the bot application is running locally and it includes the status codes of the HTTP requests to the external services from the application. depicted between the two red lines.3. High-level testing Figure 6. it can be clearly concluded that the user had a problem and may not be happy.3. 6. thus the low consecutive sentiment scores of his replies. shown in the LUIS Web interface LUIS request handling can be depicted in Figure 6.3 Chart representing the number of requests per day made to the LUIS application. The function can be split in 3 main areas. although an increase of the response time was observed. A test was made by sending a high number of requests to the LUIS application. a transition from a high sentiment score to a low sentiment score can be identified.4. Chapter 6 Figure 6.4 Sentiment Score function of all conversations A very high-level way of testing can be done by analyzing the chart illustrated in Figure 6. with the help of the two red lines. which resulted in a success rate of 100%. In the first area. The starting point of this area represents the greeting of the user to the Technical Support Chatbot. In the second area. 51 . As it can be clearly identified an ascendant slope. meaning it is a successful test. As the above analysis proceeded. In Figure 6. it results that the user overall satisfaction is increased after the interaction with the Technical Support Chatbot.5 the function for “TVIssue” intent is illustrated. meaning the user satisfaction is increased (through the ascendant slope).5 Sentiment Score function of specific intent conversations The same principle can be used to analyze the function of specific intents. the result is a successful test. but it has a lower slope than the overall function. 52 . Figure 6. Chapter 6 The third area is the most crucial for testing represents the possible problem solution. Installation Manual The administrator that installs this solution has to satisfy the following requirements: Software requirements: .Access to a LUIS app directory website [20] where the LUIS external application will be imported .Server – Azure Portal (recommended) – here will be deployed the whole Technical Support Chatbot solution.1. being able to publish it and to manage the SQL database . User’s manual 7. excepting the external services 7.1 Deployment Diagram 53 . Chapter 7 Chapter 7. Deployment Diagram Figure 7.Microsoft Visual Studio 2013 or newer – for opening the application locally.Access to a Bot app directory website [21] where the final bot will be made publicly available Hardware requirements: .1.1. Afterwards.The Admin Database.1. together with NuGet packages .Publish the Bot Application to the TechnicalSupportChatbot Web App Service 54 . The files are the following: .sql” file .Deploy the Admin Database to the Database Server in the same manner (in this case.Publish the Reports Web Application to the Reports Web App Service (in this case. The next step is to open the Technical Chatbot Support solution with Microsoft Visual Studio and change the data connection strings. Mainly. the following operations should be done: . .Deploy the Entities Database to the Database Server (in this case.json” file .1 the deployment diagram is illustrated. to the admin-reports2) .The entire Technical Support Chatbot Solution (folder). Figure 7. id-db) . exported as „Admin. the administrator has to make sure that he has all the necessary files in order to install the complete solution. 7.2 all the necessary resources to be added to the Azure Server in order to continue installation can be seen.2.2 All resources of the Azure Portal Server [22] In Figure 7. Publishing the application Firstly. Chapter 7 In Figure 7. exported as „Entities.The LUIS Application. the Technical Support Chatbot Solution is installed on an Azure Portal Server.The Entities Database.sql” file The second step is to configure the Azure portal Server or any other server which will provide a public endpoint for the Reports Web Application and the Bot Application. but further details will be given in the following subchapter which will provide all the necessary steps to publish both the Reports Web Application and the Bot Application (note: the publication of the Data Access Layer is not necessary because it is referenced in both previously mentioned applications). the local Entities Database was deployed to the tsc-db-server under the name of tsc- db). exported as „Call Center. the result should look exactly as depicted in Figure 7. with „customURL” being similar to the URL mentioned in Figure 7. For the endpoint URL.3 Homepage of the Bot Application endpoint URL [21] As illustrated in Figure 7.4. set the endpoint URL to one similar to the one illustrated in Figure 7. the value „customURL/API/messages” should be entered. Chapter 7 At this step.4 Overview of the present channels on which the Technical Support Chatbot is published [21] If added to Skype.3. The following step is to import the LUIS Application from the LUIS Web Interface. The next step is to add a new bot in your bot directory. For the Bot Application to work properly.3 Figure 7. the Bot Application should work well if run locally (partially. Figure 7. at this point in the publishing process. only the Reports Web Application should be working and should be tested. 55 . Finally. the Technical Support Chatbot should be added to the Skype channel or any other channel the administrator values. publishing of the LUIS Application is still needed and adding a new bot in the bot directory.3. because the LUIS application and Text Analytics are called from the cloud and the database is accessed from the server). User Manual If the Technical Support Chatbot Application is in production use of a certain company. 56 . As a general rule. This means that the user should contact the Technical Support Chatbot only when he has a problem or requirement that is relevant to the Technical Support Chatbot. Otherwise. Android . Software requirements: . without spam or other intrusive ways. IOS.The Skype application Hardware requirements: . Chapter 7 7. no special requirements need to be satisfied. the user can just search for the “Technical Support Chatbot” in the Skype Directory and add it as a contact.2. the user has to the application in an educated manner. then a special requirement that a Normal User has to satisfy in order to be able to use the application is to be a customer of that company.Desktop or mobile device that satisfies the software requirements mentioned above After having installed all the software from the mentioned requirements.An operating system that supports Skype: Windows or Mac OS. it gets all the information of the user from the database. all the results obtained during the development process of the Technical Support Chatbot relative to the objectives that were presented initially in the second chapter of this document and finally. also other context information such as the subscription for which the user is paying for. in the case of the first release of the Technical Support Chatbot. the user information from the database includes. from his last 3 replies. Conclusions This chapter contains a brief description of contributions. More explicitly.1. 8. the above-mentioned function provides a back-up solution to all of the functionalities available to the Normal User that may provide dissatisfaction. which was mentioned in Chapter 4.1. like a Microsoft account. For chatbots that do not have such rich user information. In this way.1. meaning that the chatbot can satisfy the exact needs of the user. the function that identifies if the user has got angry acts 57 . In order for a user to be able to add the Technical Support Chatbot to the Skype directory. provides a basic model that basically finds out when the user is angry. algorithms and best practices within the domain of Technical Chatbot Support or even to a higher domain of Chatbots. Real-time decision making The analysis of the function that identifies angry user replies. Contextualization The approach of the Technical Support Chatbot differs from the majority of chatbot implementations with the advantage of storing much user information that can provide a base for more contextual decision-making algorithms. Contributions This subchapter highlights the contributions added to an already existing plethora of frameworks.2. whenever the chatbot is not capable of understanding the user’s requirement or is providing wrong solutions that do not solve the user’s problem. further development points. the Technical Support Chatbot can provide uniquely modeled responses to a user. Whenever the chatbot is added to the channel of a user. 8.1. For example. Coming back to the Technical Support Chatbot easy contextualization. beside identification data. The advantage of having such an accurate knowledge of the user’s feeling with respect to the Technical Support Chatbot interaction in real time is enormous. not from a broad natural language perspective. a solution may be to make use of a sign-in card and take profile information from a popular account that most users have. 8. In this way. but from the specificity of the provided answers. Chapter 8 Chapter 8. A possible reason for other chatbots not being able to implement such a contextualized experience for users is the privacy policies from the majority of the instant messaging channels. the user information is added in the database when a new user is registered and it is available right-away for the Technical Support Chatbot. he has to be a customer of the company that the chatbot provides support for. The security of the Reports Web Application is ensured through ASP. if the TVIssue Intent is chosen. mostly through natural language understanding improvement. If the function ends in an ascendant slope.The positive impact of the Technical Support Chatbot by the objective analysis of the overall sentiment scores chart and identifying the ascendant slope has been successfully identified and is detailed in subchapter 6.An accurate display of the specific sentiment scores chart based on intent classification is also provided in the Reports Web Application . 8.NET MVC Identity Authentication .Correct user identification was achieved by accessing the Technical Support Chatbot from a mobile Skype application .An accurate display of the overall sentiment scores chart is done with the help of Google Charts and complex scaling of conversations to 20 replies in the Reports Web Application . any new IP that tries to access it is documented) . Chapter 8 as an interrupt.Multi-channel integration is supported currently on Skype and Web Chat . Natural language understanding improvement The Reports Web Application has the main objective to improve the overall performance. then the administrator of the Technical Support Chatbot should access the LUIS application and add a higher number or better utterances in order to improve the natural language understanding capability of the Technical Support Chatbot regarding this intent. 8.3.1. drops all the other possible response modelling.3 .Correct identification of the start and end of a conversation is achieved and further detailed in subchapter 4. formulates a default response and forwards the problem to the human technical support. user problems that may appear correlated to the TVIssue intent are well covered.The Technical Support Chatbot was published in the Skype Directory (in preview) by the name “Technical Support Chatbot” .Correct ticket forwarding process to the human technical support team was successfully implemented 58 .2.The Database security is enabled by the firewall from the Azure Portal (only a provided list of IPs can access it. with the main focus of identifying if the function ends in an ascendant slope and overall range.2. Achieved results Based on the detailed list of objectives presented in the Chapter 2.The Reports Web Application is published on the Azure Portal . More explicitly. If that is not the case. the specific chart would provide the function of all conversations regarding this intent. The sentiment score function can be analyzed.4 . This is achieved through reading and analyzing the charts of the sentiment score function of a conversation with a specified main intent. a thorough analysis of the achieved results is listed below: . showing all the intents. A great enhancement would be to add new intents in the LUIS application and their corresponding responses in new Dialogs. but only within the intranet of the company. With the current authentication implementation. the input text classification error drops significantly. there may be security risks and providing the application would have access to a vast amount of important information. those security risks should be addressed. In the case of using the Technical Support Chatbot in production by a certain company. The Administrator has the possibility to label the unknown utterances very fast. the LUIS application can be further externalized and the Administrator would not need credentials to that application anymore. resulting in a faster improvement of the overall performance of the application. the application would not be publicly available anymore. Possibility to label unknown intents from the Reports Web Application – with the help of enhancing the user interface of the Reports Web Application. It would provide a much more secured approach. 59 . Provide a rich user interface through the use of cards – by making use of cards. Chapter 8 8. The Technical Support Chatbot does not have to identify the intent in these cases. For each utterance. provided there is an internet connection. Switch to Windows Authentication for the Reports Web Application Currently the Reports Web Application is accessible from everywhere. a dropdown menu should be made available on click. By switching to Windows Authentication. Ability to add utterances directly from the Reports Web Application – enhancing the user interface of the Reports Web Application and implementing the possibility to add utterances from the UI would centralize all the process of the Natural Language Understanding Improvement. unidentified utterances can be shown to the Administrator in a Table or List. switching to Windows Authentication would be recommended. Future work Extending the intents and responses .3.Currently there are only 4 intents that may be mapped to user input requirements. In this way. because the user has choices to make. at/elizabot/eliza.masswerk.iam.appliedis. 2017. "IBM Watson Workspace. [Online]. A. Weizenbaum. [4] B. Available: https://blog. A. C. "Natural Language Processing. "Evaluating Quality of Chatbots and Intelligent Conversational Agents. Zientara. Chowdhury. Available: http://www. [15] © 2017 Poncho Inc. [6] Ubisend." 2017.ibm. G.com/create-servicenow- incidents-from-sap/. [Online]." 2007. All Rights Reserved." 2017. L. J. S. Jung. [Online]. [2] A. 489–516. [7] A Harris Poll on behalf of OneReach. Available: https://poncho. 51-89. Shawar and E. "The Era of Cognitive Systems: An Inside Look at IBM Watson and How it Works. 2005. vol. [Online]. [5] J." 2017. Available: https://idaas. [9] J. "Delivering High Availability Services Using a Multi-Tiered Support Model. a. "Implementing an online help desk system based on conversational agent. "The Bot Framework.html. Jason D." Applied Information Sciences." 2016. p. [12] IBM.com/2016/08/16/the-bot-framework/." 2016. "Using corpora in machine-learning chatbot systems. M. [Online].com/internet-facts-stats-2016/. C.is/. B. Kongthon." in Proceedings of the International Conference on Management of Emergent Digital EcoSystems." International Journal of Corpus Linguistics. [13] A.com. 2005. 10:4. S." 2015. [10] N." 2014." 2002. Windley. [14] R. [16] STA Consulting Kft. "The High Demand for Text Messaging. Available: http://sta-technologies. "2016 Mobile Messaging Report. Phillip J. Available: https://hostingfacts. "Poncho.. High. 60 . [3] G. "Create ServiceNow incidents from SAP. pp. " A Computer Program For the Study of Natural Language Communication Between Man and Machine.." Annual Review of Information Science and Technology. [Online]. M." Redguides for Business Leaders. "E-commerce and Conversion Statistics 2016. G. Williams. "Rapidly scaling dialog systems with interactive. Inc. Bibliography Bibliography [1] P. Radziwill. 2012. Atwell." 2005. Z. 2009." 2017. [8] L. Stevens. [11] D. G. "Writing SMART Objectives That Fit the ROUTINE. "Language Understanding Intelligent Service. "Using Scorables for global message handling and interrupting dialogs in Bot Framework. [21] Microsoft.co. [Online]." 2017. [Online]. [Online]. [18] Microsoft. Available: https://dev.garypretty. Available: https://azure. Available: https://portal." 2017. "Bot Framework. 61 . [Online]. Available: https://www.uk/2017/04/13/using-scorables-for-global- message-handling-and-interrupt-dialogs-in-bot-framework/.luis. Bibliography [17] G.com.luis. [Online]." 2017.com.azure. Pretty. [19] Microsoft. 2017." 2017. [Online].com/en-us/services/cognitive-services/text-analytics/. [20] Microsoft. [22] Microsoft .ai/applications . Available: https://www.microsoft.ai." 2017. Available: http://www.botframework. "Microsoft Azure. "Text Analytics.