Building Bots Part 1

it's about time we did a toolbox episode on BOTS hi welcome to visual studio toolbox I'm your host Robert green and joining me today is Sam bazoo of progress hey Sam hello welcome to the show welcome to the cozy confines of Studio B and that's nice it's changed but it's nice yeah we're just gonna sit back and chat about BOTS it's about time now so bots is one of these new things that there's a lot of buzz about a lot of cool demos of bots running in Skype or you know BOTS calling and making appointments for you and there's just an amazing amount of stuff that's going on and that's all cool of course but what I want to focus on is you know us as traditional enterprise minded business developers whether we're doing desktop or web or mobile or whatever what does it take to build one of these things because it turns out it's not that hard to get started and then what are some of the uses that we might think about where we would actually take advantage of these in the types of businesses we're currently in mm-hm so we're gonna do that this is a two-parter so obviously this is part one of two so tell us about bots let's jump in yep let's switch to slides so we can talk through so what you see I think over the last couple of years is human-computer interaction is evolving mmm-hmm be it through a RvR be it through any other ways and you see the impact of AI so we are not only interacting differently with computers we're also expecting different things for computers and this is where BOTS come in and BOTS aren't new they're about maybe two three years old now but they're evolving fast so I think it's kind of the next frontier of apps so let's kind of dive in so as Cortana Alexa Syria those bots sort of their agents okay let's dive in all right so Robert already introduced me I'm Sam Basu I'm a telluric developer advocate at progress I kind of focus on the dotnet side of things across web desktop and mobile and BOTS is something that that actually does excite me nowadays so I think one of the fundamental questions we need to ask is what exactly are we talking about because yes BOTS is a very generic term we're not talking about robots we're not talking about like the spider BOTS that Bing or Google used to kind of crawl the web there are different types of BOTS what we are focusing on for enterprise application in particular are chat BOTS more conversational BOTS bots that we can actually have a conversation with be it over voice or be it over text and what can they do for us how smart can they get so actually automate things for us and can actually help us okay yeah one of the fun things that I've actually seen many people do whenever we talk about conversational BOTS back in 2013 I think there was a movie that came out it was called her it right hook in Phoenix in them in it and it got a little weird quickly it was about conversational BOTS but this lonely writer he started having a relationship with his conversational bot because it was so smart and it kind of got him very quickly right so that's kind of the AI aspects of being a bot and how smart it can get so let's kind of break down what exactly is a bot and how we can build it okay okay I think one of the reasons why building conversational BOTS is interesting and challenging and fun in a way is because we're all trying to make a cake we have a lot of ingredients there is no fixed recipe because it really depends on what exactly you want your body to be okay so you have to kind of break down what are my ingredients and how am i exactly going to build it so it suits my needs so let's kind of break down where conversational BOTS stand today and we're possibly looking at a slice of it but bots are apps okay that's fundamentally it I mean these are web-based apps that we can communicate over HTTP and they can be housed in different things but they're fundamentally apps but there are complex apps because they have lots and lots of integrations that's what makes them smart so a bot by itself if it was just talking to you and had no brain will be pretty dumb it was only going to go off if else statement and a bunch of rules and not be able to a whole lot so the real benefit of having a bot is all of these integrations that make it smarter okay so that's what we're talking about so there is a lot of plumbing there is a lot of different things that make up a bot so to your point are the voice assistant spots in a way they are but they are in the front end to the intelligence that's behind so when you talk to Alexa when you talk to Cortana or Google assistant you are firing things off over voice but things that you are firing off beat and as your function wheezed in an Amazon lambda that's what the intelligence lies so those are BOTS okay you're just using an agent to get to the bots yeah and I mean with Cortana you can type it in so I mean essentially is a conversational bar over over text right yeah so these are agents and then we talk about platforms because this is where BOTS can recite and this is kind of a big benefit of having a bot it really is platform agnostic what you're building is very fundamental and there are multiple platforms who can all talk HTTP so you can have your bot house inside of Facebook Messenger or filio or it's slack or team wherever your users are you can have a bot kind of integrated in there so it's easy for them to find it and talk to it okay so these are the platforms and then we have agents so like we said BOTS by themselves don't do much it's what we do behind the scenes with boss that makes it interesting so first up is there's got to be some sort of rules engine that tells you about what to do what questions to ask what path to kind of guide you down as we're having a conversation with your bot next thing is natural language processing if your bot is literally going off a bunch of strings to say find me a flight out of Seattle then that's the only thing it will understand if you say get me out of Seattle it's going to get confused but that's kind of how we naturally speak to each other so this is where natural language processing comes into play and all of the voice agents do it as well so when you say get me out of Seattle they find so it's an utterance intent what exactly are you trying to do like get a flight out of Seattle and then we'll find the parameters like what airport are you flying out of and then it's going to trigger the corresponding contextual thing that's going help you flying the fly so in that way it's it's a step or several steps beyond just voice enabling an app right like you could voice enable an app so that instead of having to pick from a menu choice or click a button you could say open the report open the monthly report right right so at that point there's one thing that the app does it opens the monthly report there is a command and you say open the monthly report but if you went in and said what happened last month then obviously the app has no idea what you mean because you didn't hardwired in a bunch of commands so the the beauty of the bot is that it can be trained to be more natural more cognizant right of what you're saying instead of just expecting if you say these five things I will do these five things if you say anything else I have no idea what you're talking about okay so there is a command there's a thing that the bot knows how do you know it's how you get to it because I think that's that's an important distinction because I think the very first time I saw a bot I was thinking oh well I can voice in and I can voice enable apps and I've been able to do that for a long time what's how is this better right so it's a bunch of things that your bot knows how to do and a bunch of different ways that the user can come to it okay yep so that's a natural language processing then there is some state management there's some logic that you will have obviously behind it what if you told you about order me the pizza that I had last week right now by nature bots are supposed to be stateless like it's one turn-off you saying something and the bots getting back to you but how is it going to remember if you what pizza order last week so you can actually persist some state in your BOTS and that kind of goes back to wherever you want to store your data and how you want to authenticate and authorize the user but it's possible to maintain some state so the bot is more of a conversational and how do humans will interact sure yeah so these are the things that kind of make up the bots intelligence and then they're about frameworks because we don't want to do everything from scratch right difficult so there's Microsoft bought framework which which we'll talk about exclusively today I actually have not played a lot with others but there's there's a lot for sure there's Facebook what AI Amazon makes Lex IBM makes Watson and dialog flow from Google so there are lots of these frameworks they all are about the same and they have their SDKs they have their ways to kind of bring you on so all of this is kind of geared towards helping developers build BOTS ok that's it so let's kind of dive into Microsoft bot framework and then see what it can do for you before we do that to answer your initial question of what what can this do for enterprise apps mm-hmm so yeah all the skype bots are fun but when it comes down to what line of business apps can fit as bots it's a lot of little things that can be automated workflows and I'll give you two classic examples where we actually have customers doing this we have a hospital where people call in and they want to schedule an appointment it's a very basic task and yes somebody has a calendar open every doctors calendar is in sync and at that point you're really just choosing a bunch of things and click and you're done you have an appointment that's something that can be automated and we have in a hospital that actually has it sounds bad but they've been able to cut one human and kind of having two other things but over like six months they were able to train their BOTS enough that is good enough to schedule simple appointments it's all about like what can we automate before you have to talk to a human being right yeah another example is support systems if you have like an IT Help Desk that's trying to help you kind of go through problems and roadblocks maybe the initial things maybe your basic problems can be dealt with with a bot it just knows the basic things that can go wrong and then if you're having more problems than transfer to a human being okay so some some common examples so that's going to talk about the Microsoft side of the world because this is where we live and breathe so this is what such a kind of opened up a couple of years back conversation as a platform this is the next interface this kind of helps AB developers have ways new ways to interact with their system as an end user so it's it's a new frontier for us the benefits are obvious we get to have increased productivity we have to we we are able to reduce the time that a human has to spend over just trivial things things that can be automated and the big benefit is being able to reach customers where they are at so if you are already plugged into messenger or Twilio or slack then we'll meet you there you don't have to install a new app we just kind of have a part that's embedded into what you are that's where we stand that's the benefit of pause okay okay so now let's dive into what we can do with it what benefits we can get and how we can start building it so based on who you ask Microsoft bot framework is a big thing I think it's comprising of two fundamental things one is bought builder this is the SDK this is how you actually build your apps your bots you don't want to do it entirely online unless it's something super simple you want to have it locally and that's how you will have it in your IDE and you code it up and you make it work and you test and then he'll push it out to wherever you want to have it hosted and there is a developer portal which kind of helps you get started it's gets you the SDKs it's a place where you can actually host your apps as well a host your BOTS and then it has all of these little connections which help you get your app into slack into Skype into Facebook those things are really nice and then this is just a bot part of it and then we get into what's called cognitive services which I don't think it's really a part of the bot framework at all it's it's AI suite right but this is what really gives your body intelligence these integrations I think we are about maybe 30 or so restful api is right now you don't have to be a data scientist you don't have no AI you're literally making restful endpoint calls to use cognitive services but these things can be trained if you are into machine learning a little bit and then your bot starts getting smarter and smarter okay okay so that's where we are cognitive services these are Microsoft slides but we we know what they do it's a lot of things it's vision it's speech it's a big part of it is latch language Louis which is learning natural human interactions language and an azure is very very good at this so all of these api's are part of common e-services and this is what helps us make our BOTS smarter okay yep that's what we're going to integrate with I won't go too much in details but just to kind of give you a hint of where things are at and then there is a Shabbat service so you don't have to use this essentially a bot is a restful endpoint you can have it hosted anywhere you want but this comes bundled with a whole bunch of goodness that kind of helps you get started quickly helps you have your bot nice and hosted easily and then have all of these connections so you can put your bar anywhere you want okay yeah so that's the adabot service it has those SDKs that we talked about and it has templates it has integrated ways in which you can test your bot and then you can deploy our bots through the channels and then once you are successful building your bot and having a deployed then you kind of take it to the next level let's do see ICD pipelines let's do as you're functions that trigger other things based on what your bot is doing so all of those extra things can be built in and how you scale your part based remand and so okay so as you're Bob service is a very compelling offering when you have to get them get started with BOTS okay so that's enough slides I'm getting bored of slides altogether so let's let's look at some demo all right so the first place I will point anyone to go to is do a search on bot framework and you will end up land on dev bot framework comm this is the landing page for Microsoft offering work and it has a lot of goodness to it so take a look at to what it can offer and it talks about all of your audiences all of your channels and how you cannot get started with documentation this is actually really really well done this is a Shabbat service that you're looking at but some of it can be local so we're going to walk you walk through some of these things that you can do to get started so notice how right out of the gate you have different ways of getting started you can do bot service which is directly in Azure or you can be local in c-sharp and og okay so before we go in right now the Microsoft bot framework is at v3 mm-hmm we started about 2016 so we are at v3 right now v4 is what's coming next and we have a preview bit of it and v4 is substantially different actually in some ways but we will talk about v3 because that's the present stable production release but v4 is coming sometime this year are already next so in terms of languages and programming IDs and and tools that you can use to build your bot you are looking at C sharp and nodejs right now okay so pick your poison C sharp or JavaScript and then go at it in v4 you will be able to do Python and Java okay but we're not there yet okay so this is the best place to start and this is kind of guide you through let's look at the Azure side of the story so I'm going to go to portal - accom so this is a sure and if I make it a little bigger so we can see if you're going in here and say create a resource what you want to pick is a I in machine learning and then in there is a whole bunch of things but you want to pick web app BOTS mm-hmm once you select this it's gonna ask for a bunch of things what's your bot name where what subscription resource group you can create a new one if you have a whole bunch of bots maybe you kind of group them into one resource where do you want to have this hosted pricing and and so on templates we'll talk about that in a second it does use some address storage maybe the state management in particular does use Azure storage app insights and so on so you kind of set this up and then you here to create and it'll go ahead and create a bot for you so instead of having as you're kind of waiting waiting on Azure to spin this up let's kind of look at what we have so here's a simple bot that I have and I have not changed a word of this it's just purely what the template gives you but this thing is spun up it's called the the Sam echo bot so it's just a simple echo engine but this thing is spun up and it's hosted and hosted in a charade now so this is what you will get when you hit the create button so before we even look at anything else if you go do this test in a web chat so this is a bot that's ready and it's deployed so when you hit that test it's actually gonna bring up this little console it looks like a little web chat engine and this is where you can test so I can write things here like hello and it'll it's gonna come back and said you said hello right so we can say more things hi from Redmond it's gonna come back and say you you said - Redmond so this is literally a echo bot that kind of repeats what you said okay okay but it's a quick and easy way of you being able to deploy this and be able to test this right away so you know exactly how how your app is working so this is what's called the web chat and if you are using this and if you're using this from a web application you can just kind of plant this and embed this as is so this web chat can be taken and embedded in your verification so all of this chats functionality is available from your web app it's just one measure you don't need to know about it it's just a web chat okay okay so when I go into this thing called channels this is what makes apps interesting this is where you get a whole bunch of things that are easy to configure web chat is already running which is what we saw and then Skype this is another channel some of these channels are kind of featured and already configure for you like Cortana Skype they're very easy to get started same with teams if you are going to Facebook Messenger or slack or Twilio there are some extra like authorization and things that you have to do to make sure your app is registered with Facebook you're carrying acts like authentication tokens back and forth but it's not that difficult and when you kind of go through this Wizards they kind of step you through like go register your app bring back nap key and have it set up but essentially this is a restful endpoint that knows how to get an HTTP message from you and then echo back what it just said okay so you notice the Skype thing here which is also the same Bart running in Skype because I already went through this but what it does is let me pull up Skype show essentially that echo bot now becomes a contact in your skype list so this guy here once you configure it and open up skype it's gonna say hey you have a new contact so your bot now becomes a contact so you can talk your bought just like you talk to a human being so in here we can say hola it's gonna come back and say hopefully you said so same exact part that we saw on Azure through a web chat now it's not a venture it's part of Skype now and this is the big benefit that you get with all of these channels because you can deploy your functionality in a platform agnostic way through whatever channel your user is using okay yeah so that's just just a Skype app Skype channel there are other things you can do like obviously Skype for business and slack and Facebook Messenger so this is the nap that's running let's take a look at build and that'll show you what it's doing behind the scenes so it's saying hey we have an app a bot it's running in Azure but you have not made any code changes yet you literally haven't even seen your code you went from a template and you said give me a basic bot and this is what we spun up for you there is some code behind it how do you want to work with it so you can say I want to download the zip file which is your entire solution can download it a zip file you open it up in Visual Studio and then when you're done with your changes you push it back up - that's one way or you can say I actually do want to do continuous deployment engineers bills and deployments so this is where you say no I want to push my code to github or VST s team services and then when every time I do a push and I commit you do a build and even I should treasure so you can do CI city but if you are working with something super simple like this one you can do the online code editor and what that does it just opens up your solution inside of a browser and if you kind of recognize the look and feel of this visual studio code it's actually aware that right it's an electron app so you kind of see that shell you see the same code intellisense and some of the benefits that you get so this is what as you created for you when we said file new project and then create a new bot this is what it did and it's gonna say hey you can change things here when you're done just hit build CMD we can build it and deploy it back for you so there's a whole bunch of things in here that are not important the only thing that's important here is this echo dialogue CS this is literally the one file that's controlling what your bot does so this is going to standard for any Microsoft bought framework bot so there is a message received this is fired when your bot receives a message from the user so the users say something it comes to us and you can actually reset it so you'll notice that we actually have a counter that counts how many times you say it and then if you say reset it's just gonna reset the counter so there's a prompt to make sure you're you're confirming it but otherwise it's just gonna say contacts start post async you said that okay so this is literally the one line of code that's echoing back what you're saying that's it that's the one so just one line that can echoes you back what you're saying so this is a bot that's deployed when you go to a sure and just do file new project give me about service ok so this is a bot because it's built on the bot framework but it's not a bot because it only does one thing yes ok so if this is all you wanted then wrapping it around a bot maybe a little bit of overkill yes but this is a nice template to give you some basic plumbing and then you can build on top of that but I mean this is a good starting point because not only are you creating a bot you're also hosting it in a sure and you already have those channels that are configured so you can put that bot in Skype and slack so again I would point people to the documentation which is really good if you go through the QuickStart here this is what we did creating a bot with the bot service but if you want to get the bot builder SDK then right now you have dotnet and nodejs so if you look at what the dotnet side of the story is you need vs 2017 and then you need the bot template c-sharp I mean it's a template extension that gives you this template simple echo bot mm-hmm so you'll get the bot framework and then you start with that and then it spits out the same code that you just but just in c-sharp and it's it's local to you and then you can test you about right now this simple bot is just web app so you can kind of fire this up now is Express and run it locally and then we'll talk about a different type of way in which you can test your bot that's the emulator so same exact idea you can do it with a dotnet and c-sharp what I like being on a Mac today is the node.js side of things because if you're in JavaScript you can do the exact same things the bot framework is time from agnostic that way so what this expects you to do is have NPM installed and then you get this part builder package essentially so it has all of the bits that bot builder can do for you so let me show you what that looks like so I'm gonna go in here into a folder and bring up my terminal alright hopefully this is big enough so I'm going to go into my folder here this thing is called the node echo bot and if I kind of fire this up if I say node fjs I'm firing up the app you don't see in it anything because it's just a console app but I can say the same exact things mm-hmm so it's just echoing back what I say so same exact idea it's just running locally in a console app so if you look at what this is essentially so I'm going to fire up vs code and look at the same folder essentially this being a node app we did a NPM in it so we have some package JSON we have the node modules in here but this is that one file that's making the bot work so essentially with node.js you have dependencies so we're depending on the bot builder and we're saying we are building a console connector and whenever you kind of fire off the message received thing we're going to take your text and kind of said you said that okay one line so this is no js' this is pure JavaScript running on the server side or wherever you have it hosted but it's it's your bot written in node.js now this is just local this is a console app they kind of step you through there are some extra packages called Resta Phi which kind of helped translate this console app into a end point because the bot has to be reasonable in mind so if I do an NPM install of rs.25 which is what the docs will actually walk you through so this is Resta phi is what we're using it's a very simple package that kind of helps you make restful endpoints out of your node.js code whatever your app is doing so we're using this and then once we have it we can change a little bit of code here this one is requiring Resta phi and same bot builder but we are creating a little server here so we can host it as a restful endpoint and then we can take into the cloud or whatever web endpoint it is that's our our port number and then we are creating a chatbot the app ID and password we'll talk about this so every part that's deployed to that robot service has to have a unique identifier okay and as your create that for you but otherwise it's just gonna say you said that so if we actually go back and run this now instead of app jeaious let's run node restful app dot j s this one said hey I'm not a console app I'm running at this port because you use Resta Phi so I'm running it as a restful service on 439 78 so how do you test this you can go to your browser or you can use this little thing which also comes with the bot framework this is called the bot framework emulator it looks kind of a little bland without having any inputs here but this is a local emulator that will help you test your BOTS locally before you're ready to deploy it to whatever service so in here I can actually go to that same port that we are on again it's going to ask for app ID password you can skip it for now you can test this locally but once you have to go to a show you do need the app ID so this is the exact same app and now you see that working here in the inside the emulator so this is the restful endpoint I don't have it hosted anywhere but I'm able to invoke it and the last thing is this one kinda shows you all of the JSON that's coming back and forth about service and and you so this is a super simple emulator but it does a lot of things and actually the emulator on Windows is a little more sophisticated than the emulator on Mac because one of the things that you get into with bought development is kind of conversations and you have to test your conversations and sometimes you have to kind of start midway in a conversation to start so it's like a little bit of template where you can pick up a conversation as just simple JSON and gonna start Midway and start testing so it'll let you do all of those things right yeah so that's the emulator that's one way of running it so let's let's close this that's just a quick example of doing it locally in no chairs let's kind of go back to Azure and I'll show you the other side of the story here so this was our echo bot so let's go home if I did a new resource again if I did a web app and then you look at these templates so it's not just the basic template which you can do it comes with five or so templates which kind of help you down the right path of building a bot first thing is formed this is one of the most commonly used templates this is where you have to order a pizza or you have to book a flight out and those repetitive things because essentially when you do this on your web applications today you are going through a wizard none of this is new right it's how you want to interact with your computer are you more comfortable doing a more natural conversation rather than just clicking around on your website okay so we can kind of walk you through that wizard in a bot so these are like a series of steps then language understanding this is where we're actually going to call into Louis in cognitive services because this is what actually understands what you mean when you say natural things okay so behind it it's going to be an azure Community Services app which has to have its own app ID and it's needed it needs to be registered and then that's when you're going to have to train your model so when you say booked a flight out of Seattle what are the 20 different ways in which you can say it so they're your bot understands okay and then passes it back to the bot so essentially what we are getting is from the bot framework you are pre-configured to call in to Azure cognitive services and then as your community services makes sense of what you are saying and brings it back to the bot and you can do this from C shop or no js' okay so just a quick example of the form thing which is very easy to look at I do have a form bot here which is called the sandwich bot because that's just a normal example so let's go ahead and look at this thing here we can test this in a web bot or web chat window so here if you notice the little messaging here I can say hi I'm up here and the bot knows that it's a specialized bot and says hey it looks like you're trying to order a sandwich because that's what I know what type of sandwich do you want so these are now Pickers okay so you get to pick what you want I would like a steak and cheese now it's gonna go through a list of things now you're getting two rules right so you said this then that then this then that so and you can kind of back-trace sometimes if your user does not give you the right response you can say oh I didn't understand that I wanted this from you so you can kind of go back and forth a little bit so 6-inch sub what bread you want that and then within breads with in Italian breads you have two types what do you want what type of cheese so these are all Pickers then you get to pick a whole bunch of things so these are like just numbers that you can pick cucumbers what type of sauce do you want let's do honey mustard and that's it so it's essentially said I went through these six steps with you and I made you choose things every step and you gave me this sandwich and once you are ready is this your selection you said yes and then said okay that's your sandwich I'm going to send it off to the shop and they're gonna deliver whatnot okay so just a super simple way of using the form template which actually kind of gets you quite a bit into these like flows okay how to ask the user and if you look at what this does for you I'll give it a second all right so now this one has a little more stuff to it so there's a model which is our sandwich this has all of your options like what type of sandwich what was it what type of bread what type of cheese it's all in here and then you have your dialogues as to how you actually control what you ask the user that's message controller so this is where your tasks and your your your post and you your awaiting the message from the user all of that is going to code it up here and then you kind of let bot kind of drive the conversation as to what exactly it needs to get all the information from the user so we can go about it so all of this is kind of bundled in and it's part of the template that you get it's really easy to call into most cognitive services like and good examples of this are like vision api's if you have to have a bot that knows how to understand or make sense of images from the user you hand it off you take the image you hand it off to the azure Community Services vision API we have another customer of ours who uses the vision API to do quality control on shipping so they have a little camera that's mounted on top of shipping containers mm-hmm and they do quality control as inside going out of their warehouses you can also do sentiment analysis with with common services so as you are talking to your bot you can kind of sense that the user is getting angry right and you can sense his sentiment and say oh maybe we can do other things to make you happy and so on so all of these things are intelligence that you're adding to your bot and this is what makes BOTS smarter and they kind of let you automate workflows so this is this is where we are this is where we start some things that I really like pointing out is the thorough documentation that the bot framework has there are tutorials on how to do just about everything let me show you some things that are really interesting the concepts here let's do Congress services language understanding let me do a how-to I thought it was really nicely done they talk about channels and the thing that I was trying to get to is how you can hook this up to language services that's something we'll do in part two yeah we could so essentially I mean just just start here essentially is what I'm saying okay and they have documentation on how to use vision API and language services all of these things that help you go from your bot to other places that give you about intelligence okay that's it so bots are nothing more than apps you can a restful endpoint but it's platform agnostic so you can put it in whichever channel that your user is using and start automating little things in your workflows that don't need human interaction very cool yeah yeah I mean if you look at some of the most common things that we do on most websites it's a bunch of steps that we follow wizards and this is where BOTS can come in and make it more conversational and more natural right yeah all right that was a good introduction we'll end it for here in this part and we'll come back in part two and and see more about how to build them absolutely so again this was all about BOTS you can find me at SMI div that's my social handle on Twitter and github in everywhere but thanks for watching all right and we will see you next time on visual studio toolbox [Music]