Building Bots Part 1

it's about time we did a toolbox episode on BOTS hi welcome to visual studio toolbox I'm your host Robert green and joining me today is Sam bazoo of progress hey Sam hello welcome to the show welcome to the cozy confines of Studio B and that's nice it's changed but it's nice yeah we're just gonna sit back and chat about BOTS it's about time now so bots is one of these new things that there's a lot of buzz about a lot of cool demos of bots running in Skype or you know BOTS calling and making appointments for you and there's just an amazing amount of stuff that's going on and that's all cool of course but what I want to focus on is you know us as traditional enterprise minded business developers whether we're doing desktop or web or mobile or whatever what does it take to build one of these things because it turns out it's not that hard to get started and then what are some of the uses that we might think about where we would actually take advantage of these in the types of businesses we're currently in mm-hm so we're gonna do that this is a two-parter so obviously this is part one of two so tell us about bots let's jump in yep let's switch to slides so we can talk through so what you see I think over the last couple of years is human-computer interaction is evolving mmm-hmm be it through a RvR be it through any other ways and you see the impact of AI so we are not only interacting differently with computers we're also expecting different things for computers and this is where BOTS come in and BOTS aren't new they're about maybe two three years old now but they're evolving fast so I think it's kind of the next frontier of apps so let's kind of dive in so as Cortana Alexa Syria those bots sort of their agents okay let's dive in all right so Robert already introduced me I'm Sam Basu I'm a telluric developer advocate at progress I kind of focus on the dotnet side of things across web desktop and mobile and BOTS is something that that actually does excite me nowadays so I think one of the fundamental questions we need to ask is what exactly are we talking about because yes BOTS is a very generic term we're not talking about robots we're not talking about like the spider BOTS that Bing or Google used to kind of crawl the web there are different types of BOTS what we are focusing on for enterprise application in particular are chat BOTS more conversational BOTS bots that we can actually have a conversation with be it over voice or be it over text and what can they do for us how smart can they get so actually automate things for us and can actually help us okay yeah one of the fun things that I've actually seen many people do whenever we talk about conversational BOTS back in 2013 I think there was a movie that came out it was called her it right hook in Phoenix in them in it and it got a little weird quickly it was about conversational BOTS but this lonely writer he started having a relationship with his conversational bot because it was so smart and it kind of got him very quickly right so that's kind of the AI aspects of being a bot and how smart it can get so let's kind of break down what exactly is a bot and how we can build it okay okay I think one of the reasons why building conversational BOTS is interesting and challenging and fun in a way is because we're all trying to make a cake we have a lot of ingredients there is no fixed recipe because it really depends on what exactly you want your body to be okay so you have to kind of break down what are my ingredients and how am i exactly going to build it so it suits my needs so let's kind of break down where conversational BOTS stand today and we're possibly looking at a slice of it but bots are apps okay that's fundamentally it I mean these are web-based apps that we can communicate over HTTP and they can be housed in different things but they're fundamentally apps but there are complex apps because they have lots and lots of integrations that's what makes them smart so a bot by itself if it was just talking to you and had no brain will be pretty dumb it was only going to go off if else statement and a bunch of rules and not be able to a whole lot so the real benefit of having a bot is all of these integrations that make it smarter okay so that's what we're talking about so there is a lot of plumbing there is a lot of different things that make up a bot so to your point are the voice assistant spots in a way they are but they are in the front end to the intelligence that's behind so when you talk to Alexa when you talk to Cortana or Google assistant you are firing things off over voice but things that you are firing off beat and as your function wheezed in an Amazon lambda that's what the intelligence lies so those are BOTS okay you're just using an agent to get to the bots yeah and I mean with Cortana you can type it in so I mean essentially is a conversational bar over over text right yeah so these are agents and then we talk about platforms because this is where BOTS can recite and this is kind of a big benefit of having a bot it really is platform agnostic what you're building is very fundamental and there are multiple platforms who can all talk HTTP so you can have your bot house inside of Facebook Messenger or filio or it's slack or team wherever your users are you can have a bot kind of integrated in there so it's easy for them to find it and talk to it okay so these are the platforms and then we have agents so like we said BOTS by themselves don't do much it's what we do behind the scenes with boss that makes it interesting so first up is there's got to be some sort of rules engine that tells you about what to do what questions to ask what path to kind of guide you down as we're having a conversation with your bot next thing is natural language processing if your bot is literally going off a bunch of strings to say find me a flight out of Seattle then that's the only thing it will understand if you say get me out of Seattle it's going to get confused but that's kind of how we naturally speak to each other so this is where natural language processing comes into play and all of the voice agents do it as well so when you say get me out of Seattle they find so it's an utterance intent what exactly are you trying to do like get a flight out of Seattle and then we'll find the parameters like what airport are you flying out of and then it's going to trigger the corresponding contextual thing that's going help you flying the fly so in that way it's it's a step or several steps beyond just voice enabling an app right like you could voice enable an app so that instead of having to pick from a menu choice or click a button you could say open the report open the monthly report right right so at that point there's one thing that the app does it opens the monthly report there is a command and you say open the monthly report but if you went in and said what happened last month then obviously the app has no idea what you mean because you didn't hardwired in a bunch of commands so the the beauty of the bot is that it can be trained to be more natural more cognizant right of what you're saying instead of just expecting if you say these five things I will do these five things if you say anything else I have no idea what you're talking about okay so there is a command there's a thing that the bot knows how do you know it's how you get to it because I think that's that's an important distinction because I think the very first time I saw a bot I was thinking oh well I can voice in and I can voice enable apps and I've been able to do that for a long time what's how is this better right so it's a bunch of things that your bot knows how to do and a bunch of different ways that the user can come to it okay yep so that's a natural language processing then there is some state management there's some logic that you will have obviously behind it what if you told you about order me the pizza that I had last week right now by nature bots are supposed to be stateless like it's one turn-off you saying something and the bots getting back to you but how is it going to remember if you what pizza order last week so you can actually persist some state in your BOTS and that kind of goes back to wherever you want to store your data and how you want to authenticate and authorize the user but it's possible to maintain some state so the bot is more of a conversational and how do humans will interact sure yeah so these are the things that kind of make up the bots intelligence and then they're about frameworks because we don't want to do everything from scratch right difficult so there's Microsoft bought framework which which we'll talk about exclusively today I actually have not played a lot with others but there's there's a lot for sure there's Facebook what AI Amazon makes Lex IBM makes Watson and dialog flow from Google so there are lots of these frameworks they all are about the same and they have their SDKs they have their ways to kind of bring you on so all of this is kind of geared towards helping developers build BOTS ok that's it so let's kind of dive into Microsoft bot framework and then see what it can do for you before we do that to answer your initial question of what what can this do for enterprise apps mm-hmm so yeah all the skype bots are fun but when it comes down to what line of business apps can fit as bots it's a lot of little things that can be automated workflows and I'll give you two classic examples where we actually have customers doing this we have a hospital where people call in and they want to schedule an appointment it's a very basic task and yes somebody has a calendar open every doctors calendar is in sync and at that point you're really just choosing a bunch of things and click and you're done you have an appointment that's something that can be automated and we have in a hospital that actually has it sounds bad but they've been able to cut one human and kind of having two other things but over like six months they were able to train their BOTS enough that is good enough to schedule simple appointments it's all about like what can we automate before you have to talk to a human being right yeah another example is support systems if you have like an IT Help Desk that's trying to help you kind of go through problems and roadblocks maybe the initial things maybe your basic problems can be dealt with with a bot it just knows the basic things that can go wrong and then if you're having more problems than transfer to a human being okay so some some common examples so that's going to talk about the Microsoft side of the world because this is where we live and breathe so this is what such a kind of opened up a couple of years back conversation as a platform this is the next interface this kind of helps AB developers have ways new ways to interact with their system as an end user so it's it's a new frontier for us the benefits are obvious we get to have increased productivity we have to we we are able to reduce the time that a human has to spend over just trivial things things that can be automated and the big benefit is being able to reach customers where they are at so if you are already plugged into messenger or Twilio or slack then we'll meet you there you don't have to install a new app we just kind of have a part that's embedded into what you are that's where we stand that's the benefit of pause okay okay so now let's dive into what we can do with it what benefits we can get and how we can start building it so based on who you ask Microsoft bot framework is a big thing I think it's comprising of two fundamental things one is bought builder this is the SDK this is how you actually build your apps your bots you don't want to do it entirely online unless it's something super simple you want to have it locally and that's how you will have it in your IDE and you code it up and you make it work and you test and then he'll push it out to wherever you want to have it hosted and there is a developer portal which kind of helps you get started it's gets you the SDKs it's a place where you can actually host your apps as well a host your BOTS and then it has all of these little connections which help you get your app into slack into Skype into Facebook those things are really nice and then this is just a bot part of it and then we get into what's called cognitive services which I don't think it's really a part of the bot framework at all it's it's AI suite right but this is what really gives your body intelligence these integrations I think we are about maybe 30 or so restful api is right now you don't have to be a data scientist you don't have no AI you're literally making restful endpoint calls to use cognitive services but these things can be trained if you are into machine learning a little bit and then your bot starts getting smarter and smarter okay okay so that's where we are cognitive services these are Microsoft slides but we we know what they do it's a lot of things it's vision it's speech it's a big part of it is latch language Louis which is learning natural human interactions language and an azure is very very good at this so all of these api's are part of common e-services and this is what helps us make our BOTS smarter okay yep that's what we're going to integrate with I won't go too much in details but just to kind of give you a hint of where things are at and then there is a Shabbat service so you don't have to use this essentially a bot is a restful endpoint you can have it hosted anywhere you want but this comes bundled with a whole bunch of goodness that kind of helps you get started quickly helps you have your bot nice and hosted easily and then have all of these connections so you can put your bar anywhere you want okay yeah so that's the adabot service it has those SDKs that we talked about and it has templates it has integrated ways in which you can test your bot and then you can deploy our bots through the channels and then once you are successful building your bot and having a deployed then you kind of take it to the next level let's do see ICD pipelines let's do as you're functions that trigger other things based on what your bot is doing so all of those extra things can be built in and how you scale your part based remand and so okay so as you're Bob service is a very compelling offering when you have to get them get started with BOTS okay so that's enough slides I'm getting bored of slides altogether so let's let's look at some demo all right so the first place I will point anyone to go to is do a search on bot framework and you will end up land on dev bot framework comm this is the landing page for Microsoft offering work and it has a lot of goodness to it so take a look at to what it can offer and it talks about all of your audiences all of your channels and how you cannot get started with documentation this is actually really really well done this is a Shabbat service that you're looking at but some of it can be local so we're going to walk you walk through some of these things that you can do to get started so notice how right out of the gate you have different ways of getting started you can do bot service which is directly in Azure or you can be local in c-sharp and og okay so before we go in right now the Microsoft bot framework is at v3 mm-hmm we started about 2016 so we are at v3 right now v4 is what's coming next and we have a preview bit of it and v4 is substantially different actually in some ways but we will talk about v3 because that's the present stable production release but v4 is coming sometime this year are already next so in terms of languages and programming IDs and and tools that you can use to build your bot you are looking at C sharp and nodejs right now okay so pick your poison C sharp or JavaScript and then go at it in v4 you will be able to do Python and Java okay but we're not there yet okay so this is the best place to start and this is kind of guide you through let's look at the Azure side of the story so I'm going to go to portal - accom so this is a sure and if I make it a little bigger so we can see if you're going in here and say create a resource what you want to pick is a I in machine learning and then in there is a whole bunch of things but you want to pick web app BOTS mm-hmm once you select this it's gonna ask for a bunch of things what's your bot name where what subscription resource group you can create a new one if you have a whole bunch of bots maybe you kind of group them into one resource where do you want to have this hosted pricing and and so on templates we'll talk about that in a second it does use some address storage maybe the state management in particular does use Azure storage app insights and so on so you kind of set this up and then you here to create and it'll go ahead and create a bot for you so instead of having as you're kind of waiting waiting on Azure to spin this up let's kind of look at what we have so here's a simple bot that I have and I have not changed a word of this it's just purely what the template gives you but this thing is spun up it's called the the Sam echo bot so it's just a simple echo engine but this thing is spun up and it's hosted and hosted in a charade now so this is what you will get when you hit the create button so before we even look at anything else if you go do this test in a web chat so this is a bot that's ready and it's deployed so when you hit that test it's actually gonna bring up this little console it looks like a little web chat engine and this is where you can test so I can write things here like hello and it'll it's gonna come back and said you said hello right so we can say more things hi from Redmond it's gonna come back and say you you said - Redmond so this is literally a echo bot that kind of repeats what you said okay okay but it's a quick and easy way of you being able to deploy this and be able to test this right away so you know exactly how how your app is working so this is what's called the web chat and if you are using this and if you're using this from a web application you can just kind of plant this and embed this as is so this web chat can be taken and embedded in your verification so all of this chats functionality is available from your web app it's just one measure you don't need to know about it it's just a web chat okay okay so when I go into this thing called channels this is what makes apps interesting this is where you get a whole bunch of things that are easy to configure web chat is already running which is what we saw and then Skype this is another channel some of these channels are kind of featured and already configure for you like Cortana Skype they're very easy to get started same with teams if you are going to Facebook Messenger or slack or Twilio there are some extra like authorization and things that you have to do to make sure your app is registered with Facebook you're carrying acts like authentication tokens back and forth but it's not that difficult and when you kind of go through this Wizards they kind of step you through like go register your app bring back nap key and have it set up but essentially this is a restful endpoint that knows how to get an HTTP message from you and then echo back what it just said okay so you notice the Skype thing here which is also the same Bart running in Skype because I already went through this but what it does is let me pull up Skype show essentially that echo bot now becomes a contact in your skype list so this guy here once you configure it and open up skype it's gonna say hey you have a new contact so your bot now becomes a contact so you can talk your bought just like you talk to a human being so in here we can say hola it's gonna come back and say hopefully you said so same exact part that we saw on Azure through a web chat now it's not a venture it's part of Skype now and this is the big benefit that you get with all of these channels because you can deploy your functionality in a platform agnostic way through whatever channel your user is using okay yeah so that's just just a Skype app Skype channel there are other things you can do like obviously Skype for business and slack and Facebook Messenger so this is the nap that's running let's take a look at build and that'll show you what it's doing behind the scenes so it's saying hey we have an app a bot it's running in Azure but you have not made any code changes yet you literally haven't even seen your code you went from a template and you said give me a basic bot and this is what we spun up for you there is some code behind it how do you want to work with it so you can say I want to download the zip file which is your entire solution can download it a zip file you open it up in Visual Studio and then when you're done with your changes you push it back up - that's one way or you can say I actually do want to do continuous deployment engineers bills and deployments so this is where you say no I want to push my code to github or VST s team services and then when every time I do a push and I commit you do a build and even I should treasure so you can do CI city but if you are working with something super simple like this one you can do the online code editor and what that does it just opens up your solution inside of a browser and if you kind of recognize the look and feel of this visual studio code it's actually aware that right it's an electron app so you kind of see that shell you see the same code intellisense and some of the benefits that you get so this is what as you created for you when we said file new project and then create a new bot this is what it did and it's gonna say hey you can change things here when you're done just hit build CMD we can build it and deploy it back for you so there's a whole bunch of things in here that are not important the only thing that's important here is this echo dialogue CS this is literally the one file that's controlling what your bot does so this is going to standard for any Microsoft bought framework bot so there is a message received this is fired when your bot receives a message from the user so the users say something it comes to us and you can actually reset it so you'll notice that we actually have a counter that counts how many times you say it and then if you say reset it's just gonna reset the counter so there's a prompt to make sure you're you're confirming it but otherwise it's just gonna say contacts start post async you said that okay so this is literally the one line of code that's echoing back what you're saying that's it that's the one so just one line that can echoes you back what you're saying so this is a bot that's deployed when you go to a sure and just do file new project give me about service ok so this is a bot because it's built on the bot framework but it's not a bot because it only does one thing yes ok so if this is all you wanted then wrapping it around a bot maybe a little bit of overkill yes but this is a nice template to give you some basic plumbing and then you can build on top of that but I mean this is a good starting point because not only are you creating a bot you're also hosting it in a sure and you already have those channels that are configured so you can put that bot in Skype and slack so again I would point people to the documentation which is really good if you go through the QuickStart here this is what we did creating a bot with the bot service but if you want to get the bot builder SDK then right now you have dotnet and nodejs so if you look at what the dotnet side of the story is you need vs 2017 and then you need the bot template c-sharp I mean it's a template extension that gives you this template simple echo bot mm-hmm so you'll get the bot framework and then you start with that and then it spits out the same code that you just but just in c-sharp and it's it's local to you and then you can test you about right now this simple bot is just web app so you can kind of fire this up now is Express and run it locally and then we'll talk about a different type of way in which you can test your bot that's the emulator so same exact idea you can do it with a dotnet and c-sharp what I like being on a Mac today is the node.js side of things because if you're in JavaScript you can do the exact same things the bot framework is time from agnostic that way so what this expects you to do is have NPM installed and then you get this part builder package essentially so it has all of the bits that bot builder can do for you so let me show you what that looks like so I'm gonna go in here into a folder and bring up my terminal alright hopefully this is big enough so I'm going to go into my folder here this thing is called the node echo bot and if I kind of fire this up if I say node fjs I'm firing up the app you don't see in it anything because it's just a console app but I can say the same exact things mm-hmm so it's just echoing back what I say so same exact idea it's just running locally in a console app so if you look at what this is essentially so I'm going to fire up vs code and look at the same folder essentially this being a node app we did a NPM in it so we have some package JSON we have the node modules in here but this is that one file that's making the bot work so essentially with node.js you have dependencies so we're depending on the bot builder and we're saying we are building a console connector and whenever you kind of fire off the message received thing we're going to take your text and kind of said you said that okay one line so this is no js' this is pure JavaScript running on the server side or wherever you have it hosted but it's it's your bot written in node.js now this is just local this is a console app they kind of step you through there are some extra packages called Resta Phi which kind of helped translate this console app into a end point because the bot has to be reasonable in mind so if I do an NPM install of rs.25 which is what the docs will actually walk you through so this is Resta phi is what we're using it's a very simple package that kind of helps you make restful endpoints out of your node.js code whatever your app is doing so we're using this and then once we have it we can change a little bit of code here this one is requiring Resta phi and same bot builder but we are creating a little server here so we can host it as a restful endpoint and then we can take into the cloud or whatever web endpoint it is that's our our port number and then we are creating a chatbot the app ID and password we'll talk about this so every part that's deployed to that robot service has to have a unique identifier okay and as your create that for you but otherwise it's just gonna say you said that so if we actually go back and run this now instead of app jeaious let's run node restful app dot j s this one said hey I'm not a console app I'm running at this port because you use Resta Phi so I'm running it as a restful service on 439 78 so how do you test this you can go to your browser or you can use this little thing which also comes with the bot framework this is called the bot framework emulator it looks kind of a little bland without having any inputs here but this is a local emulator that will help you test your BOTS locally before you're ready to deploy it to whatever service so in here I can actually go to that same port that we are on again it's going to ask for app ID password you can skip it for now you can test this locally but once you have to go to a show you do need the app ID so this is the exact same app and now you see that working here in the inside the emulator so this is the restful endpoint I don't have it hosted anywhere but I'm able to invoke it and the last thing is this one kinda shows you all of the JSON that's coming back and forth about service and and you so this is a super simple emulator but it does a lot of things and actually the emulator on Windows is a little more sophisticated than the emulator on Mac because one of the things that you get into with bought development is kind of conversations and you have to test your conversations and sometimes you have to kind of start midway in a conversation to start so it's like a little bit of template where you can pick up a conversation as just simple JSON and gonna start Midway and start testing so it'll let you do all of those things right yeah so that's the emulator that's one way of running it so let's let's close this that's just a quick example of doing it locally in no chairs let's kind of go back to Azure and I'll show you the other side of the story here so this was our echo bot so let's go home if I did a new resource again if I did a web app and then you look at these templates so it's not just the basic template which you can do it comes with five or so templates which kind of help you down the right path of building a bot first thing is formed this is one of the most commonly used templates this is where you have to order a pizza or you have to book a flight out and those repetitive things because essentially when you do this on your web applications today you are going through a wizard none of this is new right it's how you want to interact with your computer are you more comfortable doing a more natural conversation rather than just clicking around on your website okay so we can kind of walk you through that wizard in a bot so these are like a series of steps then language understanding this is where we're actually going to call into Louis in cognitive services because this is what actually understands what you mean when you say natural things okay so behind it it's going to be an azure Community Services app which has to have its own app ID and it's needed it needs to be registered and then that's when you're going to have to train your model so when you say booked a flight out of Seattle what are the 20 different ways in which you can say it so they're your bot understands okay and then passes it back to the bot so essentially what we are getting is from the bot framework you are pre-configured to call in to Azure cognitive services and then as your community services makes sense of what you are saying and brings it back to the bot and you can do this from C shop or no js' okay so just a quick example of the form thing which is very easy to look at I do have a form bot here which is called the sandwich bot because that's just a normal example so let's go ahead and look at this thing here we can test this in a web bot or web chat window so here if you notice the little messaging here I can say hi I'm up here and the bot knows that it's a specialized bot and says hey it looks like you're trying to order a sandwich because that's what I know what type of sandwich do you want so these are now Pickers okay so you get to pick what you want I would like a steak and cheese now it's gonna go through a list of things now you're getting two rules right so you said this then that then this then that so and you can kind of back-trace sometimes if your user does not give you the right response you can say oh I didn't understand that I wanted this from you so you can kind of go back and forth a little bit so 6-inch sub what bread you want that and then within breads with in Italian breads you have two types what do you want what type of cheese so these are all Pickers then you get to pick a whole bunch of things so these are like just numbers that you can pick cucumbers what type of sauce do you want let's do honey mustard and that's it so it's essentially said I went through these six steps with you and I made you choose things every step and you gave me this sandwich and once you are ready is this your selection you said yes and then said okay that's your sandwich I'm going to send it off to the shop and they're gonna deliver whatnot okay so just a super simple way of using the form template which actually kind of gets you quite a bit into these like flows okay how to ask the user and if you look at what this does for you I'll give it a second all right so now this one has a little more stuff to it so there's a model which is our sandwich this has all of your options like what type of sandwich what was it what type of bread what type of cheese it's all in here and then you have your dialogues as to how you actually control what you ask the user that's message controller so this is where your tasks and your your your post and you your awaiting the message from the user all of that is going to code it up here and then you kind of let bot kind of drive the conversation as to what exactly it needs to get all the information from the user so we can go about it so all of this is kind of bundled in and it's part of the template that you get it's really easy to call into most cognitive services like and good examples of this are like vision api's if you have to have a bot that knows how to understand or make sense of images from the user you hand it off you take the image you hand it off to the azure Community Services vision API we have another customer of ours who uses the vision API to do quality control on shipping so they have a little camera that's mounted on top of shipping containers mm-hmm and they do quality control as inside going out of their warehouses you can also do sentiment analysis with with common services so as you are talking to your bot you can kind of sense that the user is getting angry right and you can sense his sentiment and say oh maybe we can do other things to make you happy and so on so all of these things are intelligence that you're adding to your bot and this is what makes BOTS smarter and they kind of let you automate workflows so this is this is where we are this is where we start some things that I really like pointing out is the thorough documentation that the bot framework has there are tutorials on how to do just about everything let me show you some things that are really interesting the concepts here let's do Congress services language understanding let me do a how-to I thought it was really nicely done they talk about channels and the thing that I was trying to get to is how you can hook this up to language services that's something we'll do in part two yeah we could so essentially I mean just just start here essentially is what I'm saying okay and they have documentation on how to use vision API and language services all of these things that help you go from your bot to other places that give you about intelligence okay that's it so bots are nothing more than apps you can a restful endpoint but it's platform agnostic so you can put it in whichever channel that your user is using and start automating little things in your workflows that don't need human interaction very cool yeah yeah I mean if you look at some of the most common things that we do on most websites it's a bunch of steps that we follow wizards and this is where BOTS can come in and make it more conversational and more natural right yeah all right that was a good introduction we'll end it for here in this part and we'll come back in part two and and see more about how to build them absolutely so again this was all about BOTS you can find me at SMI div that's my social handle on Twitter and github in everywhere but thanks for watching all right and we will see you next time on visual studio toolbox [Music]

Brad Anderson's Lunch Break s9 e6 Mark Gregory, CIO & EVP, Premera Blue Cross

- It's lunch time and this is Brad Anderson's Lunch Break. Here in Redmond we're visited by some of the smartest people on the planet. Pretty much every day. Every chance I get I meet up with them for lunch. (upbeat music) Today I talked to a very smart guy who gets his hair cut at the same place I do. Mark Gregory the CIO of Premera Blue Cross. So you're here from Premara - Yeah - located right here in Seattle. You guys are the biggest provider of insurance in the Northwest. - Yeah. - I didn't realize this but Premera is 85 years old. So what's it like working for a company like that, has these incredibly old roots and history and culture going through the digital transformation. - The history is great. Companies that have been around a long time like Premera, our challenge is to take care of all the technology that got us where we are. - Mm-hmm. - But also advance it and get prepared for the future. There's four basic problems we see in health care. The first one is, I don't get the care I need. Second one is I get care I don't need. - Yeah. - Which is waste but also - Well it's time - Can be dangerous. - Sure exactly. - Right. The third one is really around value. I pay a lot, do I really get value for what I pay? And the fourth one, Brad, is experience right. So how many times have you had a bad experience with health care of some type? Part of what we try to do at Premera and how we leverage technology is to solve those four issues. - So how do you use some of the technology that address those four concerns? - We sit on large quantities of data. - Yeah. - Right? So, if I can or we collectively at Premera can engage the customer with tools that let them see their options right. Where can they get the best value for their healthcare dollar? Where can they go to get the best health care outcome? Right? And that all comes through data and engagement tools to help them solve that problem. Also on our side, how do we help lead them there? - Not only do we have the same haircut, (laughs) but we also share, you know we have names that are fairly common. So I did a search on LinkedIn and there were multiple Mark Gregorys that I found. - Right. - So let's play "Which one are you?" - Okay. (laughs) - Do you play keyboard in the band Hot Tub? - I do not. - Back in the early 2000 did you own a small chain of Pizza restaurants in Alaska? - I did not. - Did you have a role in the movie Adam and Eve, Meet the Cannibals? - Oh man, no. I wish I did though. Didn't that sound like a great experience? - Seriously right? - I'm tellin' you man, there is a lot more interesting Mark Gregorys out there than me clearly. - Next time on Brad Anderson's Lunch Break - When we introduced Cloud to the organization we did not introduce it as a cost play. We introduced it as a value play. - Yeah, yeah. - Really, right? So how quickly can I bring up the capabilities that move the organization forward. I wanna bring it up lighter, quicker, faster. And then I want more options with it.

Brad Anderson's Lunch Break s8 e3 Raja Ukil, SVP & CIO, Wipro

- It's lunch time, and this is Brad Anderson's Lunch Break. (chipper music) Here in Redmond, we're visited by some of the smartest people on the planet, pretty much every day. Every chance I get, I meet up with them for lunch. (chipper music) This week I'm talking with Raja Ukil, the Senior Vice President and CIO of the massive Indian IT company Wipro. (piano lounge music) Wipro, you guys are massive, 175,000 employees. I don't think there's very many companies in the world that have been in IT for that long. - [Raja] We started off in the vegetable aisles and then there were market opportunities. We diversified, and in the 70s, we took that opportunity, went into hardware and just grew from there. I think the industry was coming into shape and, you know, those are good old days. I would like to look at security more comprehensively around identity, around data protection, and privacy. We are all getting ready for the GDPR, which is coming in May in Europe. So your strategy is very comprehensive, right? And if you look at some of the things which are happening of late, it's essentially somebody falling prey to identity theft. It starts from there, right? - That's where it starts. And then, just by protecting identity will not be enough. You have to have a comprehensive protection strategy, but which doesn't kill user experience, and that is where I think your strategy is what I love. You're protecting all of this, but not killing the user experience. We had pieces of this, not comprehensively, in traditionalized research solutions and they were infringing into user experience into privacy. - In college, you played cricket for your university. - Yes sir. - As Americans, we kind of struggle to grasp the rules and the concepts of cricket. So, we're gonna play a little game here where I'm gonna give you some cricket terms, - Yeah, okay. - and have you explain it for those of us who don't understand. - Okay. - The slog sweep. - Oh, the slog sweep, yes. So the slog sweep, there is a reverse sweep and a slog sweep. - Of course. - When you can't figure out what the bowler is going to do and you're desperate, you just bend down on your knees and just sweep it. - Among all the possible sweeps, is that the best one? - Yeah, I have seen many batsmen close their eyes and just play it in my day. - OMG, close your eyes? - The silly mid off. - The mid off is a position, how do I explain it? It's like, you know, when you are batting? It is like at an angle of, maybe 30 degrees. - That makes sense to me. How about the golden snitch? - Golden snitch, I think is a golden duck. (laughter) - I played a trick on you. That's a Harry Potter game. (laughter) - You need to ask my daughter. - Oh, OK. Alright. The dibbly-dobbly. - Well... (laughter) I need to get back to you. - Gotcha, okay. - The Duckworth-Lewis. - The Duckworth-Lewis, this one I can do. It is one of the most illogical things to happen in cricket. When you have a disruption in a game, the team batting second actually gets a limited number of overs. And, for some strange reason, gets a target which is steeper than the original target. So, this is a thing which is prevalent now. Nobody wants it, but they have never been able to find out a better alternative. - OK. The pace of innovation that we are delivering for you, are we meeting your expectations with Microsoft 365? - Yeah, absolutely. Every time, Brad, I come here, I just come here to learn. It's a different Microsoft. You are incredibly fast today. You think of it, and it is there. I know we have spoken for so many years. You have explained your strategy and what you are trying to do in a few days. I can't believe that the product is ready with millions and millions of customers today around the world. It's just incredible. Coming from the days where you used to have releases once every year. - Yeah. - So, this is fascinating. Another thing I must compliment you and your colleagues, is that you are listening to customers. Obviously, there are different use cases all of us will have, and you are focusing on creating those use cases for all of us part of a common platform, and that is fascinating. - Wow. That just makes, that just makes my day. You're one of the largest Windows 10 customers in the world. You know, with 170,000+ employees, you're one of the largest Office 365 customers in the world. Help me understand, as you think about, as you embrace these new technologies, even if you just talk about Microsoft early on. What are the benefits that your community, your employees see, and what is the benefit that IT sees? - Employees simply love it. Right? It's ... today's world, if you look at the employees' average age, we have a lot of millennials, right? These employees don't engage in the way, possibly, I engage. And so, - They're digital natives. You and I are a little bit older. - I wasn't going to say that specifically. (laughter) So, what happens is you need new tool sets, you need applications which are intuitive, re-imagined, simple, and essentially, you know, you empower every front line employee so that they don't waste time, and they can do things very simply. Now, to do all of this, it was not possible previously. The experience was broken because of the technology was not there. Today, thanks to Microsoft, you have a complete end-to-end story. It was the right strategic direction. And mobility - 50,000 of our people are not in any Wipro office. So, erstwhile, deployment architectures are a thing of the past. You can't manage such explosion with that kind of a thing. I'm happy Microsoft has focused and brought it to a shape where it's a real blessing. (light music) - Next time, on Brad Anderson's Lunch Break - I'm proud to say that every dollar we earn, 40 cents goes back to charity. - Wow! - Yes - I did not know that. - Is it explaining cricket someone in the US, or trying to teach a dog Calculus? (laughter) (chipper music)

Brad Anderson's Lunch Break s8 e2 Rhonda Vetere, CTO, Estée Lauder Co. (Part 2)

- [Narrator] It's lunch time, and this is Bard Anderson's lunch break. Here in Redmond, we're visited by some of the smartest people on the planet pretty much everyday. Every chance I get, I meet up with them for lunch. Today I finish talking with the CTO of Estee Lauder, a very smart and always fabulous Rhonda Vetere. So now let's take a look at your team itself. And, you now, the IT department. How has the morale gone now that you've moved so much to the cloud, how has that morale gone up? Do people say that their jobs are more impactful, or have they this, this fear that they have, well maybe my job gets eliminated? What's been that impact? - That's been an interesting transition. And really worked with a team on the mental model shift. Their mental model on growing and spring boarding into what we call the new technology world. The morale is up. At first you have what I call going the valley of despair. And then it goes back up, just like things in life. But as you lead and show them how things come more productive, how they're giving back, how they can free up their time, quite frankly, and work on more fun things. And not so fingers on the keyboard on operational issues. Their morale goes up. And we're proud of that. And there's a lot of rock stars that have developed out of that. - So that topic of staying relevant is really important. You know, during your career, you spent time at CompuServe and WorldCom and MCI. You've had a front row seat at companies who were at the pinnacle of driving different portions of the industry, and then kind of fell away, right? What can technology leaders learn from that? What did you take away from those experiences? - Being relevant and that you become obsolete overnight. And not realize it. No matter what field that you're in. Being in technology, you can be merged, acquisitions, everything happens. And things are coming at you from all angles and you need to be able to manage that change. So I took that as a learning experience, and stayed through all the different merges and acquisitions to be that constant denominator to drive that change in making the companies more relevant as the world changed. - I read an article where you listed your top ten list of business standards. And there were three of them that just like really hit home with me because I value these three as much as you do. One of them is what you called managerial courage. - Very good. - Recognize your team. And then know your metrics. - Yes. Recognize your team. So I'm always about it's the team. And recognizing them. It's celebrating success as most people forget to celebrate. - It can be little successes that you celebrate. That have a tremendous impact on the culture. - Absolutely. - Give me an example. You talk about leadership courage. - Leadership courage is the ability to step up and say the power of no in a nice way. I don't agree with this, this is what it means, we're doing the wrong thing, and even on a team and partners I look for that. Someone who has a challenging spirit that can tell you you're going down a wrong way. - All too often, people are afraid to say no. And most times, you know, who you're talking to, they just wanna know what the answer is, and no is okay. But it takes courage, and for many people, that's a courage that has to be developed. That's actually one of the interview questions that I asked when I'm interviewing senior leaders. Tell me about a time when you had to say no and were convicted that the right answer was no. And you knew it was gonna be a challenge to get the rest of the organization aligned. I'll tell you, if people cannot come back and give me an example of exactly when they've done that, they haven't had enough leadership time. They haven't had enough leadership experience. - Great point. That's great. - It has been a pleasure. - It's been a pleasure. Thank you. All right. Thank you. - Thank you so much. Bye. - [Brad] Next time on Brad Anderson's lunch break. Duckworth-Lewis. - The Duckworth-Lewis say, I can give a, it says one of the most illogical things to happen in that regard. Erstwhile, deployment architectures, you know are a thing of the past. You can't manage such explosion with that kind of a thing.

Brad Anderson's Lunch Break s7 e9 Dr. Nicole Forsgren, CEO, DORA

- [Brad] It's lunch time, and this is Brad Anderson's Lunch Break. Two of my favorite things about Microsoft are the smart people that constantly visit campus and the great fleet of shuttles. Whenever I can I try to take advantage of both of these things, and grab lunch with some of the tech industry's best and brightest. Today I'm driving around with Dr. Nicole Forsgren, a researcher, DevOps expert, and CEO and Chief Scientist at DORA. (jaunty music) - Hey - [Brad] How's it goin', Nicole? - I'm good. Sweet minivan. - You know, I wanted to harken back to the early days when we needed to have a minivan in order to put our kids in it. It's like a walk down memory lane here for me. In software, we're all about data and we all use data to make decisions, and you've brought this uncanny academic rigor to how you use data. In fact, you have so much passion you formed your own company. So tell us a little bit about the company, your background and this passion on data. - Sure. So DORA is DevOps Research and Assessment. We've been doing research looking at how organizations can use so many fantastic capabilities to drive improvement in their ability to make software with both speed and stability. That's really the key central component in helping organizations perform amazingly well. And I say perform, so that's profitability, productivity, market share. - Yeah so it's both internal and external because a big part of performing is employee satisfaction, employee passion. - Also, decreasing employee pain, decreasing burnout. - Because we talk about this in-depth DevOps approach to IT and to Dev. How does that impact morale? - When we study DevOps or technology transformation we take kind of a holistic approach and it includes culture, it includes technology, it includes Agile in measurements, and the piece in Agile includes how you develop software and team experimentation. And so the way it affects morale and team culture is we see greater communication, greater information flow, risks are shared, novelty is encouraged and implemented, morale is improved, and that ends up feeding through and having a positive impact on the entire organization's performance. For so long, have you heard before, 'IT's a cost center, IT's a cost center,' - That's definitely transitioned but that's the way it's been talked about for years. I'm tired of it. - Right? Seriously. But what we find is that it's really an investment. If you see it and if you view it as an investment it pays back dividends tenfold, because it makes your technology better. It makes your employees better. It makes your employees happier. It delivers value to your business. Again, profitability, productivity, market share. - [Brad] What's the most common pitfall organizations fall into in this area? - So there, I think there are a couple. So one is not measuring the things that matter. Right, so we have, sometimes it's just local optimizations. Right, sometimes people are like 'I'm gonna do CI, Continuous Integration, or I'm gonna do Continuous Delivery, because Continuous Delivery,' and then they just get spun up on something. Right? Or they'll measure something because it's convenient, or easy, and then it shows up on a report and that's all they pay attention to. And I'm sure you've seen that. - Oh, a hundred times, and then there are even things that I've asked to be measured and then realized it was the wrong things. - Which is fine, because it's continuous improvement, continuous learning, so then you could still keep it if you want, but at least pull it off the report. - Yeah, but start measuring, and then you're gonna realize that you're gonna get better. Just the numbers, when you first start doing the measurement the numbers are gonna be dirty, they're not gonna be clean. - Realize the power of an honest baseline. If it's bad, that's even better because you have nowhere to go but up. - At least you got a place to start from. That has been this journey, I think, that many of us have gone through. That it is all about getting started and then constantly learning, constantly getting better, constantly refining, but you gotta start. - And it gives you something to talk about and communicate about. Suddenly, if you're on a different team, I can talk to you about it. And, if I think your performance or your metrics or something sucks, that's okay because I have something to talk about instead of yelling at you, we can yell about the thing. - That makes it tangible and real. You can have a conversation and then say 'Okay, now how are we gonna go get better on this?' - Exactly - Next time on Brad Anderson's Lunch Break: - If you wanna be super capitalistic about it, we find so much research everywhere showing that diversity of background, diversity of thought, diversity of gender, diversity of race, diversity of all the things leads to better outcomes. You make better products, you build better software, you get better outcomes. It's worth the investment. (jaunty music)

Brad Anderson's Lunch Break s7 e8 Mauricio Guerra, CISO, Dow Chemical (Part 2)

- It's lunch time, and this is Brad Anderson's Lunch Break. Two of my favorite things about Microsoft are the smart people that constantly visit campus, and the great fleet of shuttles. Wherever I can, I try to take advantage of both of these things, and grab lunch with some of the tech industry's best and brightest. (sunny music) This week, I'll wrap up my conversation with Mauricio Guerra, the chief Information Security Officer at Dow Chemical. (sunny music) So I heard this amazing story about the founder of Dow Chemical, Henry Dow. - Mm-hmm. - He came with this idea to use electrolysis to basically separate chemicals, but the financial backers thought that was a crazy idea, and actually fired him! - Mm-hmm. - So, he turned around and used that technology to create Dow Chemical, and the rest is history. So what technology do you think organizations are not using enough of, to keep their company safe? - I don't want to say that it's a particular technology but that will be different, say, pointing to technology A, or B, or C. What I would like to stress, is the need to have a comprehensive cyber security program. Understanding that cyber security is not installing, implementing a couple of firewalls here, and this, and that, and that. It's making sure that from the executive level down, the companies understand, and properly manage the risk. And it sounds obvious, or too simplistic, what I'm saying, but what I'm seeing, the industries, that there might be a better opportunity for companies to take a more, more, more structured approach into cyber. - Kind of a footnote in history, you know, Henry Dow actually went back and bought the company that fired him. So, words of wisdom, be Henry Dow. - Oh, okay. (laughing) (jazzy music) When we see winning innovation, that will bring big challenges for people that are in like we will be dealing with way more automation, we'll be dealing with more data, more devices, more connectivity - it's more complex - More clouds, more complexity, huge complexity. And normally when you drive innovation is when you bring risk. And I'm not implying that's a negative, that's the way it is. - That's reality. - That the reality. New stuff, new challenges then we either need to adjust our security poster or figure it out how to mitigate risk. To summarize my point, yes it's a bigger challenge. Not only managing the well known threats, but also enabling those capabilities that companies like Dow will be using to drive innovation and being more and more and more competitive. I mean better leveraging IT tools. (sunny music) - A big part of Mauricio's job is evaluating the security aspects of what IT wants to deploy. - Mmhm. - So how do you challenge the partners that you're working with to make sure that they're delivering the right balance of value and security? - At the end of the day, there is not a perfect solution. They need judgment, different perspective. So it is all about helping businesses making educated decisions. By educate I mean informed decisions - Yeah, sure. - Personally, I spent a lot of time on visiting companies, participating in industry forums that gives me good feel on where the industry is going. But, at the end of the day, there is judgment. Judgment, but then, you need to establish a strong network and be credible with the businesses. - Yeah, so you have to have all this signal coming into you so that you're constantly being aware of what the innovations are, what the attacks are, and then you have to help synthesize that and share it with all the leadership. - Exactly. (light upbeat music) - This has been awesome. - For so long, I mean have you heard before: You know I'd use a cost center, who's heard that? - It's definitely transitioning but I've heard that - And there's been so many times right? If you see it and if you view it as an investment it pays back dividends ten fold. (light upbeat music)

Brad Anderson's Lunch Break s7 e4 Ben Kepes, Public cloud analyst (Part 2)

- [Brad] It's lunch time, and this is Brad Anderson's Lunch Break. Two of my favorite things about Microsoft are the smart people that constantly visit campus, and the great fleet of shuttles. Wherever I can I try to take advantage of both of these things, and grab lunch with some of the tech industry's best and brightest. (upbeat music) Today I'll wrap up my conversation with Cloud expert and New Zealand's finest, Ben Kepes. Out of all of those hobbies, what's your favorite? - To be honest, running is my thing. I've got two sons, my eldest son is about 17, and he's also a runner, so we head out and do ultra marathons together. So there's nothing better than hanging with my kids. - So what's the most miles you've logged in a single day? - So I've actually got a 100 mile race coming up, in about 10 days, so that will be my longest race to date. Previously I've done 100km. (light swing music) - So why did we have to send Uber to bring you here today? - Well I actually did suggest that we go out for a run, but you weren't up for it! - I was chicken, I was totally chicken on it, because I knew I couldn't talk and run at the same time. - Well who needs to talk? (laughs) We could run slow. - You know, one of the really big pivots that I've seen in the last year, is the focus that IT is placing on the end user experience is dramatically greater than what it was in the past. - Totally, and that's a generational thing. At the end of the day, if you're a millennial going into the work force, and you've spent your life on Instagram or Snapchat, or Facebook or whatever. You're not gonna use those lengthy enterprise systems that are just horrendous. - [Brad] A couple of years ago we really focused our design on how do we develop an environment that IT can deliver that is loved by users, and trusted by IT. And doing both is really hard. - It totally is, and there's this tension there, right? And the reality is, you need give people like a sliding scale, you want to give people that optionality, to go all in on the user love, but that's gonna cause more IT pain, if that's their choice for them. Or the other way, as an organization, I guess Microsoft needs to think about what is the line at which you won't go? - So there's a balance, and every organization is gonna have a different balance on that one. - [Ben] It's funny, I spend my time in analysts' session, sit around with a bunch of analysts who've been in the industry for 50 years or whatever. How many of them have owned products that a billion people use? None. So the reality is, it's all very well to say this is what this vendor should do, this is what this product should do, but at the end of the day, if you've got a billion users to think about, you've actually got a responsibility there. It's not quite as easy as it looks. - I think people would be amazed if they understood the level of the investments that we make on backwards compatibility, and making sure that we don't leave the customers behind. It's a stunning amount of our investment. - For sure. - [Brad] We're gonna play a game that I call "This or That?" So I'll give you two subjects, and I'm gonna read you a list of descriptions, and you tell me which of the subjects it applies to. - Okay. - [Brad] Public cloud, or caffeine? Don't try running a company without a lot of it. (deep breathing) - Caffeine first, but both. - Yeah, I agree. Its benefits and drawbacks are under constant debate. - No, I'm gonna say cloud. - When you first start using it, you have to build up a tolerance, or your system will freak out. - It's an interesting one, isn't it? Really is an interesting one, I think ... I'm gonna go with caffeine. - It's easy to get hooked and start adding it to everything. - Yeah, definitely both. - Now one of the things you related to me is how intriguing and how innovative Azure Stack is, I'd love to get your perspective on Azure Stack. - Yeah, so it's really exciting for me, and for me the reality is there are a lot of purists who say the public cloud is it, and there's no validity to the private cloud. If you were talking to someone in the banking space, or maybe someone in the medical space, or in a particular geography, or someone who's running a cruise ship company, and has boats going out on the high seas, well that abstract kind of idea doesn't really work in practice, because they've got times where they can't, they're not connected. So absolutely hybrid is the reality. I'm super excited about Azure Stack, where you've got something that is the same public cloud, private cloud. You can have interplay between the two, your applications can run on both. That's kind of the future, and I think it's really disrespectful for the pundits who say public cloud is the only way, and private cloud isn't real, because at the end of the day, that doesn't respect the reality for a number of organizations. - Well you know, the next time we're gonna do this I think we're gonna do it in New Zealand, we'll go for a run. - Sure, so you sent me a shuttle. Next time send a Microsoft corporate jet, pick me up in New Zealand, and we'll go do some stuff. - I'll get workin' on that. - Awesome. - Hey, thank you so much. - Thanks, man. - Bye bye, man. - When I first met them they showed us how they can ingest a page of text. It just happened to be a page of Harry Potter. And not only read the words and define the words, but comprehend what was going on. (laughs) But I do get right and left mixed up, and I get port and starboard mixed up. - Do you really? - Yeah, all the time. So I'm hoping that no one ever has to launch those missiles. I don't want them to go up the wrong side. (upbeat music)

Brad Anderson's Lunch Break s7 e3 Ben Kepes, Public cloud analyst

- [Brad] It's lunch time, and this is Brad Anderson's Lunch Break. Two of my favorite things about Microsoft are the smart people that constantly visit campus and the great fleet of shuttles. Whenever I can, I try to take advantage of both of these things and grab lunch with some of the tech industry's best and brightest. This week, I'm meeting up with one of the planet's foremost cloud experts, Ben Kepes. He's also an ultra-marathoner and part-time firefighter. - Hey man, so I ordered an Uber Deluxe. You send me a shuttle? - You know, this is like the top of the line shuttle here at Microsoft. - So you know I come from New Zealand, and not only do we get slow internet speeds, but we get the dodgy car as well. - [Brad] In addition to owning your own industry consulting analyst firm, you write for Computer World, you're one of the leading experts on the cloud, but you also run ultra-marathons and you're a volunteer firefighter? If there's ever a forest fire a hundred miles away, you're the man to call. - You're fun. I was actually a paramedic, - Oh, so you dialed it down a little, you said. - If we have an accident while we're driving, I've got you sorted. (laughing) - [Brad] Okay, so when you consult with businesses that are in the process of modernizing, what are the characteristics of the companies that really are able to make that transition successfully? - Yeah, so like I've already said it's about culture. So the culture being adept probably would change. - I think so many of us overlook that. - Yeah, and it's pretty interesting. I mean, if you look at an organization I find, 'cause I do a lot of consulting with these large organizations. I can walk into an organization and pretty much I can get a sense of what it's like. I look at the torques they use, I look around, I look at simple things like what it's like to get into the building, what it's like, you know, if they've got a receptionist there who's kind of about blocking access to people, you know, those things. - That's old school. - Yeah, it's really those things. And those things, you know, people would say that has nothing to do with my technology adoption. - [Brad] Your company invested in a lot of startups. Okay, so we're gonna play a game that I call Startup Game. Okay, so I'm gonna give you some names some are real startups and some are fake. Okay, so the first one is Thundera. Thunder with an A. Real or make believe? - Fake. - That is a fake one, you got it right. Okay, the next one is Foodler. F-O-O-D-L-E-R. - Yeah, that's gotta be a Sunken Valley true company. - That is a true company. Okay, the next one, WubbaLubDub. - See, that's too many letters for a URL, so that's fake. (laughing) - You are a hundred percent right, and I think your logic is probably right as well. Flingr. - Yeah, now that works, that's true. - That's a fake one. But it could've been a legit name, right? - Yeah. - BlaBlaCar. - I'm pretty sure that's actually a real company. - That is a real company. - But it's a bad URL. - Man, you did, by far, the best anyone's ever done on that game. - There you go. - [Brad] If you were talking to an organization that really is kind of a little bit hesitant to enter into the cloud, start using the cloud, what would be your biggest piece of advice to them? - So I think the first piece of advice is that this is happening. This change is there, you don't have any option. It's kind of this carrot and stick. So the stick is you have no option, you have to change. The carrot is, there's ways that you can damper with this, there's ways that you can do some other little things, and just showing people what you can do with things like, you know, Help EI, when it's in the cloud, for example. Just showing people the value that that unlocks. In a small way, in a discreet part of their business, we'll get them to kind of develop the muscle memory, like it's getting to feel a little bit more comfortable, until they can start moving more or mass into the cloud. - [Brad] You know, one interesting thing I was thinking about as you were talking, you know, we made the decision as we were building what I oversee in Enterprise Mobility and Security, back in 2012 they were going to do it entirely from the cloud. And I remember in 2014, you know, about one out of two customers would tell us, "Hey, I'm not comfortable doing this from the cloud." 2015 it was maybe 20 percent. Now, it's rare that I ever hear an organization say, "Hey, I'm not totally comfortable with my management security coming from the cloud." - You know, it's totally funny. I mean, just the awareness. I've been doing this for, I guess, ten years since the cloud's been around. And four or five years ago, I was at a function, I was introduced to an elected member of parliament in New Zealand, and someone introduced me as this global cloud expert, and this dude said, without a word of light, "You study the weather, do you?" Fast forward to today, only a few years later, and there's no one that doesn't have at least a vague notion about what the cloud is. And it translates into organizations who understand it, are starting to use it, to some extent. And it's the ridge, right? We're kind of midway up the angle of the ridge, and it's only getting lighter. It's the reality. - And there's literally no limit on where this can go. - And it's cool for those of us who have been kind of, you know, saying, "this is the future," for ten years. It's so cool to kind of be vindicated. - [Brad] Next time, on Brad Anderson's Lunch Break: - I think it's really disrespectful for the pandets who say, "Public cloud is the only way and private cloud isn't real." Because at the end of the day, that doesn't respect the reality for a number of organizations.

Brad Anderson's Lunch Break s7 e14 Tim Anderson, Journalist (Part 2)

- It's lunch time, and this is Brad Anderson's Lunch Break. Two of my favorite things about Microsoft are the smart people that constantly visit campus and the great fleet of shuttles. Whenever I can, I try to take advantage of both of these things, and I grab lunch with some of the tech industry's best and brightest. (bright, happy music) Today I'll finish up my conversation with veteran tech journalist, Tim Anderson. (car revving) One of the challenges that just about every organization in the world is now grappling with is, as the data moves out to the Cloud, and the data moved to mobile devices, the traditional perimeter-based security model they've used is just no longer effective. - Sure. - So what we believe is in this mobile and Cloud-first world, identity becomes the new perimeter, and we believe that, in terms of Enterprise Mobility, identity is like the most foundational piece. And so, as you give advice or as people ask you questions to get your perspective, 'cause you've got so much historical perspective, talk about the importance of identity in this mobile-first, Cloud-first world. - Well, so it is what we care about, identity, every organization - Yeah. has got data that it cares about, and what we care about most, in terms of security, is who has access to that data. - [Brad] Exactly. - And also we, perhaps, care about who doesn't have access to that data, - (laughing) well said. so clearly it's, identity is at the heart of that. - So as we take a look at the data breaches, that it seems like we read about just about every week, - Mhm, yeah. like 70 percent or more of those eventually trace back to somebody's credentials having been compromised, a phishing attack or they had weak credentials. In that world, most organizations basically have to take an assumption that they've been breached, that they have identities that somebody is using to steal or attack them. How can organizations protect themselves? - I think, one thing I've noticed, is that the answer to that question varies according to the size of the organization, so in the smaller businesses, I still see a lot of very poor practice and security because it's not really understood. I think a lot - Yep. of education has to happen then, I do think that the Cloud is an enabler here, because the Cloud services are giving smaller businesses a taste of enterprise level computing, which is a lot better than they've perhaps had in the past. Now in a large organization, I mean a lot of it is around about handle credentials and authentication, so we need to do multifactoral - Yep. authentication, and try and move on from passwords. - Yeah. So Tim, you know, you've had this incredible perspective for 30 years in technology, so think of someone who's like at the early to midpoint of their career, and they're asking you for advice on managing and growing their career and impact. What advice would you give 'em? - One of the things I've been a bit surprised by is that even though computing is, in one sense, the cutting edge of modern life, of great change, IT professionals can be quite conservative in their attitudes, - Yeah. And I think it's important to keep learning, I think it's important to go to events, to read books. Keep enjoying it if you are a developer, for example, remember the magic of it, remember the magic of programming and seeing something before your eyes turn into a project that has intelligence and has a user interface and delights users, and just keep a sense of the magic of it that will keep you motivated to do what you do best. - That's great advice. Tim, it's always a pleasure to see ya, I always look forward to seeing every time we get to meet up - No it's been great, at an event. - thank you very much. We gotta make sure we go do this again. - Sure, yeah. - Alright, alright thank you very very much. - Thank you. (door closing) - [Narrator] Next time on Brad Anderson's lunch break, - So your two terms are transatlantic flights, or World of Warcraft. - Okay. Okay, do you know much about World of Warcraft? - A little bit. I had to stop playing - Alright. so I could finish - Work? (laughing) writing my dissertation, yes. - Okay, alright here we go. You would do something unspeakable in exchange for an upgrade. - Transatlantic flight. - You know, I've got hundreds and hundreds of engineers who have built ARM Print products for, for some of them decades, we've had to go through that same cultural transition internally that many customers right now are facing, as we're helping them and pushing them to move to the Cloud, but we've had to go through it first. (happy, bright music)

AI After Hours Feature Search with GitHub Copilot

okay Erin one more time are you sure you can do this you're confident right you know what the menu item is called yes I have a secret weapon okay what's your secret weapon well I am currently searching through menus to see which one has the Vibes I'm looking for yeah not good Aron the secret weapon you no no no no welcome to after hours with the visual studio team and today we're going to show upcoming improvements to feature search in Visual Studio it takes your search query and uses GitHub co-pilot to elaborate on instructions help you find settings and more because we as developers all know the visual studio is quite powerful and sometimes you just forget what a menu item is called uh new capabilities get added to new versions and there's always maybe a better way uh to complete a task in visual studio right well with this being feature search which is already an empty text box sometimes text boxes that you don't know how to use or aren't giving the results you're looking for can be a little bit intimidating yeah absolutely like do I talk to it in full sentences do I type keywords like I usually do of control q and expect co-pilot to make sense of it like do I go straight to the AI to help me find what I'm looking for in Visual Studio um how is this better than feature search itself these are all things I need to figure out all good question so let's explore the experience and the engineering behind the upcoming improvements we're making to feature search in Visual Studio I'm Leah and I'm a product manager for search in Visual Studio I'm Ain a product manager working on get a co-pilot in visual studio and I'm Sandy I'm a developer who works on search in Visual Studio here I am in Visual Studio I'm going to go ahead and hit the search button up here that's going to bring up feature search which can also be summon with control Q this has been around for a while you may be familiar with it you can type something like I want to see all of my breakpoints so you type the breakpoint window and there it is hit enter there your break points wonderful uh but we know that sometimes users don't know the name of the thing that they're looking for especially newer users they might bring up feature search and they might type something like uh I want to change my short Cuts so let's edit shortcuts but the responses that come back aren't really what the user is looking for it's a bunch of you know commands shortcuts that exists under the edit menu so we're going to try this new button that has shown up under their search we're going to click ask co-pilot and that same query that the user types is being sent to co-pilot chat and co-pilot chat is going to respond and give actually a pretty accurate response about exactly where to go go to tools options find the keyboard section of the options dialogue it's described quite thoroughly knows about the assign button uh it even offers follow-up questions so actually a really useful way to ask questions without having to leave the IDE um you know these are nuanced questions um another similar sort of scenario would be maybe you've come from you know net core and you're used to doing stuff on the command line and you don't actually know how to do a nougat ReStore in the IDE maybe it's one of these things you could click it and explore or you could go ahead and you could just ask co-pilot same thing send it straight to the chat wait a little bit and co-pilot's going to respond and let us know that you can just right click on the solution and that there's a restore n packages option uh it also tells you other ways that you can achieve the same functionality even talks about using the package Manor console if that's sort of a way that you like to develop historically co-pilots you know mostly known about kind of long-established features in Visual Studio but you know we're adding things all the time and so you know we've updated co-pilot to be able to understand about newer features coming into the IDE so for example if you are a um uh user who has you've seen on blogs or on Twitter or something about rainbow colored parentheses seems really cool so you come into the chat or you come into the feature ser search and you type rainbow pns and there's nothing useful here that's because you don't know what it's actually called again you ask co-pilot send the question to chat wait for its response and all of a sudden it says yeah yeah you can totally turn on Rainbow parenthesis but in Visual Studio we call that brace bear colorization says go to tools options go to General under the editor and Bam enable brace bear colorization so that is um co-pilot chat integrated with feature search hope you enjoy it so I know that when I watch any demo as a developer I have to wonder how do I know that this AI works well for my situation or that it can be trusted to provide answers that aren't going to confuse me more uh so I'm curious um what have we done to the future search in visual studio with get up co-pilot that's different from a web search and different from other AIS yeah definitely one thing that's uh different from online search engines is that with online search engines there's a much broader scope of things that it's looking at so if you're trying to search for something that's a feature in Visual Studio it's really easy to come across results that are actually about Visual Studio code instead because it's the similar naming scheme and there's even confusions with people online who don't know the difference between the two products so the answers that you get from places like stock overflow or elsewhere might not be entirely accurate and there's also a lot of out ofd information online which we don't have in the model as much because we're able to provide it with upto-date information about what are the features that are currently available in the version of Visual Studio that you're currently using yeah and because of that uh additional knowledge it knows more about the newer features that the other models and other uh AI models doesn't know about because it's trained on much older data and doesn't have that direct access to the information that we have I thought it was really interesting that you've mentioned up to-date and specific information that you've added to the ask co-pilot feature in visual studio uh because I can relate to out-of-date information and settings or feature names changing so could you talk about exactly what it is we've added um to bring this upto-date awareness to get up Co pallet yeah well I think a lot of Visual Studio users are are pretty used to being able to uh navigate to settings uh commands new features what's new all those sorts of things uh using the control Q feature search that we've had for a long time um and so what's going on here is we've got an integration between that existing feature search uh and co-pilot chat where if you are in control q and you're not finding what you're looking for now you've got a button to take you to co-pilot chat to have a more nuanced discussion and then on the chat side uh we're able to pull in all of that stuff that Visual Studio already knows about itself uh and make that context available instead of um you know as Leah mentioned when you're talking to chat GPT or another uh AI it's trained on older data because training is an expensive thing usually the models are fairly out of date so you might get really great information for things that have been in Visual Studio forever but you might not be able to get good answers about the latest and greatest and so that's where bringing in the information from feature search helps you better find Niche features new features uh things like that and um yeah you get a little more um uh you get a more carefully crafted sort of EXP experience um that's really tailored to what a visual studio user is probably asking about whether they're starting in the search or in the chat V so it sounds to me like we've done some rag like retrieval augmented generation where we added an awareness of keywords of settings and features from feature search um and also other resources like what's new information that someone might have written for a new release is that right right um you know it's a we're starting with uh settings and commands and uh some other sources uh what's new is coming down the pipe and we also have aspirations to bring in other uh sources using what you mentioned retrieval augmented generation where you you bring in uh known good information into uh the AI and so it's able to give more accurate responses uh for a really specific scenario in our case asking about visual studio um so yeah I see and when I often look up instructions for something on the web I notice that I have to first understand it and then figure out where to click in the editor um or how to do something myself because they may say go do a new get restore and I have to figure out hey what does this mean how do I do that um and so could you tell me more about how this feature um helps me spend less time translating something that I've understood and actually making something happen in visual studio um sure well the idea is that you know even if you don't know the name that Visual Studio uses for some feature um one thing that an AI is really great at is understanding what you're trying to say and so having that um pipeline from you know your your brain uh to what does a feature kind of look like in an IDE and the AI understands that to bringing in the specific uh terms for things in visual studio uh we're able to provide nice accurate instructions you know even if you don't really know uh how the right way to describe what you're looking for as long as you get close uh having an AI helps you get a little bit closer instead of spending you know 20 minutes on Google searches and still not quite finding exactly what it is you're looking for right and these link outs of these settings we we link out to the actual setting right so if you tell me to change a setting there should be a link for me that will take me to that setting yeah that's the idea um you know when we detect that you know something in the in the response from the AI is is a setting that we know about we provide a link so that you can get directly there without necessarily have to follow step one two three four five although sometimes it's nice to see those explicit steps if you're trying to understand uh how to teach uh the feature that you're uh looking for right and so Switching gears a little bit from the data science work to the experience itself um from what we're seeing today in preview what do you think is left for room for improvement that we want to build or work on in the future yeah there's a lot of ways to look about it so there's the basic feature search experience that we've already had in the product so there is definitely room for us to improve to make sure that it's covering the right scenarios there make sure it's handling differently formatted queries correctly and that's like without AI even in the picture but on the side with the work that we've been doing with AI there's definitely more opportunities like we discussed before to bring in more context from different sources to make sure that the answers we're providing is accurate and it provides the right details but at the same time we don't want to provide too much information to overwhelm the model or confuse it or bias it in a certain way so there needs to be some work done to make sure that we're picking the right context to send over to them model and provide correct responses from that and on the that note there's also um more ux work that we need to finish up with the flow that we've been working on because there's some things that aren't uh exactly smooth with the experience so we wanted to work on with the uh the links that it provides the way that it works the responses we want to make sure that it's actually providing helpful information there and actionable information that the user can figure out what to do with and it's not just providing things that they don't know how to act on yeah I I mean that sounds great because we would want feature search to work well within without co-pilot and it sounds like any improvements that we make to that will also improve co-pilot and you know speaking with the team that's building a feature that helps you um resolve any unit test failures they they also spoke about you know being really selective with your contacts so you can provide a good precise answer for our listeners if you haven't already tried it the default shortcut for using feature search is control Q um so do give that a try um is there any specific feedback or kinds of feedback that we're looking for from our customers um as they try feature search and feature search with get up co-pilot that can help us build a better feature yeah so uh the accuracy of results is definitely one thing that we're continuing to figure out how to measure and figuring out ways to improve with a feature search on its own that is available in the product for everyone to try out but the integration with co-pilot is still an internal only feature because we're trying to iron out some of the details with the user experience and make sure we're providing uh results that are high enough quality that we're ready to share to external users so uh one way to I guess test the accuracy the AI is to try asking the chat directly if it's able to figure out things but that wouldn't have the context that we're bringing into search I'm not really sure what else we can answer with this question gotcha so in the meantime give control q a try see if it's able to provide a result for something that you're looking for and if you're not leave us a comment below or use the report a problem feature in the top right of Visual Studio so that we can keep track of scenarios that aren't working well for you um and that's also one way um to make sure that we have a direct line of communication with you as a user thing that we'll do is that if you subscribe um or if you take a look at the description in this video uh once we do have um what we demoed available for you to try we will have the instructions in the description um so Switching gears a little bit to the overall search space it's great that we've got AI to improve feature search but what about code search I know that sometimes when I'm hopping between different solutions and projects that I'm not always familiar enough with a codebase to know what keywords to use uh with the existing code search and control T um but also find um and Def find and replace experiences um so can we speak to perhaps any thing that we're thinking about with L natural language code search yeah definitely for our team specifically since we own a lot of those search features like the uh code search experience and find and files we're sort of actively thinking about ways that AI might be able to fit in and help users find the answers that they're looking for without having to do a normal keyword search because there are some scenarios that are better form as a natural language query or are sort of too complex to figure out a keyword search or even a regular expression query to find the answers that you need so we definitely want to look into that space in the future and explore what technology is available to help provide those answers and figure out the right user experience for it as well because we don't want to interfere with the existing scenarios that people have grown really accustomed to with finded files and the existing code search so we need to um sort of ideate do some more investigation work on our end to figure out what's the right way to bring in these sorts of AI powered experiences without to taking away from the existing keyword searches today gotcha so it sounds like the keyword searches are very well loved and that we are thinking about natural language code search but if we do it we have to make sure that um it doesn't disrupt the current experience while feeling in the user's flow right and what about some of the other experiences outside of control T um what's happening there Sandy yeah so I mean there are other places where natural language search is becoming a part of the visual studio experience so for example in in co-pilot chat you can you know make a direct reference to your solution uh and this works uh across languages um the context from your solution is kind of brought in when you make a query uh to co-pilot and so you're able to ask a question like like uh where's the codee that processes uh customer invoices um and you know that might that code might have an obscure name and you're not sure how to find it in control T because you're new to the codebase but by asking a a contextual question like that and by bringing in that reference to the solution uh the AI is often able to help and and not tell you not only like where it is but to give you a nice description of how this code works and what are the methods and other types that are involved right and even though when we think about code navigation we have find all implementations I feel like there's still a gap where hey I'd like to find all my synchronous methods because I'd like to refactor them to be asyn or maybe I'd like to find all asynchronous methods that don't take a cancellation token or maybe something as simple as hey where are the various authentication handlers because I want to understand the different flows of authentication token in my code base that we have started releasing internally and we're going to be asking people to dog fotage soon to help us uh build that to uh quality and make sure we're getting it to a point where it is actually ready for users to start consuming and providing feedback to as as you know we still need to do a lot of work to make sure that the model is providing accurate responses that it's utilizing the context that we're providing correctly and that the user experience for um handling responses is smooth and seamless as well so that users are actually able to act on it and not AR aren't only provided with a response that they don't know what to do with right I feel like one of my frustrations with using search engines or other forms of AI is some when it gives you an answer that's actually inaccurate you burn Cycles trying to figure out whether you are wrong or right um but in the meantime since this uses feature search would it be valuable for users to provide feedback on feature search scenario so control Q uh that doesn't give them the answers or settings that they're looking for yeah definitely we're actively working on building test cases and exploring different scenarios to make sure that the work that we are doing is actually impactful and is actually solving real customer problems so any feedback you can provide about what scenarios aren't cover today in the ID or scenarios that you would like to see covered would be really helpful for helping us determine what to do next

Add a Parameter to a .NET Core Project Template

last time on our template mini series we discussed creating a template and getting it to work in visual studio now we're going to talk about how you can continue expanding on that by adding parameters so learn more on this episode of visual studio toolbox hey everybody welcome back to another episode of visual studio toolbox i'm your host leslie richardson and i'm joined once again by senior pm syed hashimi welcome back hello great to be here thank you yep long time no see so in the last episode we talked about creating a template and then getting it to work within visual studio so what are we going to talk about today yeah that's right so today we're going to we're going to talk about you know how can i take an existing template uh and add you know one or two parameters to that and then we'll see uh how that works in visual studio as well all right well uh i'd love to just dive right in because i'm sure everyone wants templates that go beyond just the hello world type setup right there's no doubt about it all right so let's go let's go and jump right into it here so i've got my uh github open here and a reminder for how to get to this repository is ak dot ms slash net core templates let me basically just kind of show what we're going to do here so let me go into the to the my command here and inside the template.json file is where we're going to start adding these parameters so everything that i'm going to do here today is basically just right here inside this uh inside these files here i'm just going to snap it here let me open up my uh let me open up the the demo solution here that's got the templates and all that sort of stuff all right i'm going to go ahead and open up my template.json file here and here's where we're going to go ahead and start adding some parameters here and by the way the the first parameter that there's a very kind of special parameter here and that's for framework so what's the target framework for this project and uh you know it's important to add this for a number of reasons one um whenever you have parameters inside of a project and if you want those parameters to appear in visual studio uh the framework parameter always has to be uh it always has to be declared there and uh and it's also good for users just so that way they know you know what uh what target framework is going to be used here to create is going to be used uh when they create a project there so let me just go ahead and copy this content from here and then then we'll we'll kind of just describe more about this so symbols and paste that in let me uh double check my project here real quick so if i go take a look at my cs proj i can see the target framework is actually not five i'm going to take that and then we'll go ahead and fix this so instead of the choice being that core app 3.1 it should be net five and target net five and then the same for these replacements here all right so this is basically uh it's called the choice parameter that has one choice it's like a drop down with with one particular value there all right so now let's also go back and take a look at what can we do for other parameters here so in the code uh the the demo kind of project had an author name uh somewhere inside the code and then there was a replacement that was happening for that let's go and add let's go and add something like that to the code that we have here so let me just go and grab author new author name here you could say created by author name period all right so now what's going to happen is i'm going to create a parameter that users can uh specify a new value by who this has been created by right and you know what i'll do is let me take this and maybe we'll put it into there you know what i'll do is i'm actually just going to put this inside my um inside my project file as well maybe like you know as a comment or something all right there we go great all right so now what i'll do is let's go back to the template.json we're going to go ahead and add the the new parameter here author name uh the type here is parameter and you notice i'm just kind of intellisensing my way through all this stuff right yeah must be super helpful not having to remember every single piece of syntax yeah it's very difficult to remember all these things you know absolutely so i just specified a default value so um when this template gets instantiated you know either from or actually when this template gets used from.net new if that parameter is not specified this default value will be used and then that will be the default value and visual studio that gets kind of plugged in there as well cool so if you put author name anywhere else within this template it would also pick up on the schema or this uh rule would be applied to every instance that's right that's right that's right also file names as well so if you got a file that's called you know author name blah blah blah then author name part of that would get would get modified to whatever the user specified here and uh there's also clean up that happens to let's say for example uh you let's say for example the user specifies some value that can't be used in a file name the template engine will basically take care of fixing that up to where it would be a valid file name there right sorry so now i've added uh two parameters here we've got the framework parameter that's a choice and it's just got one choice for.net five we also got the author name parameter and the default is author demo and it's a text let's go ahead and reinstall we'll we're going to do uh pretty much everything here so i'll say reset templates and then i'll do a semicolon and i'm just going to copy the additional commands here so we can just do everything in kind of one go here and also with the correct order here all right what i did was i just combined those three commands with a semicolon so this will run and then this will run and then this will run all right we can see it reset the template uh and then it created the nuget package and then install the nuget package and we can see our we can see it's here so if i were to do dot net new syed con h i should see those new parameters here i see i've got author name here framework doesn't uh automatically appear here because because it's it's like um there's just one value there i think and i'm i'm really not sure why framework doesn't show up here i think it probably should this is probably a bug i'll kind of talk to the team about that so now i just reinstalled that template so i have to uh close visual studio and then reopen it so that way visual studio gets kind of you know the latest definition of that let me go into create a new project and if all goes well we should see uh in the additional info page those those parameters that we have defined here all right so now i see oh yeah that's right that's right that's right okay so yeah i see the target framework but i don't see my author name parameter so there's actually an additional step that needs to happen here let me open up that that solution so demo.solution so that's another thing that i forgot to mention here uh framework parameter you know that one's kind of special case then if you've got the framework parameter in your template.json it will just appear in visual studio itself uh but for the other parameters you know in visual studio where we're not sure what parameters should and shouldn't be shown so what we do is we have to actually put an indication in the ide.host.json and let me show you where that is in the samples here so all you have to do is basically define this simple symbol info property here and it's an array of values here so let's go ahead and do that so i'll say symbol this is not in the wow it's not in the intellisense for some reason okay so this is a another kind of bug here i definitely want to get that one fixed so i'll be bringing that up with the team of course all right so this is just an array of objects here so we give the id and you know the id here is uh author name and we'll check this real quick so id author name and then the the name is author name and then is visible to true so yeah that looks all good to me the schema declaration looks good too i'm not sure what's going on with the intel sense here but that's okay let me go ahead and close out of visual studio and then we'll go ahead and rerun that command to get it reset and then reinstalled yep author name okay so we'll say um demo author here okay and and uh we can see that there's just one choice in the target framework and i don't have the ability to uh to change that but you know if you wanted to you can definitely go through and and create a template that supports multiple target frameworks but you know it's definitely uh some additional work for those types of templates let's take a look at what we get out of here let's go ahead and just run this with ctrl f5 let me just run it here all right so we can see hello from console one created by demo author it's going to take a look at this make sure the replacements happen so we saw that the namespace and then this the string here got replaced with the project name as expected nothing see demo author has been uh replaced as expected and then inside the cs proj that string was replaced also a demo author here so that is really about it for you know how to create uh how to create parameters here uh and then get them to appear you know not only in dot net new but also in visual studio and uh one thing that i want to kind of point out that that i did point out in the previous video but let me go ahead and point it out one more time is you know let's say if you do want to support something that's more kind of complex right like if you want to create a template that supports more than one target framework or if you want to create a template that has different types of parameters right like maybe you want a checkbox instead of a text or or stuff like that the best thing to do is to go take a look at the existing asp.net core templates that have been created by microsoft here so this is the repository where all our kind of source templates appear and most of them that you'll probably be interested in are under project templates so that's definitely a great resource for how to create you know kind of a more kind of complex template uh but yeah you know we'll also kind of cover some some additional kind of info in the uh and the upcoming video as well that is exciting and yeah that's great once again it seems super easy to be able to continue expanding your template like that yeah definitely yeah exactly so it's kind of a grow up story here you know i wanted to make it very easy to kind of get started with a basic template but then also make it kind of easy to uh to grow that template as well and you know that's kind of what we're doing through these uh series of videos here yeah yeah so what's next what will be next is uh you know i think what we're going to cover next is uh kind of troubleshooting and analyzing templates here you know so what are the kind of common pitfalls and then what can we do to avoid those and and what's available so so i created a a.net core tool that that template authors can use uh to analyze their templates so i'll show how to install and use that and then uh go through just kind of the some of the uh the gotchas there yeah yeah that sounds like some good stuff to know because i don't think anyone ever gets it right on the first try you know yeah definitely definitely sweet so until then happy coding that's it thank you you