Building Blazor applications on a Mac

hello so welcome to the building fleas their applications on a Mac and definitely check out the weather app that Scott hunter just mentioned it's really cool but we have some kind of simpler stuff to get started to talk more about what blazer is and take you through an intro app how to get your feet wet and then we'll move to a more realistic app and make some changes there as well yeah oh and I'm Kendra havens I'm a program manager on dotnet and Visual Studio and I'm Dan Roth I'm a program manager on the asp.net team cool okay so Dan what is blazer is boy so much talk about it and so much excitement so Blazers a modern web framework for building client-side web apps using c-sharp and net instead of JavaScript it's cross-platform it's open source and it's based only on open web standards now we've had ways to build web apps with dotnet for a long time a traditional don''t web app you take your dotnet code and you put it on the server and then the browser sends requests to your server and it responds with dynamically generated HTML or Jason that's pretty typical dotnet web application but if you wanted to do anything that ran in the browser well that meant you had to use some Java scripts like using one of whatever the popular JavaScript framework of the day is and that's cool I mean Java scripts great I mean yeah it's great you kind of have to switch context though and that can kind of slow you down so they can stay in dotnet code that's kind of convenient if you can use the same language the same frameworks the same tools same build system across your entire application that's what blazer is all abouts enabling you to build full stack web apps using Donette and c-sharp that's what we've been working on sweet okay so Blazer you build client-side web apps using just net and c-sharp code it also comes with a component model so you can build reusable web UI components you can package them up as NuGet packages and libraries and share them with your friends it also enables you to use the same code on both sides of the wire if you have some common logic like validation logic or data models that you want to use both on the client and on the server you can do that with Blaser it's full stack web development with net at the same time if you have existing JavaScript code or you want right so we're sharing dotnet code so even like working and like f-sharp or Visual Basic if I have libraries that I want to call from blazer yeah you can do that it's just normal donít assemblies that you're calling is your into in your blazer application absolutely and also if you have JavaScript you can still use that as well you can call into existing JavaScript libraries from your blazer code and even call back from the JavaScript code back into net through JavaScript Interop you can use that to also leverage all of the native functionality of the browser okay so if I'm not-like or if I'm slowly migrating my app to blazer alright trying it out in some section I can still use javascript elicit don't have to throw your JavaScript away you can just you reuse it in your existing blazer application how does that work is that possible I know it seems like magic well there's two ways that blazer apps function the first mode we call blazer server the way blazer server works is that we set up a real time connection with the browser we use signal R to do that it's basically a WebSocket connection Brady has a talk on all on signal once we've got that real time connection with the browser we run your blazer UI components on the server any UI events that happen in the browser we send them over to the server dispatch them to your components the components run they render and then blaze are very cleverly figures out just the parts of your UI that have changed and see arises that diff back down to the browser to then be rendered and updated so it's a kind of a thin client model you're you're still running your UI components on the server but you get that rich interactive field that you would expect from like a JavaScript based single page application ok and in server all of my c-sharp is compiling on the server still not at all inside the browser all the compiled compilation happens at Build time but your build code is gonna be running on the server on dotnet core it's not actually going to execute in the browser right but you still get the rich interactivity of being able to handle all the UI events and update the Dom seamlessly without having to do a full browser refresh but I have heard about things you started to do that you know I are not compiling but executing executing that's right in the browser with blazer webassembly your web assembly what we do is we we can download with your app a full dotnet runtime that's implemented in web assembly now web assembly is an open web standard it's basically a bytecode for the web if you can compile your code to web assembly that code can now run in any browser on any platform at near native speeds native as in like browser natives well pretty close to the wire like it's almost like an assembly language that's been standardized for the web so it's pretty fast super fast and it's so we built a dotnet runtime in web assembly that you can bring with your app it's small and compact and then you just download normal dotnet assemblies into the browser and execute them using that runtime the same come you I components the blazer components in this case they're running in the browser itself they do they handle UI events they calculate the diff in process in the browser client side so I see our little slide here says preview that's right so please our web assembly is in previews lays our web assemblies in preview laser server however is ready to go for production I want to use which one here server versus web assembly so Blaser server shipped with dotnet core 300k also shipped with three one LTS LTS is a long-term support release so Blaser server is ready for production use today you can put it in your websites put it out in production it's fully supported with a long-term support life cycle blazer web assemblies still in preview isn't quite done yet but we're working real hard on it we expect to have that finished up in just a few months later this this year how do you pick between the two well there's there's some some trade-offs with blazer server the benefits there is well your your well first of all its production yeah if you can actually use it right now but if your code is also running on a full dotnet core runtime so it's super fast on the server and it treats the client as a basically a thin client it has less demands on what the the browser or the client device needs to do so for example if you want to support older browsers like IE 11 you can do that with the blazer server application because there's no requirement to support web assembly blazer web assembly on the other hand gives you that option of pushing and offloading load from your server onto the client because it's a true client-side application it can support things like offline scenarios that's still in preview so it's not ready for use today but it will open up some really cool scenarios in the future oh my gosh I'm so pumped now regardless of which model you pick though like don't don't get too hung up I'm picking one versus the other because the component model is the same you write your components once and then you can use them in either hosting models so you could start out for example with the blazer server app and if you want to switch the Blazer web assembly in the future that's pretty trivial maybe when it goes to production we can switch our components over if that's what you wanted a oh my gosh I'm so pumped shall we get let's get started ok you say laser I know you can get started by going to blazer dotnet so we have a really simple URL to remember just Blazer dotnet and it'll take you to our main blazer and I guess landing page and we can go to this get started documentation and this will walk you through everything that you need to do to get started with blazer all the stuff you need to install and if you're on a Mac like I see that you are there's a tab there that gives us to do for Mac that has all the steps for getting Visual Studio for Mac set up with blazer so it's already installed dotnet core 3.1 and actually since you're here I won't read this I'll just I'll just launch Visual Studio and you can walk me through since you've already done it you've already installed Visual Studio from that you have dotnet core 3-1 on your machine which comes with Visual Studio for Mac so you're all ready to actually to go with blazer development right now alright so since I got 3.1 installed I already have a blazer server template and if I didn't see that there I could actually just find that in just the regular app templates blazer server under dotnet core that's right oh ok didn't see it's already targeting 3.1 they have a little authentication tab so on the latest preview of Visual Studio for Mac we they now have support for the authentication options for blazer server so you can create blazer server apps that support auth using asp.net core identity cool all right I'll go ahead and initialize one I called it blazer at seven that's a one seven just my favorite number it's nothing to do with the precise number of times we practice nothing at all just lucky it is so this will create you a new blazer server application it's all ready to go a blazer surfer app is just an asp.net core application with a couple of additional things wired up to support blazer server why don't we go ahead and just run it so we can see what the application looks like yeah that sounds good so the visual studio can now build your application get it running you can debug you have full support for blazer development within Visual Studio for Mac once this gets building and actually running we should see a sort of you know standard single page app style style UI and give it a second it's coming slowly will blame blame that on the browser yes you can tap around you have different tabs for different different pages this is all being handled through client-side routing on the home page you have some simple static markup nothing too fancy going on there on the counter page though you have a a button that you can click and as you click the count goes up there's no page refresh happening here the Dom is just getting fluidly updated normally that would require some JavaScript to do but that's all happening using c-sharp and on that neat not to see how that's working yeah and then this is fetch data page is basically a dynamically generated table of weather forecast data so let's take a look at that counter page so that's implemented in this counter dot raiser file and you can see it's pretty simple it's mostly just H standard HTML markup at the top it says that this is a routable component so app page says that we browse to slash counter we should end up here so if you go to the browser you should see in the URL at the counter tab yep slash counter that's where we're at and then you have some normal HTML markup we've got a button that we can click and there's that onclick attribute normally that standard HTML onclick attribute would have some JavaScript in it but here we're using razor syntax a little at sign to say no no I would like to use some c-sharp we've got c-sharp code here go down to that and it's pointing to your c-sharp method increment count that's incrementing the current count field which then gets rear-ended on the page now to show that that really is running c-sharp code whenever every time we click that button set that breakpoint let's see we can hit a breakpoint counter page yeah there you go we're in our C sharp method neat writing client-side web UI using using blazer let's see what else is going on here well the nice thing about these components like all these don't razor files they they get turned into normal dotnet classes as part of the build and now you can reuse them like you if you wanted to have another counter component someplace in your app you can do that by just adding that component wherever you'd like it to be so for example if we wanted to add a counter component to the home page like let's say we wanted two counters one on the counter tab bone on the home page if you go back to the let's go to the index dot razor file that's the home page for the application okay so here it just has markup there's nothing really interesting going on but let's start typing bracket counter angle bracket counter on here so yeah and you can see you get awesome intellisense in Visual Studio for mac and now it's showing up purple that's that's signaling to you that this is a component yeah you know what a blazer component if we rerun the application is purple or brown I think I might be a little bit color dot net purple everyone you learn new things about how you perceive the world yeah okay so I added the counter component and I'm going to go ahead and f5 again and so now when I go to the index yeah it has a completely new component a part of it so I didn't need to like copy in the counter code or anything it's just inheriting that component a reusable chunk of UI you can click on it it works just like the other one did and if you go to the other counter page you see it has a an independent count that's separate from from counter on the home page you can also pass data into components like components can have parameters to define a parameter on the on a component let's let's go to counter and make it configurable you parameterised how much it counts with each click so to do that you just add a property and the property can be made a component parameter by attributed it with the parameter attribute gotcha I have a little tip I want to show in here Visual Studio for Mac does indeed have snippets so if i just type prop and tap Todd OH it automatically loads in my property okay save some typing what st. yeah and let's call it increment amount I think would be good and let's default the value of that property to just one yeah give it give it the parameter attribute to that's that's cool that's what's really gonna make this a parameter for the component that we can pass in so by default it'll be one and now in the increment count method instead of incrementing by one each time doing a standard increment will increment by increment amounts of current count plus command amount perfect you get lovely intellisense for your c-sharp code inside your razor for local variables now go back to the index page and let's use that parameter let's pass in an argument so if we go to index now just add some space after they are and you can see incremental mount shows up wait way you set parameters and blazars just using standard attributes in tax and now the counter on the home page or in commit by 10 whereas the one on the counter tab should just default to the standard increment domenica the one i just kind of chose 10 I hope that's a good number that is a great number 10 times better that was before and we should see okay let's check this yes and now it's incrementing by increments of 10 whereas the counter on the the counter tab should still be doing by 1 because we didn't pass of past any parameter value the houses working like we said this was all based on that magic real time pipe we can actually see that happening if you yeah inspect the in the browser dev tools let's look at the network and if do a do a full reload so we can see all the stuff that's being downloaded holding a shift key or something there we go so you can see that this app's pretty thin it's not downloading very much stuff it's just a little over 400 kilobytes of download and all the magic is really happening in that first request that underscore Blaser WebSocket request yeah you see when you look at the messages you can see there's all these binary messages flying on that WebSocket connection yeah it's updating as you speak if you clear it lets clear them all out and then go click on the button on the sorry there's lots of try the yeah a lot of work yeah then click clear I'll click me thank me but nothing you clever every time you click we send the UI event to the server your counter component gets the message it rear Enders itself and then the Dom updates get sent back to the client that's what's happening here this is how that's how this app is working this is ablaze every base that cool so that's that's counter now I know if you go back to the code real quick on the counter dot razor file some people really like you notice that there was that act code block at the bottom yeah right there so at code that's basically a chunk of code that's going to get added to the generated class for this counter dot razor file it's like a bunch of class members and some people really like that that code lives in the same file because all your UI codes in one place other people would prefer to have that in like a normal C sharp file like in a code behind file which is totally cool you can do that in blazer as well what you can do is add a let's add a c-sharp file to the pages directory let's call it counter crazier dot CS that's sort of this the standard naming convention for a code behind file look good yeah it's good what it automatically put it underneath I put a whole bunch of they got Razer license information I was messing with my header demo later today for the productivity talk a delete that we won't you don't get to see this yet you have to wait till this afternoon I'll talk about how to put header and we don't automatically license like all your c-sharp files I'll get at it no no no that would be bananas right specifically chose that mighty license cool well we have a counter a class now now this matches the name of the class that would be generated from counter dot razor so let's make it a partial class ok grab you grab that code go ahead and grab that that's fine go ahead and copy that out of there and we're gonna move it into here we don't need the constructor so we can just copy over the constructor but make the class a partial class so it'll be combined with the generated class for the component and then you probably need to resolve some namespaces right and I can do that with alt enter so that opens up the little light bulb that I get so I can automatically add using beautiful so I guess all the normal C sharp productivity enhancements and features now are available to you in this I'm getting another little suggestion so this actually might have annoyed some people and I did this originally use a compound assignment it was all part of my plan yeah so vo Cermak gave me the suggestion that I can simplify that it does look better yeah well so this should be the same as what you had before like there's no difference in their code it's just that we've refactored they run it to prove it yeah go ahead and run it to make sure it's still working that's pretty nice like the C sharp so I've got a lot of nice features my gosh all of the little things that's what I work on that's nice t-shirt little things productivity that looks like it was before so she should still have like you know income increment looks good what other c-sharp stuff can you can you do and in yester Mac does it support all the the visual studio it does so right now we actually share the same editor so basically everything that we've added actually latest and like Visual Studio and Windows 16.5 preview 2 I believe is what we're on all of those are actually already piped into Visual Studio for Mac I can demo those later today with Mikaela there's a whole talk on that yeah there's like a shared editor so it's all piped in okay yeah so like like adding an overload like is that is that oh yeah so if I do alt enter um our keyboard shortcut to open the little tooltips I can't override sorry terminologies do something like as you can see all of the component life cycle events in this UI like the on initialize on parameter set these are all standard events in the life cycle of a component so yeah if you wanted to like run some code when this counter component is first and after it's initialized you would override on initialize or on initialize async if you have some async logic I think the fetch data component actually does that so this is this is the component that was generating that dynamic table of weather forecast data and if you look at the bottom all the way at the bottom there's a yeah there so you see uninitialized async there's exam distracted by how cool it was to have a c-sharp for each blue printing out your people we're cool okay continue so yeah there's uninitialized async that's that's when this component gets initialized you see it's using that weather forecasts service to get all the weather forecast data where's that forecast service coming from if you scroll all the way back up to the top there's this @inject directive what @inject is doing is it is a way to do dependency injection into your components you can get services that you've configured here we have a weather forecast service that we're injecting into this particular component and then below it's just normal razor syntax lucky where you were saying it's mostly standard HTML markup but intermingled with C sharp logic statement yeah if statements for each loops like we're just looping over each weather forecast instance to generate the rows in that table that's how that weather forecast table is being generated super cool so we started sort of to get more into realistic scenarios this was just the file new template is the basics they're like a level 2 app that we can use and show us short so I've been working on a recipe app in Glazer to like you know manage the list of recipes and do recipe ratings and those types of things why don't you pull that up yeah that's my repo you give you guys one if you want to download it and see my code like ok got it best for you recipes best for you recipe all right so let's see if I can open this up now usually I've been developing this on window is actually on Visual Studio but it opens great on Visual Studio for Mac builds runs debugs all those things just work as expected once this opens up why don't we just go ahead and run it first just to see what the app is capable of I'm gonna wait till I load my workspace documents perfect and maybe restore my new get packages that's probably always kind of always that restore your new get packages before you build and run its lifestyle choice okay so I'll try to it's saving might have gotten a weird thing well we're yeah we for that we can go ahead and look at the home page of these let's sit let's open on index dot dot razor so we can see a little bit what the when we see the app we'll have an idea of what the code looks like this is the home page for the application and you can see at the top it's mostly just there's a search box and then a little bit of an if else then just to see if the recipes have loaded and once they've loaded it's just generating an unordered list with the list item for each recipe that it loads from a recipe store and again we're using dependency injection to get a recipe store service and to initialize the the list of recipes so that's all that's doing that search box at the top is a custom component for handling the searches once you scroll back up to the top and we can look at that real quick yeah so you can see that on the search box we're passing in normal HTML attributes to like specify a placeholder but we're also able to specify a delegate for when a search has occurred so that we can hook into that and actually then run the query against our store so looks like the app is now up and running yeah so here's that home page that they actually read beautiful nice nice images design is not my forte you can put down recipes you can see details about the recipes some basic functionality you sort of search bar search for chocolate stuff yeah now notice that as you type the search is updating like you're not hitting enter as your house your typing is just looking at the current state of the search query and running a search but it's not doing it on every single keystroke it's like a little visible pause then it waits to see if you're done typing and then it does a search and that's nice because it's throttling all the searches to your database you're not just hammering the database with every single keystroke so it's interesting to see how that's implemented in Blaser if we look at the search box component that's where all that logic is encapsulated so search box that razor there's off now this is that component that just implements the search text box and it's handling all that debouncing logic where every time you search it'll wait a little bit until the even two you're done typing so mostly you can see that this is just an input like Scrolls we'll see the mark up at the top is an input and that's not about timers yeah and it's bringing in the timer's namespace the attributes attributes attribute there what that's doing is allowing you to grab any additional attributes that are passed on the component and sort of splatting them wherever you want them to be that's how that place total holder attribute is getting on to the input and then that app bind is binding the value of the put to the search query text search group property so as you type blazer we'll see that the text has changed and then set that text on to the search query property which I think you have below yeah right there it's actually even specified as a parameter so if you want to pass it in you can do that and we know that it's gonna do that on every single keystroke because after at bind back at the top at the end of the the input all the way the end see that at bind : event that's specifying which event we want to use for the bind like by default we will do it on change here we're saying no no no I want it on every single input every time you type please update the value of the search query uh property and then now let's look at search query cuz there's some of that's where the little bit of cleverness is happening search query when it's set like when the value gets bound it just starts a normal net timer and this is just you know like you saw up above system timers or whatever whoever the name space is it's a normal dotnet timer from the base class library where it will stop the timer if it's already been started and start a new one and if that timer completes then you know there was a pause without any keystrokes hit and that's when it'll actually call back into the the search query changed then call back that was passed in so I thought that debounce parameter there about 300 milliseconds that's that's the the amount of time we want to wait and you can see in an on initialize there's that component life cycle event again that's where we're moving up the timer setting up the defense time frame and wiring up the event handler for when the timer actually completes so all the event handler does is call into your delegate that you passed in and then you get your search results that's the balancing logic and you know 50 some odd lines of code all done in c-sharp no JavaScript required that's great and so should we go back to slides are we short on time I think we want to make one change oh we got five minutes okay let's make a change to this out okay so um if you go back to the yeah let's go back to the app and look at the reload oh right I stopped yeah so the we can click on the recipe that brings you to a recipe details page okay and most of that's just markup and there's even a form where you can do star ratings but there's also these tags and currently if you click on a tag they are links but they don't they don't go Oh like they don't think okay so what we want to do it is it is going to a new URL it's going to slash tag and then the name of the tag that we that you clicked on let's wire that up so that we actually end up on the search page but with the correct list yeah of recipes okay so go back to index dot razor what we want to do is grab that value out of the URI so in order to do that we're gonna add another route so add another app page directive and make a slash tag and then slash now if that's in curly braces cuz this is the value we want to grab out and we'll call the value that we're gonna grab out tag okay now to get that value we're gonna bind it to as a parameter on this component so add a new parameter remember we added a property before to create a parameter do in the code block let's create a new property so property anywhere thank you drink call it tag okay so and then put the parameter attribute on it so now what the Blazer will do is it will take the value from the route and set it on that property so now we can just pass it in to the get recipes yeah it's no it's not specified yet then we probably just will pass empty that will give us all the recipes when the attack is initialized one more thing is up above in the in the search box let's make sure we populate the search box with the tag if there's one there so set the search query parameter a little space in there [Music] sign and tag oh so we're actually getting into the c-sharp property there you go that's all you so now it should populate the tag so go ahead and run it okay let me start it and that should populate the search box with the tag as you click on tags that will filter the search to just that tag whenever you click on a tag like on the page and it should be just that easy okay so click on a recipe let's go down to the tags let's know fruit there's only one food that that works and it populated it you just added a feature to my heart right you're welcome now top back to slides real quick sure so that was getting started as blazer and some blazer in action um we know the number one question on a lot of people might people's minds if we have time for Q&A and stuff as um what is an expert blazer webassembly when wouldn't cute when we get it when will it be out of preview yeah so so this was all blazer server work that we were doing with Visual Studio for Mac Visual Studio for Mac fully supports blaze a server which is great blazer web assembly is still in preview it hasn't shipped yet it is the the focus of the blazer team right now and we are planning to release it in May of this year as a supported release and that will include full support in Visual Studio for Mac the initial release of blaze our web assembly will be based on dotnet course redone in fact we plan to add the template in an update to the dotnet core 3.1 SDK it will be a current release not on LTS release so it's not gonna inherit the LTS status of the rest of dotnet core 3.1 it is the first release of blaze or webassembly we want to give it a little bake time before we bless it as an LTS release to try and make that clear we're going to update the versions of the packages to be 3.2 we've actually already started doing that so the latest previews our version does 3.2 after may blazer web assembly will become part of down to 5 and will then be just part of the normal dotnet update release cycle so you just need to remember to keep clicking YES on that update prompt keep updating and look in May for the the released version of a blazar webassembly got it so in summary we talked a little bit about being productive in blazer on vs or mac blazer server is what we show today it is in production you can go and use it with dotnet core 3.1 and get excited for May 20 2002 a time question in turquoise that's exactly I'm glad I actually start with a statement somebody said I won't Don Ross Visual Studio you know what we have some exciting t-shirts for this event and we'll be giving them out as part of our party so make sure you stick around for until the end of the day so notice but more and we actually have some questions as well so the first one is habeas from a is there any way I can explore on blazer webassembly in Visual Studio Mac that we were just talking about that you can actually open a blazer webassembly app in vs for mac today you'll need to do is the a smith core hosted version of the template the standalone version of the blazer webassembly template isn't working just yet mb/s for mac but it will by the time we ship blazer webassembly in the May time frame you don't have the ability to create laser webassembly apps from vs for Mac just yet but that is coming as well so you can create apps from the command line as long as they're hosted in a nascent core app and run and develop your components and all that the rest of the functionality will be cut will be coming later as we get closer to the may release that's great well there you have it and next one is just close this one down is there performance benefit using blazer server over a blazer webassembly yeah so blazer server runs on a full dotnet core runtime your components are running on the server so you get all the performance benefits of dotnet core which is honestly one of the fastest stacks on the planet it really treats the the client machine also as a thin client so it has very little requirements on the client side of the of the application blazer webassembly runs on an isle interpreter based runtime today so if you have components that are doing really computationally intensive stuff it can drag a little bit when you're running that type of code in the browser so you might want to offload that to like a web api somehow on to your server so you're running on the folder core runtime perfect I think we have time for just one more question so yep question is is there a benefit using blazer over angular react or some other spa oh yeah I mean you see sharp yeah context you get all that syntactic goodness like productivity features in Visual Studio yeah it's a full stack solution for web app development with net and c-sharp I think also there's just a lot of really nice simplicity that comes from the the blazer programming model and tooling like it's really easy to get started file new project and you're up and running in less than five minutes so that can be really nice that said if you if you love angular and react in view and you want to continue to use those for your front-end development you can absolutely still do that and still use net core for your back in api's G RPC services signal our hubs and so forth great well thank you very much for your time it was fantastic to hear about all the Blazer cool myths out there and I'm looking forward to the 2020 build announcements that we're probably gonna have

Building a Windows 10 IoT + Azure Cloud Apps

[Music] hi welcome Visual Studio toolbox I'm your host Robert Greene and joining me today is Daniel Jacobson hey Daniel hey how's it going good how are you I'm good I'm good Daniel's a program manager on the visual studio team that was correct and you are an expert in building Windows 10 apps sure yeah absolutely um you this app that you showed it net conf on channel 9 you also showed it at one of our internal technical events that I thought was just so cool that I wanted you to come on the show and talk to us about building window 10 apps using the universal Windows platform in the context of this actual application because I think it really kind of drives home why people should be interested in this and what the possibilities are it's not just about the surface it's not just about the phone so you know the universal Windows platform runs anywhere Windows runs and that opens up some really interesting possibilities such as sure yep you're going to show yeah so the real beauty of the universal Windows platform is being able to write one application and run it on any one of the Windows 10 device families which right now includes IOT as you see here does top mobile surface hub xbox hololens you name it so one app runs everywhere it's really things that no doubt are being worked on that we're not allowed to know about yes probably who knows so the inspiration for this application actually came to me when a friend approached me saying that they were interested in starting their own small business in Seattle a coffee shop if you will of course coffee Seattle make sense yes so what they wanted to do to optimize their chances of success being a new business was be data-driven be smart about the way they run their application so I thought this is a perfect use case for Microsoft technology I could build universal Windows platform applications that target multiple devices Usenet technology and Azure and pretty much build an end-to-end solution that would help them out so what I built here is a device that will help determine when someone enters or exits their store then send an event to an azure hosted asp.net web application which will be listening for those events then store it in a database and then when they want to review that data over time they can use a desktop or mobile application to actually look at graphs see how is my store trending how many people are in my store at different times in the day what's the busiest day of the week what's the busiest time of the day that way they know when do I need a higher help what days what times how successful are the promotions that they're running do they see a spike in traffic when they run promotions etc oh so that was the idea behind this and as you can see here the app design has quite a decent plethora of parts but it's actually a relatively simple application when you get into the nitty-gritty details for all of these apps so the IOT device really only has one class of code that actually does anything same with the Azure hosted web API it's really just a single class that does everything that's like 200 lines of code and then the client app is a little bit more involved we're not going to go through all the code but I'll point you to the github link after this so you can take a look at it yourself so the way this is going to work is on the IOT device which is what we'll spend some of our time it's it's it's more unique in modern application development era phones have been around for quite a while now same with desktops but these are up-and-coming so what this is going to do is based off of these devices I'm going to fire an entry or an exit event saying someone has entered my store or someone has exited in my store and when that's done it'll send an HTTP request sorry an HTTP request to my Azure hosted Web API listening for a post response which will then write the event information to a database including what type of event is it and at what time did it occur okay and then it will actually send a push notification to the device running the application so one example here is Sam in the back um it's a small shop I'm doing inventory work or I'm doing bookkeeping I'm the only one working at the time right someone walks into my store no one's there to greet them well I'll get a notification instantly saying hey someone has entered your store you better better go take their order so it's another neat way to reduce the number of people I need working yeah this is the modern Bell there you go who needs a bell when you have active smart phones and then finally when the push notification is sent the client at the data visualization app that has some graphs it's notified - hey update the number of people in the store and then it refreshes the data cool yeah so I guess we'll just go ahead and get started yeah so in Visual Studio I have a solution that I've already pre-populated with some of the infirm I'm not going to spend too much time on the asp.net application but as I mentioned this is hosted on github and only cut at the end but what I want to show with the asp.net application is how easy Visual Studio makes it to publish it to a live running web well web app so here's my API I'll look at the one class that I've actually modified most of this is from the template but here's my single controller of interest which has some metadata about my Azure services these are just keys and other private information that's all office gated from the code through a resource file then I have two methods of importance git which is just going to get a list of all the documents and post which is going to post a new document so it actually takes in the parameter my custom event from my portable class library so this is actually shared code and then it writes that to the database and that's pretty much it so two classes you can find those online but what is Keys hard-coded in the code somewhere yes so keys is in my portable class library and a resources file okay I don't want to show that to everyone cuz I can mess with my service here yeah so what I'll go ahead and do is show you how easy it is to publish so if I right-click on the application itself and I have the web tools installed all I have to do is find publish right here I sign in with my credentials which I've already done so it should cache it I can see my profile so I would select Microsoft Azure app service yep it would populate with all of my services select the servers okay and if this was a production environment I would just hit publish that's it but since I want to actually debug this later I'm going to go ahead and change the settings to make sure it is debug in ECP which I did that before so I'll publish and that's it so now this is actually going to send this code to my live running asp.net web application hosted in Azure yep so just to test that as soon as the finished pub finishes publishing we'll debug it real quick make sure this is working before we get into the uwp applications and while this is going the reason I want to focus on this one first is because as you can see both the IOT device and the data visualization app have dependencies on the azure app right so the IOT device is reading to the web app the client app is up sorry got that backwards the IOT device is writing you have a document through through this cloud service and then the client app for mobile and PC that's visual as I visualizing the data who's reading from it okay so as you're a speed on that first and then you WP publish succeeded I can see now in my browser here is my running application so what I want to do now is actually attach the debugger to that and make sure it works so I'm going to press a breakpoint on the post event go to cloud Explorer another tool within Visual Studio I don't have to leave the IDE navigate to my web apps shop analytic service right click attach debugger and then that'll try to find the live running process to attach to in my Asscher service so it will launch the service and if I return to visual studio once I see the breakpoint is ready to go I should actually be able to send a post method to my live service and hit this breakpoint make sure it's working so I'm using a another application called postman which is just a simple web debugging tool that enables you to send HTTP requests to any website you want and what I have here is a pre filled out body so I'm going to send an event type event to my live service that is event type of true which means someone has entered the store and I'm going to send it for today which is September 1st it is so let me go ahead and change this oh if it will let me that's interesting post body oh that's why I was in the wrong one September 1st and this is 8 p.m. global time so 20 o'clock ok I will hit Send I should hit the breakpoint I hit the breakpoint as expected it is loading the information so now I can actually take a look at the variables that are being passed in so a new event crew a fight time you'll see 9 or September first 20 o'clock I can continue to execute this code and now I should actually see a push notification on my device so this device had been previously registered for the push notification service um so even though the app isn't running the data visualization app isn't running because I had run it in the past I'll get the push notification so what that means is if I've got my Windows 10 phone and I don't have the app up and running but someone enters my store I'll still know that they're entering my store right which is pretty cool so I'm pretty confident that's working now I'm not going to worry too much more about the asp.net application let's actually build the uwp application so one thing I'm going to do is actually continue to attach the debugger to this running service and I'm going to build the IOT app in another instance of Visual Studio you'll see why in just a bit so let me open another version of Visual Studio with the solution open there we go and I'm going to actually start from scratch I'm going to add a new project to the solution I don't have an IOT client project tip so I'm going to right click on the solution I don't need cloud Explorer since this is a client up I don't need to worry about this controller anymore now we'll minimize that so let's go ahead and add new project and I will just select blank Universal Windows platform application and I will call IOT client so as soon as I create the project it lets me select my target platform version in this case I'm going to choose the anniversary edition I like to play with the latest and greatest stuff go ahead okay so to get started with this app and for those of you who have worked on IOT devices before what you'll recognize is a lot of the code is just boilerplate code that essentially defines where all these devices are registered on the IOT device and the breadboard so I can say this device is here I want to communicate with this device is it an input or an output device it's really not any logic it's just kind of getting getting everything set up so I'm not going to write that code that codes boring I do want to write the logic so what I'm going to do now is navigate to the main page of my application woops that's the wrong one and I'm going to include that code so let me go ahead and find that code does stops it bits and I'll walk you through what this code means it's just not as exciting to write it cool so let's start at the top I see a lot of squiggly lines that's because I'm missing some references so to start with keys as I mentioned before is in my portable class library so I can add a reference to the same portable class library that's included in my asp.net application I'm sharing resources across both apps the next thing I will see is missing is GPIO pin I can hit control dot to add a reference to the class if I go further HTTP client is missing the references I'm just going to resolve all my references so GPA what someone opere keys actually have to add a reference to the portable class library so GPIO status is actually not something I can resolve immediately what this actually is is a representing a text block on the UI so because it is just a Windows 10 application in order to help me debug that I'm doing everything correctly I just added a text block to the UI of it okay so let me go ahead and navigate to the zamel and for those of you familiar with sam'l development this is just as a molap I will add a textblock X colon name whoops equals GPIO status beat the designer but that's alright horizontal I'm an equal center vertical alignment Center and then I'm gonna make it really big so we can actually see it 72 now I should be able to reload the designer there we go now let's since I closed my tag so I'm not going to see anything here yet but when we run run the application you'll see that so if we return to the code behind that results down no squigglies and all that remains is to event handlers and I'll explain those in just a second but that's what makes a IOT development with Universal Windows platform really easy Wow so I'm just going to create empty methods to handle those for now okay and let's take a look at what this is doing and then just deploy it and see what happens so if I go to the right top the first thing I'm doing is defining the base URI for my azure Web API and this is essentially going to be used to send the post message so I'm going to construct my HTTP client object with this as my base URI and then when I send the post message it knows ok target the right live running servers right these pins if we take a look at the schematic for this device are just actually mapped to pins on the Raspberry Pi so you can find diagrams for the the specific bread where the specific IOT device that you're targeting that'll have a pin mapping for all of the digital devices that you'll be connecting to it ok so if I take a look back to the code 5 and 17 refer to the pins for my sensors so that's what we have here we so that's what we have here yup connected to a breadboard on metal board and then the breadboard has wires connecting to the Raspberry Pi device and correlating to those pins that I specified so the black wire here is ground red is hot so it's the voltage and then the four colors are just different digital devices connected to the pins that enable me to control them through the app okay yep so for somebody who has is a good c-sharp coders done built apps it hasn't really played with the Raspberry Pi know how long look how long does it take to get to the point where you could wire up sensors like this like this yeah probably a day maybe thank you okay yep so I built this in an afternoon and the only previous IOT experience I had was Arduino based it was not uwp it's you don't have to be electrical engineer done soldering I mean so so though in the past right the commonality that all of these devices have is that they're all just digital devices okay they're all very they'll behave similarly so even though I'm using an IR beam sensor here it behaves just as a button would I mean it's going to have a value of 0 or 1 depending on the state of the device so if it's a button and it's pressed it'll have a different value than when it's depressed just like when this is covered versus when it's not covered it'll have a different value okay and is there good introductory material on let's actually show you where that is so if I just search for Windows 10 IOT core on Bing what I will find is tons of documentation so if I just select learn about Windows 10 IOT core there's really great information about what it's like to develop for these boards including examples that people have built so an air-hockey robot and then some tutorials as well so if I click on documentation and I look at the samples hello blinky is essentially the hello world of IOT development so I could select c-sharp C++ does it show you and it shows you exactly how to buy everything ok so to get started it tells you exactly the the hardware you need how you hook it up together and here's the pin mapping that I was referring to before so this is actually the same board that I have and as you can see here GPIO 2 and GPIO 3 are just the numerical pins that I'm referring to here so 5 17 4 and 6 are the ones I'm using so 17 5 6 and then 4 I think is here so all in the same thing yep so it's not too difficult so that's what those are referring to those are just the constant integers that define where I'm plugging those into my raspberry now these are the objects that are actually going to reset represent those pins and then these enter and exit boolean's are going to be used for the actual logic of the code so we'll explain that in detail as I'm writing the code and then client is my HTTP client that is going to be used to send my HTTP web request to my live running service finally the last thing you need to do is actually initialize the GPIO controlling so you'll pretty much see this method in any of the tutorials you go through for an IOT device development but once you have all of the GPIO pins determined first you make sure that a controller exists on the device and what this means is if I were to deploy this application to a desktop or a phone it would hit this line hit this if statement and GPIO would in fact be no there's no GPIO controller on my surface yeah so then it would return out of the method yeah at least nothing that I can control um so then I would send there is no GPIO controller on this device and my application wouldn't do anything that's it so nothing would crash I'm only using the universal API set right meaning I don't have any platform specific API is everything is just common and then I'm actually opening the pins for those four sensors so as you can see GPIO pin I'm defining the sensors and opening the pins okay so after that I need to say is it an output device sir is it an input device so these LEDs are output devices I cannot interact with them the values never going to change I can write a value to them I can say it's on or off but I can't read anything from it so when I set the drive mode for the lights I set it to output and then the sensors are the opposite they're all they're going to give me a value back when I cover it or uncover it so those are input devices just like a button would be okay um after that the debounce timeout is really just to filter out any noise from the IR beam sensors so for some reason the the value changed for 50 milliseconds really really quick like unregister below something that was undesirable it wouldn't actually cause anything to happen after that I just create value changed event handlers and that's really what IOT programming is all about when something happens the value is going to change I have some sensor that changes the value and then I respond to it so what I want to do is I want to place breakpoints on both of these and deploy my application to this device so first I'm going to make sure it's my startup project set a startup project I'm going to select a debug and I'm going to choose armed because I'm debugging to an armed device Raspberry Pi 2 has an ARM processor would finally - electro machine it's hooked up to the network and I'm going to copy the IP address from before and to play so while this is deploying the very first deployment to a Raspberry Pi takes a little bit but it's a $30 computer so you can only expect so much after the radio first f5 oops I it speeds up drastically and you know you'll observe in just a moment so while this is deploying I do want to explain the logic of how this device is actually going to work so if we go back to the PowerPoint I can see this is kind of the overhead view of my device so if I were looking at it like this yeah this is what I would be seeing okay and I can see I've got my customer ready to enter the door it's my ninja cat riding a t-rex with the Microsoft flag and I can see both of the boolean values are going to start at false right meaning they have not crossed any of the sensors so as soon as my customer enters through the first sensor so they've gone half way through the door yeah I change the value for my enter device I know the value of this sensor has changed I can respond to it and I can change the boolean value that I'm tracking as it continues through the door I can now see both of those boolean values have been set to true or in this case both of the sensors have been broken right and I know an entry event has occurred because entry sensor was fired off first and then exit sensor right now if the reverse were true if it went through the exit sensor first and then the entry sensor that's an exit event someone's walking straw just went like that and messed with you so someone so actually if you look at the github repository and someone goes like this and messes with you so that it messes it up I actually have a timeout after a couple seconds and nothing happens it resets the state of the device another example could be say a fly goes by and it messes it up right yeah so so there are ways to counteract that and if you just know some simple c-sharp programming it's pretty easy to do okay but for the purposes of this we don't worry about that case so as soon as the event is triggered though I'm going to immediately reset the state of the device so that when the next customer comes through it's ready to go it needs to respond instantly so let's see so the application is almost done f fiving should take just a couple of second it's kind of like using an emulator takes time the first time yeah I understand right get everything over there you're good flanked to you a device sits and then this app along with dotnet runtime the subset of it yep in all likelihood that runs on this device right dotnet saw dot nets already on here right the runtime so you're copying the literally you putting on it just the executable so in this case it's everything that comes with the application so all of the references all of my new get packages so the it's interesting you mentioned the DA at runtime so with the universal Windows platform you can actually package your dotnet runtime as a part of your application okay so since this is a debug application it's not a store application it's actually deploying but net basically okay however if I were submitting an application to the store for example that shares the same exact net runtime as all of the other apps in the store obviously we're not going to include that version of the.net runtime with every app so there are shared frameworks that you can take advantage of okay um but you can actually change the version of the dinette libraries you want to reference through nougat it's pretty cool cool and then how much storage is on this device how many so it's like this can you get on there it depends on your SD card so I actually have a micro SD card on here AC but that actually has the operating system as well so if I want to flash this device with a new operating system I can take out that micro SD card plug it into my computer download the latest preview build of IOT core and plug it back in and ready to go alright yep so great so the application has gone and it looks like a breakpoint was already hit probably when I reached over yeah so I triggered one of these sensors and we'll go into more you to get the exit further you've reached through I think you actually exited the store I think so I always get it mixed up which ones entrance and which ones exit so now might just have to turn it around but okay I trigger one of them first but what's interesting is I just need to know what is happening when that breakpoint is hit what are the values that I'm tracking at this point so in this case I can see the value has changed for my exit sensor and I want to track the arguments that are being passed into this event so I can just use Visual Studio debugging tools to look at the local variables and see that args that edge is equal to falling edge and I know that seems kind of cryptic almost but this is the the parameter that you're going to track with most of the digital devices that connect to these IOT devices and if you did more of the tutorials that would become more obvious but even if you don't do the tutorials you can figure it out just some clever debugging right so what that means is the instant that that breakpoint was hit meaning means the sensor is broken right and when the sensor is broken args that edge is equal to a GPIO pin edge falling edge so that's how I track if it's broken if it's unbroken it's going to be equal to rising edge okay and those are just enumeration values so I can stop otherwise I'm going to throw an exception and use that knowledge to finish my logic so this is the important part and then for reference the beam is broken and now else the beam is unbroken there are really only two states for this device your long effectively so now I can write the logic for my application on here using those if statements yep so as soon as the beam is broken that's when I want to set the boolean value to true so since this is my exit sensor value equals value changed I need to set exit equal to true now if the enter sensor had already been fired that boolean value would already be true meaning this one was fired second that's when I know what event has happened right so if enter now I'm going to say customer has entered true that means someone has entered my store that method doesn't exist we're going to write in just a moment okay else well exit is already equal to true otherwise I really don't need to do much so in this case actually want to break out of my method and not do anything else did the backwards from how I normally do it normally I put exit equals true at the bottom but it shouldn't really matter okay so the only other thing that I want to do if enter is equal to true is reset the state of the device as soon as I fire off the event so I reset both of those boolean values to false so now it's ready to go for the next person so the code in customer has entered is what's going to send a message send the event to Azure correct okay yep so now um if the beam is unbroken I actually have a light to track when the beam is broken and I'm broken for debugging purposes that's all that's going to happen here so as soon as the beam is broken let's turn the light on so exit light dot right GPIO pin valued low and if you do blinky that the hello world of IOT devices letting that line of code we're pretty similar the light so the LEDs here's red and yellow LED in the device yeah so now I'm just going to copy and paste that to here and I'm going to write the value of hi to turn it off and that's really all I need to do for this so I can just copy and paste this segment into the customer has entered method oops sorry no the entry sensor value changed method and let's take a look here so let me lower the output a little bit so we can see everything so now if our edge equals GPIO pin edge falling edge they're the same exact devices this is the same exact behavior this is right the only thing that's going to change is now I'm working with the entry sensor right so enter light enter equals true whoops and free light yes entry light now if exit so if exit was already fired I'm actually going to be sending an exit event so exit was fired first and then enter I'm still going to reset the state of the device that is the same else entry light dot right GPIO pin value hi so the last thing I want to do is remove the not implemented an exception here and just place a breakpoint to make sure that the events are working as proper or as we expect before we before we send a message to the edge of your website let's make sure we're getting the right expected behavior right so now you'll see deployment should be much faster this time deploy succeeded that was much faster yes there you go so the breakpoint is ready to go so the first thing I'm going to do is just cover one of these oops sometimes when you have some bright lights like a studio light it interferes with the censors but if I cover this one pretty closely I can see the red light is turning on the lights blinking yeah so now if I cover this one the yellow one I can see the lights actually staying on because a break points it so the code is not continuing to execute to turn the light off all right so here I can see in this case V is false that is an exit event so that is going out of the door that means if I were to continue execution of the code and go the other way it should be true so let's try it let's cover yellow first and then red so again breakpoint and now I see V is equal to true yeah red stays on because again the codes not going to turn the light off until it continues to execute right so that's all I need to know so my entry and exit events are working properly on the device now let's actually do something with that information so again just using c-sharp code I can create my object triggered event which I need to first add a reference to my portable class library models folder let's call it event T event equals new trigger event and I'm just going to go ahead and create the event in line so event time equals state time dot now event type equals V I'm thinking I might actually I have a theory that when I'm gone my cat just sleeps all day long I don't be able to find out yeah that you could actually use proximity sensors instead of IR beam sensors so instead of tripping a beam if it goes near it you could also trigger an event right so you can track things at the home you can use this as a security device or some type of monitoring if you're one of those data obsessed people as well you could figure out where your cat spends the most time in the house I know her just by all near the bed so I already know that but a sense in another bed and you could actually calculate how much time spent under the bed versus other other places so lots of funny stuff you can do so I've got my event now I just need to actually create the content to send to my HTTP a web application so HTTP content equals new string content the string itself is just going to be T event to string and then I'm actually going to use some of the overloaded parameters to send the media type so I'm interested in telling my asp.net application that this is Jason that I'm sending because of my tea event to string method I overload it I can actually navigate to it to return the json serialization of that object and that's that it works properly with my asp.net app so let me just do encoding dot utf-8 and then application jason and one thing that i always accidentally do is do forward slash or backslash always get it confused so I can make sure I do it right application slash JSON perfect cool so the only thing I need to add now is I just have to post it so this is the one I want content should actually give it a name and now all I need to do is HTTP client which already have open sorry client I already have my HTTP client I don't need to recreate it dot post async the request UI is going to be API slash event and I'll show you where that comes from and then the actual content so the API event is just what's appended to my base URI so my base URI is shop analytic service as your website's net API event is what's at the end so now if I were to do this so how does it know where did you specify the base URI so I specify up right yes so I specified the base you are I lost debugging to my life service sometimes it times out so I'm going to test debugger again and you'll see why very very soon so up above is where I specified the client information base UI equals keys that has your web app AP UI and then when I actually create the client in main page load sorry at the bottom clavicles narrating yeah ok client equals new HTTP client so yeah so as soon as this page loads what I do is I create my HTTP client with that base address and then I initialize the board right yep cool plot explores um let's try it touching one more time sometimes you get finicky behavior with live writing services so anyways let's go ahead and actually deploy this application to the device and set a breakpoint when the customer has entered the store what or exited depending on the parameter passed through and just so we can track the entire this codes running on the device this code is executing on the device and you've set a breakpoint it's going to show up in visual studio because you are debugging on the remote machine I'm deep but I'm doing remote machine debugging on the IOT device yeah so remote machines bugging has existed for a long time I could do PC to PC yet but now I can do PC to xbox pc to IOT PC to surface of Xbox you name it let's see what's going on with this one my life service is not behaving so well but that's okay we'll go ahead let's see a breakpoint perfect this should be true someone has should be coming into the door so on my local variables load V equals true yeah I can hit f5 again and now what I should see is as soon as it sends the event it has executed properly I should get a push notification that's where I was saying my life service my other reel so a little bit delayed but someone has entered the store example seconds later now it's interesting is I can actually view all of the events that I sent sent so far again in Visual Studio so if I go to document DB accounts loving that because it's just unstructured data and no need to spin up a whole database so this is I'm actually using yeah it's just a list so I'm using document DB which is a no sequel database all it is is a JSON document for the demo I just have a single document but one of the things we're working on in the repository is making it more scalable to use the the no sequel query that is supported so creating a new document for every event and then a collection for every month for example and then we could kind of parse it however okay but you'll see why I use a single document when I copy and paste all my fake date and it's really convenient so if I find my document C document EB is very powerful but I just have single document I can go ahead and load and here we go so at the very beginning we did 20 o'clock ah a clock global standard time and I said true and then I've done one real event that I sent through the IOT device which happened just a couple about a minute ago which is 144 and it's now 145 yep so that is the real event and I didn't have to leave visual studio at all for that so now just for convenience I want to show you another tool that you can use for document DB to copy and paste the data so you can actually see the data when I run the next app okay so in the azure portal i can again go ahead and refresh this document this is the live document here are the two events that i was just working with now let me go ahead and find my fake data i have a lot more than just that and this will actually make the app look like something there we go many more events so I can go ahead and save that okay and not my database has been updated with that data so makes it really easy to schedule in going linked better than going about 1,500 times exactly although we get lots of steps on the band my mom would get really tired though so I'm actually going to break out a Visual Studio 2015 for now but first I want to deploy the IOT device so I'm not going to debug it I'm just going to hit ctrl f5 to deploy it without debugging and that will just continue to run it on on that IOT device for pretty much ever okay um so I think one funny example is I went on vacation and deployed it before vacation came back a little over a week later it's still running fine I came back and swung my hand across all the lights on the push notification cool it's pretty reliable no crashes or anything like that so deploy is succeeded I can go ahead and close this instance at Visual Studio let me go ahead and get rid of this instance of Visual Studio I don't know why that debugging wasn't working that's okay and let's take a look at Visual Studio 15 preview which which preview version is this so this is the next major release of Visual Studio or agile studio 15 preview 4 is available to smore this is actually beyond 4 but now okay so this is latest and greatest sigh you never know what could happen hopefully everything works properly so 4 is available publicly and this is something past that yes something password is available publicly right this is one of the beauties of working at Microsoft is getting to play with daily bits of everything it's awesome so let's go ahead and take a look at this application this is what's going to visual the data mm-hmm I really just have one page that I'm doing there's a lot of code that processes the data behind the scenes and that's where again we can optimize our database structure to just do that in the cloud using querying but for now we're not going to worry too much about it up here you go we might have some intellisense bugs with this version of Visual Studio but I don't think we need it if we're not actually gonna write too much sam'l but the designer works so that's good so let's take a look at the designer first to see what this looks like so on a phone I can see I have a grid here mm-hmm and it's a little easier to view the structure on a desktop first because of adaptive code so if i zoom in I can see I effectively have a grid within a grid so here's my main grid is the outer entire rectangle is the first main grid and in that grid I have two rows this is the first row and then this entire thing is the second row so the first row just contains my coffee shop name the hours and I'm open it'll also give me information about when the data was refreshed then in here I have this picture that tells me how many people are currently in my store okay it's really busy and people are like lining up to the door or something I want to know how busy it is and then I have three graphs that will be displayed in quadrants and what I do is I actually write this adaptively so that if it were on a phone I would just be able to scroll through each of those charts so I can see here's one of the charts below and then it's not going to show you the rest because it's cut out of the UI but I could scroll through it and I'll demonstrate that as well so I'm going to go ahead and just deploy this application and what I want to show here is one of the new features that's coming out in Visual Studio 15 for zamel development to really help you build performant zamel apps okay so this should take just a second I'll also show you a couple of tools that have been out for a little while now but a lot of people still don't know all about them they've only been out for a little over a year so this application is running so let me close some of this so what I want to show you first is what is new so inside of the Diagnostics tools window we have these Diagnostics wall debugging tools that run just while you have five rewrap there's a new one called UI analysis for uwp tools now this is not on by default since it's brand new so for those of you watching if you want to turn this on all you have to do is click select tools check on UI analysis and then you'll have to reset the debugging session okay it's pretty painless but what that does is it gives me a whole bunch of different events about my application and the performance of my application so the number one is using binding instead of compiled bindings I'll have to go ahead and fix that at some point but the one that I want to show you is one that actually surprised me and I found out about this because of this tool so let me restart my application here's the app you can see all the data let me restart my application and describe what I'm doing that is bad that UI analysis told me about that I had no clue about it's actually pretty cool so when my application first launches I can see I don't see much except for this loading ring right and what i'm actually doing behind the scenes is i'm creating these objects and i'm just hiding them so i'm setting their visibility property to visibility collapsing and what that means is on startup it's still loading that zamel control into memory it's having to do the processing to create it and just not doing anything with it right and what UI analysis told me is let me find one of them here we go element numpy 'pl in store of type 1 RT it's the the charting control was collapsed at load time so it told me that collapsing an element or setting its opacity to 0 will not prevent the element from being created so it is creating the element it's using processing power it's slowing down my startup time well that's something I definitely want to fix so not only that but I can actually this is going to work it might be broken because of this it's the problem with daily builds so when this is out if I double click on this it'll take me to the line in zamel that is causing that to break mmm so I can actually just go ahead and navigate to it so numb people in store right here so here's the chart so now what I want to do is change one of the properties so it's going to be hard without intellisense but I can do it so X : the flirt load strategy is the way to fix this oh I do have intellisense lazy cool just I don't have the highlighting I guess so I'm going to set a property called X : to furloughed strategy equals lazy on this chart and then I have to do one other thing so I have to navigate to it in the code-behind up one other tool that I always do this myself I've just gotten so used to it samo edit and continue is another new tool in Visual Studio 15 as you'll notice this application is still running I'm still debugging is app and I just added that property to my zamel without having to stop anything and I just did there because I'm used to it now so I think that's really going to transform the way people create sam'l applications yeah so I will stop it because not going to have to edit the code behind let's go ahead and here and let me make sure I get the right property name so X not num people in storm so the way I'm going to get rid of that warning and UI analysis is I'm going to set that property and then I'm going to call some it's somewhere in the code where it actually makes sense to finally load that element so it's not going to be read at startup it's going to be when I have the data available to actually make that chart meaningful so I'm going to look at my code and I can see on main page loaded I get the data and then I set the data context and as you can see I already did it for a different control before but if I do this dot fine name num people in store this doesn't actually do anything but it tells the code hey I need to create this object I'm trying to do something with this object so it will create an airline so this is going to retrieve the object that has a specified identifier name so in trying to retrieve it it's telling the sam'l hey you need to create it so now if I run this again I should only see five I guess infractions of that specific UI analysis time so here we go so now if I go back here and I sit select filter you analysis it's down to 48 instead of 49 visibility collapsed so as you're debugging what's amazing about this is since its while you're debugging you'll catch these as you write your code so rather than waiting until the very end doing your post-mortem profiling you're at all ready to go oh wait you have 60 performance things to fix on your UI threads well now as you hit these issues you just go ahead and fix emit persuade yeah so the last thing I want to show is UI debugging is my UI debugging that has can you get those before running is there a no so those are actually calculated at runtime so there's a process running that will tell you that this issue is occurring okay so the last thing I'm going to demo is actually in Visual Studio 2015 with this same application and I want to show it because not many people know about it Oh hardcore example developers definitely know about it now but there's still a lot of new developers that are not aware of this feature and I want to make sure it's pretty evident to them that they have this tool toolbox available to them so Sam will you eye debugging is kind of the precursor to zamel editing continue which I showed you before and I think it might be a little busted in Visual Studio 15 but that's what happens when you play with today's build yeah so let's go ahead and hit f5 it should still work I've seen that warning before it's because I forgot to add a reference to my portable class library but the binary output of this folder is enough to get it to work because in another project I have the reference it's confusing but yeah the app works I just want to show the same with your ID buggy so live visual tree now I can see it so here I've got this actual Dorner in my running application let me go ahead and actually while this is running let me show you the adaptive UI nature so if I make this really skinny after a certain point it makes everything in a single column and I can just roll down so the behavior on the phone is I just go ahead and swipe and I can see everything working but now let's say I wanted to change some of the styling on this say I don't like the size of contoso coffee shop I can go and click on the adorn er is that always been collapsible like that uh since update three yeah Wow cool and it's also you can do it start it showed up so you can also hide it I didn't notice collapsed this one is right so you can get rid of it entirely if right interfering with your app so are on the topic of getting rid of things go back there yep why is this decision tools options so that's a global setting a frame I never ever wanted I never ever want it unless I explicitly turn it on and I shouldn't have to go into abductors oh yes for every single one and get rid of it yes sure that's good for you we've actually had this feedback before there's there's two sides of the fence there's the people who actually do use this information line that's why examples options you said it globally alright I'll take that back to my time thanks it's a great feature but when I'm building an applications give them away I didn't know if that if the thing works yet I'm frame rate is one of the last things I'm gonna check that's true never the first thing yep okay but you can see kind of see that's one of the benefits of having a show and inviting there you go close to the product any prior wisdom so what Sam what your ID bugging does is it lets me select that control and then when that control is selected in my effectively this is kind of like the Dom for those of you have to web debugging I can see all of my visual elements that are at runtime and this could be programmatically generated controls if it has this next to it it means it was user generated in a page okay so I can actually click that to navigate to the control in the sam'l so let me actually just show the same oh so that's the textbook that I'm actually looking at here so this version of Visual Studio does not support sam'l edit and continue but the power of that is I can use them or you I debugging to find the control right then I can use the live property Explorer to modify local properties of it so I can set that to 30 to make it bigger I think that looks a little bit better and now if I were to stop this application and run it again it would revert back to 24 right but with zamel edited continuing editing I can then after testing am a lot property explorer I could then edit the zamel file while it's debugging changes the values of the property hit save and then just keep going and I don't even need to stop the application so that's cool yeah it's pretty powerful one of my favorite new features in Visual Studio 15 oops don't know what I did there Ike's and I hit alt tab I wanted to go back to this one oh it's still gone but yeah that's those are some of the new tools cool any questions umm yeah one question so the the PI Raspberry Pi sends a message to the azure service yep which then use notification hub to send a message out to the client right yes okay so some someone a event-driven model where an event is fired the azure service kind of wakes up that sounds like what Azure functions are so did you consider using Azure functions ah for the server side so I initially didn't consider using Azure functions mostly because I wanted to demonstrate the power of sharing dotnet code across all of these different platforms so I'm using uwp I'm using asp.net and I'm using the same code for all of them yeah but as your functions would also serve the purpose of what I'm doing okay cool yep but just to show this one last time as I'm running this it should trigger an event and I should see current people in store either increment or decrement I don't decrement that was exiting so now I don't want people in the start so I got this way then it should go back to zero cool and it's pretty fast so ready oops I might have missed oh yeah they got one so takes two seconds three second clearly notification and update it so you could have that you can have a screen like this in the window of the store shows the number of people in the store and what the average wait time I could eat that too yes I can hook this up to my register so so one of the things I was considering just additional functionality is having register functionality in this app and then I could do how quickly our orders processed how much is the average order I could show correlate this data with money instead of just how many people in the store if I'm running a promotion maybe I'm giving discounts so more people might not equal more money yeah it's such a but the last thing I did want to show is a couple resources for this mostly the github page I think that's what's all this all the code is up in github yeah for both the client and the server and if you have any issues with it or any questions raise an issue or tweet me at pm at Mike this is going to be a living repository for awhile who makes a good sample of all the different services available to you and did they open the coffee store are they using this are they using it today now so this is still just v1 we have to first create our printed circuit board and put it in not um no not yet okay maybe Sunday thanks so much yeah all right cool this is a great example thanks for coming on and showing other problems yeah lot of fun alright hope you enjoyed that take a look at the code play around with it let us know what you think and we'll see you next time on Visual Studio toolbox [Music]

Build productive Python web applications with Visual Studio Code, Azure and Azure DevOps

hey everyone welcome to Microsoft Connect my name is Nina Zakharchenko I'm a senior cloud developer advocate at Microsoft focusing on Python and today I'm going to show you how to build productive Python web applications with Visual Studio code Azure and as your DevOps let's get started I've set up a repository that has all the code and the steps that you need to follow along just grab it from the URL get do slash connect appear I'm I've already set up my local development environment I've cloned this repository I've also set up a virtual environment by running Python 3 - M the end and then activating that virtual environment by running source and have been activate if I do a pip freeze we'll see that I've already installed all the packages from requirements txt in this environment let's take a look at this project in Visual Studio code code has an amazing and easy shortcut that lets us open up a whole workspace at once all we need to do is type code dot in the directory we'll see I have all my files here if I look here at the bottom I'll notice something interesting yes code has automatically picked up my Python virtual environment I didn't have to do anything to get this app running locally you're gonna have to set up a few secrets the guidelines for that live in the em-dash sample file here you'll see everything that you need to setup in order to get this pop working in production I've already set up a Postgres database I've run my migrations on it I've loaded in a little bit of fixture data let's make sure that our app runs locally to do that hit ctrl and backtick to open up the python terminal we'll see that V s code automatically activates the virtual environment for us we don't have to do anything to check out our app locally we'll need to run Python manage py run server to bring up the django web server and that's kicking off now let's take a look Oh we'll see we have a local Twitter application have our tweets we can even say a new tweet if we'd like hello connect everything is working as expected now if we run into issues how do we debug our jingo app going back to vs code let's kill our development server for now let's take a look and see what our request looks like by setting a breakpoint tap to the left of the line number two create a red breakpoint dot in the gutter next open the debug panel we'll see that we have no configurations here all we need to do is click add configuration and yes code knows that we have a Python project open it creates a new Python configuration for us super easy if we want to debug our Django what we need to do now is select the Python Django debug configuration and then hit this green button right here as our debugger starts we'll see that the bottom bar has turned orange instead of blue now if we hit our URL again we'll see that the debugger has stopped at the breakpoint we have a few options here we can continue we can step over we can step into or step out but I want to show you one of my favorite features and that's the debug console if we click here to the left of the terminal we'll see this little window pop open and I have a fully featured debug environment here I have autocomplete if I wanted to take a look at the request I can just type that in there I can also interactively work with this object I can click on the arrow here I can take a look at the body any of the content parameters examine anything that I want to about the request I can also take a look at our user Bob and check out any attributes about him I'm going to continue here that's great but what about debugging Django templates our templates can get really complex and hard to navigate thankfully the s code provides that too let's open up our index.html we'll see some template code here all I have to do is click to the left of that and that our debugger breakpoint triggered here I can look at the user in the debug console like I did before I can also setup watched variables here so every time a breakpoint stops these watched variables will populate I don't have to do anything I always want to see what user is coming up on my index view I can set up the user ID or the user in the watch here super-easy it makes debugging Python web applications a complete breeze now we've got everything working locally what if we want to deploy our application to the cloud well thankfully yes code has an answer for that too check out the extensions tab here on the left hand side in order to kick this off you'll need to install a handful of them the Python extension of course along with as your account to sign into your azor account and as your app service to do the deployment once you have these extensions installed you'll see on the left-hand side here a new icon that as your icon if you click into that you'll see your app service subscription all I need to do to set up a new deploy is hit this plus button right here I need to enter a globally unique name and I'm going to select an operating system Linux along with the Python 3 7 runtime see in the corner here that the web app is being created if an resource group isn't found it'll create one along with an app service plan let's just give this a few minutes to run here I have a pre-existing deployment what did I need to do to get this going I needed to configure our application settings so our production environment needs to know how to connect to this database you you can set these application settings straight from vs code here I click here you'll see all the ones that I've set you can even right click on an individual setting and hit edit rename delete it or you can do it from the portal to easily access the portal component of this application all we need to do is right click and hit open in portal and that will pop open the portal for us and open up the specific app service if I'm once my deployment is completed to set up that deployment I would want to right-click and configure the deployment source I would want the deployment source to be local get all I would need to do is right click again and hit deploy to web app that's it just a few clicks and you have an application running let's take a look at my existing app okay right click and hit browse website and it will go straight to that URL for me here we'll see our tweet our application running in the cloud on as your website's net this is great for smaller applications smaller uses maybe you're doing a little bit of work in development or QA what if we wanted to set up a CI NCD pipeline for our application we can do that with Azure DevOps pipelines to set those up go to dev azure calm here I have an existing as your DevOps project with a few pipelines set up I have two individual pipelines one for my continuous integration one for my continuous deployment let's take a look at the continuous integration one if I edit it here we'll see that there's a yamo file path as your - CI - pipeline Dhamma that exists in or git repository we can take a quick look at that here so the siamo file describes everything that we need to do we're going to be using in a bun - image we're going to be using Python 37 we're going to install all of our requirements and then we're going to run our tests and make sure that those paths before continuing onwards with a deploy going back to our pipelines let's take a look at the continuous deployment pipeline this is also configured with another llamo file this one is also in the repository right here the deploy pipeline we're using Ubuntu again Python setting up our Python version and then setting up a few environment variables our deployment URL our deployment username and a deployment password we set those up in the portal going back to our project here opening it in the portal going to our deployment center and grabbing those environment variables from here we're going to grab does get clone URL and then we're also going to make a set of user deployment credentials setting a username and password that were then going to configure in our pipeline another important thing about this pipeline is the triggers here we'll see that we're disabling continuous integration for our deployment pipeline but we're depending on the build completion step of our CI task having set this up once I deploy to master my CI build kicks off if it goes successfully I get a deployment super quick easy and really understandable that's all I have to show you today if you want to learn more about Python as your check out this URL you can also take a look at the Django on visual studio code tutorial or take a look at the demo repo all the steps that you need are right there all laid out and I hope you all have an excellent connect it was great talking TTA bye

Bringing Big Data Analytics through Apache Spark to .NET

hello everyone welcome to bringing big data analytics through pachi spark to dotnet I'm Bridgette Murtagh and I'm a program manager here at Microsoft on the dotnet team so let's start off with what is apache spark so big data means that there's an increase in volume velocity and variety of data so let's take for instance a factory there can be thousands of internet-of-things sensors in a factory each producing petabytes of data now while it's great to have that much data so you can understand how our factory is performing and ways to improve the equipment how can we actually process it all when we have that much and more than just that how can we process it all quickly and efficiently well welcome to the world of Apache spark so what is Apache spark Apache spark is a general-purpose distributed processing engine for analytics over large data sets typically terabytes or petabytes of data put a little bit more simply Apache spark is your great tool we can use to analyze a large amount of data and a quick and easy to understand way so we don't have to be data science experts to understand or use it there's quite a few different things that we can do with the Apache spark that are all super interesting and exciting but just to touch on a few of them one of them is spark sequel which means analyzing data that's structured in some way so maybe data from a CSV or from a database there's also spark streaming which means analyzing data in real time as it's being produced so in our factory example it means analyzing data live as it's coming from those IOT sensors so we can go and detect maybe if there's a malfunction on our data and we can go and address it right away there's also machine learning capabilities with Apache spark so you can combine the powers of big data and ml to scale and have faster more efficient training and prediction of machine learning algorithms so to understand how Apache spark works there's only three main components that we really need to take a look at the first one is the driver the driver consists of the user's program so for instance if you wrote a c-sharp console up that would be part of the driver the driver also consists of a spark now what the spark session does is it takes that users program so for instance it takes that c-sharp console up and it divides it into smaller pieces known as tasks now those tasks are divided amongst our second component which are the executors or the worker nodes and the executors are the workers are on something known as a cluster so each of those executors takes one small task so one small piece of our users program and finishes executing it and the third component of our architecture is the cluster manager which helps with dividing up the tasks and allocating resources amongst our driver and our executors so how can I use the pachi spark it sounds super great super useful so how can I get started with it so it's quite a different there's different api's that we can use that are popular with spark and they're written in languages like Scala Python Java and AR but up until this point there weren't any dotnet api's for spark sort of I wanted to use Apache spark combined with my pre-existing dotnet knowledge or extensive code base and business logic well we now have an awesome tool that we can all use and it's known as dotnet for Apache spark so dotnet for Apache spark is a free open source and cross-platform big data analytics framework it allows us to reuse the knowledge skills and code we already have as dotnet developers so anywhere that you maybe have an extensive C sharp F sharp codebase now you can go ahead and introduce big data analytics within it dotnet for Apache spark is also designed for high-performance the overall goal for dotnet fair patchy spark is to provide dotnet developers a first-class experience when working with Apache spark so we have had several customers expressed a lot of interest and actually see success those net faire Apache spark and one of them is the Microsoft search assistance and intelligence team who's working towards modernizing workspaces in office 365 their job is to work with different ml models on top of substrate data to infuse intelligence into office 365 products their data resides in a deal and interns gets fed into their models why they were looking towards dotnet spark was because a lot of their business logics such as the different features or tokenizer z' were all written in c-sharp meaning it would be ideal to be able to use big data analytics still within the dotnet ecosystem and so far their experience has been extremely promising stable and they've really loved the vibrant open source big data analytics ecosystem within the dotnet community the scale of their jobs has been about 50 terabytes so quite a bit of data and they really started seeing success with it so now that we've seen a little bit about what dotnet for Apache spark is and why it's such an exciting new solution for us let's take a look at a few different scenarios that we can complete in some really exciting applications we can build using dotnet for Apache spark so one of the most fundamental Big Data apps is batch processing so what is batch processing or what is batch data batch data means that we're working with data that's already been stored so for instance we could be doing something called log processing which means looking at and gaining insights from logs from maybe a website or a server or a network of some sort so we can understand what actions our users are taking or what pages of our website are the most popular we can also do data warehousing which means taking in data maybe from a variety of different sources and then performing a large-scale analysis on it maybe that data is all stored in Azure storage and then gaining different meaningful insights from it so in the example that we're going to be looking at today we're going to take a look at some github projects data so you can see here in the snippet of that data that our projects data includes the URL of our projects the author a description what language it is things like that and we want to know on average how many Forks does each language have and the number of times each project has been fork is represented by that column each there so let's go ahead and take a look at our first coding example with dotnet fair Apache spark so I'm going to open up visual studio 2018 here and we can see here that I'm just dealing with a c-sharp console application that I've already created and I've already installed the Microsoft spark nougat package oh there it is cool and we can see that it's also installed because I have these great jar files over here in the solution Explorer and at the top I'm using Microsoft spark sequel and Microsoft's piped sequel functions because as I had mentioned spark sequel helps us work with structure data so if I'm reading in github products data that data does have some sort of pattern or structure to it so I want to use spec sequel so to start off with in my main method here the way that we start off any dotnet fair pachi spark app is by creating a spark session that's what's dividing our program into the smaller tasks to be distributed amongst the executors so we can see I've created a spark session called spark I went ahead and built the session and I just called my app github and spark batch pretty appropriate name now after doing that the next step we typically want to do in our apps is to actually read in our data so I have our data is stored in a CSV and we want to read that CSV into an object called a data frame so a data frame is going to be the basic object that we store our data in when we're working with structured data in spark so if I open up this region here we can see that I'm working with a data frame here and I just called it projects DF to stand for my projects data frame I go ahead and called the read method and then I also called schema which means that I'm working with whatever pattern my data has so for instance I know that my data has an ID column a URL an owner ID I also know the type of data that's stored in there whether it's an inter string or something like that and it's a rather long schema because I do have quite a few columns and then I can call dot CSV since I know that my data is stored in a Commons comma separated values file and then another popular method that's good to use is dot Show which allows us to actually print that data frame to the screen so I continue on another popular step that we'll want to do when working with batch data is to do some data prep or data preparation and that means that we're kind of cleaning up our data so maybe if there's some Knoll or missing values or if there's some extra values maybe like a few extra columns we don't need we can go ahead and remove those so that our data is easier to read and easier to work with so one of my first steps of data prep was working with the data frame na functions and with that I chose to drop any values that are null so any role any rows that have missing or null values I chose to removes that when I go and perform calculations later I'm not accidentally trying to perform calculations on a missing value also with my data prep I chose to drop a couple of columns that I don't think will be important for my final calculations so I dropped the idea the URL and the owner ID columns so after doing some data prep we can actually go ahead and perform the functionality that we wanted to which was finding on average which languages have been forked from the most often so the first thing I wanted to do was group my projects by language so what I've done here is I've created a new data frame that'll represent now my group two data and I called the group by method which allows me to choose which column of my data I want to organize by or to group by so I chose the language column and then whenever I do a group by I also need to call aggregate or this dot AGG method and with aggregate it allows me to perform some sort of functionality across every row or every entry of my data so in my case I performed the average of the forked from column on each row of data so essentially I'm grouping by language and then finding on average how many times each language has been forked and then finally I don't want to just display my data as is I want to do one final step to make it a little easier to understand and read so I've chosen here to order my data frame in descending order so that way I have the top forked languages at the top of my data frames so I can see those first and then a final good step to do is to go ahead and stop our SPARC contexts so I went and went ahead and called sparked off stop just to clean up resources and make sure everything finishes executing correctly okay so now I have a few steps here that I'll need to be able to build and run my program so one of the steps and working with the.net for patchy spark app is to make sure that we have one of our environment variables set correctly so there is a dotnet assembly search paths variable and we'd want to make sure to set it to specifically my apt so in this case batch and then the bin debug folder and then whatever version of net core app that you're using and then also one other thing you can check is the level of logging that you have in your output so I go ahead and open this up so there's a file called log4j dot properties and here I've set the logging level or whatever is going to be output to my console to be the error level so rather than displaying maybe some extraneous warnings or info or debug messages I'm only going to display messages that are actually an error which will help make sure that my output isn't too confusing or crowded in my console okay so now it can actually be time to go ahead and build and run our program so fortunately I've already done that with for us here to save some time so I'm going to open up the terminal not that one that one will be later okay so here I moved into my batch directory using CD batch and then I just went ahead and built my project using dotnet build we can see that build succeeded so now let's go ahead and see how we actually run a dotnet for patchy spark app we use something called spark submit or the spark submit command and it every time we use spark submit there's a few different components to it so we you say spark - submit and then we're also going to reference the dotnet runner we're going to specifically reference one of those jar files in our case for using apache spark version 2.4 and dotnet for apache spark version zero point 4.0 and then also we want to have a path to our apps dll so it can actually build and run correctly so after running sparks submit let's see how our program did so first we're expecting just to see the data frame with our csv data in it so just the raw github prod so let's take a look alright so we can see it here we see that it has all the columns we expected but we can also see that the data is kind of overlapping with itself there's a few too many columns like updated at is all the way over here instead of continuing to the right and then there's also a lot of null and missing values so it seems like it was definitely a good idea to go ahead and do that data prep we did if I scroll down now we can go ahead and look at our data prep result and so now this data frame is a lot easier to read and understand we can see the data doesn't overlap with itself and all the data actually exists there's no longer all of those null values so it's a lot easier to work with so next we can go ahead and see the output of when we were trying to calculate the average number of times each language has been forked so let's take a look and so we can see we have a column here for language and a column here for the average number of times it's been forked and it all looks correct and we can see that it did sort in descending order because language is at the top havin forked on average more often than the languages down here and then it's also worth noting that for all of those data frames it's only showing the top 20 rows and so this is really useful because in case we're working with terabytes and petabytes of data we want we don't want to be stuck trying to show those data frames and then having it take forever or getting confused and too crowded on the console okay so we have successful yet run our first dotnet for a patchy SPARC app so let's go ahead and go back to our presentation already so we have already done that demo so let's move on to our next scenario which is combining machine learning with Big Data so when we combine machine learning with Big Data it means that we want to scale the training and prediction of machine learning algorithms one great framework we can use for the machine learning when we're combining ml with Big Data is ml net which is a free cross-platform open-source machine learning framework in our example that we'll be looking at we're going to be performing sentiment and Alice which means that if we're given a piece of text we want to determine if it represents something positive or something negative so in our case we're going to analyze a set of online reviews and we want to know which are positive and which are negative if we were given reviews such as I love dotnet for Apache spark that would be considered positive and we could maybe see either a true or a one depending if we're using a boolean to represent positive or negative sentiment if we saw a statement like I hate running inefficient big data queries that would be considered a negative sentiment so let's go ahead and take a look at our sentiment analysis demo where we combine Mission ml donut and dotnet for Apache spark okay so I've opened up Visual Studio 2019 once again and in this case when I look at the nougat packages I've installed I haven't only installed Microsoft spark I've also installed Microsoft ml which is the nougat package we need to use ml dotnet and then we can see at the top here I'm using statements both related to ml net and net fare pachi spark so you can see Microsoft ml ml data we can also see Microsoft spark sequel if I scroll down here we can see that just like we had done in the batch example we start off by creating a spark session for our program and I've just gone ahead and given my app a different name compared to the batch app next it's also going to be similar to our batch example since we are still technically working with batch data we're just taking it a step further by also performing machine learning so in our case we want to read a review data into a data frame so I have some Yelp reviews so I have it in a Yelp csv file and i've also set a few options for my data frame here so for instance I know my data has a header and so the two different columns in my data which include the text and if it's a positive or negative review those columns are labeled that's a header and I don't want spark to treat that header is part of the data because I could throw off my results and then I just went ahead called show so I could see my raw review data as is before we go ahead and actually predict using ML dotnet so now it's time for the fun part where we can actually start combining machine learning with big data so how would we actually start calling the ML dotnet code so how we can do that is using something called a UDF or a user-defined function and so UDF's are a popular solution so that we can perform some sort of function on let's say each row in our data frame so if I open this up here we can see that we create a new UDF by calling the UDF method and then register and within the angled brackets I have string to represent the input that I'm working with which is text or reviews and then boolean to represent what my output is going to be which is a true or false for negative or positive sentiment I've decided to call my UDF ml UDF and what I'm doing within this function is passing the text into a method called sentiment so we may be asking so what is the sentiment method where do we where do we create it what do we do within it so within sentiment if I scroll down here we can see that sentiment actually contains our machine learning codes the code that was generated from ml dotnet and I got this code and I actually also trained my sentiment analysis ml net model by using something called model builder the model builder is a really useful UI tool that we can use within Visual Studio that helps us train and work with machine learning and a much easier and understand way so just to see what model builder looks like if I right click on my project and say add machine learning I can see that within here I can choose a scenario so I can choose things like issue classification sentiment analysis price prediction so my case I would have chosen sentiment analysis and then I can just go ahead and choose my input file so I could choose my input review dataset to do some training on and then m/l net does all of the training for me and generate some really awesome code for me so if I go back here this was actually code generated from that melt from model builder using ml and what it's doing is essentially creating a way to start predicting so it calls the ml model that was trained and created and then it creates a prediction based on whatever strings I pass to it and then down here I've created classes to represent my review data since I do need to pass that or work with that when I'm using ml dotnet okay so now that I've gone ahead and created a function where I can call that ml dotnet code I want to actually call that function so what I've done here is there's this really neat functionality in dotnet fair Apache spark where we can actually execute sequel queries so if you're familiar with sequel syntax at all we can have those sequel queries within our code so in my case I've gone ahead and selected column 1 which represents my input review text and then past column 1 so past each review to my ml net method and what I can do here is then just go ahead and print that out essentially and then I called Show so that I can see the output of my operations and then similarly just like we had done using the batch app you'll want to make sure you set your environment variable correctly so that bin-debug net core app folder and then we can go ahead and go into our apps directory and build and run it so let's take a look at how that came out ok so you can see here I had built my project and built succeeded and then I ran sparks submit using pretty much the same types of parameters just in this case it had to be to my current apps to yellow if I scroll down what we see here for the first data frame is that was just the raw review data so this is these are all the reviews that were in my Yelp dataset and this is the true answer if it's a negative or a positive sentiment so we can see when someone loved something that represented with a 1 that means it was a positive sentiment and if someone said something was not good that was a zero so negative sentiment so now the next data frame that we're going to see is going to represent the prediction from our ml net code so let's scroll down and see you and we can see here we're dealing with the same reviews but now this is the predicted sentiment so you can see here that it looks like it was pretty accurate we can see when someone loved something if his predicted true so positive and then again when something was not good it was false so negative sentiment so you can see here that we were successfully able to combine dotnet fair pachi spark and ml dotnet let's go back here it's now that we've done that demo we have one final quick scenario to go through and this is with structured streaming or real time data analysis so in structured streaming or real time analysis we're working with live data so data that's maybe coming in from a sensor so like an IOT factory sensor or a phone or a network and structured streaming uses the principle of micro batch processing so essentially it takes our continuous stream of data and divides it into smaller little chunks so maybe every five seconds represents a new batch and then it can perform functionality on each of those smaller batches and then appended the result essentially to a table that already exists so then if let's say another five seconds passes well have another batch perform functionality on it and append it another batch should perform some functionality on it append it and so on and so forth as our stream still exists so in the quick demo that I'll show you here we can actually do live or real-time sentiment analysis so still using don''t for pachi spoke with ml dotnet now we can see that if I type a string into a console let's say we can determine in real time if it represents a positive or a negative sentiment so let's go ahead and take a quick look at that demo okay so I still have the Microsoft ml at Microsoft spark new get packages installed I start off creating a spark session but instead of reading into a data frame from a CSV or stored data now I'm doing something where I'm reading a stream and I have to set up the host and port information my stream is coming from and then I still use ml net with a UDF and I can still call the ml dotnet code in a very similar way and then finally as on like working with the data and displaying it I used something called a streaming query and I can go ahead and call write stream and determine that I want to write my stream to the console and instead of saying spark stop we can do query dot await termination so I go over here I can see here that I've set up a quick netcat terminal so just an easy way to read to or write from a network connection and for instance I could write something like I love spark and then in my other tab over here I've already built and run my dotnet for Apache spark app if I scroll down you can see that I've been working with my different batches here so every time I hit enter is considered a new batch and it determines in real time if my line was a positive or negative sentiment so you can see when I said I love spark it was considered true a positive sentiment so that's awesome we have real-time streaming working ok so a couple quick steps for how you can get started with dotnet for a Pachi spark we have if you go to the dotnet website so do T net slash spark you can go ahead and read even more about dotnet fair patchy spark you can go through a really neat getting started tutorial we have for you can get up and running with dotnet spark on your local machine in 10 minutes or less and you can also visit our Docs and see some other learning resources we have you can also visit our github so github.com slash dotnet slash spark so you can go ahead and view some of the documentation you can see how things are implemented you can participate in the open source community it with spark so thank you so much and now I guess we'll turn to questions fantastic so that was an amazing presentation Brigid now help me out because I am a little dense spark isn't like a database is it is it where the data is stored or is it a medium for transferring data over help me outs to set the context from these questions sure yeah so spark isn't a database so it's not where the data is stored so you'll already have your data stored in something like somewhere in Azure so maybe and Azure data leak storage or in a blob or something like that so spark is actually kind of like the framework or the tools that we can use to start analyzing that data so it allows us to read it in and to process it more quickly and to make different calls to it so that we can gain meaningful insights rather amazing so it's like that it's like the pipe then that takes data from any scenario and then moves it over is that right yeah that's a good way to think of it that's good so you mentioned a lot of stuff with with.net we didn't have dotnet isms for spark when did this start and if you were to point because you pointed out a tons of cool things but if you were to tell people one thing look at this first and see why it's powerful what would you suggest to them okay so we started this project we first started a towards the beginning of this year so I would say about in April even though we did have a predecessor to this project a few years ago but yeah I would say as of this year's when we actually started having like these awesome dotnet bindings to Apache spark and for something really awesome I would say if you check out that landing page though d-o-t net slash spark you can see really that you can really start processing terabytes and petabytes of data at a manageable scale so you don't have to spend like days and weeks and months and years processing all this data you can actually start gaining insights from it in like a matter of hours that's pretty cool does dotnet for Apache spark support support using f-sharp instead of c-sharp yes I believe it does so dotnet ecosystem in general fantastic well this is amazing and thank you so much Bridgette now here's a couple of things though before we go I want to remind everybody about I'm not finding it but the the actual tech challenges that are available let me get my might handing notes out here because they're they're pretty good and if you have more questions for Bridgette make sure you get them in make sure you participate in the technical treasure hunt it's happening all day tons of technical problems that you can solve maybe even do a little bit of code if you solve all of them you will get a ton of wonderful prizes it's pretty cool to go to dotnet comp front slash party that's what we're gonna if you go to Dinah comp front slash party sorry they're talking in my ear you'll be able to see all the cool things and there's a ton of things happening today make sure you go to the Apache spark coolness for net which i think is pretty cool we're gonna go to a commercial break here in a second but and after that we're gonna have more servers less with a Jeff Holland is going to talk about durable functions 2.0 server less actors orchestrations and a stateful functions