First, I want to say thank you for spending your Monday morning with us. It's a real pleasure for me to be here in Dublin today. It was 31 years ago in 1985 that Microsoft got its start here in Ireland. We were one of the first technology companies, as you may know, to recognize that by coming to Ireland we could not only plant our flag in a place with great talent, but we could work with people here to serve all of Europe. Little did we know that when we made that investment to put our first big data center here in Dublin, we would also eventually become involved in an issue that brings us back here this week and today to launch a new conversation. Little did we know that this data center in Ireland would lead to litigation that would involve a case we filed against our own government, litigation that would be followed by many people across Europe and around the world. Little did we imagine that when we started that case, it would wind its way through the courts, literally taking us years, but little did we know that if we persisted, we would find, as we did this past July, that we would actually win a case that, as Fortune Magazine said, created not just only a good week for Microsoft, but a great one for the cloud. We brought that case because we thought it involved an important principle. We brought that case because we thought it was important to establish that no government through its own unilateral legal process can reach into other people's e-mail located in other parts of the world. We brought that case because we thought it was important to stand up for people globally. And that in many ways is what we wanted to talk about throughout Europe this week. As a company we look at where technology is going, and we believe that we're, in fact, at the dawn of a new Industrial Revolution. And that's why we think it's important before this just continues to go forward at such a rapid pace to ask ourselves, how do we ensure that this cloud not only serves the fortunate few but actually serves the world as a whole? That's what led us to put together our best ideas and share it in a book that we're publishing today. We do this not with a sense that we have all the answers, because we know that we do not. No one has all the answers. But we think that there's certain tenets that will all be served well if we can recognize together. The first is that as promising as all these technologies are, the reality is that with opportunity comes challenges. There are challenges that we see in the world today that are linked to globalization, and they're also linked to technology. Technology is creating new jobs, but automation can challenge and even replace some traditional jobs. There are new challenges to privacy rights. There are new questions about income inequality. There are new opportunities but also challenges for people with disabilities. If we don't acknowledge explicitly at the outset that this new era creates both opportunity and challenge, the truth is we'll do less than what we should to make the most of the promise that lies ahead. We in short need to ensure that we move technology forward without leaving people behind. And that requires that we accept a sense of shared responsibility. Those of us who work in the tech sector, those of us who work in the private sector have an important role and responsibility, as do people across civil society, as do governments around the world. If we're going to come together and do what it takes to make the most of this new era, we need to appreciate what we each can contribute. But we need to work together with some clear goals. From our perspective as a company we see three sets of goals that I'll talk a little bit about today. In the book that we've published we've outlined 78 policy recommendations. And the really good news for all of you is I am not going to read all of them to you this morning. It starts with ensuring that this new era of cloud technology relies on a trusted cloud. As a company Satya articulated our principles about a year ago as he said that we need to focus on security, on privacy, on transparency, and building a cloud that ensures that people can rely on us to secure their compliance with law. It's about ensuring that the kinds of traditional protections that we've all had, not just for years, but for centuries for information that is stored on paper actually persists when we move our information to the cloud. To make that effective, we need to do several things societally and with governments. We need to keep information secure. We need technology that keeps information secure. We need government policy that keeps information secure. We need to address these problems in a way that both acknowledges and ensures that data can move across borders. And there are few places in the world where the global nature of this technology is more apparent than here in Ireland, when you see a company like Microsoft using our data center not only to serve the people of Ireland but to serve the people across Europe and in Africa and in the Middle East. That's why we were so focused as a company on the development of the new Privacy Shield between Europe and the United States. It's why we were the first company to stand up and say that we would certify that our data centers would comply with the new EU Privacy Shield, and it's why we were the first company to then go do the architectural work to make that certification effective Trust has really been in the headlines for three years now. But as we've thought about the issues, we've concluded that, in fact, our needs are a lot broader than trust alone. We need to build a cloud that's responsible as well. We need to build a cloud that respects and protects human rights, even amidst the equally important work to protect public safety. It requires new steps that we all take together to protect people online. It requires new steps that we think about the implications of a cloud-based era for the environment and for sustainability. That's why we as a company announced earlier this year new principles that would guide our work as we go forward. We committed ourselves every year to get better when it comes to our use of renewable energy. We committed to be more transparent. We shared with the world that today 44 percent of the electricity that flows into our data centers comes from renewable energy sources, either solar or wind or hydro. We committed that by 2018 that number will cross the 50 percent threshold, and early in the next decade it will cross the 60 percent threshold, and we will continue to get better and better from there. But we need government policy to focus on this as well. And given the promise of artificial intelligence, this too is an area where laws will need to be modernized. We need laws that promote the use of artificial intelligence, but we need laws that recognize the vital, ethical considerations that need to be at the heart of how we societally move AI forward in the future. If we do a good job in this area, it helps actually I think create the foundation for the third and perhaps the most important issue as well. You can argue that one of the defining issues of the year 2016 is, in fact, one of the defining issues of our time. How do we ensure that we can have innovation and economic growth that is genuinely inclusive. If one wants to have a healthy digital economy one needs to build a learning economy. One needs to start by investing in the skills that the next generation of people will need to succeed, skills like coding, fields like computer science. But one cannot stop with the next generation. We live in a time when many people of all generations are going to be called upon to learn new skills and sometimes find new jobs and even enter new careers. And we need to think about how technology and the private sector and civil society and government can come together to make that easier for people to pursue. Similarly, we need to ensure that this new technology benefits not only companies that are older and larger, but new companies and smaller businesses, as well. We need to ensure that the benefits of technology and economic growth are not confined to people who live in the major metropolitan areas, but that they actually do what they should, they reduce distances and barriers for people who are working in more rural areas. And as we think about inclusion, as we think about the needs of the young and old, as we think about people in cities and in towns, there's another vital group we need to think about, as well. It's people with disabilities. Did you know that there are 1.2 billion people in the world who have some type of disability? There are 300 million people in the world who are visually impaired. There are other people who have other forms of physical disability. There are people who suffer from some kind of cognitive disability. And yet what is amazing about every one of these people is the incredible potential they have to do great things in their lives. We simply must do our best work and we need to take new steps to come together with governments to ensure that technology advances the lives of this vital group. Technology can do amazing things. I think it should inspire us to raise our own ambition. I think we should aspire not only to contribute to scientific knowledge, but to find a cure, to create a day when people who are blind can use technology to see in new ways, when people who cannot walk can benefit from technology to stand on their own two feet. Fundamentally as a company when we think about what this fourth industrial revolution can offer, when we think about the clear goals that come out of creating a cloud that is trusted and responsible and inclusive we know this. Progress will come not by building higher walls, but by constructing broader bridges. We need to construct the kinds of bridges not only between nations, but to the future, that will be broad enough for everyone to move forward. Thank you very much.
Wednesday, 23 October 2024
Behind the Scenes of Bloom Open Space
I assume that at some level all technologies have that capacity to make something that we never imagined before. I'm always trying to work with things to find out what that is. What's the secret potential of this technology? What can you do now that you could never have dreamed of doing before? Bloom came out with a little experiment where it was just creating a shape on the screen, having it expand and disappear. Really, it was just a very simple tech demo. There's something quite charming about it quite instantly. The Music X Tech program is about empowering artists to create and experience music in new ways. With Peter and Brian, we had the opportunity to take their application, Bloom, into mixed reality. The first big difference is scale. To be in the middle of it is an entirely different experience. This is my first toe in the water with augmented reality. I like this thing of still being in the world with another experience added to it. I think it's a bit of a sense of magic actually. Being able to actually touch something in the air and see it become a real thing in front of your eyes. I know how the technology works, but I still find myself running off to these things in the air. Trying to touch them. They seem so compelling and solid. It really is truly mixed reality. It's blurring the lines as to where one starts and one ends. One of the things I've always liked with Bloom is that it gave anyone, regardless of any experience of music, the ability to feel what it's like to be an improviser. In this situation, you are not on your own. So, you are in a sort of orchestra here, rather than being a soloist, when you have the app in your hand. It is quite magical actually when the two of you start creating things in the space around you. I don't think anyone's experienced anything like that before, actually.
Azure Speech Service Vision Keynote Demo Microsoft Build 2019
ONE SERVICE I WANTED TO SHOWCASE TODAY IS THE AZURE SPEECH SERVICE. NOT ONLY IS THE SPEECH SERVICE GETTING BETTER AND BETTER WHEN IT COMES TO SPEECH RECOGNITION, IN FACT WHAT YOU’LL SEE IN THIS DEMO, IS EVEN FOR COMMODITY HARDWARE, TO REPLACE ANY MICROARRAY SETUP SO THAT YOUR SPEECH RECOGNITION IS WORD CLASS. THE MOST INTERESTING THING WHEN YOU COMBINE SPEECH RECOGNITION WITH LANGUAGE MODELS THAT ARE SPECIFIC TO YOUR ORGANIZATIONAL DATA, YOU CAN START PICKING UP ALL THE JARGON. SO IMAGINE A TRANSCRIPT THAT GETS CREATED THAT HAS THE ABILITY TO UNDERSTAND THE LOCAL JARGON THAT'S SPECIFIC TO YOUR ORGANIZATION, YOUR INDUSTRY, THAT WAY MAKING THE TRANSCRIPT THAT MUCH MORE USEFUL. LET'S THROW IT TO OUR TEAM OUT ON THE GALLERY TO SHOW YOU SPEECH TRANSLATION AND TRIPS. THE PROTOTYPE DEVICE YOU CONNECTED TO A CLOUD SERVICE THAT PROVIDED LIVE TRANSCRIPTION AND TRANSLATION. WE ARE PROUD TO ANNOUNCE TODAY THAT WE'RE MAKING THE CONVERSATIONS TRANSCRIPTION CAPABILITY WITHIN AZURE SPEECH SERVICES AVAILABLE AS A PREVIEW RELEASE. COME ON, LET ME SHOW YOU. YOU MIGHT ALSO REMEMBER THIS HARDWARE FROM LAST YEAR, WHICH WE'RE MAKING AVAILABLE AS DEVELOPER DEVICE KIT. TODAY, MY COLLEAGUES AND I ARE GOING TO GIVE YOU A DEMO OF NEW RESEARCH THAT WE BELIEVE WILL MAKE MEETING TRANSCRIPTIONS MORE EASILY AVAILABLE TO EVERYONE IN THE FUTURE. WE ARE GOING TO SHOW YOU THIS DEMO USING JUST THE MICROPHONES BUILT INTO THIS LAPTOP AND THESE TWO SMARTPHONES WE HAVE IN FRONT OF US. WITH THESE, WE CREATE A MICROPHONE ARRAY IN THE CLOUD THAT ENABLES AZURE SPEECH SERVICES TO PROVIDE ACCURATE, IN-PERSON MEETING TRANSCRIPTIONS EVEN WITHOUT A SPECIAL MEETING DEVICE. >> ALSO, YOU WILL NOTICE THAT I DIDN'T BRING MY PHONE, BUT IT CAN STILL RECOGNIZE MY VOICE AND CORRECTLY IDENTIFY ME, BECAUSE I'VE GIVEN IT PERMISSION TO USE MY VOICEPRINT TO TRANSCRIBE WHAT I SAY. THE SECOND THING WE'RE GOING TO SHOW YOU IS THAT THE LANGUAGE MODEL OF THE AZURE SPEECH SERVICE CAN BE TRAINED ON THE DATA IN YOUR COMPANY'S MICROSOFT 365 TENANT SO IT CAN LEARN THE UNIQUE VOCABULARY OF YOUR INDUSTRY OR COMPANY. THIS IS AVAILABLE IN PRIVATE PREVIEW. SO BASICALLY, FOR THE NEXT TWO MINUTES, WE'RE GOING TO HAVE A RAP BATTLE OF SORTS. BUT FOR ALL OF US GEEKS HERE IN THE ROOM. HEIKO IS A PRINCIPAL PM ON THE SPEECH TEAM AND HE'S GOING TO GIVE US AN EXAMPLE OF SOME DEV SPEAK. YUSEF IS IN HEALTH CARE MARKETING AND HE'S GOING TO DAZZLE US WITH HEALTH CARE TECH JARGON. SO WHILE THEY SPEAK, I ENCOURAGE YOU TO FOLLOW ALONG WITH THE TRANSCRIPT ON THE SCREEN SO YOU CAN SEE JUST HOW POWERFUL THIS SERVICE IS. HEIKO, TAKE US AWAY. >> AZURE SPEECH SERVICES ARE BUILT WITH VMS RUNNING ON AZURE HYPERVISORS USING UBUNTU BASED DOCKER CONTAINERS THAT ARE ORCHESTRATED WITH THE AZURE KUBERNETES SERVICE. AZURE SPEECH SERVICES ENABLE A VARIETY OF TECHNICAL CAPABILITIES INCLUDING ASR, NEURAL TTS, MICROSOFT TRANSLATOR AND RELATED CUSTOM SERVICES. YOU CAN ACCESS THESE USING YOUR FAVORITE PROGRAMMING LANGUAGE, SUCH AS JAVA, JAVASCRIPT, NODE.JS, C++, OR C# AND OTHERS. THE BAR HAS BEEN SET. OKAY. NOW IT'S YOUR TURN TO GIVE US A BIT OF THIS HEALTH CARE JARGON. >> MICROSOFT TEAMS CAN PROVIDE EHR INTEGRATION THROUGH ISV VENDORS INCLUDING INFOR CLOVERLEAF, REDOX AND OTHERS VIA THE HL7 FHIR STANDARD. HL7 FHIR IS HIPAA, MARS-E AND GDPR COMPLIANT AND IS BASED ON A MODERN TECHNOLOGY INCLUDING HTTPS AND RESTFUL PROTOCOLS AS WELL AS EXTENSIBLE APIs. THE FHIR OPEN SOURCE COMMUNITY MAKES THEIR SOURCE AVAILABLE ON GITHUB AND THE MICROSOFT TEAMS FHIR IMPLEMENTATION ASK ALSO ALIGNED WITH PROJECT ARGONAUT AND FOLLOWS THE U.S. CORE PROFILES FOR ALL THE FHIR RESOURCES IT CONSUMES. WELL, THAT WAS FUN. I'M GOING TO CALL THAT A DRAW. WHILE WE'RE GOING A BIT OVERBOARD, WE UNDERSTAND THIS IS INCREDIBLY IMPORTANT SO THAT EVERY COMPANY WITH THEIR SPECIFIC JARGON CAN HAVE ACCURATE TRAN DESCRIPTIONS. WE'RE EXCITED ABOUT WHERE THIS WORK WILL TAKE US AND OUR FUTURE AMBITION IS TO ENABLE TRANSCRIPTIONS FOR ANYONE ANYWHERE AT ANY TIME.
Automated Testing of Shader Code
so without further Ado I will introduce the first Speaker Keith Stockdale who's going to be telling us about automated testing of Shader code welcome Keith Hello nice to be here hello all right take it away all right then okay so this is automated testing of Sher code at rare my name is Kei doil and I'm a senior software engineer at rare and I've given a this a talk on this topic um just at this year's GDC but this is going to have a bit more of a C++ spin on it so a little bit about me first um this is me in case my audio my uh video quality is a bit poor and if you're wondering the accents from Northern Ireland that's where I'm originally from and I've been working at rare uh for the last six and a half years on the engine and rendering teams I quite Bas I basically my whole job is just writing things on the GPU that probably shouldn't be r on the GPU and at my time at rare I have worked on CAF thieves seaf thieves is a game that we launched initially in 2018 it is our first onreal engine game and it is based on a very heavily modified onreal engine 4.10 so quite old and I said we initially launched in 2018 because we have been consistently putting out updates um throughout the years just last week we released a major update and today in fact fact we released for the first time on PS5 which we're all very happy about um so still Lots going on with CF th still very much in development and alongside the development of CF thieves we also have ever wild and ever wild is our upand cominging game at rare and it's a little bit different with CF thieves in that we are trying to stay up to date with the latest and greatest Unreal Engine So currently we're on Unreal Engine 5.2 and we're in the process of of upgrading to 5.3 okay with all that out of the way the problem and the problem that we have at rare is that we are writing more GPU code and this GP code is not your typical GP code that I put pixels it's more general purpose simulation code so just in C of Thieves we have two very prominent GPU part um simulations so the first is our fft water simulation so you can't really have a game called CF THS while having having a very good looking C so that has all been simulated on the GPU and then we also have our GPU particle simulation so anything to do with fire or Sparks fireworks smoke bubbles water spray all of that type of stuff that's all being simulated on the GPU and with all of this general purpose code being executed on the GPU we like to test it and the reason for that is because we as a studio love adoma testing and we love it so much that we have given a number of talks on it this one included um one particular talk was automated testing of gameplay features and see of Thieves by Rob Mella way back in 2019 at GDC this these are some links if you want to go to it if a QR code's easier there's a QR code otherwise some words um but this is the mean slid that I felt was the most prominent thing on his presentation where he's talking about all the benefits that we have reaped from autom testing so we've got things like um redu time to verify bills so we get our content out to players much quicker um lower bug count with comparison to previous rare titles and observably reduced crunch as well so these are all the reasons why we like to do it but Rob here was talking about testing of gameplay features but I'm here to talk about testing of Sher code and if you don't know how to get started with testing Sher code this is a fantastic article by Bart ronsky um it's called how not to test Graphics algorithm and in it he lays out two main ways of testing Graphics algorithms the first is screenshot testing which is what's we've been doing at rare since before I started back in 2015 um and then the second approach is what I'm going to be talking about today which is sheer unit testing so our unit testing framework for sheaters we just called it rare sheeter test and it is a testing framework for hlsl codes and if you don't know what hlsl is it stands for high level shoting Lang language and it is the shading language that all directex applications use as their shading language so if it's a directex app on the PC or if it's an Xbox game chances are the Shader code will be written in hlsl our unit tests are written in hsl 2015 so quite an old version and the main reason for that is because our testing framework is being used for both CF thieves and ever while so 2015 has a common feature set there so we stick to that and it runs on top of an unreal testing framework so this is not a standalone testing framework by all means it is just adding on top of what unreal already gives us um so that we can use all the nice tooling that unreal provides as well as letting us test our Sher code an example of the testing framework that we are building upon is CQ test which stands for code quality test um and because I know in the audience there are probably at most one or two people who actually know what this is um it's an Unreal Engine testing framework like I said and it's available from Unreal Engine 5.3 and on so it's fairly recent um it's quite comparable to Microsoft's unit testing framework and if you haven't seen it before um this is what's a testing Suite might look like with CQ test so you have your test class macro at the top and that defines the actual name of the test suite and then you have any number of test methods within it which are your actual test that run and within your test method method you'll have various assertions basically just making sure all of your code works as you expect it to and then in addition to test methods you also have setup methods called before each so these methods will be called before each test method as well as after each so that'll be called after each test method to do some cleanup now um before I go any further I want to just like make it clear what our coding environment is um Onre engine 5.2 currently so we are upgrading soon but very much still on 5.2 and that means we're in C++ 17 land so unfortunately we don't get the nice C++ 20 just yet um that will be coming with 5.3 so another reason why I'm very excited about 53 and we are using HL C 2015 as I said um now throughout the course of this talk I'll be showing C++ code as well as hlsl code so if you see the C++ symbol at the top right it be C++ code on the screen and if you see the hlsl doggo you will be seeing hlsl code okay then on with actually showing how to write a sh unit test in our testing framework so HL C Dogo at the top right this is hlsl so it starts off with some includes so at the minute it kind of looks like C++ um with it with the includes the first include is the header that all unreal shaders must um include so there's nothing interesting there then we have the actual testing framework I'll be talking a little bit more about that header later on and then we have our system under test header which is in this case the water particle framework and then we have the first clue that this is not in fact C++ this is a Shader so this is an attribute that you tag on to the entry function of a compute Shader to tell it how many threads to run per thread group so this is a very simple it is running a single thread so not very good use of your GPU but we don't really need it to be very fast we just need it to be correct and then we have the entry function and we name the entry function the same name is the test this just makes it easier to connect up the hlsl side to the C++ side and at rare we like to use the given when then naming scheme um for writing tests so in this case we have got giving moving water exists so we construct some water with our known velocity when the particle spawn on the surface so in this case we're passing in water into this spawn function and it returns us a water particle then the velocity is inherited so we assert that the spawn particles velocity is in fact the velocity that we expect it to be and that's quite that's basically it so you've got your standard given when then um that you might see in any other testing framework with any other um language now this is a Sher so we need some C++ code to dispatch the code itself self and we try to make sure this is as minimal boiler plate as possible so we start off with the header of the testing Suite which looks like CQ test test class macro except it has an extra argument and that argument is the path to the actual Sher test themselves and then we have any number of um test methods so again it's just giving the method and giving the name of it so you can search that name and you'll find the C++ code as well as the HL cell code in your code base now our the Shader I just showed is extremely extremely simple it has no external parameters to bind or anything like that there it'll just run with no nothing to provide in the C++ side so all we need to do is call run compute Shater test with the dispatch configuration so if you're familiar with direct xit all this will be equivalent to calling dispatch 111 right so it's um dispatching a single thread group on your GPU so it's a single thread group and that single thread group will be running a single thread so not very fast not very not making use of your GPU in any way your perform but that's fine because we are the main thing we're consider we're concerned about here is testing our code on the actual Hardware that it would be running on during your game now at this point I want to talk about the goals of the framework and initially I was going to talk about all of them but um just for the sake of time I'm only going to talk about two the first is we want our test to be fully automated avilable so I mentioned that we have screenshot testing for CF thieves and we've had that for ages um but the problem with it is it's not fully on amable we do require our test team to verify um screenshots from time to time to make sure everything is still right however that's not ideal we'd rather them just get a green tick for happiness and red tick for sadness um so to demonstrate that this is actually something we can do now um we have this feature in CF thieves which is our calm water system and what it will do is as you get closer to an island it will smooth out the fft water simulation um so in the distance there you can see it's all very choppy then as as you get closer it Smooths out quite considerably and we have a full test Suite just for this feature and the nice thing about building on top of our testing framework in unreal is that we get all the nice unreal um test running capabilities as well and Reporting so we can just open this up in our session front end like anything else um hit run on all of them so there's 29 tests and they run lightning quick so there are 29 tests here and they're running at in about 0.3 seconds so about 10 milliseconds a test and that is lightning quick especially if you consider how slow screenshot test can take um so yep fully automatable we're happy with this and then right to goal number five error driven development and what I mean by this is if you get an error as a developer in the while rating the test in the framework that error should tell you how to proceed so it tells you what you did wrong and how to fix it now to explain to give a demonstration of that um we have a slightly more complex sheeter test here um so instead of just passing in the dispatch configuration we are passing in a setup function and that within that setup function you can set some parameters on the actual Shader itself I I'll go into more detail on this later on the talk but suffice to say developer writes this it looks right um but you'll get a compilation there and it's not a massive big Litany of template errors it's actually a single line which says expected a single parameter of f sheer test context reference so it is very very common very easy to just miss out an ampersound right how often have we seen like performance regressions because we accidentally um forgot theant in a sorting functor right um so we we do as much static reflection at API boundaries of functions that are templated as much static reflection as the language currently allows us we're not quite at the land of static reflection proper yet but I'm very hopeful for C++ 26 to get us there um so yeah as much as we can and then we static a search and give nice error messages now you'll notice here that I'm not able to actually format the message I'm not able to say what they did wrong I can just tell them what they need to change something to or to make sure that the parameter is a reference rather than just passing by value and that's because we don't really have user generated static assert mesages right it's currently not possible these have to be string literals you can't do stood format or anything like that with them um at least it's not possible for us um depending on what your development environment is it might be possible for you right now um because it got accepted into C++ 26 very happy about that we'll get there soon maybe possibly um and it's mented in K 17 at a minute and I think it'll be making its way to gcc1 14 um still no word on msvc yet which is the compiler that we use um but some people might be able to use it and I'm very excited for them um these are the links to the discussion art basically seeing the progression of it and also a link to the paper um it's all of the points that quarantine brought up in his paper are basically all the things that I want so yeah rational solid there um definitely worth a read now binding extra sheeter resources in the previous example I talked about binding extra parameters um and we do a lot of static reflection whenever we're binding extra sheeter resources um and to give more of an example of what that actually means is if we have this test Shater here called test with extra parameters very imaginative name and we have these parameters themselves so we've got a biess buffer and we've got an inst float so if you've got a non-static global variable in HL cell code that is a sheeter binding so that is an object that you can bind to on the C++ side to pass in information to be used within your Shater and here we're just asserting on them essentially so to be able to do this you need to a littleit of extra work on the C++ side and the nice thing about unreal is it provides some nice macros for us to do that so if you've written any rendering code in onreal engine you'll have seen this macro here so begin Sher parameter struct we give it a name and then we just pass in all of the parameters that we are going to be using so in this case it's our bodess buffer our uent and our float and we end it off with an end Shader parameter struct however this is wrong and we'll get a nice compilation error here out of this it will first say the test Shater that this test uses has a parameter struct which does not contain the Bas Shater parameters it might shock it might not Shock you to believe that um the test framework itself requires extra parameters be bind because whenever we are running a Sears what that's doing is logging some information to a buffer behind the scenes and then we read back that on the GPU on the CPU um so it needs its own parameters and whenever we just use this begin shter parameter struct for ours we're not including the ones that are required for the test so how do we get around that well thankfully the error message continues says to fix this please use begin CQ test sheeter parameter struct so it's telling you that the thing you need to change so change begin sh parameter struct to begin CQ test sh parameter struct and that will fix things for you and then it continues and says the line below this one is the test that uses this inal theator so we're telling the developer that it is the sh it is the um test that is signified in the next line of the error message that is the problem so we're telling people where the issue is what they did wrong and how to fix it so this is what I strive for in this framework just to make it as easy as possible for people to write tests without having to go diving into the documentation of somewhere else um so all of this is predicated all the static reflection is predicated on being able to detect invalid expressions and in C++ 20 this is quite trivial um you just use Concepts so you have a concept here I've just called it C has sheeter test parameter strict and we have our requires Clause here and within our requires Clause we will Define some constraints and essentially if you pass in a t uh to this concept and these constraints hold so they are valid constraints with this type T then the concept will evalue to true and everybody is happy so you could just static aert on this because the evaluation of a concept is a Boolean expression and if everything is fine turns true everything's good if it returns false then the whole concept framework gives compiler writers the opportunity to provide good helpful um compilation errors however I did say that we don't have the luxury of C++ 20 just yet um so what people typically do in this situation is lean back on void T um but there's another way that I want to talk about today which is what I would call emulating C++ Concepts so it's trying its best to make it look like it if you squint hard enough so we start off with a non-templated struct called C Shader test parameter struct so it's the same name and then we have a templated requires function declaration and in this we're just passing in a type T just like with our concept so if you start squinting it's starting to look almost there and the return type of this requires Clause is a decal type and the deal type will evaluate to this expression so it's our constraints so yeah if you squint hard enough you will see that this is somewhat concept e like um one of the difference here is that we're using the comma operator rather than having semicolons and the reason we're using the Comm operator here is because we're sort of abusing it a bit um the result of the Comm operator is the left um upper hand and the right upper hand and then it Returns the right upper hand but if any one of these Expressions is ill formed then the whole expression is ill formed therefore the type that's returned by The Deco type expression is ill formed so the whole requires function is thrown out it is ill formed it does not exist SP kicks in and then to make use of this in a static assert you don't obviously get the automatic um conversion to an operate to the operator bull like you do with Concepts we instead pass this concept in to t- models which is the driver of this feature and you can see here this is our lovely error message whenever you get this um now I'm going to go through the actual implementation of this because I haven't actually seen it anywhere else although then again I haven't really dived into too many um temp meta programming libraries but I thought it was pretty neat so I'll talk about it so it starts off like any other uh templated meta function it's a a template struct called t- models and it has a function here which will return a reference to a character array of of two elements so two characters if the concepts requires function is well formed with the arguments passed into it so if everything's well formed everything passes then this function will exist if it doesn't resolve if it's if it's if the requires Clause is ill formed then this function will be created instead and the difference here is that it returns a a reference to an array of one character so there's a size difference here and you can probably see where I'm going with this the output of this expression is then um B asserting that if you get if the result of resolve is of size two if it's of size two everything's well formed everything is good and it goes even further you can do a little bit of I'm not going to call subsumption because subsumption is very complicated and it's um it's got It's got well- defined rules so I'll say inheriting so you can inherit concept using the refines Clause um and that's about it really and I just want to talk a little bit about a future goal I have with this framework before I end um and that is hlsl 2021 I mentioned the start that we are stuck on 2015 for now but I really want us to get to 2021 and the main reason for that is we get templates and I love me some templates but the main reason why templates is a great thing is because it allows us to program to an interface rather than to types and if you've written any amount of aom um testing in the in the past you'll know that dependency injection is your best friend right um and with templates we get static polymorphism which is nice way of getting dependency injection somewhat into the language and it allows us to write much more expressive code and um it is easier to write and share hlsl libraries and but the problem is it's a breaking change to go from 2018 to 2021 so um I'm going to fight that battle but I'm not I'm not sure how I'm not um that sure how far I'll get with that one um but if you do want to see what a testing framework in hsl 2021 looks like look no further than mine so I have uh an open an open source project on GitHub any go and see it um and it's running Sher tests on GitHub actions and all of that fun stuff so it's got a full CI and everything going running Shater code tests um and the nice thing about it is it is well the the main goal of it is just to experiment and see what a testing framework can look like in this language and I'm heavily basing everything off catch 2 and by that means by and by that I mean I just I'm just taking every feature that I can for me catch to I really like the the framework and I want to see how how far you can get to it in shoters um but yeah that's basically it um so I just want to leave you with an a nice thing I like to say Shater code is code please test it and that's about it so if there is any questions or if there's even time for questions I can answer them hey yeah thanks very much it was really interesting uh we have a few questions and a few minutes for them uh the first question is from V OA says asks um Can this code be used with GSL glsl yes glsl theyan I see no reason not to it's just I haven't used glsl since University days I'm I'm sorry I've been a rare too long another question from Leslie Li is how does how do assertions work on the GPU um so assertions so I I go a lot more in depth into this into in my GDC talk but effectively at the bottom of an assertion if an assertion fails it'll write some data to a buffer um with some sort of information on it so um the type of data that failed the data itself and what kind of assert that fails and then we read that back on the CPU and then assert on that data um using the underlying testing framework that we're building upon um so yeah it's just it's all it's all big lie around a readback buffer essentially cool um question from bsdb is screenshot testing still a thing why not just grab the state of the rendering pipeline just before all the pixels are colored uh that would be huge screenshot testing is still very much a thing and it's something that we rely on quite a lot at rare especially on CF thieves it has been extremely beneficial throughout the development of CF thieves um for us like it has shown up so much stuff um for us and has caught a lot of things before getting to release so may I can speak for us at least it is very much a thing for us I don't know about other Studios cool and one from D candy um can this be extended to glsl which you also already answered but also spear V open GL or Vulcan yeah that is something I want to I sort of want to try it because yeah you can't you can compile uh HL C down to spear V and then run it in Vulcan and that' be something really cool I have a I have a pipe dream of getting that to run on my Raspberry Pi 5 like I think that'd be really cool to get that to run just like on a like on my own Rosary Pi server um something I want to try that sounds cool and uh a talk sorry a question from Peter nmo for me is uh speaking of catch 2 in the future will there be more support for testing Frameworks in Visual Studio yes we're still very much working on uh other test T in Frameworks um for the test Explorer in in visual studio and uh crossplatform testing uh remote unit testing all that kind of stuff so yeah keep your eyes out for more in that space uh and then I think we've got time for one last question from Michael voling who asks have you all had issues with developers forgetting to register tests that's my biggest fear with manually registered tests uh these are not manually registered as soon as so as soon as you write that uh macro test class it's that's it in it's a it's a static registration on yeah so that that can't happen you can you can opt out so like you can say this test should not run on server for example and if it's a client only thing um but it's Auto registered yeah great all right well thank you so much for joining us Keith and uh hopefully see you around see you
Monday, 21 October 2024
ASP NET Core Health Checks
thank you so I think I'm life yes you hopefully cool so let's start yeah I will talk about the asp.net core health checks today but first I will quickly introduce myself so my name is Sharon Gooch I'm from southern Germany I'm a Microsoft MVP for several years five years I think I'm blogging writing a lot leading it on a user group since years and speaking at several community events and meetups and I also started a stream on Twitch not regularly but from time to time and I'm a software engineer at the U in Basel so I'm working Switzerland living in in southern Germany hopefully the sound is good last turn last year the first 10 minutes were awful it's it's going well this time so I have a quick agenda oh I need to quickly close not close minimize Skype window as well I see two versions of the agenda currently so yeah I will quickly quickly introduce asp.net core health checks and then I will show a basic implementation of the health checks and there are also a lot of checks in the community so we'll have a quick look there and how to visualize the health of subsystems of an application and if it's some time left I will show you quickly on an enhanced scenario so I built something with docker containers to show how this could look like so all the sources I will show today are will be available on github so the repo already exists but I will push it afterwards so what's next exactly about the health checks what are the health checks so that health checks are kind of a framework inside of the ASP and framework to check the health of application subsystems maybe databases or services which are not inside the application or you can also check your your system you can check memory you can check your space of your hard drive whatever you want to check you can also handle the health of your subsystems we get a feedback and the checks give you the state of your subsystems and you can handle them we can show the states in your weekly application and you can provide an endpoint to show the information about the health of your current application and of the subsystems of the current application so the namespace is Microsoft add extensions that diagnostic start health checks you'll find all the stuff here and yeah a little bit of history so I'm was pretty much involved into health checks so the development started at 2016 I think I'm not sure when exactly but during the MVP summit in 2016 LAN Condren asked via Twitter if there's anyone who want to attend the asp.net hackathon and help him through write checks for the asp.net health checks the first version I think it was a draft at that time so Damien said yes I said yes and we met Glenn Conner in an Andhra nurse at the asp.net hackathon and started hacking some checks then it moved through some repositories until it makes it into the product so it is in the product still in speed on that code 2.2 I think what can you do with health checks you know maybe mention it it is monitoring the health of parts of your application or yours or subsystems of application imagine it databases some Web API is G RPC service is any other services or the environment of application you can check memory performance drive space and so on and you can provide the health state of your application including the health of the subsystems you want to check so you get an endpoint that that tells you about the health of the whole system the entire system I think yeah that's almost about the theory why should you use the health checks yeah you can handle unhealthy states in the application you can maybe provide information to the users that some of the subsystems or databases aren't available currently you can keep you a partially running in case of an unhealthy state of an subsystem maybe there's yeah a specific database not running and you can keep your application partially running without this database for example or you can tell the users about the unhealthy state and so on I can switch to alternative subsystems in case of an unhealthy state of this specific system subsystem maybe you have a different one and what I didn't mention here in the slide is you can provide a an end point I think it's the exactly same endpoint to the what is called Essure a load balancer thing as I can connect your Asia environment to this end point and the load billets I can is able to to switch the applications or they the switch between the instances of your application okay let's share some code and this section I will demo you a simple implementation of health checks so this is the kind of agenda of this section just created a spill don't care a SP donate core MVC application using the.net CLI I already did this then we will configure some basic simple health checks without any logic in it we will implement a custom health check and we will use the health health check service to to curl the checks and to to visualize the states of the subsystems this is the basic configuration of the startup of your SP donut corportation you can add the health checks services and the configure service method and you need to provide the the endpoint I mentioned this is the endpoint which provides the state of your application including the subsystems so we say abuse health checks is the mom exactly I will switch to the console hopefully the font size is okay we can increase it a little bit and I'll open yes code and go to the startup and this is common asp.net MVC application already added the health checks here I'll check your services and I added the endpoint this is a little different than in the slides because in asp.net core 3.0 we have the new endpoint routing and the middleware which provides routes are now registered here with map something map controller route or map health checks this is the endpoint I mentioned so this is the basic configuration just to add health checks or just to have the possibility to bad health checks so now I want to add a few health checks so this is a way to add the liquid based Orlando based health check pretty simple this doesn't add us nothing here it just outputs unhealthy state so we can say add check yeah I'm sum over I to you you can do it the delegate way or we can add an health check instance we don't see this later we can do it the generic way to add an health health check type here Oh as well with thee with a funk we get a health check result so we return the result that's that so that's one I can also return an unhealthy States or some options some things to return we can return healthy unhealthy or something but in between degrade it so we leave this and I have another quick sample which is more useful I think it is a check to ping a subsystem okay this one starts a ping oops wrong keys here I did you know that you can press windows start to add emojis here yeah what works okay we create a new ping I said send the ping to localhost because it's currently it's hard to find a service on Azure or somewhere which responds to a ping I think all the edges services don't respond so I can't ping dead confident nets are also kind of ping my own block and so on so I need to find find a host to ping so if it's not a success state then I will return unhealthy if this needs more than 100 milliseconds I think yes then it's degraded it's pretty slow then and if it's all fine I will return and healthy state so in case of errors I will also return an unhealthy state I think that's with the delegate a checks but we can also create more your more complex checks using classes so it's this one you can create an eye health check so I need to find the interface it's not implemented correctly so I need to get this one the cancellation token and now it's implemented so it's almost similar to their to the lambda expressions here so I in this so we will return a health check result and get a context and we get a cancellation token so this is more complex way we can check the state of something so this is just an example I set it to true so it's healthy and if it's if the subsystem is healthy or we return our healthy state and otherwise unhealthy state so here this line you can put your checks whatever you want to check I will show you a more useful sample later okay so we need to add this you say add check I'll use that generic way it's so I don't need to add a name next ups let's get be null I think and here we need to add some tax a different keyboard layout here so now it's working so I'm switching between German and US American layout okay so this should work but I this should work if I call the endpoint let's try it I press ofin the five also the language is switching between German and English I'll place only if something bad with the with a stream font size audio and so on okay if this should work buzzer OH open up in the wrong window so yeah yes so it's running if I press f5 um I see something's going on here and the end and wash think it's unhealthy yep it's unhealthy the complete application is unhealthy because of this here if I switch this to the healthy state I've said it okay I'm to restart it it will be healthy so this is the endpoint okay you can't extend the endpoint a little bit so I need to find the snippet for this yep there's some options we need to set and configure and we can have this lines here say predicated true say response writer s d-u-i response writer to find the using so this isn't there okay let's check the namespace check options response my turn okay let's remove this I was sure in the next day more than okay that's about the one endpoint but what about visualizing so to visualize the the health of the epic of your application now I can use their health check service and you can call all the checks on demand and display reserve and the results in your view so to do this I'll go to the home controller and inject the health check service so like this this is already in the DI container so I can use it wherever we want and I'm going to add a new view so I also need to add the views here so just copy the privacy and call it health and put some code in it so okay so we use the diagnostic self checks here our model is the health report and then we write out the the status of the models of the model is the health report we return this from the control action we can display the duration of all the checks and we can iterate through the checks and display the state of all the checks we have okay let's change this back to unhealthy turn a better message and restarted so paid there so we haven't created a link for the health I will do it so I think I did something wrong at the end point end point and if you do have the same URL so I will rename this end point no I'm wrong I did it wrong so I shouldn't have changed the the end point here it should be under home health exactly so that's it so the state is unhealthy yeah needed five milliseconds seconds or milliseconds than that so the foo is healthy the bar is not healthy healthy and we have the ping here you should have set the name and we have the simple health check so the ping is healthy don't we have a name here tag [Music] should be pink they don't see it yet I don't know why anyway so it's working we have the the health state of our sub substance here and we can also add an indicator to the page to see if the entire application is healthy or not so to do this open the layout see is HTML pennants injection here so you can do a dependency injection in your view I inject the health check service and I use it in our code block let's put it here so we await that check health async armed with this we get the health health state so I think this is a health state response our health result so as you can see it here this is a health report okay that we have the same in the layout and we can check the states here so as SATA background come current in case the state is unhealthy degraded or I don't set it in case it's all healthy so and I'm able to set it here and let's see what happens I need to repeal it I think yep so let's close this one now the head is red and unhealthy and if you changed this and let's say this is decorate it to rebuild it again it should be orange them or whatever set here orange yes so under and healthy state doesn't have a background color so it's it's white okay do we have any questions until yet if not that will continue with the slides out there and let's have a quick look into what the community did so what was expected after I had another look into the health checks health checks um a post playing around with the health checks for a year or so uh didn't had a look until this time this period and after had another look expected some health checks created by Microsoft but there are almost no health checks the reason is that it's pretty simple to implement some and the other reason is that the community already created a lot of health checks so if you see this slide yeah if you entered interested go to this repository that's better it's a burial I'll pronounce it right he created the asp.net core diagnostic health checks package or packages on the right side let me see a list of nougat packages available so you can check the system cat Network sequel server MongoDB and so on ready event store secret light a lot of other stuff also um Amazon Web Services can't be checked and fire a signal or and so on Kuban eighties and I think the list will increase a lot so this project is port from beat pulse and it also provides UI features so well show you in there more enhanced steam off so please have a look also you some of these packages to check sequel server or or Seco light and so on um also try to check circulate I think this doesn't work currently I need to check it again maybe I did something wrong and if not I will a pricing issue you also it wasn't possible to connect to and secure light database you know inside the docker container that's didn't work okay so about the health check your ISO also ends of this project they created a pretty nice health check UI so you can add it with with this service registration and with these middlewares here and this also provides another endpoint which is specific for this UI and it also provides health checks UI endpoint with them with a nice HTML UI so let's have a look what I did for this demonstration I created something like dummy weather statistics application downloaded a lot of weather data from Washington State I have implemented four different with the station spree mutton Cedar Lake and Monroe and a health monitoring app and what's missing currently so where the data visualization app but this is not important here and it's all running on docker I'll close this one here and go to the console this one oops openly is coached there so these are six applications this one doesn't count it's not implemented yet at at a common library to share classes also shared the the dbcontext this is this is the health monitoring app and we have the four different weather stations here I'll go through the f qor to connect to the secret light database so this is all wired together with docker compose this is the visualization of the health health visualization export some ports it depends on the day and therefore stations and also need to link the docker containers together it was talking it working stuff here and I created four different docker files per container one to run so so let's quickly look here this is the configure services of the of the check visualization app so I created the dummy foo health check just to check whether it's working or not and edit a database context check I think this is the only health check provided by Microsoft the only finished health check then added a check for URI so this is called your group it's in the URI package you can set your I want to check give a name like with the other ones you need to tack this or it make sense to tag this let me to filter it later on so I created for ping checks and for data checks there's this one for checks only check whether the services are available or not and the other one are really loading data and check whether there's a result or not so I also added the health checks UI and also set it to the to the after the apps here so this is the health checks this is what I wanted to show in the previous demo you need to define a different response writer and with this you can enhance your own endpoint to add some more information I will show it quickly and I need to add the hell check CIA middleware so I don't want to start it here with Mia's code I used the console one and run docker come powers up it's all compiled and it only needs to start so all the services are available so we see the name docker container here tennis here this one is the monitoring app and the other ones are the stations so it's called 5,000 that's not HTTP and it's running this one is just a single Web API application and this is our endpoint we extend it so now we get a JSON result we get this state here it's not a longer a simple text and we get a lot of more informations so the duration we get all the entries these are all the checks we have they have the checks I also provide it's an endpoint this is almost the same but needed for the for the UI so and this is the after checks UI provided by these guys by the community so and here we see all they all the services that are running so these are running in the docker container this is in the alternate docker container but in the end the visualization yep and this is our dummy justice to check so what happens if we kill one of these ok a little bit so here are the containers running so looks a little bit bad I will resize it all the way around so like this ok I want to kill one of them let's say the weather station in canvas well don't have container killer okay some it was the idea it's just darker kill it's it's not docker container kill really try to lift up your containers there you go yeah no it's killed thank you almost almost done let's just this last demo okay you can see now there that station in Kent was offline its offline and you see the checks are unhealthy no okay so that's pretty much it um I say thank you and hopefully you enjoyed it I thought I really liked seeing the level of community interaction here with the health checks you're right they're so simple to write and already so many of these that really help with those other integrations and services that we want to know the health of mm-hmm yeah absolutely there's a lot of implementation see a lot of documentation and from the community yep the user interface right we saw the same thing with the folks who do swagger stuff to have health checks you want yes it's something every user administrator is going to need yeah absolutely yeah I didn't know that I'm I definitely need to make sure I had that to my applications going forward thanks to thanks for hosting thanks for having me for champagne
AI and the Workforce The Opportunities and Challenges of Introducing AI in Manufacturing
(upbeat music) - I wonder if in your conversations or in your research, there are examples of building a skills pipeline or developing the workforce of the future, to ensure that the workforce is ready for this disruption. - I always like to start with the people aspect because we are at the junction, at a point in history where, I like to say, virtually there's nothing technology cannot do today, Greg. It is up to the leader, having a very different vision of the future, the leader's ability to execute and create a different culture and bring others along. So to that end, empowering the workforce and dealing with workforce transformation is one of those very, very important pillars. (upbeat music) - What is the dream at MTorres of how AI can be used in the future? - Yes, well, our vision is that our machines that are, very automatic, but they still need one operator of the machine, using them, as the process is very special, this operator has also to become an expert of the process to be able to run the part and lay out the part. So, this knowledge you have to transfer to the operator. Okay? And the operators are very, moving between different, in an industry, between different lines and it's always difficult to have some, experienced workers there. So our dream is that, the machine can be self-aware what is happening in the environment, in the material, on the machine, on the process, on the whole plant, be able to self-verify, auto tune, and even anticipate problems and make decisions or at least propose decisions to a human. Okay? So we currently see that, our machines will not have humans operating the machine, but human supervising and coordinating the work of the machines (upbeat music) - AI is essential not only to business operations, but to society. Give us a sense of the conversations that you're having with customers. - The number one conversation that we're doing a little bit of, kind of creating awareness and education around, is this job elimination. - Yeah. - There's that myth. Our research suggests that yes, jobs will change and 75 million jobs in the manufacturing space will go away, but 133 million new jobs will be created. As we move from agriculture as a society to industrial, we had, Greg, perhaps a hundred years ago, 50% of the workforce in agriculture. Now we have like less than 2% but we have historically very, very high levels of employment, because what we did as a society, we changed the profile, the demographics of the workforce. Now it's time that we do. We go through another wave and then we have to go through that together. (upbeat music) - How do you think about ensuring that Tetra Pak has the right skilled workforce for the future? What solutions, what recommendations do you have? - We have a program called Future Talent where we bring in quite a lot of people directly from university every year. We try to, we recruit from slightly different places now of course with slightly different education with these type of, you know, data science and other type of information technology educations. Then when they start working, I think it's very important that they quickly realize that they have to learn the practical reality of industrial processes as well. Because you can't put together fantastic algorithms that solve a problem that doesn't exist of course. So if you don't understand the problems that do occur in real manufacturing processes, you will designs, you know, the likelihood of designing something that actually solves a problem, is not so high. So being interested in how it is, you know, spend time on factory floors, travel the world, we always introduce them and send them to customers. They have to be on the factory floor, they have to operate machines so they understand what type of problems. They have to work with people who understand that. (upbeat music)
Activating Civic Power for Social Impact
I just want to thank all of you for being here today and particularly thank the PopTech and Microsoft Envisioning teams for gathering us here. This is really, really juicy and I'm really appreciating what I'm learning already this morning. I wanted to say a word about the work that I do. My name is Eric Liu, I'm the founder of Citizen University, which is a non-profit. We're based in Seattle, but we do work all around the United States. Our work is fundamentally about trying to democratize understanding of how power works in civic life. We've got a variety of programs that do that and I'll describe some of them. But I want to say actually a word in the first place just about our world view. And this flows in part from the conversation we've already been participating in this morning, and some discussion we had last night, some of us over dinner. I think it's really striking how much resonance there already is between the presentations we've heard, and one of the things I want to name and just call out that's just a thread among all them and is core to our world view at Citizen University, is the ecosystemic view of things. Our work ... I wrote a book several years ago called, "The Gardens of Democracy" and the title of that book contains a bit of an argument. One that we've actually heard elements of this morning and the argument is this. That so much of the way we think about work, the economy, community, civic life is through the metaphor of a machine. Right? We're hearing [inaudible] from and others about this kind of obsession with efficiency and this mechanistic view of the purposes and the operations of all the ways that we work together and deal with one another in public or not in isolation. And the argument of this book was essentially that while that metaphor of the machine is useful in some ways, and it's illuminating, and when you actually start paying attention it's ubiquitous, right? We talk about the economy as a job creation machine, we talk about whether the Fed should tap the brakes on growth, we talk about priming the pump of economic growth. All these mechanistic metaphors and in politics as well, we talk about political machines and we talk about the ways that we relate to government in sort of the sense of a frustrated consumer who's put coins in a vending machine and then didn't get our candy bar, right? So we kick the machine. There is some utility to that metaphor. But I think this morning really underscores the ways in which that metaphor also blinds and it limits our sense of what's actually going on around us right now. And that it's properly understood. An economy, a workforce, a neighborhood, a community, a polity, these are gardens not machines. They're complex adaptive systems. They're networks that have organic life to them, that bloom and cascade in the ways that we're seeing images of all this morning. And the role of the worker, the role of the leader, the work of the manager and the work of, certainly of the citizen, is to be a gardener. Right? Gardens when left entirely to themselves will grow like gangbusters, they'll be awesome. And a laissez-faire approach to a garden, yields initially beautiful blooming growth. But then when you continue to be laissez-faire, it yields an opportunity for noxious weeds to take root and eventually take over and kill the garden and tip things over til they're all dead. Gardeners garden, gardeners weed, seed and feed. Right? And I think I wanted to kind of dive into that metaphor a little bit by way of opening because it seems like one of the ways that we segway from this conversation about networks in general and networks in workplaces and networks in a beautiful instant and yet evanescent polity like Burning Man, in all these cases, we're thinking ecosystemically and we're thinking and behaving like gardeners at our best. Right? So that is the spirit of our work at Citizen University. And one of the things that [inaudible] asked me to do here was to try to connect some of the dots between work and civic life. And the fundamental question before you here, which is the future of work, and I really like the way that Anton described this theme. That it's not about just enterprise workplaces and about context of work at your job, but just what it means in fact to, as my friend Howard Gardner at Harvard says, do good work. Howard has launched a thing called The Good Work Project that is looking at what is the meaning and the essence of good work in any domain. Whether it's a profession, whether it's your life as a citizen, whether it's in your life in family or faith, what is the essence of good work? In our work at Citizen University, I've learned several lessons that I wanted to share some reflections on today about the nature of networks and the way that networks in this day and age are activated for civic change. I think it goes without saying, I think we all recognize no matter how politically or civically engaged you might be personally, that we live in an age of citizen power. We live in an age of bottom up upheavals and that's not just the politics in the United States of the last year, it's the politics of the United States going back at least to 2010 and the arrival of the Tea Party, Occupy Wall Street and other such movements that have since bloomed here. But of course it's not just the United States, it's the Arab Spring, it's the Maidan revolution, it's the Umbrella revolution, it's all of these movements in which without permissions, without the authorization or even knowledge of hierarchies, citizens in self-organizing, spontaneous ways are forming networks, activating those networks, and changing the frame of the possible in their communities and societies. So we live in this age. And our work at Citizen University is about trying to capture the essence of what it means to exercise power in a time of bottom up upheaval like this. And we've learned three lessons in our work and I want to kind of describe each of these about what it is that makes a network particularly effective in civic life. The three things that I want to reflect are in the first place mutuality, secondly moral clarity, and then finally, imagination. So let me just say a word about each of these and give you a little bit of example of some of the stuff that we're doing. So mutuality. I think you know those classic slides about [inaudible] and the strength of weak ties and you know the triads and the rest, I think are such a great vivid reminder, not only of the dynamics that unfold that become mappable and visible when you do a network map, but about the underlying pulse of social norms that actually feed any kind of network. And the most central of those is a spirit of mutuality. Right? There is no Burning Man without mutuality. This idea of that the gifting, for instance, is done with this kind of unspoken sense of reciprocal mutual response. Right? So at Citizen University, one of our projects is something called the Civic Collaboratory. And what this is, on one level, is a network. It's a network of civic innovators drawn from all across the United States, all different silos of civic work, and from across the breadth of the political spectrum. Right? It's folks who run the gamut from co-founders of the Tea Party to co-creators of Black Lives Matter, to folks who are leading advocates among the Dreamers and undocumented immigrants to people who are working on voting rights, people who are working on civic tech, people who are working on civic education, people who are thinking about veterans and activating veterans as civic assets here at home, people who are thinking about civic work in all these different ways. And yet the reason why we created this Collaboratory was our perspective revealed to us that relatively few of these folks were busting out of their silos. That relatively few of these people in their domains had not only opportunity but any knowledge of opportunities to play with one another across the lines of their different domains. And so in 2011, we convened the first meeting of the Civic Collaboratory. And we learned a very interesting lesson here. We did it kind of the old-fashioned way. We had a big meeting, we called it actually at a place that's Microsoft descended in a way, and the Gates Foundation headquarters in Seattle. Bill Gates Sr., Microsoft Bill's father and Justice Sandra Day O'Connor, retired Supreme Court Justice, joined us as co-conveners of this and with two co-conveners like that everybody we invited came right? This great list of folks showed up, they wanted to be in the room with ... the rumor had happened with Bill Sr. And Justice O'Connor. And we had this great day and a half gathering and during much of the gathering all these people were meeting each other and learning about their respective kinds of work. Our intention as the curators of this gathering, was to say, "Okay folks before this day ends what are we going to do together, what are we going to commit to doing together," right? And at first, there was this high sense of purpose and energy and people were like, "Yeah what are we going to do together?" And then as the day went on, it started sinking in with people that wow we're coming up with all these interesting plans of how we're going to connect veterans and immigrant rights activists and how we're going to get Tea Partiers to work with people on the left, and we're going to do all these things. But it started to dawn on folks, I'm not getting paid to do any of this stuff, right? It started dawning on them that I don't have time to follow up on any of the commitments that I'm sort of making in front of Justice O'Connor and Bill Sr. here. It started dawning on folks as well that there's some of my competitors in the room here. I'm not sure it's totally in my self-interest to dive in and do this stuff in this way when other folks in the room might take advantage of my distraction or my non-focus on my core mission. And so by the end of the day, being attuned to that room, I sensed that there was this kind of sapping of commitment. People went through the motions still, and went through and kind of made a list of commitments but there was something less than fully satisfying about that first meeting. So, we went back after that first meeting of the Collaboratory and we said when we gathered it, we don't presume that there will be a second meeting. Right? We don't know. But we felt like there was enough there that there was something to with this, but that we needed to activate a different kind of energy. And what we identified was what we needed to do was to shift the terms of self-interest in that room. And to really make a set of rules and a set of norms that would unleash, that would invite and bring into the open, a sense of mutual self-interest. And indeed remind folks at the end of the day, that all self-interest is mutual self-interest. And so in the next meeting that we did, it's a couple months later, we devised a new format that we called a rotating credit club. And the rotating credit club is this very simple format that's in some ways adapted from processes that unfold in a lot of unbanked immigrant communities around the United States, where a circle of would-be entrepreneurs who don't have access to capital or traditional lenders, will put their savings into a pot and then they'll take turns getting the benefit of the entire pot in order to capitalize their small business, their dry cleaner, their restaurant, whatever it might be. We adapted that format so that every single time we meet at the Collaboratory, five or six members of the group will take turns in rotation. And they'll present to the rest of the group a project they're working on, an initiative they're trying to launch, a challenge they're facing. And the rest of the group has to offer not just commentary or critique, the rest of the group has to make hard commitments of capital. Social capital, ideas capital, relationship capital, in some cases because there are funders in the room, it's money capital, though it's not primarily about that. But this idea is that you will lean in and you will make those commitments of capital because in the best sense, what goes around comes around. Right? It's going to be your turn to be in rotation next quarter and you're going to want folks from this very disparate network of activists and catalysts and civic change makers to want to lean in and commit back to you. That's one big lesson that we've learned from our work at Citizen University is the power of mutuality. The second lesson here is about moral clarity. And I want to describe another project that we launched actually four days after the election. This is a project called Civic Saturdays. We launched this project as I say, right after the election, and what Civic Saturday is is essentially a civic analog to church. It's not church, it's not synagogue or mosque. It's not a traditional faith institution or gathering, but it is about American civic religion, about the norms, the values, the texts, the creed that those of us who live in the United States have both the right and the opportunity and the obligation to actually inhabit and revivify by our choices and by our deeds. And our gatherings have the arc of a faith gathering. So we get together and there are, there's song. We sing together. You turn to your neighbor, you talk about a common prompt in response to a question. There are readings of what you might think of as civic scripture, texts from the American tradition, well known and not well known. There are sermons that thus far in Seattle, I've been giving but others will be giving as well. And then afterwards, there's a social hour like you might have after church or something, which started out as just literally tea and cookies and the rest, but has evolved organically into sort of a self-organizing activism hour. Right? When we did this first one four days after the election, it was just out of the strong instinctive sense that there was this yearning, this need for people to try to make sense of this moment of upheaval. And we thought well, on four days notice it'll be great if 50 people show up at the basement of this bookstore that we booked for the event. And over 250 people came to that first one. And [00:14:00] we realized wow, we had tapped into something here. And what we tapped into was not just post-election need to make sense of things, because we've now done these periodic gatherings every month since then. What we tapped into was a desire that people had to be in a community where together they could seek moral clarity. Right? And so it's one thing to have a network of affinity and a network of shared skill or strategy or objective in public policy, and that is what connects Black Lives Matter to the Dreamers to the 15 dollars an hour movement to the Tea Party, they all have some purpose in which they are able to kind of use that purpose to organize in a seamless non-directed way. But what gives their networks life in an enduring way is moral clarity. Each of them has a strong moral point of view. You may not like their point of view, you may not agree with it at all, but there's no ambiguity about what Black Lives Matter stands for. It stands for the idea that black lives ought to matter in this country as much as non-black life. And that in our institutions, and in our criminal justice system, in our education system, and in employment, black life does not matter as much as non-black life does. And that is a strong moral point of view, right? The point of view of the Tea Party and the successor networks that have grown out of the Tea Party, organizations like Citizens for Self-Governance, is similarly a point of view that says this country was founded to overthrow tyranny. And that the federal government today is itself the source of tyranny and we need to unwind and rewind and scale back the reach and scope of the federal government. Again, like, dislike, agree, disagree, that's fine. But there's no ambiguity about the moral purpose that's driving that network and that feeds that network magnetically in a way that helps it grow. This brings me though to the third and final element that I wanted to say a word about and thinking about social civic networks and the ways in which they live and thrive. And that is this idea of imagination. Now you've got a good dose of that in both looking at images of and hearing talk about the beauty of Burning Man. But I think one essential element that is true at a Burning Man, that's true in pick your movement today of civic action in the United States that's effective, and that's certainly going to be crucial in any work place setting where leaders like you are thinking about how to form and activate networks that cohere and give people a sense of purpose, is this notion of play, of how do you awaken a spirit of imagination and creativity. [00:16:30] And I want to tell you about another project of ours at Citizen University that we launched last year called The Joy of Voting. And this is a simple project that recognized there used to be in the United States, a culture, around local elections in particular, that was about joyful, raucous, robust participation in voting and elections. It was about street theater, open air debates, dueling parades, bands, bonfires, the culture in which it wasn't just do your duty, eat your vegetables, cast a vote, but it was about join the club, join the party, join the tribe and participate in public in a joyful, creative way that expresses your ownership of the republic. Right? Screens, whether television initially or now you know our other screens have pretty much killed that culture of joyful engagement in voting. But our hypothesis was there's no good reason why we couldn't rekindle that, particularly at the local level. And so in working with the Knight Foundation, which as some of you know, operates in a couple of dozen cities around the United States where there are or used to be strong Knight [inaudible] newspapers. We partnered with them to catalyze Joy of Voting projects and they're very simple projects where in each of these cities that ran the gamut from Akron, Philly, Wichita, Miami, now St. Paul, Charlotte, Grand Forks, and other places around the country, big and small, blue and red, was we convened artists, activists, musicians, everyday citizens, neighbors, folks who were just engaged in the life of the community. We came and invited them to practice imagination. We invited them to think about what would it look like to have a project at a small scale, that you could create in election season that would awaken people's sense of belonging, that would awaken their desire to be part of something greater than themselves and would awaken most of all their spirit of imagination and play. In each of these communities, people brainstormed ideas, they went out to their networks and generated and harvested more ideas, and then over the course of weeks we ultimately chose in each of these cities, five projects and gave them micro-grants so about thirty five, thirty six hundred dollars. And we seeded all of these little projects around the country. And last year, in what was one of the least joyful, most grim election seasons in living memory, we were able to spark this spirit of imagination and creativity in civic life in these cities. And now we're expanding to a group of other cities in the United States. I tell you about this project because whether it's about voting or it's about showing up in other ways, when you think about what it means to activate a network, it's not just about mutual aid, it's not just about a sense of moral clarity, but there has to be activating the spirit of imagination and purposeful play. And that is certainly true in civic life, you think about the Women's March, you think about all the ways in which people have been figuring out, and the March for Science, ways in which to create this sense of spectacle this joyful sense that we're not getting beaten down by the threats and menaces to our democratic systems and institutions, we're going to revive them by expressing the full breadth of our humanity with spirit, purpose, play and joy. Right? And that is the spirit in which we've got to operate in applying everything that we do in one domain, in my case civic, to everything that we do in another, perhaps in your case the workplace. So the final word that I want to say is this. In thinking about networks, in thinking about the ways that we learn from each other in activating our networks both civic and workplace, one of the most basic lessons that we've learned at Citizen University and that I've tried to express personally. And it's actually in the book that the good folks were good enough to put in your gift bags, my new book "You're More Powerful than you Think," about this age of citizen power and how we practice citizen power, is this simple insight. Once you take stock the way that these images and slides today have shown us how to take stock, once you take stock of your network and your network of networks and the network capital that each of you has individually, collectively, institutionally. Once you have that assessment, you've made that visual map and you've taken that accounting, you face a pretty binary choice. The choice is this. Shall you hoard? Or shall you circulate? That's it. In civic life today, in the United States in this age of nearly unprecedented inequality and concentration of wealth and opportunity, we face this choice. Shall you hoard the power of your network? Or shall you circulate it? And I'm here today just to beg of you, as you think about how you're going to get smarter about networks, to make a personal and collective commitment to circulating your power. Thank you very much.
ABP Open source web application framework for ASP NET Core 3 0
right I shared my screen yes you do have a chair give me one second okay all right everybody we are live now we got Holly on here talking about the ABP f-- framework open source so let's let's take it away hey leo oh there you go okay hi everybody today I will introduce an open-source project the ABP framework probably some of you have already heard about it or using it in your applications in any way I believe that this talk will be usable to understand the need for such a project and what ABP is offering with its next version I am a Libra whom Kalkan a Software Architect and co-founder of palo soft i am the lead developer behind the ABP framework ABP is an open source web application framework for the latest asp.net Corie 3.0 ABP provides a complete architecture and the strong infrastructure to create modern web applications it follows best practices and conventions to provide a solid development experience first we should understand why we need to such a project and why we have created it if you are building your applications on top of the plain asp.net core you mostly find you on repeating yourself to build a similar infrastructure again and again you need to decide on an architecture say a layered architecture based on domain driven design or clean architecture principles you will write some action filters or interceptors to automate cross-cutting concerns like authorization validation exception handling database transaction management audit logging caching and so on you definitely don't want to repeat these logics in every source method you need to create base classes abstractions integrations extension methods how purrs library workers to simplify your application code you will deal with all this infrastructure and much more stuff again and again for every project for that reason big enterprises are building their internal application frameworks on top of asp.net core but it's hard to build maintain and document it this is why most of such works finally failed or no one knows how it works because of the missing documentation beside the big enterprises most of the software development companies don't have such development resources to build the infrastructure themselves we have created the ABP framework to address this problem and provide a full stick application model and infrastructure for everyone it's hard and time-consuming to select the right tools and we'll the clean architecture while starting a new application as a Software Architect you need to close the followed software trends tools principles and patterns you have to believe the fundamental solution structure and train your team to align them to follow the same principles and coding styles to create a consistent source code when you use a VP fundamentals of the architecture will be already decided and implemented for you a VP solution model is based on the domain driven design principles and patterns DDD offers four fundamental layers domain application presentation and infrastructure a VP startup templates implement this layering by providing a multi project solution structure a b.p framework provides infrastructure for your application to implement TDD easier and finally a VP documentation supports you to truly implement TDD in your source code a beefy framework is designed to create fully modular applications it provides infrastructure to distribute your entity's services api's UI components and page into isolated modules in the development time and bring them together to act as a single application on runtime we already created tribute modules like account management Identity Management and talent management those are ready ready to use in your application documentation and guides help you in building your own modules micro-service is another training approach to create scalable systems both in development time and on run time NIV ABP has been designed at micro services in mind it provides infrastructure to implement common micro service communication patterns easily it's integrated to the island server for for authentication pre-built application modules are designed so that they can be deployed to separate servers or containers with date they're dedicated databases as independent micro services we have also created a complete micro service solution sample and documented it to be a reference model you can check it on the website multi-tenancy is a fundamental architecture to create software as a service solutions ABP framework and modules provide a complete multi-tenant infrastructure tenon determination automatic data is relation dynamic database selection channel pays the setting system and much more are already implemented by the framework it not only supports to develop multi-tenant applications but also makes your codebase mostly unaware of to multi-tenancy pre-built tenant management module provides the application logic and basic UI to manage and create your tenants ABP framework is not a new project actually it's the successor of the asp.net boilerplate framework asp.net boilerplate has been around for six years it has been more than 7,000 stars on github 1.5 million downloads on new yet it has been actively used by tens of thousands of developers around the world we have been working on the next generation of the asp.net boilerplate framework for a few years and it's finally here the first stable version will be released in a few weeks based on the asp.net core 3 it's a complete rewrite so we don't use the log name anymore it's the ABP framework and the new website is ABP dot io it's now more modular and micro-service compatible supports relational and on relational databases provides more options to create composable and distributed systems in addition to simple monolithic web applications we are very excited to introduce it in the same event Microsoft launches asp.net Court 3 now I want to show you how to create a new project based on the ABP framework and make it running in a few seconds first I'm entering the ABP our website in the getting started page we have two options to create a new project via a BP command-line interface CLI or we can directly download from the website once we enter a project name then we can select application template or module template to create a reusable module I select the application template we have two options as the UI framework for now MVC and angular and we have now two database providers and two framework or supports most of the relational databases at MongoDB and now we can create a new project but in this demo I will create the project via CLI ABP new project name its grace the complete solution in a few seconds yeah the solution has been created I am opening in the visual studio I have a preview one it has a mom it is a multi project solution consists of the source code and related unit tests it's layered based on domain living patterns and principles and layers it's also well documented on the website so you can easily understand how solution structure is design it when I run the application the solution template has is include some of the pre-built modules like tenant management and identity management when I run the solution yeah I see the account module I can enter the user name and password at the startup time place uses two modules as I said before identity management is used to create and manage roles then we can set permissions for each role that means we can allow to do something in the system for the human role also in the user side we can set rules for the for a user we can create a new user it also supports multi-tenancy out-of-the-box we can create tunnels and use multi-tenancy as well it's localized to a few language okay that's all the solution is stuff actually doesn't contain any user interface component or any functionality I have shown it uses nougat package for all the functionality related identity management talent management account login and register page okay let's continue now I want to highlight some features of the ABP framework as a brief the main goal of the AVP framework is to make you focus on your own bussiness code by handling repeating tasks by conventions it handles cross-cutting concerns and provide services you need let's see a typical application service class to understand how ABP can help you in this very simple code block AVP provides base classes for ddd elements in this example class inherits from the application service application service base class provides properties and methods commonly used in the application layer AVP automatically registered this class to dependency injection container as a transient service you can use the generic repositories to work with the database declaration authorizing tribute can be used to check permissions which are auto policies actually it is completely integrated into asp.net course authorization system and extends it unit org system automatically creates and managed database connection and transaction for you it rollbacks if your metal drops and exception otherwise it saves all changes to the database as a single transaction it works out of the box by convention it automatically validates the input and throws exception if it is invalid so you can safely work with the palette in put inside your method it automatically creates and saves audit logs for each method call so you can then get add attack tailored report to know which user called which method audit log system is very detailed detailed and configurable it can also save entity and property changes as well you never handle exceptions manually ABP handles all exceptions and send an appropriate result to the client ABP extant asp.net calls localization systems to make it more dynamic and extensible object mapper is used to make objects it has Auto mapper integration out-of-the-box and can be replaced by another mapper AVP also provides some other services usable in the application source in this way you only write the code related to your business without repeating yourself if your application service basically performs quadruped operations then you can also inherit from the root of service to get it fully implemented for you you can then override and customize any method you need once you create an application service you type collie want to expose it as an HTTP API by writing an API controller AVP can automatically create API controllers by convention based on your method names input and output types so most of the times you don't need to manually create API controllers arrange routes and HTTP methods okay you have created your HTTP API and or let the ABI a VP framework dynamically create it for you what about when you want to use the API from a client application AVP can dynamically create JavaScript and C sharp proxies you can use the client proxies to use the remote HTTP API just like calling a regular method in the client side you don't deal with authentication exception handling JSON serialization deserialization and point configuration and so on notice that it doesn't generate any code it all handles in the run time you don't have a large machine generated code that you don't understand what it does here to example code that uses the dynamic client proxies JavaScript code calls the method just like calling a JavaScript function c-sharp client injects the application service interface which is shared with the server the injected service is a dynamic proxy object which internally performs HTTP calls to the remote server then you can use any service method just like calling a local local method and you get the result if provided ABP provides a distributed event pots out-of-the-box publishing and even is pretty easy just injected I distributed even post service and use the publish method subscribing and consuming an event is also easy just implement our distributed even hunter interface and fill in the handle even method the rest of the communication and retry mechanisms are handled by the ABP framework it's integrated to RabbitMQ as a first implementation we will add more options in the feature a BP is framework UI framework agnostic you can use different UI frameworks like asp.net MVC angular react whooshes or blazer it currently has a missing and angular startup templates more will become by time in the section I will show some of the ABP features for a missile age razor face UI option in a modular application you typically want to em and make your UI components like wheels or javascript file CSS files and image into the LS or may get packets you may want to create application independent reusable modules with their own entities services API and UI elements ABP provides a virtual file system that acts as a single file system but internally can read contacts from various resources like physical files embedded files or even from dynamic files created created on runtime this is an essential feature when you want to work with files distributed into modules we have created tackle programmers for most of the bootstrap for components like buttons models cards tops and so on in the sample you see a button and the model the dialogue code writing using a BB tag helpers it produces native bootstrap code on runtime in this way you write less code to create bootstrap components in your views it makes possible to get the power of the intellisense when you want to directly work with the bootstrap code you are free to write native with dropped HTML which is cooperates nicely with the AVP toc helpers in addition to wrappers to bootstrap talk helpers we taught that creating a regular form requires a lot of boilerplate code to reduce the code we type we have created a VP input tag helper which generates most repeated localized and validated input controls it understands the input type and can work with the existing attributes to customize the HTML output time we thought why we should define an a VP input tag for each form element when we want to create a simple form what if we create a single tag helper that gets the model and generates the complete form for us ABP dynamic form tag helper does exactly this creates a complete form for you with a single line of code it gets all the information from your view model class handles validation and localization to the last feature I want to highlight in this top is the bundling and minification system provided by the ABP framework you may think that why we need to create another bonding and unification system while we have a speed-up net course bundle and minifiers or gulp or grunt GS the main reason is that none of them are modular none of them can read CSS files inside the dll none of them can this arc the bundle content on the fly and actually none of them are easy use which bundling and minification system can be simpler than this code it's just a tack helper directly used in your page or view it adds given files individually to the page in development time bundles and minifides on runtime it works with the virtual file system out of the box so you can add an embedded or dynamic resource into your bundle it can automatically include dependencies to the bundle that means if you included the library to the bundle that depends on jQuery it automatically includes jQuery to the bundle if jQuery was not already added to the current page before that means it rearranged bundles on-the-fly can't determine the bundle content conditionally it allows you to create named bundles and reuse it in an application or even across multiple applications it has many more features that are needed to create modular web applications check you can check its documentation so I highlighted some AVP features that are easy to explain in such a short talk then there are me many built-in features and extensibility points waiting for you to disk or visit a BPO and follow tutorials to learn it deeply as the last word ABP doesn't try to reinvent the well instead it tries to fill the gap between the plain asp.net core and real-world enterprise application the not scenarios it helps you to code in best practices to build maintainable software solutions okay that's all for my talk if you have any questions I'm glad to answer them yeah somebody was that's a great great story the Helena somebody was asking Ritesh um Tamara carcasses are they any real use of a BP at the moment yes actually a BP is the next generation of the asp.net boilerplate framework so it's widely used by thousands of developers for the ABP IO the next generation it's in the preview know we have just upgraded to asp.net code 3 yesterday immediately upgrade nicest asp.net core so we are using it in production but we don't call it 1.0 for now because it has some locking documentation and we we want to test more so some of the custom some of the developers are using the NIV a BPO in their production and one but we don't such as for a few weeks more a few weeks more Dom we will release the first stable version for the BPO awesome awesome yeah people been saying great presentation there's a comment here from Oz Bob Western Australia I've been reading a VP documentation all did a support slash buckle integration and open API and swagger UI Wow definitely gonna give it a go so yeah your you did a great job for sending all the features of your framework that's awesome so I want to thank you for taking the time talking about it and showcasing it I know everybody were excited so we're gonna hang up here folks and we're gonna get Martin who's gonna talk about indexing nougat with a sure functions so again Ellie oh thank you so much thank you very much and thank you to everybody listening me okay oh so much have a good one I will be back in a bit folks
Building Bots Part 1
it's about time we did a toolbox episode on BOTS hi welcome to visual studio toolbox I'm your host Robert green and jo...
-
>> It doesn't matter if I'm short or tall. >> Anxious. >> Or obsessed. >> Loud. >> Quiet...
-
hello my name is Ron Lou and I am a program manager on the video studio team in the previous video we talked about how to inst...
-
hey you're watching after hours with the visual studio team and I'm really excited about this episode because when I...