I know that uh uh you are all here to see our headliner um my personal friend for the last 19 years uh our technology director at ID software the smartest guy you will ever meet uh it is my honor and privilege to introduce tonight John carac [Applause] [Music] [Applause] [Music] [Applause] hi everyone so I know last year uh I wound up talking for three and a half hours and overrunning some of the tournament broadcast so we're going to the idea was to attempt to shorten it down a little bit but when I went and took notes for myself about all the things that I wanted to talk about there was like 300 lines so we'll see what happens Tim will probably cut me off if I wind up running too long I so there's an amazing amount of exciting things going on in technology now that affect gaming and the larger ecosystems that we work in and you know a lot of it's just fascinating in so many ways but the thing this year that's sort of the elephant in the room that we can't go without talking about is the console cycle where we went a lot of years with the same console platform and it was good for developers and console players but uh it was if anything getting a little bit stale considering how long it had lasted relative to previous generations and now we have the The Heavy Hitters with Microsoft and Sony coming out now obviously I'm still under NDA and I can't talk about really juicy technical details but there's enough out in the public record that there going to be some discussion about it and it's uh you know it's interesting it's almost amazing how close they are in capabilities how common they are and uh that the capabilities that they give are are essentially the same we can talk about differences in memory architectures but the bottom line being that they're a multicore AMD processor with AMD Graphics is it's almost weird how close they are where I'd love to know the back room stories between the different uh companies about who knew what when about how close they were coming together whether they converged intentionally or if they were just surprised that they wound up having almost identical specs so in the near term it's clearly going to be a good thing for developers Gamers uh an excellent thing for AMD uh you know we've got it's it's really interesting considering three or four years ago when we were looking at where we thought the next console generation was going to go at the time we were really thinking that there was a possibility that we might wind up with an all Intel world where Intel with larab as their uh CPU oriented GPU might go out and push really hard on the console vendors and maybe even sweep the field you know there was a thought that Intel has resources that almost no other company has they have certainly the best processed technology they have some of the very best silicon Engineers uh and they've got boatloads of money that if intel wanted to go in and make a Mark here that they might go in and potentially even buy the winds to the you know make offers that the vendors couldn't refuse and larabe was very interesting when you looked at it as uh you know something that was different than the CPU talking to the GPU world that we've had for a long time now uh it would have opened up the possibility for some novel rendering architectures for sort of Reinventing the wheel and making it look a little bit different uh things like uh voxal or splatting architectures and some of these things that are not triangle rasterization based might have been better suited there but uh as we all know by now larabe was eventually dropped as a GPU product I you know I know some of the people that worked on the project and a lot of the hardware people will just say oh it just wasn't quite ready because of the software but the hardware was good uh the hardware does live on as a high performance Computing part and the Zeon fi I but still that thought that several years ago we thought it might all go this way and it went almost diametrically opposite direction uh we could have wound up with still a you know fragmented world like we've had for every console generation where different vendors have different uh different architectures and approaches and Developers are forced to choose the common subset between what the different platforms can do which always leads to less optimal development unless you're uh you know a sole platform you know single platform game on a console where you can really optimize for that you're always making uh compromises and on the PC you're always making compromises across the the breadth of your installed base but now we did wind up with something that looks like uh it's again a PC based architecture I you know AMD has they've traditionally lagged Intel and raw performance on the CPU side but having a lot of cores here having it something that's going to be prevalent on the consoles means that developers will favor that as an optimization and that will give AMD a little bit of a boost relative to uh Intel clock for clock probably I mean you'll optimize the code specifically for one micro architecture and that will probably become somewhat prevalent on the PC Intel still has a significant enough performance lead that I don't think they'll outstrip the in raw performance but it'll help close the gap and makes it useful when you're comparing price performance and some of the other things there so in the in the near term it's clearly a good thing for gaming uh in the longer term we can maybe worry a little bit about uh whether a monoculture in the ecosystem there is really a good thing I suspect that we're at a mature enough point in game development that it's probably not particularly harmful that making radical different changes are so traumatic that uh we can Coast for quite a while without having to really fundamentally re-evaluate the things that we're looking at but it still could have gone differently you we could have wound up with another Real Player for real contendant for contestant for uh console Generations would have been super mobile Hardware rather than uh kind of productized PC Hardware we can imagine uh something that had maybe 16 arm cores with a whole bunch of power VR Graphics cores and you could have made a pretty good console with that there's a lot of uh there's a big ecosystem for that there's a ton of development work going on you get good easy performance out of them in a lot of ways and I suspect it was the the 64-bit not being quite cooked enough on there that really made the difference uh and the fact that huge amounts of memory are obviously a big deal for the console platforms now and that's probably even more so than the last factor of raw performance tons and tons of memory is the thing that makes game development a lot easier here I mean we are currently going through the crunches to make uh Wolfenstein fit on existing platforms and it's that you know it's always that final 90% when you think you're done with the first 90% of the development it's the last 10% that turns out to be just about as much work to make it all work inside the limited resources so lots of memory is going to be nice on the upcoming consoles uh it's going to make things a little bit easier there's a ton more that we'll do visually and I to the games there but I think think that I think that the writings on the wall that pushing things to their absolute limit is is not really tenable for the most part in development because there's just too much that you can do it was one thing when you had a system that you could sort of hold the entire reference manual in your hand and you could say well we're going to do as close to the absolute best we can imagine as possible on this but you know nowadays no one person even has the entire capabilities of one of these modern Platforms in their head you wind up ists in you know the simd code and the compute code in the GPU and the memory architectures in the file system the user experience the interfacing with the operating system all these things there's you know there's too much to wind up with this crystal jewel of perfection like you might have at least diluted yourself into thinking you were approaching in previous years so development is a lot more about trying to you know get the good value out of the time that you can put into it and Mobile's really been like that for a while now uh most mobile developers are not anywhere close to trying to max out what they can do on their diverse ecosystems and platforms it's really about trying to come up with something fun make it happen and iterate on it quickly and that's I you know there's a part of me that always is you know always does want to do the absolute best possible but the reality is that's just not the smart play to you know to look at the Modern game systems uh we've got the comparing the two big players right now I mean we should first say that the other bit players it's interesting to look at where the Wii U uh I always thought that the Wii U and the Vita would be great targets for dun3 BFG you know we should be able to to bring that over directly but it's not something that's generating enough interest on the publishing side of things uh because it's just they're somewhat marginalized platforms and looking at the adoption of them it's certainly got to be a little scary even for the mainstream consoles for the big ones whether there will be the uh relaxed expectations of what they can move through on there uh because clearly there's a difference on the handheld platforms being in the post smartphone World while the 3DS is doing okay the vaita is really not doing particularly well and a lot of that has to come from the fact that we have not quite as good gaming platforms but everybody's carrying a pretty good gaming platform already and selling a new one into there is a little bit of a harder sell uh but the uh the smartphone developments on there are going to certainly matter about how you approach development how you look at the different platforms that you support but the big two are clearly coming out with both barrels loaded and they're going to have a big impact it's going to be very very interesting to see how it goes over the the next couple years if they meet the adoption of the previous ones or if something's fundamentally changed in the market now everyone would like me to come out with some I you know some a overb comparison about the two platforms and uh to be completely honest I haven't done really rigorous benchmarking on them so even if I didn't have NDA protection I couldn't give you a a really completely honest answer but they are very close they're both very good uh Sony it's no it's no secret that in the previous generation I favored the 360 over the PS3 uh while there's certain things that Sony did well for a game developer the 360 was a nicer platform to work on uh it was a little bit more powerful it had much better development tools and it just it just felt better uh Sony has made large strides uh their development tools came out of the gate much much better than they ever have uh they've made some somewhat more gamer focused uh decisions in their their strategies and architectures Microsoft is putting a lot behind things like connect that I you know I'm I'm still not really sold on connect I recognize that what my needs and desires as a game developer or what I use the techn ology for might not cover the the broad consumer base that they're looking for so it's their play to make but I think connect still has some fundamental limitations with the the latency and the frame rate on it and interacting with it is still when you when you interact with connect some of the the standard modality interactions there about like position and hold and position waiting for different things it's fundamentally a poor interaction and one way that I look at it is I used to give Apple a lot of grief about the one button Mouse When anybody working with a mouse really wants more buttons are helpful there and connect is sort of like a zero button Mouse with a lot of latency on it where I where you know you move around and you select something but because you can't you know like snap your finger at it and make something happen you have to like wait for a time period or have some other trigger section there and things like the the PS move where you're maneuvering around and You' actually got buttons on it uh or something like the the sixense razor Hydra where you've got position tracking but also uh buttons to click I think have some fundamental advantages now it's possible that as we get finer and finer grain control of uh the tracking that maybe you can start telling you know intentional little Taps of the finger in Virtual space and that will you know that will help some of it but I'm I'm still not completely sold on it uh the other thing with connect is that relative to touch which has been a revolution in user interface and it's just it's absolutely amazing to watch a two-year-old play with an iPad and just have things moving around and that was clearly such a huge win where that is a a big big positive to user computer interactions because you're directly manipulating things now connect as far as waving your hands around and moving things it is still at least a directly propor interface like a mouse where you can say I'm moving 6 in here and it's going to be proportional to that 6 in of moving which puts it into a better category than like analog thumb sticks where you're integrating your motion over time uh but you're not directly manipulating things because you're moving your hand here but you're watching the screen over there to see what's happening now it's possible that you could have uh if you had a viewer relative game if you weren't using connect as sort of your Social party game where multiple people are playing it at the same time but you were looking at a screen and you tracked where the viewer's eye was and you knew where their hand was in perspective space you could then sort of do okay I'm touching that I'm moving that over there you would still have all the latency issues to deal with there but it would be a much more direct interface and things like body tracking like that will have more more benefit when you've got things like a virtual reality system where you can you can see your rendered virtual hand interacting with rendered virtual ual things so it's a technology that absolutely has a future and Microsoft is you know is pushing it hard they've done a lot of excellent research with it but I'm still not completely convinced that that's the you know that that's the Cornerstone of the next gaming platform but they're still doing all of the other things that they've done well right so they are you know they are a solid force to be contended with no matter what but it's it's anyone's game as far as I can tell right now and that's that's exciting uh I think that we're always healthiest with at least a strong duopoly if not a full three vendor sort of three vendors fighting it out uh you know the so the game development on there it's not changing as much as you might think going from previous generations you know we have we have our ways of doing things and because it is a conventional approach that's not tearing up the rule book most studios are pretty much evolving the code that they've got you we're not throwing away our couple million lines of code and doing something completely different uh is still using your cores to run code to generate vertexes and run fragment shaders on it and that's not uh that's not changing all that much you just get lots more of everything and I again think that most of the time a lot of it will be spent on just making our lives easier so we can do a better job with the games so some of the things like Community backlash against the Xbox one uh it was interesting to see the power of the public reaction on it and there's the a couple things that have played into the the public Consciousness on it one of the things that was up early before even the the major issues with the the used games and always on and things like that was people being a little bit freaked out about connect being on all the time and you know this my game system is going to be watching me at all times and who knows what that can do and and I had a lot of you know conversations with people that really just didn't think that was right that that feels incorrect uh I am completely confident that that's a very temporary sort of vision about things where if we go back 10 years the idea that everyone would be carrying around a phone that has your GPS located position at all times would cause you know a lot of the tinfoil hat crowd to go absolutely crazy the idea that yeah but the government's going to have back doors into all of these they can turn them on and uh track everything where you're going and well yeah that's pretty much the the situation but uh you know we just kind of uh carry on I you know the situation with connect there's already a ton of uh laptops where the the webcam is kind of there and you have issues with people you know remote hacking things and turning them on and I wouldn't be surprised if there's not some absolutely terrifying hack that happens at some point that is a huge media issue with it but we'll we will just get used to it a lot of these things are inevitable like the people talking about Google Glass and the issues with people carrying around having cameras uh recording things all the time I think that's going to be a net positive for society having this sort of ground truth that's uh that's retrievable in many cases in a lot of things uh but it is it does have sort of social transition issues that we're going to have to feel through uh you know as those are adopted but there's there's a tide of technology that I think is in many cases stronger than uh you know than what even regulations can hold back in a global world where you can have sort of uh local places where you'll have stronger privacy regulations stronger uh you know preventions of like I guess in in some European countries you can't leave the devices on all the time so Xbox certainly will not be required to operate in that mode you can turn it off but they have a vision for how it's going to interact with you and we don't know yet it it's people have to try these things to find out if they're you if they're really going to be good uh the thing that of course got Microsoft the most uh in the most hot water was the issues with always Internet connected being able to not having the the used game transfers that whole issue and I do sort of fall onto the side of I think the Witch Hunt was a little bit unjustified there uh there are legitimate grievances that people can raise about how they would choose to have uh you know their game systems managed uh I personally am extremely fond of having all of my digital purchases you know in a curated Garden all of the all of my iTunes all of my Amazon stuff all my steam things uh and that's it's a positive thing I mean yeah you can have better and worse ways of doing that but we are very quickly going to be past the age of having a game that you hold in your hands on optical media I think that was another thing that was open for debate this generation was were we going to have Optical media at all or was it going to be all digital distribution I I would it's probably won't be many years before we wind up with SKS that just have the optical drives deleted uh and everybody will just be getting it uh through the net I think that's again a clear case of the future future is the future is obvious right there and it will be good for us in general I go over this a lot with books where I'm sort of a a biblio file where I love books I love my I have an encyclopedia britanica set that I hardly ever open and all these things where we have a fondness for our our physical the fetishes of uh of the objects that we've bought but really it is just so much better having everything in the cloud and being able to get at it like that now the other platforms that I'd love to be able to develop on 3DS and some of the other other things other small platforms I would like to see the the game show up on the Vita or the Wii U just to kind of play with some of the different characteristics that they've got there but it's it's extremely unlikely to happen because there are there's a finite amount of time and there's only so many things that you can focus on and there's you know the fact that you can't be you can't have complete knowledge of even one of these platforms even the handhelds now have so much going on on them if you know all of the intricid instruction set and their different extensions and the characteristics of the gpus and the the different memory mapped IO registers there even that's getting to be a little bit too big for somebody to you know to get a full grasp of or what happens is you're just finally getting the hang of everything five years into it when the console cycle changes again and I can look right now with some fondness at the 360 and the PS3 and say we've got a really good handle on this we can we can do some great stuff in it but it's time to put that aside and go on to the you know the next gener generation where we're still groping around figuring out what's you know what's going on uh one of the other dire other platforms that is in my opinion not going to be making a a big impact right now is the sort of micro consoles where you look at the OAS and the things that are just tiny little things that plug into your TV um in the long term it's it's possible that things like that may become very very important as they're coupled with Cloud rendering and uh the distribution like that I don't think that it's going to have a super big impact right now but it is uh amazing to see how uh the relevance of Android has the title wave of Android just pushing almost everything else out of its way uh you know I'm still obviously an iOS user for most of my day-to-day stuff but I did finally just in recent weeks spin back up a little bit on Android development where I looked at it three years ago and I thought it was really pretty awful uh but it's it's especially compared to Apple but you look at Paces of relative change and apple continues to make things better but you know Google has made things better at a faster rate to the point where developing like a Java app on Android is uh is really not that bad right now I you download their uh the ADT bundle and you plug in your device and it it pretty much just works in many ways without the certification hassles of iOS and things like that can actually be a lot easier uh it's only when you get into the native development kit that things really fall apart in terms of not not really looking well cooked at all but the you know when you look at the scope of Android and what they've done it's the power of free you know there's uh giving away the platform has been such a good thing for the you know the entire sort of electronics Industry because if we look back five years ago we had every little handset company trying to write their own operating system essentially and doing a very very bad job at it uh I mean now we still have the danger that everybody they they go take Google's code base and they mess it up in various ways while adding value you know we've got that classic problem of I you know that's happened with video card drivers in the bad old days and it certainly happens with Android platforms now but it's still world's better than we would have been if they had to be rolling it by themselves so that's been a great thing it still has you know it has a lot of the the messiness of Linux underneath it I mean that's still what's down there and the development tools reflect it there's also some of the magic of it where when I'm poking through something and I realize it's like okay this is going into the you know the system Library oh I can just go get the whole source code for android.jar and and step in through that and then I can look at the kernel drivers for at least the reference implementation of things so that's you know that's pretty wonderful uh it's you know it's not clear when uh I'll have an opportunity to to do iOS development again to do a fair comparison but the taking things in threeyear gaps is great to to see progress where when you're sitting just kind of sitting with it you might not notice how much better things have been getting but the Android stuff looks you know looks at least usable now uh so we've got things like Nvidia Shield which is certain basically a repackaged super smartphone in a high quality handheld gaming format and that's uh you know it's pretty neat on there it's got excellent uh ergonomics and uh good controls for it it's not just a really crappy Bluetooth uh gam pad bolted onto a phone uh but you can still see some of the warts of Android sort of showing through like I was playing Sonic on it and you know Sonic is not supposed to drop frames that's sort of the essence of what the Sonic the Hedgehog experience is about and you do get an occasional stutter that I would lay the the blame at Android and you can go when you ad into an Android system and you do a PS and you look at pages of stuff scrolling scrolling by like what is all this crap and how is it helping make my experience better I it's a shame that that's not more nailed down now some of the problems may be not just scheduling but also the the drive for power management where you'd think that if you have a quad core mobile system that we could just be saying well let my game use two of those cores put nothing else on there and just let me be like on a console and have things where they schedule and you wake up in a fraction of a millisecond but it just doesn't work out that way you can have four millisecond difference in you know when you wind up waking up the next time and I'm still new enough to all of this I don't have the answers at the bottom of it but the you know the platform is not you know it's not as much as you could hope for and while it's all solvable you can certainly look down and say well what do we need to do at the kernel level what do we need to expose to make this better I unfortunately it probably won't get solved like that it'll eventually be solved by just a complete surit of extra resources you know maybe when we have 16 core arm processors we can actually have eight of them to our yourself and you know and let all the other stuff kind of use up the other processors and that's sort of what the uh you know what Microsoft and Sony are doing on the big platforms is reserving processors for uh for the other stuff they want to do and as a game developer if we know okay we've got you know five six seven whatever cores that are really ours we can work with that and we would rather work with that than having kind of random unknown things popping up and bothering us uh you know cloud gaming again I I've been saying this for years I think it will have a big impact and while know oh I on live hasn't set the world on fire for anything uh things are still progressing it's again people will look at bandwidth or latency and think uh it's not getting good enough fast enough but years go by and things just get a whole lot better you know we have a ton of bandwidth coming to a lot of houses now and while it's not everywhere just like you don't expect to be able ble to play your Xbox in the car uh there will be places where cloud gaming is just not appropriate you know in your Backwoods cabin with the satellite internet link it's not going to be a good idea but there's going to be plenty of places where it's going to be practical and it's going to offer really strong advantages for the consumer being able to kind of pop in and out of all of your games as fast as you switch web pages being able to architect games for huge back-end clouds that could take care of resources where it never has to go to flash it's always swapping from you know somebody else's Ram in the cloud I being able to have completely Fair Network play without any local prediction being able to I you know run them essentially as split screen with people in multiple places so there's a lot of advantages there and I think that there's a I I I would I would I would bet that there will not be another console generation like this where we think about the Microsoft box the Sony box uh after this one this one's going to last this generation is going to last a long time and I think it's likely that some combination of mobile devices generic players Cloud streaming and some other devices will wind up sort of fragmenting that around a lot of different front and back ends rather than having the platform that's being developed on but you know whether that's basically you'll have to come back in probably eight years from now to see whether whether that prediction comes true because this console generation is going to have a lot of legs to it so we've got all this power now that we've got on the the PCS you know when you can put in multiple Titans into a PC box and render just absolutely incredible amounts of flops and vertexes and tex so what do we want to do with it uh I think that the most important thing is trying to make the game development faster and easier more iterations on creativity because I do think that we are fundamentally creativity bound at this point we can the graphics a lot better it's easy to imagine the limit case so if you if you look at some fabulous blur game trailer that's made to just make something look unbelievable the state-of-the-art in offline rendered graphics and they look pretty sweet you know there's amazingly good stuff there but a lot of what's amazing in game trailers is the director's view it's the the scripting and the layout of what's happening there something that I would love to see and it would be expensive and I nobody's probably going to do this to satisfy Idol Cur osity but I'd love to see somebody take just a random snippet of someone's game just you're playing this game let's just take 5 minutes somewhere Midway through your game and give it the F and the full offline rendered trailer treatment on there and if you said if we just applied unlimited Graphics power how much better would the game experience be and I suspect not all that incredibly much we can already do a credible job of rendering anyone's Creative Vision in the game we still a avoid some of the hard cases some of the things that don't come out really well you know we're not going to do a ton of uh you know furry creatures and deformable trible worlds and things that uh that just that are still kind of hard but we can do a good job at anything that a creative director can say this is going to be the greatest game ever help me draw it on the screen we can always make things a lot better there's a lot of advances going on there will be continuous improvements but it's not going to be set the world on fire that this is the game that you have to buy because the graphics are prettier uh there are a lot of uh things that we should stop putting up with though things that we are sort of used to like it's it appalls me that we still have people that are targeting 30 HZ games for Next Generation I you know just there's it's not that hard it was hard to hit 60 on the current generation there's you know there are scars in the codebase and on the developers that uh you know that let us make it to the the quality and performance level that we did but we've got a lot more power now and yes you can throw it at doing some crazy pixel Shader to get some like final little bit of physically correct Lighting in there in the onto the screen in the pixels but it's going to be lost in the difference in non-calibrated color between people's TVs and I think it would just be much better for people to concentrate on smoother game experiences and I see this where you see somebody talking about uh a very Advanced rendering technique talking about you know subtle differences in the specular loes of the brdf of this material and there's a demo going with a tear line flying up and down the screen where this is just not priorities are out of whack and we do get Tunnel Vision on our own little thing I mean Graphics programmers as a whole and then you know rendering people in specific there but stepping back and looking at the whole thing you know we should have complete consistent frames presented you know we shouldn't have a tear jumping around uh a problem with that is that it means that you are limited to these nachy frame rates 30 60 120 in some cases and it means that it's harder for performance subtle performance differences to show I mean benchmarking is all about do you get 32 or 47 frames per second and if you're playing the frame rate game properly the locked frame rate you know that's that's not a valid distinction between them you know maybe you start doing like we do with resolution scaling but you know it it takes away one of the one of the bragging rights things I mean that's sort of always the fun of the the PC gaming rig culture of just the hot rotting for the modern days uh but it's not necessarily in the best interest of the full gaming experience rather than just throwing more of a DI you know moving the dial a little further over uh we see a lot with some of the graphics things that we still have dxt compressed textures that you know that turn a lot of things into garbage and it doesn't matter what you do with the fragment Shader there if you know garbage in garbage out but even if you step back a little bit more where the normal map that we use for everything are everybody bakes 8bit normals and interestingly even with the first GeForce that supported uh sort of normal mapping Nvidia uh recognized that 8 bit normals really aren't good enough and they had the 1616 high low format to uh to have like if you want to do a smooth compound curve like an auto body you can't use an 8bit normal map on it um you know that's referenced in global space it's just not good enough uh so making some of these transitions where okay we've got a lot of memory now we can just stop doing some of these things that we're so used to use some more uncompressed textures I you know don't use certain formats uh you know getting to gamma correction for everything and a lot of these things that will make subtle differences but they're not going to they're not going to change the face of gaming these are things that it'll make gaming better but it's not going to be something something on the level of some of the breakthroughs that we've seen in the last couple decades as we got our first 3D experience our first six degree of freedom exper experience you know as we moved to gpus in the first place and started taking some of those strides I honestly don't think that there is another stride of those Lev of that level really left eventually we will get to fully physically based rrac rendering it's not coming in this generation but it's coming eventually and it will win in the long term I think there's almost no argument against it um and it'll be great it'll be better than what we've got but won't change the world uh some of the things like you know that we still see funny in graphics engines when you're doing super sophisticated things animation and looks you know looking at games and you see something that's this gorgeously lit local subsurface scattering uh and everything and then you see like know a box on something just slide right through an arm or all this interpenetration of things that are just blatantly and grossly unrealistic you can say oh but my light transport is on the nose but you know when the when the simulation of what's actually going on is is completely off you're I know it's not the full experience that you're expecting there so a lot more work has to be done on uh with that to go ahead and make simulation of things better to make presentation of human and animation all these different things to climb completely out of The Uncanny Valley in all conditions there's tons and tons of work to do there and I think that the things that will make the largest differen is probably won't be in exactly the pixel rendering I you know I still love it I'm going to talk a lot about it tomorrow tomorrow on where we those final things that we do need to do to get to really really correct and right uh pixel rendering the physics of it what we can change but it's not going to really radically change the uh you know the scope of the gaming so the gpus that are giving us all this power we probably will be able to use more and more of it for things that aren't uh that aren't Graphics rendering you know being able to use it for the simulations and there's been you know for a decade now people have been talking about well we're going to be able to use this this stuff to make better and better physics or better simulations of different things and it's been Troublesome because you can't make physics a fundamental aspect to the game and make it optional or scalable it just doesn't work out that you could have a low-end system have slightly different physics solvers so all we really get to use that for is fluff uh you know you get your cloth waving in the The Breeze your grass and your uh your water surfaces and all that's great we'll get a lot more power from it so faster and faster gpus are still a good thing even if you believe that we're past the kn the benefit curve for pixels all this power will still get used for for useful things and there may be some Tipping Point where we cross a threshold and we really can do something that's important and magnificent that just wasn't possible before I don't really I I don't have a suggestion for that but it's going to be the things that people don't expect where at some level of power and simulation Fidelity we're able to do something really cool so the uh the gpus is one of the trends that's that's going on now that the consoles are than ful going to be leading the way on this is the unified memory uh PC addin graphics cards are very much swimming against the the tide of History uh separate memory buses are the way things have been on graphics for you know since the beginning uh there's a lot of reasons why that's not a very great thing we having to send things across the bus having to manage separate memory spaces we had to deal with that on the PS3 uh with the split memory architecture and life is just much much better when it's all unified uh Intel is obviously bullish on this with their integrated Parts but there's a resistance from uh from Nvidia they have their their strangle hold on the the super high-end workstation class separate addin cards and and they don't have as strong of a position with integrated parts so they they cater to their interest but leveraging the unified memory space is uh is going to be where a lot of the advances in development and interaction with the gpus goes where we should be treating the gpus just like another processor you know we have we talk with four or eight or however many cores of CPUs uh we work out our scheduling systems sometimes we do have our our critical sections and our locks and the things we have to live with on there right now with Graphics that's pretty much hidden behind an opaque graphics driver layer where we need to get to the point where we can treat the drivers can treat the gpus the way we treat the other CPU cores and that's going to involve the the vendors giving up some of their flexibility and that's a hard hard fight where right now textures are still an opaque blob and each vendor has their own way of organizing them but forces are are battling this where we have you know everybody has to support native formats everybody has to be able to render to and texture from them for you know for the Android platform if nothing else for the compositors for running on I you know whatever you're doing with Windows and the Mac so we have to have this generality at least at that level but I am I've been calling on them to document and and expose all of their you know the more the texture formats you want to use the tiled and swizzled formats because there's a lot that we can do to make things better right now it's like it's especially pathetic on Android if you uh if you want to load a texture in uh on Android you wind up loading it into Java making a native buffer passing that native buffer to the driver which will copy it into some other place that maybe will be GPU related and you've got at least three steps going through here and what my vision for the future of a lot of asset management is all of this should just be memory mapped in a 64-bit address space it's all going to be backed by flash we're not going to have this 15 millisecond rotational latency to deal with but to be able to back something with a memory map file we need to know exactly what the data looks like so we need to know what these formats are we need to know what limitations there are on tiling and alignment and so on so this is one of my big pushes I want to see this happen uh I think it makes too much sense for people to you know really argue against uh aside from Strictly parochial concerns about the you know your add-on card won't be able to do this as efficiently if it's going over pcie whatever uh but that's it's something it's a path that we will be evolving to that will make everything better enable a lot tighter interactions it'll be a good thing so the virtualization of Graphics resources finally sort of hit mainstream now with the the tiled textures and tiled resources and this is something that people that have been coming here have been hearing me and moan about this for 12 years or something it's finally uh in Hardware that a lot of you own it's still not what it should be because interactions with the operating system are hamstringing things where I come back to this I really want this please memory map this file you know please let me you know page things out and some of this battle was uh convincing Hardware people that the idea of any kind of a fault is ever a good thing and you can use virtualization in a way like we do with uh the Tech 5 megga texture in a completely non- virtualized way and non-blocking way where you know we let things go fall back automatically update as needed but to be able to use it for something like geometry meshes you really would like it to go ahead and say I don't mind blocking for the 1.2 milliseconds or whatever it's going to take for this to come in from flash uh Hardware people still tend to run screaming at this concept that you're going to idle their cores for 1.2 milliseconds but that's again one of those things that they'll they just need to get over it it's a net positive win for the the entire system to be able to enable some of these things uh I mean to be able to have an entire application where all of your media resources are just set as completely static memory mapped resources that you have pointers to yes a smart app will go ahead and pre-touch manage the paging of all of this but so many loading and saving issues and issues with uh stability issues with games come from loading and managing and caching and dealing with all of this stuff and making a whole class of stability things just just vanish that'll be wonderful uh but it's still an uphill battle and then once they you know once you've convinced them that yeah this is probably a good idea then it falls back to well Microsoft won't play ball on this I you know their their OS core OS is very very conservative justifiably so and there there's there's value to that argument but they're not I it's this extra level of difficulty and Nvidia keeps telling me whenever I suggest something esoteric that I need to start using Linux as my develop system because they'll go hack the kernel for me right there and send something over and it's it's been sorely tempting in many cases uh because that ability to just have something just hacked right in even if you know it's not production but just to be able to answer a question you know that's really great I've got a couple things right now that I'm looking at that I really want these questions answered we'll worry about if it can be product how it should be productized once we learn whether it's actually a good thing to do so another G thing that's uh interesting is the the the place for tiled renderers and this is one that I may have gotten wrong many many years ago where I was I really disliked the power VR architecture way back in the day when you had the you know the voodoo cards and the things that that were sort of friendly to develop for that just blasted right to a frame buffer while the tiled renderers they theoretically traded if they traded uh some complexity in the interface and the lack of sort of direct manipulation for a memory bandwidth efficiency in theory uh the problem with the power VR there was a lot of issues with not having the proper blending modes and there were driver uh issues and the the limits of the tiling architecture had some fundamental limits at that time where if you went up Beyond a certain amount they just broke you know those those are largely fixed now and of course imagination Technologies with power VR went on to largely dominate the mobile space with the tiled renderers and they're really darn good now they're they're highquality driver they highly performant systems you can get excellent performance out of them with good tools to back it up and there was a conversation I remember having at a a 3D effects technical Advisory Board where there was uh one of the other guys was basically saying it's a foregone conclusion that tiling architectures will win and he he was masing that on the fact that render man is essentially a tiling architecture and I'm not sure if that train of reasoning is exactly correct we have because memory has expanded at least in Pace with uh you know with the limitations of I mean the scope of memory has expanded even though the bandwidth hasn't so these massive frame buffers you don't have the same Pixar had to be tiled because of memory reasons not because of performance reasons and that's not necessarily true for us here but they've done certainly a good enough job that like I said at the beginning it was credible that you could have imagined a nextg console by Microsoft or Sony having scads and scads of tiled renderers they they scale really quite well in many cases so I'm not sure where that's ending up uh we do wind up with uh essentially untited architectures for the the current Graphics system so there's still life in them yet and that's going to be interesting to sort of follow on and I am taking way too long going through these so I still think that the the real action and the things that make a difference are in IO devices uh what we do on the screen makes a difference but what you can do that's not a screen can make more of a difference or what you can do to make it better uh displays are remarkably good nowadays I I had a couple experiences just in the last the last couple months where I was looking at something I was in a shop I saw something up there and then I was startled when it moved that it was a video signage rather than I you know rather than just something statically printed there and one time I pulled my my iPhone out of my pocket and it had a card room card key stuck on the top right inside the bezel and I spent a moment staring at this going how did this image get on my on my home screen before I realized like oh that's a physical card placed on top of the you know stuck to the screen of my phone so we are really most of the way there with display Technologies when I you know when I'm mistaking displays for reality but there's still things that that can get better on it uh it's great to see some of the things like viewing angle is almost a Dead Issue um you know there's improvements that could be made in pixel fill when you get up really close like if you look at things under magnifiers like in a head mounted display you can complain about gaps between pixels but that gets addressed as resolution gets better and better and displays are amazingly cheap uh there is a a picture of me that was in some magazine that showed me with two LCD screens Square aspect ratio that were 1280 x 1024 many many years ago I paid $10,000 each for those uh IBM displays and I you know you think about for a couple hundred dollars now gets you a good 1920 x 1080p display uh that's you know the miracle of modern electronics and that's a really really good thing about the only thing that's gotten that's not going well in displays is latency and there is I there's a trend to more and more things that happen between the bits going in the HDMI port and pixels lighting up on the screen now there's there's more adoption of gaming modes the Nvidia is pushing pretty hard on the manufacturers a chip level to make sure that a gaming mode does the right thing that it you know it feeds through directly and doesn't add latency uh it's it's a little bit tricky when you think about things like if you're feeding a 1080p signal into a 1080p display then it's obvious what happens but if you're feeding in a 720p signal and it's got to be refiltered at some level and I mean that's another thing that gets done wrong all the time I you know they're not gamma correct usually in the resampling they they may do various other filters in it but what we need is a bit going going over the data stream and this comes into the whole oh God how do we Corral all of these people so we need something at the driver level we need something at the hardware level then we need something on the TV that listens to this and actually does something different but that's what we need is anything that's generating an interactive signal something where you're doing something and it's generating an image based on what you're doing should be in no latency mode no processing because a lot of the processing that gets done it sort of assumes an imped compressed stream and games are you know are not that and we you know that should be turned off because in some some cases it's generating appalling amounts of extra latency so that's one of the things that has sort of regressed a little bit and that happens in a lot of areas actually where you get bandwidth increases at the expense of latency and sometimes clock speeds go up enough that the latency decreases in an absolute sense anyways but we've hit limits on clock speeds in a lot of cases so now as we're increasing bandwidth we're getting more latency and and there's a lot of just sloppy programming not knowing that it's important in all the in all the different cases but that's something that I push hard on and I'm hoping that you know there will be continued improvements in that uh resolution you know we're we look at uh phone resolutions when you're at Retina Display and Retina Display as it's termed now doesn't mean more resolution isn't uh isn't beneficial uh you can still see aling if you look at a wireframe rendering going on a phone at a normal display distance uh but it's you know it's pretty damn good uh I would be as an an image processing or as an image rendering producer uh I wouldn't be too sad staying at a 1080p resolution and concentrating on getting the anti- Laing right and all that uh it would be it's nice right now and it might be a brief window where 1080p usually does mean 1080p pixels uh the 720p was not like that on hardly any displays they were always resampling and that was a bad thing so I worry that more resolution if you start getting displays that have 2 and 1/2k resolution then those are going to be resampling all the input signals we're going to go through all these same problems again there's going to be more latency added more non- gamma correct upsampling all sorts of problems but it does look like that the industry push for 4K is you know is gaining momentum and it seems almost ridiculous to me that I but it's going to happen uh the electronics people need something new to sell so they're going to make 4K happen uh it's not clear to me that that's going to make a huge win on people's walls and displays uh the you know the selfish thing that it is going to great is it's going to be good for the place where we really desperately need resolution is the head mounted displays you know when you're looking at your your PC your your TV screen resolution is pretty good maybe it's not everything that you'd want but in head mounted displays we really do want a lot more resolution and we're going to get it you know not because uh the display manufacturers particularly care about that market but because they need to have bigger bigger bigger numbers on it you know it's the megapixels race on it and we're going to have uh you know it seems ludic but we're going to have 4K tablets and mini tablets uh in not too many years and that'll make a just a Dandy head mounted display for that so there's going to be some value coming out of it but I think that is kind of going beyond the the knee of the curve in benefits to Consumers again refresh rate is something that has a there is room there's room for improvement there uh a lot of displays now do 120 HZ internally for 3D shutter glasses but very few of the consumer devices take it as an input and even a lot of PC monitors you have to hook it up with just the right cable and enable it just the right way but 120 HZ display is noticeably nicer it's not going to change the way you look at things or make your your jaw hang open in astonishment but it's nicer and smoother and it affects everything everything on your desktop is better with I you know at 120 hertz so there's uh there's room for hoping that that goes uh and in our limited bandwidth budget I'm afraid that it's going to come down to more res resolution where doubling the resolution versus doubling the frame rate at this point I would take doubling the frame rate you know at different points there would be different tradeoffs between that but I just hope that whatever bandwidth we get we can choose to deploy as uh frame rate in addition to resolution the actual stereoscopic for shutter glass TV is still you know it's not a world beating feature when you set it up with all the right parameters if you have your stereoscopic screen set up with the parameters set so that it is an extension your field of view at the distance you're sitting for that size of screen the field of view is correct the projection Matrix is correct and uh the separation for your eyes is correct it can be a really powerful experience you can have that sense of I can put my hand through the screen and grab that thing there but there's a lot of reasons why you almost never actually see those set of parameters in fact one of the the console vendors has a limit on the maximum screen separation which is somewhat below what you would like to put on it and the reason it's like that is because causes eye strain as you deal with your eyes wanting to they know how much they have to tow in to to Center on something but they're focusing it a different amount this is Divergence effect and that is an issue with eye strain and we generally try to minimize it by tuning it down a little bit but you also have lots of issues with with the game content that we're putting in where if you have your monitor a certain distance from you that would mean that in game if you were doing it all exactly right you would not be able to show on screen something that's going to be closer than that uh where in first person shooters you've got the whole issue of your weapon which is positioned in this aesthetically pleasing orientation and field of view for two-dimensional rendering but it's just wrong for 3D and it's not it's not completely clear what we want to do on there where you're ideally you would have something that if you've got your screen out there you want something which would only be like holding it sort of uh you know at at a distance that's not realistic because if you hold it at normally you can project onto the screen but then you've got something falling off at the edge of the Monitor and that's another eye strain inducing thing that causes problems so you know stereoscopic stuff is not too hard to do I think it's an excellent stepping stone towards the the head mounted displays in virtual reality which I do think is one of those can make your jaw hang open and really make a difference effects on there so you can pick it up sort of along the way as a destination on its own it's still not you know not super awesome now the one exciting thing for me that happened just in recent months I is I I had to reevaluate some fundamental assumptions that I had on what's important on the displays and this is this is great for me because this is something that I you know I've never talked about it I haven't talked about in the previous uh talking about displays but the issue of uh persistence and jutter where Michael AB br's team at valve has been pushing hard on this and the first time I went and talked with them about it I was really skeptical where I'm like no I want 20 HZ display that'll fix the the problems that we have with 120 HZ oleds problem solved that was my position uh but they went and uh they built some test hardware and that the best way to win an argument is to you know build hardware and prove it and within uh and it's great the way they they did this they uh they basically knew they wanted OLED display so they they pulled apart Samsung Galaxy 4 phones and they had their Hardware hacker guy WR it make an interface board that lets them drive a pair of Samsung phones directly from a PC on and then they they added this extra circuit to be able to change the persistance globally on it you can tap a button while you're going through changing that and you know I I put on their prototype and 20 seconds later you're right this is absolutely important I retract any statements that I made about the lack of importance of uh you know of persistence it's a compelling thing uh but it makes some so and there's a couple really interesting things about this CRTs back in the old day had essentially no persistence I mean really old CRTs would have long persistence phosphors but the CRTs that we would use for monitors and gaming had extremely low persistence I the Electron Beam swept over and the the the bright spot was gone in microseconds now when I look around online there is a there's a class of people that have been really bugged by the the transition to LCDs for things like apparently watching sporting events which is something that I never really do but the people that really pay attention to watching their Super Bowl game that a lot of people thought the experience once we moveed to Flat panels uh has just been worse than it was on a CRT and I never understood really what they were talking about until I was able to look at it more from inside out on the head mounted display perspective and there's uh there's a good example you the problem is if you're looking at something uh with a CRT or most display now they're on all the time it's fully persistent that was great because there was none of the 60 HZ flickering eye strain issues the displays are solid they look like the reality signs so it was good for all of these things but when you have something move it brings up the fact that instead of uh like moving smoothly it's all here and then it's all here or worse when you're tracking something in a head-mounted display this pixel that's here and it's going to change to something else the same pixel gets dragged as you're moving your head so it smears across this area and what I track what I originally chalked up to motion blur being crummy LCD switching times when I would see like if you put on a rift and you move around you've got these you know everything it can be fine when you're looking at but when you move everything smears and then comes back St stable now it certainly is a problem there's a 20-some millisecond switch on the LCD but even when you do that with an OLED that switches in a microsc you look around and you can't read something as it's moving and you can do a test of this with any sort of smart phone device if you've got text on something you move it up and down you can totally focus on a line and read it as it's going but if you're scrolling it even at a liquid smooth 60 frames per second scroll fast and try to read there and you can't do it the same way because you've got this persistance jutter that blurs the uh the text going so it's an important deal now it's it's questionable how much of a difference it makes for a conventional gaming experience uh I did dig back up an old I old multisync monitor that I could run at 120 HZ low persistence uh which would be you'd think better much better than what this demo was uh and it still doesn't make that much of a difference to me on standard gameplay now I couldn't sync uh the the demos that I wanted to on there because it was an old Square aspect ratio wouldn't take the other stuff so I couldn't make a completely fair comparison and I think that's still a worthwhile thing to do is go back and dig out the 10-year-old Monitor and put your modern best of breed Graphics engine technology experience and see if you can make it you know look better if you can run something at 120 Herz is that better than uh than looking at the or 60 HZ low persistence looking at an existing display and I don't know but for a head mounted display it's a world of difference but one of the fascinating things that comes out of this is one of the other technical directions that might be happening with fully persistent displays that's frustrating that it hasn't is non-isochronous updates where right now we update 60 times a second Clockwork and that is that's why we have uh vertical tearing or I you know SN tearing when you swap and you because you swap in the middle in a lot of cases for a lot of displays what you'd love to do is just say well I'll tell you when I'm done here's the entire frame that would mean all of the you know the hot rotted PCS it's better to go 62 frames per second than 58 it actually makes a difference or even 37 versus 35 any amount that you go better would be an improvement in smoothness there would be a positive thing to that and I thought that could be endr run in by uh mobile devices that care so much about power consumption so that's I and that was in the DVI spec again 10 years ago but it hasn't really come about it probably will in you in coming years for mobile stuff but that's directly in opposition to this low persistence display issues because for low persistence display you're just flashing an image on a regular basis so you need an isochronous display it needs to be happening at a at a particularly relevant uh you know regular time period so there are two directions that both add benefit but are absolutely contradictory to each other which is going to be interesting to see you know kind of see how that plays out um so my hope for this the other thing with that with the the low persistence is that 60 htz is not good enough 60 HZ was really not good enough even looking at your monitor on a TV set when it's a long ways away and it's sort of in your fobal region it's okay but it always flickers when it gets out to the outside of your view I put this in a head mounted display and most of it it's covering your entire field of view and 60 HZ is completely and utterly unacceptable uh I thought that you know 120 htz is certainly good enough uh valves running their stuff at like 95 Hertz which brings up the question that I was lamenting earlier about how why can't we have game developers even make 60 htz games when there's still people looking for 30 htz saying well now we got to do 90 or 120 hertz that's that's going to be a challenge you know that's not happening in the real near future but one of the the most exciting technical research things that I did in the last year was this uh this notion of time warping where you take an image at a particular orientation and rendering and then you warp it to be uh rendered for a different position or orientation or time and that I'm hoping is going to be the solution for this where we finally have a hard real time situation where 90 or 120 times a second we absolutely have to deliver a new image no SS or butts but perhaps that image is coming from a source that's only updated maybe even only at 30 HZ so that should allow completely smooth uh like Mouse look or head look on there for transferring the uh the rotation or even slight movements but it won't handle anything else in the world so you've got uh if you have your your drifting snowflakes in the air probably won't care if they get a frame here or there and a slight stutter but if you've got some performance art thing happening in front of you that may matter so what might happen is you could imagine taking the reviled motion interpolation technology from consumer television and coupling it with the time warp presentation technology so your world is always 30 htz with one frame of extra latency there but you use motion interpolation at an image level maybe augmented by marking or depth buffers or the things that we can do since we know we create this stuff there rather than interpolating it from an impect stream but you use that to generate 120 frames in between and then you Time Warp the presentation so that you have no latency for the the actual viewer so that's something that that might actually pan out so I'm very excited at some of the research opportunities sort of in the next year looking at that where we might be able to kind of get the best of both worlds have our cake and eat it too uh on the input side of things we've got I yeah I've got you know yeah I'm only halfway through my list here [Applause] so the I you know talk about how touch is so amazingly natural it is the most natural computer interface we've ever had and watching kids interact with this is uh is really remarkable uh and to think that we had touchscreens we had light pens and things like that for so many years and it was only until the smartphone and eventually the tablet that this this made its really made its Mark and it's it's interesting to wonder how many Technologies could be like that where they itchy things right now but 5 years from now they're going to be ubiquitous and kind of the Cornerstone of the way we interact with things uh you know mouse and keyboard have you know are certainly the most precise current thing to be interacting with you've got the direct relative motion it's not direct interaction because you're moving your hand and interacting with something there but it's still highly precise know game controllers are still unfortunately really really a bad interface for certain types of things because they're not direct interaction and in the old days uh like a game p for a Nintendo you know a Super Nintendo or Nintendo Gamepad was uh was actually not an unoptimal interface because all you did was Mario go right Mario go left it's a binary interface the button was completely reasonable for this it's only when we started to do turning on game controllers and bringing analog into it that it became a really really bad thing uh or I mean it's better than nothing it's better than binary having an analog on it but it's not it's not good you're using this coar low Precision muscle in your thumb and time integrating and you watch people a lot of times playing games and they're they're even pulsewidth modulating with their thumb you know rather than trying to get the exact analog position and wait the right amount of time you know it's like Snap Snap Snap trying to kind of bump your way to the right position and it's just not good but it's what we've got to to live with there I think that you know some of the controllers where you start to see Sparks of betterness the I think the the PS move it still got kind of a latent kind sort of laggy uh interface to it but at least you're moving something and you've got buttons I think the the razor Hydra with the two tracked things it's it's unfortunate that it's got wires and it doesn't have any feedback inside it but I think something like that will be where we wind up where you've got sort of the split thing you've got two hands that can be moved and tracked independently while still having all the buttons to kind of do your intentional actions on it so I I think that there's things will migrate that way usefully and then some of the other things like the uh the PS3 gun controller where you've got a tracked gun with a a we move and uh and a thumb stick on the end of it things like that will be really excit you know work that's a good interface for things if you had that in sort of a virtual reality environment that'd be exactly what you wanted there again when you're interacting onto the screen it's got that layer of Separation so it's not perfect right I you know the tracking is still a hard problem and it's it's still remarkable to me that something that seems so fundamental about where is this where am I isn't just some nailed down solved problem but it's not uh there's still a ton of research going on again valve has done some really spectacular work with uh with Optical tracking where you've got things like the the track IR that looks at the little dots uh and has works pretty well in here and then falls apart when you get out of the field of view you have the the RF tracking things like the six sense that work more or less everywhere but are not very accurate and have different falloff characteristics uh you know there's inside out I think that my take is that eventually head mounted displays will wind up tracking with Inside Out tracking where you'll have cameras on them looking out at the world because that's how you know that's how the brain tracks and we should be able to mimic that but it's there's a lot of challenges especially for absolute positioning I suspect that there's a middle ground between absolute six degree of Freedom ground truth I here are the coordinates uh that there's a useful intermediate step between just accelerometers and Gyros where you have relative positioning to it and I think that there's potentially benefit for using optical flow uh analysis coupled with the Gyros and acceler accelerometers that are on there to get useful relative motion but this is all stuff that hopefully can be experimented with and worked on sort of in the coming year uh I have not yet tried one of the the funny standing platforms where you you know you get to stand on the treadmill rotational stuff I've talked to some people now that have used it and nobody has said glowing things about them but maybe they you know maybe they will yet uh become something interesting I'm still most excited for that type of stuff for taking the sort of indoor laser tag areas and turning those into virtual reality play spaces which has the advantage of you are completely free to put marker graphics on every wall floor and ceiling so inside out looking would never be outside would never not have a marker I think that's going to be just fabulous when that happens uh you know the audio side of things is uh you know the audio people always feel like the the poor stepchildren on the development team where they get the scraps of the resources uh given over to them and we've we've at least twice before tried sophisticated audio processing and we just kept coming back to it doesn't it doesn't win yet in game when you're when you don't know what the speakers are when you've got the broad range of things there and you've got a a screen sitting in front of people it's been I mean maybe you know maybe it's been the wrong call but we've we've looked at it a couple times and it still hasn't really won but in a headmount of display I think it's going to be amazing I think that having head tracking on it to be able to do that full you know the head related transfer functions so that you can close your eyes and echolocate inside the rooms essentially uh it's going to make an impact there so I think there's going to finally be some happy audio people that will feel the the center of the universe much more than they have uh in years past so I you on to the software side of things now the one of the things that you know I think I feel I've probably learned a lot more in the last three or four years about large scale software development than than anything else there's been the you know the work that it was an amazing learning experience to go through and take our entire multi-million line code base and get it squeaky clean with code analysis and that was an experience that I wish I could just sort of download into everybody because I still run into a lot of people that when you talk about things that are unsafe practices or things that will cause problems that's such a mushy nebulous term it's not like I can count the cycles and show you why my way is better than your way and I don't know how to like win arguments about things like that but there is wisdom to be gained for looking at those experiences but you know getting adoption across everything is still you know is still very challenging and one of the biggest things that we've got in our code base right now is the inertia of the optimization that we had to do for the previous generation you know it there are you know there are a lot of of scars involved in getting the 60 HZ gameplay experience that we had uh for rage and a lot of the things that we did there just don't make so much sense now and extricating them unfortunately isn't as easy as just saying well we're going to chop this out here and here because there may be something in the model importer that goes into the layout code that goes into the manifold generation code that finally comes into the the produced baked model loading code and they were all necessary to get what we you know what we wanted to do achieve and what we did achieve previously but there uh there are things that hold us back now that we need to get out of we need to De optimize much of the code to make it uh to make it much more uh less special case because it's the things that bite you over and over that have uh assumptions that are in multiple places in the system but that's where all the the really good high payoff systems optimizations are where you can recognize that oh if I do this over here and this over here and this here I get this synergistic wi that you wouldn't have gotten from any one place uh but those are also the things that are then harder to dig out of the code base later on and we have a lot of that in our you know our asset pipeline the the way models are imported and things that we've tried to do for convenience tried to do to make things nicer for people that's one of the big lessons is accept lenient was is just a bad idea the web World learned this I you know to the you know to billions of dollars of effort expended on that across browser compatibility from accept lenient where you really want to be hard ass about the assets that you take in especially when you control them you want to say it's either right or it's not right and if it's not right uh just don't allow it don't fix it up and let it go with a warning because nobody will look at the warnings and that's something that we're still kind of working our way through where I we need to get to the place that in the code we are squeaky clean on all this static analysis we need to get that way with all of our data assets we need to be able to say all right all of this data it comes in we apply lint for models to it and get it get it squeaky clean and it's going to be good it won't cause these other problems where right now we you know it's it's sad how much time we spend chasing problems where something comes in it's messed up in some way trying to infer what level of this munging process pipeline I you know made it that way so simplifying things in that ways is one of the big sort of strategic things that we're working on uh but in general this pushed for robustness first then predictability than performance and we've been I you know our code robustness has been really great with our stable code Branch I think that we we've had really high quality software um you know all of the import pipeline I worked on I did that I sort of triaged that when I was doing it because this was millions of lines of code I concentrated on okay here's the stuff that runs when we ship to millions of people this is the code that's going to be run millions of times in millions of different platform systems I I did not pay as much attention to the the loading code the the tools code and so we have more problems with that and we also have the age problem of our toolbase still has some of the very first code that I wrote on Windows you know all the way back to my first NT workstation was the first editor that I wrote and there are bits of that there's still fixed function openg G you know immediate mode not fixed function anymore probably but at least immediate mode open code in there that I you know that just sets things and that is far in away the stable part of our code base but you get into the question of well you know you don't just stop the world and rewrite everything we've done a few big initiatives with a couple uh a couple large scale changes inside there but the framework that it's working inside is still this creaky old thing and actually that one of the things that I spent a lot of time the first six months of this year on was actually user interface code related to user modability and I probably I can't talk about it uh publicly now but one of the big features that we've got going forward was something that I you know that I went back and wrote UI code for the first time in many many years where that was UI code was always the first thing I pushed off onto the junior programmer where all right it's like you you do the menu code and the UI code and I I'll work on this fun stuff I and I seems like I revisit that now every five years like I did uh I did the UI code for the uh the iOS uh you know Wolfenstein classic and doom classic originally and that was interesting to explore some new things and it was interesting to kind of come back now that I've been you know internalized the whole functional programming Paradigm and ways to do things there so it's I we're building up new chunks there but we still have this sort of dangerous no man's land of very old code that's been patched by 30 different people and is is problematic uh you know API wise like yeah we do still have some of the old openg G immediate mode code in there it's it's gratifying to look around and see that in the larger scheme of things across the world opengl sort of w which is I you know which really wasn't expected even uh even five years ago where we were kind of thinking that okay it's you know it's not evolving fast enough Microsoft's doing a spectacular job with what they're evolving with d3d but Along Came mobile and now webg and all of a sudden openg GL is what's you know is probably rendering 10 times as many triangles as I you know as anything else and it's you know it's far from a perfect API uh all of the the implicit state in bindings is still you know a huge wart we still want lower level access to you know directly using buffers and uh some of the other things treating it like a CPU but it's still a good thing and I I can give myself a little bit of a a pat on the back for all the the work that I I did to kind of help make that happen over the years and I think that uh the industry as a whole is still is definitely better off for that and again it's it's not bad to have Microsoft doing a great job with all the the 3D stuff uh but it's good to have a couple competing paradigms and something that was you know that wound up being free for the platforms that are going in thousands and thousands of different things so that's been a good thing I on the rendering side of things uh physically correct lighting it's wonderful to see that that's all the rage now it's all anybody's talking about for the next gen stuff and it's you know it's good it will win it will be better we've seen this you know we've seen this story before with the offline rendering crowd about being resistant at first and finding out that it actually does make their lives easier I uh I made a push for this several years ago and did not get enough internal traction with uh you know with the art team to to kind of buy into it because it requires changing things looking at it a different way giving up your favorite tweaks and hacks and the ways that you know how to sort of beat the current rendering equation into submission but uh you know we've had we're in a situation where we've got a lot more buyin now it's got public mind share uh it's going to be the way things are going we're we'll always be approximating it to some degree but trying to have somewhat plausibly rational albos and having things that make a little bit of sense is just a good thing and in the end it'll be easier to make content that works well so my uh my big software Evolution over the last certainly three years and stretching back tendrils a little bit further than that has been this move towards functional programming style and pure functions and it's long it's been long enough now that I can really take some valid data points from it where I can look at some code I just ran across this uh last week I was looking through some code and I came across this file full of a bunch of reasonably sophisticated code that turned out uh you know we're not really using anymore but it had just sat there completely innocuous not bothering anybody and uh it was it had been written in this functional style where it was all self-contained and you know I count that as a win and there's a number of things like that where you look at this and say okay this is a lot of complex sophisticated code but it's it's a pure function it's just completely compartmentalized over here you pass in stuff you get out stuff and anybody that's not calling that function does not care whether that code exists or not and that could be contrasted with so much other code that has assumptions and interactions and tendrils throughout the system where there are so many things that you know you you set a flag here or you call this subsystem to put it into this state so that something else that you're going to do later will interact with that in a different way and we have sort of the the horror show of that right now is our build game code the stuff that that goes and you say you know build game for these Maps it sets all of these Hooks and flags throughout our resource loading and file system to sort of uh be able to get call backs on these different things and then runs and starts up the map and there's different flags and console variables and it's just God awful and it causes US problems on our almost weekly basis and I've got enough of these positive cases to stack up against this to say it's it's usually more of a pain to try and write something in this self-contained functional way but you know but if you're going to use the code for years it has large advantages I just I'm seeing the payoffs more and more as I you know just on a regular basis I see these wins where that's been happy and good because it was written in a pure form that's been a real pain because it wasn't because it used callbacks and and flags and settings and all these other ways of doing things so I am more convinced than ever that it really is a big win so I've been applying this for years just uh the notion of functional Purity in C++ you know in my day-to-day writing language and I've gone and poked around at uh you know the looked at Haskell in different places but I never wrote a significant amount of code uh in the last year I set out to go ahead and try and get sort of my you know my 10,000 line of code in real functional languages to try and get something that I can talk about their merits in a concrete sense rather than an abstract sense and I had been you know it was one of these things where I programmed just enough that I always had to go review the tutorials each time I did it you know I'd go through the beginning hasool tutorials maybe three times where I'd go I'd learn this I'd write some code come back six months later forget all the ways to to set up the Imports go through it again but finally trying to write enough code to make it stick and the the project that I set out to do on this was I wanted to do something that would be enough code to be to learn about the large systems because when you look through exercises in books they're toys and there there are a completely different set of things that you learn when you're looking at system scale versus exercise or toy scale so it needed to be something with a little bit of meat on it and something that I could reference against uh other things that I've done so what I set out to do was take uh the original Wolfenstein 3D and reimplement it in hascal so I started off with uh you know loading the the assets into there which are in this God awul old format back when I was fitting things on on floppy discs and I was you know Reinventing compression methods for myself so it's some rle on top of very badly bastardized independently invented lzss uh thing that's just you know this messy binary format which You' think okay maybe this isn't the greatest thing for implementing functional Purity but it turned out that that winds up being a lovely little bit of functional code to do that it's it's a fraction of the size of the C code where you hear TCS and people talk about a tenth the size of equivalent code and that's that's probably an exaggeration but I you know I believe that some of this nicely commented clean hcll code can be uh you know a factor of a few smaller than the the C++ code that that I was porting to uh I got to the point of having um you know having the guards on paths walking around I you know running player movement at some basic levels but I was finding that it's it's really difficult for me to work on something that's not work I I you know I have this you know I would I would Pat myself on the back for tearing away and spending a whole hour on my hasal research project every now and then and I couldn't hit it on a regular basis it was just not I didn't have enough time to I couldn't I couldn't make myself take the time I mean I just have that internal itch that's like oh I could be doing something that's productive in the here and now and I know that it's good in the long term but it's a little you know it's a character issue that makes it difficult for me on that so one of the things that happened then that was sort of a a happy uh Good Fortune is I had you know I'd always been poking around one of the languages that I would look at I settled on Haskell as probably my my functional language of choice to follow up on and I I generally still agree that that's one of the strongest uh directions I'm still not completely sold on the value of laziness so the people that talk about L derivatives uh they might have a point I I haven't evaluated side by side so there there may be something there but one of the ones that I at least considered and rejected was lisp and lisp has you know there's a lot of history to lisp it's just about the second oldest well inuse language it goes back to the 50s and I've always had in my head lisp almost up on a pedestal there were things like I you know a formative book in my teenage years was uh uh hackers Heroes of the computer Revolution and a lot of that talked about the MIT computer crowd and you had the you know the building of the list machines and the list Packers and Richard stallman's last days there and that you know that's stuck with me a lot and there's things reading Paul Graham's essays reading the Unix haters handbook and the uh the people that you know that would still talk about these These Old list machines and the values of it oh and the my favorite paper title now is there was a a paper by the uh the people that became the the bracket scheme uh system and the subtitle on it is Revenge of the son of the lisp machine is about programming language environments and I thought that was just uh just fabulous for a paper subtitle I so I'd always had this sort of lisp up on a a bit of a pedestal and I thought well it'd be great to have deep list knowledge because I've heard plenty of people say oh just learning the syntax that's not going to do anything for you you need to to write enough code to sort of get the the Zen of lisp and for years you know for decades now I thought well that be nice but you know I'd pay money if I could get the brain download to do that but I just can't see myself ever having the time to do that as witness by my trying to spend time on hassal I just I can't make myself spend that many if I'm sitting at my workstation I want to work on something that you know that's relevant to the current project but I what happened is I found out that just poking around I forget exactly how it came to my attention but there's a tiny little lisp development environment that you can get on the iPad called lisping and it's it's a sort of visual editor it matches up all your parentheses and uh you couldn't write a real program in it but you can write scheme in it and uh and go through and do exercises and it shows you the uh the arrangements and it's kind of neat you I look at that as okay this is the idea of visual programming is never really taken off and we all still wonder whether at some point we can extract more of the structure of the programs Beyond just syntax coloring and we can do useful things there and lisp with its nature of everything's an S expression and the homo ion of it has a lot going for it and still this is a trivial program but what I found out that was just magical to me is that I have my iPad with me at like all of these times when I'm not at my workstation and I found there were all of these little hours where I could be doing programming if it was on my iPad because I'm at home sitting in a you know on the couch somewhere or I'm on an airplane or I'm stuck someplace I don't have my system so I don't have that same personal guilt trip about not working on the work that you know that I need to move towards the next Milestone and I could actually do you know some real programming so I I I picked up and I spent a decent amount of time learning scheme uh doing some work uh on the iPad writing a few simple programs write a KD tree uh Builder write a parser for something and eventually I did wind up with one of the the grand old books of computer science uh you know structure and interpr interpretation of computer uh programs or sip there and that's I you know it's classic MIT text it's for undergraduates it's supposed to be sort of a first MIT course but it uses scheme as the programming language and it's one of those books up on a pedestal that are on the books every computer programmer should read and you know I started going through I took I actually took some time off from work and said I'm going to do Lis immersion I'm going to spend uh this week doing nothing but list programming I'm not even going to think about working on what I should be working on and that worked I was able to go in and clock a whole bunch of hours and you know get to the point where I I pretty much do get it and I get some of the the value from it and the uh so this undergraduate book you'd think okay why why is John carac reading an undergraduate textbook on computer programming uh but there's still there's value to be gotten in you can find Value in almost anything at that level any college textbook you pick up you can probably get good value out of even if it's a topic that you already know one of the things that's that I I crossed some threshold in recent years where I've always loved sort of reading grazing on technical stuff I've always like textbooks and reading them I you know like bedtime reading for whatever but in recent years I've gotten to the point where I actually look forward to doing the exercises and part of that's because getting older you know things you need a little bit of an extra push to make things stick in your brain and I was talking with my my stepfather was an engineer I about that I mentioned that and he looks at me and says he's SE he's pushing 70 he's like you have no idea John so I'm not sure I'm looking forward to the you know the slow decline towards that but the you know working the exercises in the book uh is you is obviously useful and the thing about uh you know this particular book where if you take your uh you know learn language XXX in 24 hours you just get these very practical little exercises but here I'm going looking up traveling through Wikipedia going okay what's an acrian function what are these Church numerals and all of this math basis stuff that was kind of entertaining to learn while I'm learning the programming challenges along with it so I do think that I sort of get lisp now and I have you know a working ability in hasell and I've got a few conclusions coming from it uh one of them is that there's still the the question about static versus Dynamic and I know there was a survey just coming out recently where the majority of programmers are still really not behind static typing and I know there's the the two orthogonal axes about whether types are strong or weak and whether it's static or dynamic and I come down really pretty firmly and all of my experience continues to push me towards this way that strong static typing has really significant benefits sometimes it's not comfortable sometimes you have to to build up a type scaffolding to do something that should be really easy but there are real strong wins to it I see this in the code that causes US problems in my my toy stuff in recent months seeing the uh in in the hasal stuff that I was doing the one head scratcher where I was looking at something going what the heck is this doing why and I I desperately wanted a real debugger and you know and hasal does not have what I would consider a real debugger the one thing that was there turned out to be the part of the data that was untyped when I was still looking at these two planes of data for the Wolfenstein data and I just had them backwards and it was causing bizarre stuff there that was the only thing that really had me baffled for a little while was a case where proper typing would have removed that if it was dealing with typing throughout it and conversely in the Lis work I had a bunch of cases where I was doing some that was just wrong because the types weren't there if it was statically typed it would have caught it ahead of time now I do see more of the charm now that I've had some experience with a dynamic language I see the lure the enticement of having just throw random types onto anything about not having to you know have sort of template typing arrangements and things you're there's a there's an appeal to that but I think that it bites you in the end if the code lives a long time and it grows enough I think that the value of types is is just super super important and that's something that's it's a religious argument among programmers and I I despair a little bit about trying to win the arguments in a you know in a convincing empirical sense because these do come down to these Tendencies of programmers and if somebody's being I belligerent about it they can just say well I don't have those Tendencies and I can say as you know being a lead programmer over I or a technical director over dozens and dozens of programmers millions of lines of code it's just amazing how you know how many mistakes and how bad programmers can be you know everything that is syntactically legal that the compiler will accept will eventually wind up in your code base and that's why I I think that static typing is so valuable because it cuts down on what can kind of make it past those hurdles there uh so I I'm only getting stronger in my you know my stance on the utility of static typing static analysis I you know don't let those things just kind of happen and get fixed up at runtime so one of my uh my sort of sneaky plan for turning the the hasal work on Wolfenstein into something that was going to tie into real work uh relates to a vision that I have for using uh multicore and game logic in a different way than we're doing right now so the state of where we're at right now with game code is that we run all the game code in one thread because the idea of using locks to synchronize among all of our current game code was just absolutely terrifying uh the heavy weight lifting stuff that we do with Collision detection pathf finding I a lot of the uh the the stuff that would take up tons of work building extra particles uh animating skeletons all that gets run off into jobs that are done with deferred uh calculations it's it's a pain to do things that way you set it up uh but it's understandable where a developer can just look at this and say okay I want to do this next frame I'll set up my query please please go Trace these 50 things and I'll deal with it next time it adds all these extra uh flags and bookkeeping but it's it's still a heck of a lot better than saying oh we're going to merrily run in parallel I'm going to critical section some things that way it leads to disaster there's just no way that was going to work out but one of the things that I've been thinking might be possible is the the thing that I wanted to test on the Wolfenstein and I didn't really get far enough on this to to say for sure yet I hope to follow up and continue it and it's arguable Wen Stein's not enough to prove this one way or the other but if you are running all of your all of your actors or entities in the world independently parallel but it's functionally pure they're passed in a reference to a static copy of the world and themselves they return their new version at the end uh they can't break anybody else because they just they're they can't touch anything else it's not allowed by the compiler uh and that is by the way one of the things that I I feel pretty strongly about why why I went hasell rather than something uh some of the other perhaps more approachable languages it's the brutal purity of it you know languages talk about being multi- Paradigm as if it's a good thing but multi-paradigm means you can always do the bad thing if you feel you really need to and programmers are extremely bad at doing sort of the time scale integration of the cost of doing something that they know is negative I mean everyone will know it's like oh this Global flag this is not a good thing this is a bad thing but it's only a little bad thing and they don't think about how you know the the next five years how many times that little bad thing thing is going to to affect things so brutal Purity you have no choice you know you you do not have an escape hatch you cannot do set bang and change something just because you feel this is uh you know this is a really good thing and it's appropriate here you just have no choice absolute Purity um and so yeah the way I had that all segmented is the little bit of monatic code that happens in the main thread is just one file everything else is absolutely pure functions no you know not even any of the you know the fancy things that can help you escape in some way is uh but so I was proving out this idea that okay entity is running separately you've got the the obvious question well how do you shoot somebody if you can't affect them I you know you you say well I'm firing my gun I I hit him the world says I do I want to make him die so you have to you have to make an event of some kind that gets communicated to uh to the other entity and now all game engines have some kind of event passing mechanisms and in general I don't encourage people using them because it decouples the flow of control you can do something if you can do something right here it's better to do it right here rather than have it done by some system elsewhere but if you're in a pure functional mode that's the only way you can wind up doing effects and it turns out to actually be uh not that bad one of the things that was one of those uh wow this is really clean in hll is in Tech 5 and Tech four for that matter we have an event system where you can bundle up objects with different numbers of parameters and you've got issues with the typing and the number of parameters and it's this whole system for for passing around these bundled events in hasell it's a partially evaluated function that just takes an entity as its parameter and returns another entity there's no system for it it's just built in for the language you can have a you know a due damage function you partially evaluate it with your Five Points of damage that you're going to do and then you pass it so that it's just going to take that uh the last function parameter as the entity and you set that up on your own personal list so every entity makes a list of all the things that they're going to do to anybody and then at the beginning of the next frame you go and you gather everything together you find all of you distribute all of that to the entities the first thing they do they've got the world they've got the list of all the events that affect them from anybody else they apply all of those one after another generating a new copy of themselves as they go then they do their thinking and their processing and generate the final version that goes back into the world so there are uh with this sort of uh method there are um coherence nothing gets out of because everything sort of happens at the same time there's no sense of time ordering but you do have the question of since everybody's looking at the previous frames rendering two people approach a narrow hallway they both say they want to go into that hallway they both think it's clear so they both go into it and then well what do you do next frame they both say well I'm I'm here but two people wound up at the same place so there's a little bit of uh uh resolving that has to be done there and it may just and I haven't worked all of this out but I'm completely uh completely satisfied that it's possible that you you wind up giving some precedence on on there use physical repulsion rather than uh absolute surface collisions on there it can it can all be worked out so this was my uh my Approach in hasell this is what I was building there and it seemed to be working out well uh it may yet fall apart on something you know harder but I had to pick uh a research project that I could conceivably do even if I haven't managed to to finish it yet if I had picked Doom 3 there's just no way I would have personally been able to go through and do this um but my sneaky plan to justify all of this as actually being a really good idea was that while in C++ the idea of using everybody running the way the code is right now would would not work it would be it would be awfully crashing and would have race conditions and it would just be terrible but if you would code sort of in this style of you do your object from beginning to end you can look at everybody else but you can't touch anybody else and you could do everything theoretically all the stuff that we defer right now the tra against the world the animating your pose the building your particles that tie on all that stuff that as long as it didn't take more than one entire frame 16 milliseconds you could have straight line Pleasant beautiful linear code uh that would be clear and simple and easy to step through tell what's going on not this mess of deferred and handed off to different subsystems queried the later frames you could make code that would be nice again it would be code that looks like you know Quake 1 through three where where you're you can just say you start here you go through you do all of this and you're done that would be a really powerful thing for game development if we could make game development twice as easy on the game programming side we'd have twice the iteration time get games done in half the time it would be a huge huge win to make some doesn't have to be an order of magnitude but a factor of two would be Monumental uh so what my thought was is that okay we can't have things that just uh that just randomly look around at all this even if you say you're programming in this look only way but if your pointers let you point at somebody again if it's syntactically legal it will make it into the code base it would be almost impossible to to guard against so my scheme was is instead of taking a just your s single Heap here uh go ahead and pretend you're doing gar I mean actually do garbage collection on it which another is another thing that garbage collection is a benefit for developers it has a bad name a bad reputation in game development for intermittent pauses uh intermittency is bad a fixed overhead we can deal with and it makes programming easier and programming is often the long pole in the tent for getting uh getting games done so anything we can do to make this easier to spend our performance is going to be a useful thing so you set everything up you garbage collect every frame so you make a pass through all the objects in your game heat it's going to take discipline to make sure that none of the big things stay in the game Heap you put all of your assets resources separately managed but the game stuff you have it in the Heap you can make a walk through at every frame so it's this a fairly standard com uh sort of compacting or broken hearts um uh garbage collection where you pass through you copy over the stuff that you still need to the next frame so you it takes twice as much space for uh for your HEAP but the game heap if you isolate out all of the constant data and all of the big media it's not that large I I can guarantee that our our real mutable data you could count it it's counted in Megs not tens of Megs uh and it's probably only a few Megs so it's completely credible to walk it every frame so you walk through the entire frame you Compact and copy it garbage collect to another frame but you keep the old frame there and you memory protect it and when you're fixing up all of these pointers you know as you're doing the the garbage collection you're marking pointers on whether they're in use whether you've copied them over but each entity keeps its own pointers to itself but any pointer has that it has to something else points to the frame that was just memory protected that was garbage collected from so any pointer that you get is either going to point to your state which is mutable or if it's pointing to anything else it's pointing to a static copy so you will bus error if you try to access that it will be hard protected so it's not possible to sneak in something that's going to be a race condition and I think that under those conditions you get garbage collection which is a programming win you get the dethreading of all of these things that we do for performance reason you allow it to run all of these things in parallel so we get more scalability as we go up to dozens and dozens of cores and I think it could be a really huge win but it's a research project it's not something that I can sort of roll into the the current code base but I look at it and think it's not ridiculous that it could be migrated towards and it would have some large benefits so that's one of my you know one of my sneaky schemes for wanting to kind of move the the state of the programming forward there but there's several things that need to happen in I you know before that can really of come to fruition so some of the other things that that are appealing about lisp is well so related to programming the game stuff I have been scratching my head a little bit in recent months thinking that when I look back at Quake one with I did Quake C and that was me just totally winging it on language design you know okay I've got C format expressions and one object type that goes across that's the only thing you use for all of it and it was certainly a huge success there were lots of people that learned programming so they could do QC and that abstraction that's pretty powerful and it has all the damn parentheses and it has prefix notation and these things that people professional programmers that are used to working in C are not comfortable being shown a bunch of lists there are uh there are arguments I'm not sure there are statements I'm not sure that I believe them that novice people that have never programmed before take as quickly to functional programming as declarative programming that doesn't quite ring true to me me uh I think that might be some selection bias that maybe the MIT undergraduates that say they've never programmed do as well uh when presented a functional programming language but there is a lot to be said for the imperative nature of you explain computer programming by you do this you do this if this you do this otherwise you do that and you know from you know from teaching my seven-year-old an eight-year-old I I think that it's I I think that there is a value to the imperative nature there but I do wonder if we had had a more potent language there if we had used something like scheme what more could have been accomplished there that that wasn't in the nature of just uh just quak C and so you know an idle thought where for embedding for for doing a big application I think hasal is just a superior language for what you'll get out of it than Lis there were lots of times I was doing Lis code and I was thinking this would be shorter and clearer in uh in hcle um and certainly the typing is is a huge huge thing but for small things for an eded language like what Quake C was there's a lot of advantage to lisp is so scheme at least is so tiny you can make a make it very very small and yet still have kind of complete power you know you can in theory do anything with even the simplest of schemes there so uh using it for an embedded language is is still appealing for me and I've sort of still got my eye open for where where could I use this in some way because in the end while I wouldn't do application development in it I found it Charming in a way that's sort of I maybe difficult to convey to uh especially non-programmers but it is this very very old language that is still as powerful as you might need it to be it doesn't have all the sort of acrs that you would want in a a modern development system but it's still all there in theory and elegance uh and I I will probably continue to I you know as I carry my iPad around it's sitting in the last airplane flight I took I wrote a sodoku solver in there uh because I was sitting there with my iPad and the Sky Mall magazine whatever so that ability to kind of do random programming at random times has been useful for me it's of takenow hours that I would otherwise not be doing something particularly useful for and let me do something that that's actually creative and programmer growth inducing and so functional team wise I the idea of it's obviously not possible for me to to go and say okay everyone it did we're going to work in hell now I this would just be not good for all sorts of reasons I you know you want people to work with the tools that that they're comfortable with and that's it's actually something that's been challenging for ID uh specifically in recent years in that we have there is an industry way of doing things and there are zillions of people that are just Aces at unreal and unity uh that have skill sets that are hard one and working to something that's that's a very different you Paradigm different way of doing things different tools there's there is value lost to the world when you take somebody that's expert in one thing and make them be a novice in something else there's also the the upside from that where uh when I was going on about has I had one of the developers at ID obviously went and looked into it and he came into my office with this sort of cross look on his face he's like what the heck is all this monad crap and I you know and I sort of understand that where if you take somebody that knows how to program and you say if you take a person that knows how to program NC and you say well here's python or Pearl and you look up the you look up the the correspondences okay here's how you you know here's how you uh you know get the time here's how you uh you know call functions here's how you deal with the file system and it's just a matter of translating what you know into another language uh but going to functional programming is a different thing and it's sort of and to get the most of out out of it you have to sort of embrace newbie brain you have to be able to sit back and say I'm not an expert I'm not a master of my domain I'm going to go back and I'm going to learn something that's fascinating that I don't understand and I'm going to be groping around for a while not knowing what I'm doing but in the end there may be some insights out of it uh but you can't you know you can't drag a whole company into that you know what i' what I'd love to see somebody doing is I do think there are real benefits to it and the hasal ecosystem is improving there are some companies using it for commercial work but it's still not industry grade I but I think that there's a great opportunity for a team if you've got a small cooh hesive team to be the functional programming team and figure out how to make mobile games or make web games or do things that you can iterate on that you can do these six-month project Cycles to to learn the lessons and the things that that we learned 20 years ago with the development systems that we use now and there's an opportunity I think for that goes on for years of evolution people do things hopefully Keep Their Heads above water I you know land the the contracts or selling enough of the different things but evolve to the point where you've got a team of Ace people that can then take on a big challenging important valuable project when you are ready to write millions of lines of hasle code you don't want to go into that blind you want to have bootstrapped it up through a number of small projects and there's no amount of seminars and exercises and learning that you can get to that without going through the projects you know the whole project cycle of starting it out having your missteps figuring out where you went wrong correcting releasing postmortem and going through all of that so uh I hope that are some groups out there that are that are doing just that that are you know a core of people probably from college that I you know that are into functional programming go out do valuable things build the skill sets and turn into something powerful something that we can look at five years down the road and say man they are just kicking everybody's ass because they're uh they made some right decisions early on and have you know built up the skills necessary to do that and I guess that's probably about it so I'm not too far off time so I'll I'll take I'll take questions now for as long as anybody wants so do we have a microphone that I we can get out into the audience do we have a microphone anyone here okay you can yell really loud on say again yeah so I it was interesting a couple weeks ago I was trading some email with Elon Musk from SpaceX and I was you know talking about some like I'm excited about these virtual reality things and other stuff and then he he kind of hits me with this you know well if it's not on the path towards colonizing Mars or making the money to fund colonizing of Mars you know it's just not not that important he's like making me feel guilty for not thinking on a planetary scale I and yes Elon is serious about all that and I I respect the hell out of him on a whole lot of levels I so earlier this year we crashed the last rocket on from armadillo and I did uh spin down most of the development work for this year we're we're talking to some investors different things right now but I I made the play two years ago that I was going to going to go for it for what we thought armadillo needed uh we thought that we were within Striking Distance of the suborbital cargo markets the NASA the NASA Cruiser payloads and there were a lot of people interested in working with that you know we're working with the you know we flew payloads from people at the European space agency and uh some universities locally and it looked like it was achievable there that there was enough market to to make the business case close but we the state that we had been in we got to the point where we were scraping by I mean we had an operating profit uh but it was doing contract work for for other people and I I reached the conclusion that we just weren't going to to get where we needed to go with that there's this tempting thought that if you work for contracts people and you pick the right contracts you'll be developing things that you wanted to develop anyways that it will it will help you towards your goal and and there's companies many companies that have taken that explicit ATT and as far as I can tell it's not working it wasn't working for us we you know we were keeping the lights on we were building Rockets uh but they weren't the tasks the things that we needed to do so about two years ago uh I made the call that okay we're going to cut out all this Contracting work you know no more NASA Air Force rocket racing whatever and I was just going to dig into my pocket and say I'm going to spend what we need to on the development um you know which turns out to be something north of a million dollars a year to run the run the team I you know we had eight people on the team we have a large hanger there's Mills and materials and the the operations of going out to New Mexico to fly uh it costs a million dollars a year maybe we could have saved a little bit more on that but I went into it thinking that we can do this in two years the the arguments that I would make is we had like clockwork almost since the beginning for over a decade we had built two new airframes a year and we we learned immense amounts it was great you know we would build these crazy different things we built the Mand a plane rocket planes I and so I thought okay we're at this point we have the propulsion pretty well understood the numbers close it looks like we can do this we're just going to repackage the things that we're doing into these more tubular sounding rocket uh arrangements and it's it seemed like a perfectly good plan everything's Mak sense but what happened uh was disappointing uh and I still we have arguments somewhat about exactly what I you know what the causes of this are but when we went to do that what should have been faster repackaging of everything turned out slower and we wound up we built uh two airframes the first year and then the second year we wound up only building one airframe uh so things had had regressed and we going slower where we should have been able to turn these things out in series uh we made less progress we so we got a bunch of things flown we uh we flew to 94 km with uh with one vehicle we had uh at least one completely successful recovery with the um or one guided recovery back um on the steerable parachute which is wonderful coming down Landing the rocket in front of the pad the individual pieces came together but we didn't get a a mission that had everything uh work together properly and we were flying people's payloads on this they were sort of like at your own risk I you know you can put them up might get your payload back there's a good chance we're going to crash we had Purdue University was put a payload on practically every single rocket and they got to learn a lot because they had cases where their stuff didn't Trigger or our stuff made a smoking hole in the ground or our stuff recovered but I their stuff didn't turn on right and we went through a lot of useful learning experiences there but we also had some fairly expensive payloads from uh the European Space Agency uh folks that we were flying on there and it was they knew it was at risk and we didn't take their money when we didn't make a successful flight uh but I was disappointed that it didn't it didn't turn out better and I have you know a few theories on why it worked out that way the I you know the point that I try to avoid uh it's a hubristic point is that I just I wasn't as involved with armidillo in recent years uh I had smart guys there that I you know that I felt were competent I respected their uh their directions but uh me not being there left me in the position not wanting to second guess the the boots on the ground when I when something would come up and I'd be like I I kind of think you should be doing something the other way but if I'm heads down on software and they're the ones there every day working on things I didn't feel I didn't feel really Justified and I I never want to be that manager that's out of touch with what's going on in the engineering that's just saying oh back in my days we would have done it this way I so I I let my hands off the wheel there and we can't make an comparison about how important that may or may not have been but there were you know there were some of the things that I I term creeping professionalism that that came into it where uh in the old days at armadillo we were we were all about okay here's an idea run down to the shop you know grab the the saws and the welding gear slap something on the mill and you know and you've got something next week and it's it's fabulous but I as we did more Contracting work I we started having and as people out fulltime we would have uh big engineering drawings lovely stuff comes off the plotter with the uh all the information about it we have our revision controls you know on the and our document systems and of course NASA loves seeing all this stuff it gives everybody that's Contracting a good set of warm fuzzies to get stuff you hold in your hands on the engineering documents and to know that we had a preliminary design review and all this is this makes people feel good when they're paying for engineering but it definitely slowed us down uh some of it may also have been the the move to full-time employment where there was a when we were all a volunteer team when it was just me paying the bills and everybody had a full-time job and we came here and we we worked on Rockets a couple days a week everybody was really focused on getting the work done if you know you're you're going to be here for five hours and here for eight hours there's just you know there's not a lot of time for I you know for goofing around and we and actually we had a little bit of that when more people we sort of hit a critical level when we had a couple more volunteers and I noticed productivity actually dropping we paired out some of the volunteers and productivity went back up uh so there was certainly some of that effect when when you get everybody and it's your job you go in and you know you check your mail and browse the web a little bit in the morning rather than going first thing down into the shop uh and the people that I had this thinking that okay people are working 20 hours a week part-time and they still got a full-time job we should be able to get three times as much stuff done when everybody's full-time but it didn't work out that way at all where when rocketry was people's full-time job they got other hobbies and they did other things with their 20 hours a week on it and then your uh your 40 hours full-time a week is very rarely 40 really good hours for most people when you you wind up having all the little things you do at work they're not really work so you there's a number of things that that that may have contributed and we're still tantalizingly close the one of the bad things that we the things that we didn't do is we should have made series production of we should have made multiple vehicles at once and we did this years ago with our modular Vehicles we made parts for five of them and that served us so well you know we we got to crash every single one of them but it wasn't that traumatic because we had more pieces there and we just built another one we desperately should have done that with these two rocket vehicles I and that that was our that was our critical mistake in the last two years because we should have been able to put more of these together uh creepy performance was another thing where it's I we used to just make everything out of aluminum it's not like the highest Tech material but carbon fiber started creeping into our our development systems here we started you know we started Heat Treating our own aluminum rather than just using thicker aluminum and all these other things that and this is chapter in verse for the some of the errors that NASA has done over the years and it's heartbreaking for me to see my own team following some of these uh these problems uh but the situation we're at right now is things are are turned down to a sort of a hibernation mode where we've still got uh people maintaining the shop uh we are talking to some uh some other investors if anybody wants to kick in a couple million dollars to build Rockets uh give me a call I but it's probably if we don't wind up Landing uh you know an investor it'll probably stay in hibernation until there's another you know liquidity event where I'm I'm comfortable throwing another million dollars a year into things and I the rocketry has always been a it's been a negotiation with my wife uh understandably so where uh she has always insisted that I not be one of those celebrities that squanders all you know all of their Fortune being high on the hill and then penniless someday so uh rocketry has always had this carved out area where this is Jon's crazy money I you know while all of this is to make sure that you know the family will be well taken care of and John's not going to be you know begging begging for work sometime I but I basically expended my crazy money on armadillo so I don't expect to see any any rockets in the the real near future unless I'm you know unless we do wind up raising some investment money on it yeah oh so we do have a mic good uh thank you um you were talking earlier about um the differences in technology between uh CPUs Intel and AMD uh what are you seeing in your opinion between gpus with AMD and Nvidia with the current Technologies and also can you please sign my license plate sure I'll I'll be around here after after the talk great thank you so I uh I have not done side by-side direct benchmarking so you can't uh you know I can't make a definitive statement on uh AMD or AMD versus Nvidia on the graphics card side of things um I mean as far as I can tell when I have looked AMD is occasionally faster uh their Hardware certainly topnotch uh it has always been the the driver and software ecosystem that's made me tend to stay with Nvidia uh you know nvidia's had a really spectacular team there and they've the the AMD people are working hard but Nvidia at a strategic level has just piled a lot of really powerful people onto uh making that software ecosystem really good uh but AMD is continuing to get better Intel is getting better on the software side the you know there used to be just the you know the the horrible there used to be Nvidia with good openg GL drivers and everybody else in various levels of awfulness but now the big three of Nvidia AMD and Intel are all pretty good we can we can ask for better in most cases but I but I I I'm not unhappy with any of that on the on the CPU side we did Benchmark some recent amds for our our CL our baking stuff that we do for making the mega textures which soaks up hundreds of cores of CPU and they were still uh they still lagged Intel on the the performance standpoint a bit there on price performance uh if you could throw enough cores at it they you know they're arguably a win in some cases but we wind up being sort of box limited about wanting to throw as many cores into one box uh and a lot of we benefit from not fighting with blades versus uh having larger systems with dual sockets and everything so they're doing great work and I think that I I mean I I feel bad even having to make the statements about whenever I somebody really draws me out about okay you know which do you prefer well you know I do prefer Intel and Nvidia personally but AMD is doing great work on so many things and there's lots of cases where for it doesn't take a large other reason to overshadow the difference in performance and value metrics that I'm looking at it's great to see a bunch of competent people working on lots of different products yeah I can't I actually asked about whether I I could say anything related to the development and the answer was no yeah can we can pingpong you can come back to you yeah yeah so in some ways the I I do I do feel a little nostalgic sometimes for the Glory Days of microprocessors when I'd be happy getting my microprocessor report to hear Spark versus PA risk versus power versus I you know x86 versus Arm and now it really is x86 and arm I mean power PC is still there for for the Wii U and for some embedded things but we it's clear that we're we're getting a duopoly and I always find it remarkable how Intel had the strong arm technology from deck uh that you know when deck fell apart Intel had the world's best arm processor for years they did nothing with it and eventally divested it and now all of a sudden the arm stuff is super important I so I would betting against Intel on an engineering basis is I you know a CPU engineering basis is probably really really risky now I mean they they don't look as Invincible after sort of flubbing the larabe stuff but they still probably have the best guys at the circuit you know at the low-level circuit uh uh layer when you're looking at things at the process level and how to get the very best out of not some synthesize out of a bunch of RTL core logic stuff but when you're going down saying we're going to make the most efficient thing they are probably better at it than anybody that's making arm processors the uh the baggage of the x86 is is really not as much as a lot of people would think when you can look at it as in fact there was a there was a lovely article that came out in this last year about the trying to analyze the the real importance of instruction sets and there was a bunch of data through there I heard people contesting the the results of it but the result that was reported was that the architectural overhead of the x86 instruction set is really not that much it's offensively ugly from a you know if you look at the the manuals on there about how much is is devoted to all of it but in terms of what it costs you in terms of power and silicon it's not that large now whether x86 is going to be able to compete in a I don't look forward to a multi- CPU like Android ecosystem back to fat binaries and uh runtime translation that's not a happy thing really I and whether Intel would ever swallow their pride and make another arm core I they could probably make a really really strong armed core but that does not seem to be their plan of record uh pushing with Adam I think they're going to have adoption problems with it even when they do probably wind up starting to win benchmarks power uh Power benchmarks and all that um but they they do seem committed to it they know the the writing is on the wall this is this already far more processors are sold there and eventually the raw dollar volume is going to be larger in the AR space and right now the margins are spe spacular for x86 but that's not going to last because more and more things are being chewed up by the you know by the transition to mobile devices yeah okay we have over here back okay oh sorry um given the two new consoles that are coming out and their unified memory and their connection to the GPU have you been looking at anything concerning uh abstract algebra like monoids semigroups concerning that parallel computing power that you now have access to with low latency so uh as far as using the the gpus for the uh additional processing um that's one of the things we're a little bit behind the curve on right now we have I did a bunch of opencl work a few years ago but we are still sort of hand waving our exact strategy for how we abstract between the you know the different interactions for the open CL versus direct compute I and and possibly uh using compute shaders in different places so we're not all sorted out right there yet that's uh I'm not completely positive how many places we're going to use it for there there are complete no-brainers like building particle systems uh some of the particle system works is no-brainer work for doing compute but some of the things collision and some of the decompression stuff there benefits to be had there but they're not as huge as you might think from the flops count uh I am very excited to see how the the tight interactions using shared memory having the unified space being able to just pass buffers around having low latency compute job dispatches that aren't all Ser uh that aren't all serialized through the graphics command buffer there's a lot to be done there and I need to write more code I mean it's been a few years since I wrote any uh compute Shader things and there's there's more that that I need to personally pick up yeah uh I was wondering if you could talk about about itch 5 uh and it's been out for a few years um has there been any big changes with the engine since rage came out so we we're not dramatically changing anything the uh the fact that Wolfenstein is still cross-generational where it's still going to have a 360 and PS3 SKS uh meant that you can't make a super dramatic difference and change in there uh we are lots of things get cleaned up a little bit better we are still pulling out some of the Mine Fields of uh things that we had to over optimize for the previous generation but it's a little bit of an awkward time as we have these titles that are are shipping on current generation Hardware uh the thing that I still that's been most painful about it that I still feel the most is the the dis size limitation where uh it's if we had Infinite Space uh there would be I wouldn't feel at all bad about the decisions that we've made we' we'd quadruple up the the resolution in all Dimensions um but we can't ship that on all the optical drives I think that if we can wind up streaming that from uh from the cloud to all the different platforms there's a lot of uh possibilities there but we still have a lot of discussions internally about the uh the winds versus the costs of static baking for some things how we couple it with Dynamics so there are continuous arguments going on but most of the work right now is getting you know more physically plausible materials bringing the high dynamic range um with accurate interpolation of different things uh stuff that's uh I'm actually been for you know for a couple months now I've been sitting with the artists I moved out of my office and just kind of plopped myself down with uh uh in the middle of the art team so that I could hear them bitching and moaning about the different things and try to prioritize what uh you know what needs to be fixed what needs to be addressed because so much of it has been just education there are so many things there that that we can do a great job of if you approach it a certain way and and as we've hired lots of people like I mentioned before the issue about hiring from different Studios that have different technology pipelines a lot of people just wouldn't know the right way to properly set up your environment maps for things or the right way to tune something or what should be in the different uh the channels of the textures so that's where most of the the visual differentiation is coming out now and it has to be that way because there is push to get your latest bullet point gwiz figure giz feature of okay you need to do this this this and this to be nextg but there is so little that you can just throw additional technology at the same set of media and make a difference at right now it is not the right way to go we do need to get to physically plausible materials for for everything so that you can so the global illumination can can act properly like in Rage we had Global illumination where you could have you could drop a nice light and it could light everything up and because it was baked we could do a highquality job on on it but uh we had the the diffuse map Ms if you look at the textures most of them are are half or a quarter of the level that they should be so there's all these fudge factors going on where Global illumination had in many Maps a 5x multiplier assigned to it because the diffus maps were just completely implausible so there's this education that has to that we have to go through and I tried to do this several years ago and it just it it wasn't the right time we were too busy with rage and the other things and and we had too many entrenched uh sort of ways of doing things but that's at least progressing now and that's that's our big thing much more so than any specific technology because you can't address the technology for a lot of those until you sort of get a handle on the source data and we we haven't had it really sorted out Yeah you mentioned before that the Doom franchise to you meant demons and shotguns I'm curious what the what it software means to you as from the standpoint of a a founder and and of course longtime employee well you know I I've made the comment before that I I really don't think much about the past and Legacy I mean I I commented that I'm a remarkably unsentimental person I that's when it's brought to my attention like at quak con it comes up and we're like look at this it's like oh we did this 20 years ago this 18 years ago look at the you know wasn't it cute back then with the the thrun room at the a small hotel uh but I don't think about it much uh you know I'm happy I'm proud of a lot of the things that I've done over the years but I'm I've never reached a point where I want to really sit back and reminisce about them I have I mean just heard me talk for two hours about all these things that I'm super excited about going forward these things that I want to see happen in the coming year things that I hope I can nudge somebody into changing the way they're doing it so because it'll affect me after that but the uh you know in software with what we've done it's probably still it's going to be the Legacy of I you we invented the first person shooter uh and I Pro I resisted that for a while really because I sort of had this hope that uh you know that there could be other directions and things that that we do uh early on in the if we look back to the very first projects there we went you know Kean to Wolfenstein to Doom to Quake there was this intentional sense that we are hopping between genres because you know we want to show that we're not tied to you know to just what we've done before but in the end when things reach the scale that we're at the development team size you do kind of wind up sticking with what you're known for and you know what you've what you've done and that really is the the legacy of ID software and it's certainly not a bad one I think fpss are going to be here forever just like we've you know we will always have driving games and fighting games we will always have FPS now and and we started it so that's I I'm pretty proud of that all right so um in the past uh the PC kind of drove you know gaming and the power and everything and when you uh came up you said you know it's a it's kind of changing with the new consoles and everything with unified memory and all that um AMD and ATI merge they did that that kind of made it easier for them to kind of jump in and do the whole unified memory thing um how do you see that changing as far as consoles now things are going to get simpler easier so it's more of a of a of a streamline Thing versus PC's having to use brute force to do everything to compete so in the near term it's definitely good for everybody PC console developers Publishers it's good all across the board right now uh it will allow developers to build things in a console style one thing that this would have I never would have been thought this a couple years ago but AMD has the opportunity where it's not out of the realm of possibility that we they could expose a lower level API than openg G d3d that's here's all the buffers here's how you write it from a console because we already have a nice layering adding another one if there was a real value there there might be something to it it's not clear to me that I I think PCS will have their traditional tack of they're just going to be a lot more powerful uh right now you could certainly buy an outfit of more powerful PC than the nextg consoles it be expensive but a couple years into the time frame your run-ofthe-mill PC will be a super set in every way to what the consoles are uh and in fact you'll see probably damn near the same chips in PCS and then you'll see those with more cores and higher clock rates uh so there as always the PC will be able to soak up a fair amount of inefficiency it'd be I guess I despair a little bit of of actually vanquishing the inefficiency in the PC we still win in the end just by clubbing it with more and more root Force it'd be nice from an an aesthetic sense that the inefficiency not be there but I guess I'm a little bit more at peace with it uh but the fact that the the development of the PC and the consoles is coming closer together but both those platforms should be looking over their shoulders at mobile and web and what's you know kind of going on and driving the development issues there because those are are the things that in the larger world when you look at the billions of people playing that um you know they're not it's not an argument between p and console it's a completely different argument and those are probably going to have the broader repercussions as we look you know 5 10 years down the road yeah um do you think those issues can be attributed to Mega texture and if not are you going to use Mega texture in future ID games so the performance issues on uh like on the PC we had the disastrous driver problem and that we still have driver problems that that crop up um some of that is the fault of doing something that nobody else was doing there's a uh there's value to doing pretty much exactly what everybody else does that's the well- tested path nobody will accidentally break something uh that relies that everybody's doing we were clearly doing something off the beaten road with uh with the mega texture The Continuous texture uploading uh all that entire pipeline of things and it you know it hit the goals that we were looking for on the consoles uh on the PC it should have just been Superior to that but we had you know we had the disastrous driver problem which colored a lot of people's views of it and we still now occasionally have problems where rage starts running crappy on somebody's system um it's still not a foregone conclusion that as it stands right now yes we are still using Mega texture higher densities uh building with more texels on there having more in the back that we can uh we can call you know call as needed even if we can't ship them necessarily uh but if we don't the step that I would like to take uh datawise and we just can't afford it on the current generation of like shipping on optical media is if we could if we could ship a diffuse map along with the uh the pre-baked light then we have the best of both worlds we have no downsides to Dynamic lights uh where right now we have to just approximate them but you can also have the the full glory of the completely baked worldview and we're still not I'm trying to push more towards we have all of this but we're far from tapping out everything we can get out of it where we have some technologies for letting people matte paint directly on the mega textures that never got followed up on uh we should be able to fix every Crossing Edge in uh in the world which is something is not at all plausible to do any other way go ahead and Stitch across uh every time you have two things uh meeting don't have that edge seam there that nothing can get rid of but actually cross fade a little support texture around it uh the ability where you know where we have used it with the stamping has allowed just fantastic advances in something that has no stability issues or performance issues there's big winds to be had there that we didn't make the most of and we're still trying to to make more out of them in the end when you say long term data always wins so something like Mega texture will win in the end uh whether it's you know whether it's dominant in a coming generation or it takes all the way to the transition for cloud gaming when you look at a cloud gaming world where you're running on a data center server uh having it's completely credible to consider having a petabyte of data for this entire glorious world that's you know that's built with Cloud resources rather than some local Farm uh over and over in almost all cases uh cleverness and compression Technologies eventually fall to just put the damn data there uh it's possible uh we were pushing a little bit too early on megatexture because it just barely worked on current Generation stuff uh but I think inevitably that's where things will be your tiling textures is just a very peculiar form of texture compression uh and you you expect to be able to have your entire surface spread out and boy I really want to be able to reference the surfaces as direct 3D coordinates whether it's you know some kind of voxelization or something but I hate unwrapping pelting things and adding gutters around it and all the issues that that adds to us but eventually you just want the materials of the world what what it makes up and it's going to be something like a mea texture on the topic of uh virtual reality and uh dring a display to your head pretty much uh what's your opinion on doing something with pulling in input from the outside and doing something like dual Google Glass and having augmented uh actual 3D augmented reality okay so Google Glass and augmented reality in general the vision there is that you've got you know Terminator Vision where you've got text appearing in front of everything you know the floating people's names over their heads and that's going to be huge that is going to be a many many billion dollar industry it's going to I think take over the way smartphones have I think that uh when that finally becomes real uh five years will go by and everybody will have augmented reality uh goggles now it's important to realize that what Google Glass is today is not that Vision at all it's a tiny little cell phone screen in your field of view I there are way hard problems involved in doing the the vision of a M mented reality about outlining people and floating things and change you know adding solid objects into the world and uh abash and his team at Val spent you know quite a bit of effort on it and uh I was pushing them really hard like VR we can do today we can make this awesome we can make a difference and it'll be spectacular the AR stuff it's going to be a bigger deal than VR eventually but I don't think there's any guarantee that eventually is even going to start five years from now I think we're going to see a lot more wearable stuff it'll be the built-in cameras it'll be the little info screens what Google Glass is today will productize and will be valuable but the game World of what you imagine that about I want to put virtual people into those seats you know I want to you I want to have the crater open up in the floor in front of me that's not uh coming in the real near future eventually it will but there are many hard problems to be solved uh before we get there but the VR stuff's coming soon and fast and it's going to be great yeah um so when we look at like the future of voxels versus rate tracing just if you could elaborate on that a little bit um you know do you think the idea of the infinite detail engine even though that's largely like procedural um do you think that would be maybe a stepping stone to rate tracing as you know polycal increase or really do you think that's not even going to be considered in the future as rate tracing Technologies meet performance expectations all right so eventually rate tracing will win I think that that's a uh it's winning in all the offline space and for all the same reasons it will win in software or in realtime applications but we still have there's a lot of leg room still for rasterization especially as we look at mobile and lower power constraints uh it was actually I was stunned to find that I got disclosed on costic Graphics technology which is Imagination Technologies bought them they're one of the mobile companies and I've I've looked at three or four Ray tracing architectures over the years that have all been hopelessly naive uh it's somebody that either knows enough about Hardware to you know build some form of a array intersection engine and thinks oh we can use this for games with not understanding any of the many manifold issues that are involved with it but the uh the costic stuff is is smart they've got some uh it was one of those where I learned a couple things reading through their I their technology paper that it's still not obvious to me that it's going to be a that it's going to make games productized in this generation I I I don't think it closes on that you can do a real-time rate tracing game but it's hard to show how it's going to be better than a rasterized game my pitch to them is if you you know something that would be unique on there is you should do a uh a jewelry toy kind of like a super Beed because that's the one thing where you really do care about your index of refraction and your colors on there that would be the the perfect poster child case for uh for a ray Trace chip is you can do this awesome jewelry tool and we Jewel has sold a heck of a lot of copies so that might even drive adoption of it um but getting your foot in the door with something uh as a way to once you're there you can evolve and you can get better and you can eventually start encroaching on the uh the standard Graphics territories um and the reason why rate tracing is going to win is because it is understandable and tunable um the you know there's the the great comment that somebody had on Twitter about looking at this 800 page book on Shadows for gpus and you know they're just basically saying if if you need this to do Shadows you're doing it wrong because in rate tracing all you do is you distribute points to the area source of your light and you you know you trade off where you put them and how many you do but it's simple it's it's a loop this big to make all you know all of your Shadow calculations basically and simplicity has a lot of value there's the lesson of the year really as or of the age as we're going into is that the the things that we do we would do for optimal performance previously uh are many of them are ready to be traded away for better understandability robustness developer predictability and so on uh so voxels and in terms of other representations of things it still it still seems unesthetic to me that we're Ray tracing against triangle representations uh but it seems like it's going to get good enough like it's going to win and whether you're tracing it's a triangle or some kind of meta blob Splat thing it's at most a linear difference in performance so it's not large and I was I would have been surprised years ago to think that some form of rate tracing and it does wind up all being against triangles I thought that there would be something else uh that there would have been large enough benefits but it's not looking that way now because it comes back to the tyranny of the pipeline we have everybody knows how to make these triangle meshes now but if we go to say well now we need to build some kind of voxal thing they'd still be making triangle meshes that we voxelize at some point which is then sort of going through this additional media trans transition and if you had the ability to just use the triangles um but an interesting thing about when you do look at the triangles now and it's crazy the performance that we have on gpus that you can load up 20 million triangle models and they work you know it's it's uh it's fabulous when you think about it but uh what's surprising is how bad many of them look if they're if you've got any vertex operations going on whether it's vertex specular shading when you've got 20 million triangles and you're drawing it on a 2 million pixel screen the triangles are just randomly coming and going and there are significant additional aliasing anti-aliasing challenges to be had there to your left uh one thing that I haven't heard you mention for a while is um amd's um virtualized texturing um what what is your views on it are you would you consider incorporating into Tech 5 maybe for future games or anyway yeah so uh we've been working with that quite a bit and we were obviously the the implementation test for the extensions going going into it uh it's yeah I think it's I've been push that had been my crusade for a decade to have virtualization of these different things and it's I it turns out that the the nitpicky details about page replacement and especially feedback from it is harder than I would have thought 10 years ago you basically wind up having to almost implement the the tech 5 feedback buffer sort of thing with all that messy stuff I do wish that there was an ability to just have it demand paged to say back this by my flash drive I think that that would be a a big step forward but that involves interacting with the operating system I I think it's about as good as it we we were vetted on the extension we were part of the the specification process for it and it works uh it gives us some real benefits in that we can get proper anisotropic trilinear filtering with a single with I without having to go through our megatexture pipeline for all these different samples um so it adds some benefits it's surprisingly not quite as much of a huge win as you would think considering how much code we do on the megatexture side of things already the biggest benefit will probably be from letting us take advantage of all the memory space in uh certainly the consoles it's unclear how much the PC where we want to be able to just say okay we've got this many gigs we're just going to let our let our Pages fill up everything where right now we say you've either got a set of 4K or 8K textures and this will give us the flexibility to just take up what however much we want and there's still the the tradeoff that we have to make between how much space do we set aside for our compressed cach where it's all I you know DCT or HD photo uh compressed at very very high bit rates versus how much do we set aside for direct access by the virtual textures and one of the things that I'd love to do is and it's not clear whether this is going to be feasible even on the the current generation is I'd like to not have I'd like to use uncompressed textures for everything if it means less pages but we can have we can at least take advantage of more memory there but one of the problems right now is that we use a ycg uh color space which uh we convert to RGB before we do our calculations but you can't apply gamma correct texture filtering to ycg since it's zero it's biased into the middle I I'd like to be able to just use uncompressed 888 RGB for the the stuff so that I could use I could get gamma correct texture filtering be uh proper throughout the whole thing but that's a lot of extra memory you looking at going from uh eight bits per pixel to at least 24 if not 32 if we don't pack anything else but that's where we look at those things at the memory we've got all this memory I don't know what we going to wind up blowing it on we always wind up out of memory in the end it seems inevitable but getting rid of some of the compression artifacts there seems like it would be a positive thing um this past generation with the consoles has been one one of the longest ones uh we've had in recent history um do you think that it will this new generation with the PS4 and the Xbox one will last as long as this p console Generation Um and will it integrate augmented reality and virtual reality so I think that it will last at least as long um well there's a couple arguments that could be made I don't think it's going to be supplanted by a different architecture in that same time frame now whether the platform can evolve is another question they are getting more operating system like in the way you interact with them we don't get to you know get our grubby fingers on the bits anymore even on the consoles uh so I'm sure there are strategies for evolving the platform as we go forward uh but I don't think there's going to be much of a push for uh for another generation for a long time really we could be doing great Innovative work even on the current generation for many more years yet it's not like anybody's seen everything that you can do if you take as a challenge well but I really want to do real time Global illumination uh at 60 frames per second I can't do that Wham me on the current console uh yeah you can say we need new generations but if we're concentrating on you know on making great games I don't think that you know I'll be surprised if we see radically better games enabled because of the new capabilities on the consoles and I've thought for a while we're at or past the knee of the curve and the payoff benefit so we're going to be definitely on the the gradual sloping part of it going forward so I don't think that there's going to be a push for maybe we get to the radical Ray tracing uh platform at some point that's it's probably not worth changing off until we get to that point uh but whether we get something like cloud gaming owning large shares of the market before then uh or people using mobile systems where you just play on your mobile phone it shows up on whatever screens near you that could become dominant gaming platforms but a traditional next Xbox I yeah I think that's a long ways off and it there's a credible argument to be made that there there may not be another console generation as we know it now but don't know yeah do you think that the performance of non-volatile memory like ssds and whatnot could ever reach a level to where it could be combined with system RAM and if so how much simpler would that make everything for you so ssds have been the greatest performance thing that's happened in recent years much more so than multicore has been for us the the difference that you get when you move to an all SSD system it feels like the good old days when you just doubled the clock speed on your processor and you're like wow my computer's really fast I so that's been a really great thing and I think an interesting thing that we've seen is while it's easy to say We'll always suck up whatever resources are available and I I just you know said that a moment ago but what's been interesting to see is while there are definitely exponential Trends they don't have the same factor on them uh we are seeing clearly that people don't fill up their hard drives and people are able to migrate to smaller ssds so I think that's a good thing and having there's something to be said for the silent PC as well that you know we don't have these big leaf blower FL fans on things anymore it's it's nice to see something with no fans no air movers maybe it's not as powerful but it's not getting hot and it's not you know requiring to be blown across there so ssds have been really good now what will happen in terms of integrating it like more directly with the system I think that there's a clear path to that and that's this embracing memory mapped data structures where there is a big benefit to doing that from a stability standpoint when you actually use memory protection and you say this can't be touched we say all right we're you know looking towards the future we've got 100 gigs of something that we're going to just sort of have ma tier referenced and whether it's backed by some combination of uh cache SSD going to hard drives going to the internet or dropping the hard drive in between and just going Cloud to to SSD there's I I think that that's a clear and obvious Direction but it still requires getting the OS and the GPU people together but I think that's going to have big payoffs you can't just say well I'm going to take my latest AAA game map everything there and have it run great but the thing is it'll actually run better than most people I think would credit if you did no optimization work on that and if you just do a little bit of prefetching for setup uh I think that it'll be an improvement it because it'll make software better it'll let people build games without having to worry about how you're marshalling your data around I think it'll be Improvement in in a number of Dimensions uh thanks for humoring us and staying here for two hours I really appreciate it um I was just wondering about your thoughts about strobe lighting and how that affects motion blur with current LCD technology okay so the valves low persistence demo is is a global global shutter so it strobes the entire screen and interestingly they they think they've identified some other perceptual issues with that where you know they think that when your eye makes a very fast Cod that if it's going from a point where maybe there was no illumination there's a perceptual thing that they're fighting uh the other way of doing uh temporary illumination is with a rolling shutter where you like you're drawing the scan and it can be blanked out some distance behind instead of overwrapping and continuously being illuminated you have maybe a quarter of the screen as a bright band going down I I have a theory that I'd like to see applied to this where instead of linearly addressing the frame buffer I in a standard scan pattern if we went to a deep inter leave pattern say eight or 16 wide interleaving where you're getting scan line here here here and here and then here here here and here and they're blanking out 16 or however many scan lines a tenth of the frame behind I think that might have a a useful benefit there where you could be at no point would any part of the screen be black you would be within say 16 I 16 pixels of a lit line so perhaps that alleviates some of those perceptual issues um and you could also then look at it as a thousand frames per second uh display that's very sparsely set out and you could go ahead and render a thousand unique frames or do the time warping thing to that and that's uh that's probably the best way to to wind up doing this but the fallback plan sort of Plan B is is LCD backlight issues where uh the the current light boost Technologies on the the Nvidia light boost monitors that the high-end Asus ones they do a a remarkably good job at this where they they Flash the backlight for four milliseconds then they turn it off while the actual LCD is updating then they flash it on for the alternate eye and while this was designed to make 3D gaming work better uh to have the glasses work better but it's a really good thing for any Dynamic fast motion thing as long as you're updating at that 120 htz if you're updating at 60 and you're double flashing it's got this weird artifact to it that's perceptual if you look at it a high-speed camera it's it's doing exactly what it's supposed to but if you update 120 times a second with those blanking it's better than you know than any normal 120 HZ monitor I that's still a global strobe so the alternate approach is and I know I've seen somebody online doing the home brw project with a string of LEDs that they're strobing in order uh to do that and that's that's a that's a valid direction to go that can work even with slow switching LCDs because if you have an LCD panel that takes 15 milliseconds to switch and there's a lot of them that are more than that like the current Rift one takes over 20 milliseconds to switch and some of them are even worse but if you're down to 15 if you're over 15 there you're screwed there's no way you can get the low persistence down but even if you're at 15 you could have your scanning in changing the LCDs I but it flashes it just before it's going to change and it goes through so you're flashing what's 15 milliseconds had time to settle there and then it goes on and then you can Flash the next bar and turn that one off while it's just starting to do its long change period so I think that's that's Plan B for headmount display uh Technologies if you can't get awesome fast switching oleds on it which would be the oleds are the best way to go if if we can get that that's what we want uh failing that the L LCDs with a strobing backlight but the problem is L CDs are getting so tightly integrated that in the old days your backlight was this separate assembly that you could just like take some screws off and pull it off and look at your transparent LCD which is pretty cool but now everything is these laminated packs of super high technology and getting you know getting into there to be able to strobe them is uh is an open Challenge and whether headmount displays will ever be a large enough market to make an LCD manufacturer make some change like that when they're talking about millions and millions of mobile phones or other mobile devices I don't think you know it'd be it'd be great if the head mounted display Market had that kind of clout in a couple years but I I wouldn't hold my breath for it you talked earlier about some of the um issues with control devices uh what do you think are some advancements to be made in that field in the near future so control devices I think that the the near-term pragmatic things are going to be a split controller whether it's like a HRA or two moves uh where you've got motion and buttons and position tracking um full body tracking things like the uh the leap gesture tracking I think those are going to have value but it's kind of got the no button mouse problem with the the connect and you start having to do gestures to trigger things which becomes this very imprecise uh Direction so I don't I don't have a wish list like on the output side on display technologes I know exactly what I want I I don't have that so much on the input side if anything I'm I've got low expectations where I'd be thrilled to just get a great position sensor a position sensor that had accuracy repeatability no drift I that would that would make me super happy and it still seems to me that while the action does seem to be in Optical I am continually Amazed by the fact that uh rtk realtime kinematic GPS systems can give millimeter level uh relative positioning from GPS satellites and these are going going 8 kilm per second up in space you know multiple ones and you're getting millimeter level changes in distance that's magic you know the fact that that can be happening and that we can't do better than that on a desktop with like some really loud carrier wave emitters and place down I I still you know I've gotten fallback research that I I want I'm trying to draft one of the one of the armadillo guys that's an electrical engineer into working uh working up some prototyping stuff on that but I you know there's there's camps the optical stuff is certainly the one that's working best right now uh but it's still not a solved thing you still have to deal with occlusion issues and stray light Reflections all sorts of other problems that can be there but the optical stuff's looking pretty good yeah we know that as um Technologies uh be become cheaper and the uh capabilities of an individual grow that eventually Indie um Indie Games will be able to craft experiences similar to tripa games um I have two questions related to this um first off how far into the future well how far off are we from Indie games like a a just a single person being able to craft a similar experience to um Call of Duty Modern Warfare 3 and what do you think this spells the Doom of the Triple A J triaa game industry or do you think they'll find another Niche uh I I disagree with your premise the uh the triaa games are they're the result of an army of people I doing their work and it's not programming I mean programming is one of the you know one of the tent poles there but most of the work is in content creation and all the technology in the world doesn't suddenly turn you into an awesome sculptor uh now you can look at things like open asset libraries can start having a really really big benefit there and that's where more so than the pace of Technology the change in an ecosystem like that is what you know what you should be hoping for to to advance the state of small scale development um and there's plenty of possibilities there and when you look at what's happened with the open things when you look at the magic of the web and uh or how Androids take taking off things that can be done broadly distributed for free uh can have huge impact they can have really large effects on things um but no technology is not going to all of a sudden let you uh let you produce a AAA game we can hope more and more stuff gets open source that I you know that when you can just and a lot of games like mobile games are done like that now where you grab a grab a physics engine over here you grab a rendering engine from over here and that's going to continue and that's going to get better and better and it's remarkable what people can do in the mobile Space by cobbling things together there but it's not going to take you to Modern Warfare it took you to a different place and that and that's that's really great that it worked out this way because the the Cobble together stuff didn't take you to Quake three it took you to Angry Birds and cut the rope and these other things that are different and that led these developers that nobody knew about to become Titans in their industry I think that there's going to be a similar thing with virtual reality I think that the people are going to do the the same type of thing it's not going to be the big companies that uh that sort of set the language that other people work within later on uh but programmers today we do have the resources at our disposal both hardware-wise and what we have available in the open source world to be magicians you know you can call forth the powers of all of these different things and very much like Fantasia though most of the Wizards don't know what they're doing they call forth Powers beyond their Ken and uh but it still works out more often than not so it's a it's a good happy thing so I got the better wrap up soon so any any last important questions back there um yeah uh so now that Tango works is uh using a tech 5 how has it been um how has the experience been with an Asian developer like you guys haven't worked with an Eastern team before and there's another question um the moding scene do you see a tech 5 being thrown by It software like to something like uh udk like do you foresee taking a tech fight to that level or is it going to be on a per game basis the studio decides whether it wants to do modding tools or not so uh we've had we had a lot less interaction with Tango than we had with machine games um they started off changing the the technology uh more radically at the beginning they they had a lot of different things they wanted to integrate um I mean there you know they're quality people obviously and they're uh they've got work that's come along but we just you know I had I've had almost no interaction with them honestly uh some of the other people on the core Tech Team have have done more support but they've pretty much done things all on their own uh you know we're deep involved in Wolfenstein at this point on getting everything sort of together in the push towards shipping but the Tango has been U sort of off on their own largely uh regarding modability and things that's some of our major internal directions are are working on improving that because we you we totally fumbled that this generation and I don't think that uh I don't think it's likely that we can expect radical improvements in this current crop in like Wolfenstein and Tango uh hopefully better but there's still um there's work that needs to be done on a on a corporate level and how we prioritize things in our games how we prioritize our releases uh that effort needs needs to be expended on but that uh the thing that I talked about doing user interface work for six months that was my contribution towards how we're changing this going forward but it's not a near-term thing and there's other significant technology things that I want to change that will be better for us and also better for enabling the you know the external creativity so I think I'll I'll wrap it up now but I'll I'll Mill around for a little bit afterwards Back To Top