all right if I still have a voice we're going to do pitch a few questions to Stephen so Stephen I I just I need to ask what the last few weeks have been like for you I just have a a data point in the numbers from our perspective here at the AI and education Symposium we decided to do a virtual conference this year because we sold out the inperson so quickly we set up things to register we had we were excited we had like 34 people registered virtually and we were like that's great we're going to like really get to know them and then I went to sleep and I woke up and my email was just scrolling with people who had registered and by the end of 3 days we went from we went from 34 to 560 so what happened yeah it's been a it's really crazy little stretch I mean the other anecdotal thing that I just heard from my old friend Rufus is that he was waiting in line at a to have breakfast this morning with his son and he was convincing his son to maybe go to this thing and was mentioning that notebook LM was going to be there and the people in front of him were talking about notebook LM and playing a an audio overview just randomly so uh it's in the air right now it's pretty pretty insane it's interesting because I think a couple things happen um The Voice models are really good the the the audio technology is really good the way that the audio overview was designed it was kind of a separate team at Labs that originally kind of did it and then we integrated it into notebook LM the the prompting that elicits that conversation is very powerful but the other thing is I think that most ordinary folks had not yet experienced an AI that was grounded in the sources that you gave it like that most people had just had an open-ended chat experience and so the sense that you get when you listen to an audio overview and because they're audio they're sharable and so it kind of went viral because of that but that feeling of like oh this AI actually seems to be displaying some quite sophisticated knowledge and and most importantly I think the ability to find things that are interesting or surprising in your material so you you know people are out there trying to kind of give it the most boring or or just bizarre material you can get and the hosts are just incredibly good at finding you know some nugget of of insight or a good story and whatever you give it so I think it's one of these Technologies where like three or four things came together at at a really nice time and and that's why we're seeing the adoption we are and can we jump into why the product has gone viral so um it's actually gone viral among students quite a bit and it's because um so imagine you're having to read a full chapter of like a really boring textbook and instead what you do is you upload that textbook into notebook LM and it generates a podcast for you that's super engaging and interesting and it outlines and and actually like like highlights all of the super interesting parts of it so that you can actually talk about that chapter in a really interactive and um engaging way when you meet with your teacher or your friends so I think that's one of the reasons why it's gone so viral and so I'm curious to know you know if students are able to create a study guide create an FAQ create quiz you can create quizzes to quiz yourself on the content that you're actually uploading and then also be able to um you know listen to this super engaging podcast that you listen to on your way to school instead of actually reading the text how does that actually um uh change the way that students are going to consume text versus consuming let's say an audio or image file that is easier to consume but perhaps isn't quite teaching them the right skill set yeah I I there's so many interesting directions that this is going to go I think um and and this is happening across all of these Ai and we've seen such amazing things all all all day here at this event but on some level the the the macro thing I think the highest order thing that's happening is exploring information through conversation is really you know because that that is true of the text interfaces as well as the audio interfaces and and of audio overview um you know one of the things that I do when when I'm testing notebook LM and new prompts and things like that is that I have a notebook that just has a full text of one of my books and I'll just go in whatever the new kind of tool is and I'll just ask questions about the book and explore it and that's my way of testing like how accurate is the model are the citations in the right place and all that but I realized early on that exploring a book through dialogue is actually an incredibly interesting compelling way to get into the material it doesn't make sense for a novel it doesn't make sense for you know narrative um stories but for information for learning it is a very powerful way of learning and until language models came along if you wanted to have a conversation with a book you had to find the author which is hard so authors are scarce um or you had to have a great teacher who was an expert in the book who you could have a kind of Socratic dialogue with um but now in the age of these language models that conversation with a book is is now possible or an essay or whatever you're doing and and I think there you know one of the points that people made about audio overviews is that um it actually lets you you know if you like to listen and and remember things better in the form of listening to a conversation um you have that option and it also frees you up to like let the hosts actually ask the questions too so you don't you know when you're getting that first overview you can just kind of listen to people going back and forth on the topic and that is very Illuminating um particularly if for getting that initial survey and then you can dive in deeper with the product yeah I can imagine the conversation that students would have after interacting with this tool is just going to be so much more robust and so much more cognitively sort of higher on a higher level that I wonder if like the learning is actually going to be elevated the expectations sorry kids the expectations for you will also be elevated because you have this amazing tool at your disposal that's what's coming up for me and going back to what you said at the tip top when we think about those higher levels of the do higher levels of tax um blooms taxonomy because now you've already got the info right like you've already got the recall you've already got all of those first one two three levels so now I can ask you to go and critique that conversation I can now ask you to go in and create your own conversation based off of what you've already heard this is going to elevate and even last year when we talked about this this is going to elevate the ways in which we as Educators teach young people how to ask questions it's going to raise the bar for what we ask young people to do with knowledge which is what I think is going to drive engagement gone are the days are we giving you spelling test right I'm now asking you to apply this in order to show me this I'm asking you to interrogate this in order to create this um the collaboration that is possible across this is the question I have yeah I so agree and Katherine so like let me give you an example of how I'm using it which I think is relevant to students um so I'm trying to figure out you know what the next book I'm going to write is going to be if I ever have time to write a book again um and so I have created a a notebook called the next book and whenever I have an idea for something something you know I read an interesting article I have notes whatever I just throw it into that one and there's like nine different book ideas in there in in various sources and I'm h i what has started to happen is that I'm having this kind of improvisational brainstorming connection with the AI in that notebook where I'll be like hey was thinking last night about writing something about the anti-nuclear protests in the 60s and 70s like I uploaded a couple of like articles here like what do you think like is there book material there like what would be the good stories and it's like well yeah there was this really interesting story of something that happened in France you know but and and I'm just capturing all of that and then at the end of kind of a month every now and then I'll kind of like gather all my notes because you can save all your interesting responses from the model in a kind of like note-taking area too that's an important part I didn't even get to show you and then I'll kind of summarize it I'll be like okay let's do a week and like a month and review like what were the what were the core ideas that you know we covered this month in this notebook and it just it in 30 seconds it creates this beautiful digest of all the ideas we covered so thinking of it as like that kind of like companion like yes it's kind of an interactive tutor on some level but it's also just this kind of intellectual sparring partner where you can think out loud with something that has read all the important documents that you need to do to do your work as a student or as a writer or whatever you do Stephen I want to close with one question and then we'll go to the students and and this question is to bring us back to one of our three themes for this Symposium and it's the theme of agile curriculum development right so one of the things that mhet is we we work so hard to create learning experiences for adolescence that are agile and that are flexible and that means that means cross-checking against you know fast amazing that they are architectural actually these these competencies these Frameworks and these new things that are happening with that with with with standards we we want to be able to hit the the standards as they're changing and adapting and preparing students for the future so let's talk about like adding new sources right a whole new like the isty technology standards if we want to make sure we're like doing something in there for future Engineers how does it work when you add new sources as they're coming up this is one of the biggest pain points for curriculum yeah I mean literally we what you would do let's say we we have that notebook where I created the Jane Jacobs thing right but it didn't have the IST standards in it um and so I would take um the lesson plan that we created I would actually turn that into its own source so that it the original draft of it is there and then I would just like upload the IST standards and I would say um you know based on this new standard stock that I've just uploaded is um please suggest revisions or updates to the original lesson plan so that it's compatible with these new guidelines that I've just uploaded and it will just go through and do a kind of like almost like a suggested line edit where it'll be like well you know this is pretty good you probably should add something here you should do this here so it just will do that in 30 seconds um 30 seconds okay that kind of a move would take at least three months three people to do it well I think the one of the things that's really important is like like the the timeline um uh notebook guide that I mentioned really brief like if you're an author or you're a teacher and you're trying to like figure out okay I want to make sure I have a chronology of all the events in this history class I'm teaching or something like that assembling a timeline is so laborious like there's no one is going to win a Nobel Prize for literature for actually like C Pulitzer for like creating a timeline so the fact that you can take something that would actually take you two hours and you can do it in 30 seconds so that then you understand the timeline and the chronology and you can think higher order thoughts about the work that you're trying to do and understanding history or the flow of a novel or whatever it is that you're doing so a lot of this is saying look we're going to just take all the stuff that is laborious and painful um and free you up to think the the more complicated things and also maybe we'll help you with some of that high order thinking too because it is a kind of thought companion as well but the speed of some of these things is really just so so much time-saving potential and I'll just add the the sighting of the sources you know the the idea that we we know where it's coming coming from and that we can you know we can check to make sure it's not hallucinating and that it's actually the source itself is so deeply reassuring it in in some ways like the two state-of-the-art things that notebook LM has right now and both of them are really made possible by Gemini um is that citations flow and then audio overviews um citations doesn't play as well on Tik Tok uh you know but it is a really powerful part of the product and uh and so you know we're glad to have things that can go viral like audio overviews so they get people into the product and they can see nerdier things like citations that are actually really important as well all right this feels really fitting in great time Stephen we're going to ask the our high school student leaders to have questions and it feels really important to hear your voices since students have made this product go viral so we uh Steph hello can okay hi U my name is Alex um my question for the Google notebook Alum is how does Google notebook Alum account for fraudulent sources or hallucinations yeah it's an excellent question um so you will find if you use notebook LM that um because of the way that we architected The Source grounding and because of advancements in the underlying model of Gemini which is just very good at factuality that you will rarely see outright hallucinations particularly in the text answers um you will sometimes see the model I think the better word for it is kind of get confused that there's some like ambiguity in the sources or it can be interpreted in different ways and the model will interpret it one way or another but but it's not like it just invents something out of you know whole cloth the way that the early language models used to do um however you raised a good point if you put completely fraudulent information in there it the the mileage will vary it's hard to tell sometimes it will actually call it out and say you know this is wrong or this is offensive um there are kind of safety standards built into the product um but you also people put in their novels and fantasy worlds that obviously are not factually correct and people are getting a lot of value out of this so the model is open to the information in the sources and will try to accurately represent that information so you could trick it if you if you are dying to work with fraudulent information in your workflow um notebook LM may help you do that I'm not exactly sure why you would want to do that but uh it it will try to be faithful to the sources generally yeah uh we need a microphone here you go my question was how does a Google notebook LM accurately summarize like informational text specifically PDFs and like handwritten notes does it use like keywords to cross reference the topic with other sources yeah it's it's magic the things that it can do so um the the core thing here is that you can put um just to give you a sense of the magnitude of this you most of you probably won't need to do this but every notebook can have 50 sources in it and each Source if it's text based can have up to 500,000 words in each source so you can have 25 million words worth of information in a single notebook that the model is kind of versed in um one of the things I've been building slowly over time is a single notebook that has like 10,000 quotes from books that I've read over the last 20 years plus the text of all the books that I've written and I'm slowly building a single notebook that basically is an expert in every all the important ideas I've ever read and all the things that I've ever published and that is like my kind of second brain that I can use to to think about things so when you have a really big notebook um you know there's basically as I was saying before there's a context window of the model if you take information and put it in the context window um the model is extremely accurate at at summarizing and making sense of that information even if it's just an image of handwriting right it it just these mod mod are visual auditory they can understand all these things in an incredible way if the information is bigger than the model's context window so Gemini has a context window that can basically handle 1.5 million words as we were talking about this Mana was talking about before um if it's bigger than that context uh what we do is we we choose a representative sample of passages or images from your sources that are the most relevant to the question that you have asked or the task you're trying to do and so we have a very I mean you know it's Google so Google is quite good at at searching and exploring information but basically we just find the most relevant passages and we put those in the model's context in the model's focus and it then is able to answer the question on that so if you have a big collection of sources it's a little bit sometimes a little bit less accurate than if you can fit everything into the context and we're going to give people more control over that over time so that they can see you know how much of the context you're using and things like that but that's basically how it works one question hi my name is taliana and my question for you is because this still summarize this how my Google notebook impact a relationship to read relationship to read to reading yeah so how does notebook um affect the relationship to reading so you know one of the things that we did from the very beginning and it was kind of stressful and we still haven't perfected it yet is sure that you can always see the full text of the documents that you upload inside of notebook LM we want notebook to be a great surface for reading as well as having conversations and generating audio overviews um so you can read documents you can do your course reading and you can select a passage and say hey save this as a note um and so you can actually go through and explore just in a traditional linear reading fashion and by saving the most interesting passages manually and creating notes you can then take those notes and turn those into a podcast or turn those into a study guide um and so you know I'm an author like I I'm hugely invested in people continuing to read and not just listen to podcasts about things that they would have read before um and so we we we built the reading tools first um for a reason because we think that's an important part of it um we just think that there other we basically want to meet you at whatever kind of learning mode is best for you if you if you actually have a better time remembering things through audio we're going to create audio overviews and other formats so we'll do that for you if you really like to have text conversations with an interactive tutor we're going to do that for you if you want to just read the document and take really good notes and have the AI help you understand things as you read we're going to be there for you as well I still want to emphasize the importance of humans around this right like a tool and resource does not change educational outcomes for young people so even though we've got this incredible tool and we've heard these amazing tools all day long these are still tools you're ability your fluency ability is still going to have to be practiced right you can get these summaries all day long you can get these FAQs all day long but the development that you and your Educators work in concert around these tools around the implementation will always support the creativity and the agility and the aperture for what these tools can actually do Back To Top