GLEN GORDON: Hello, everybody. Glen Gordon here. It’s FSCast 238 for
November of 2023. You may notice that there’s no music in the background. That
is intentional because this is an atypical episode of the podcast. You
probably heard about our “Next Big Thing” contest. Many of you provided us
with really compelling entries. The top three entrants got to present their
ideas to a panel of judges. Yes, I was one of them. That all happened on
November 14th at a live event. And it’s that live event that we’re going to be
playing on FSCast this month.
In December, we’ll have no new FSCast episode because I will be taking my annual month of hibernation. We will, however, have a replay of FSOpenLine near the end of December. And then I’ll be back near the end of January with FSCast 240. So enjoy your holiday season. I hope it’s a really delightful one for you. And enjoy the replay of the “Next Big Thing.”
RYAN: Hello, and welcome to the Sharkvember “Next Big Thing” live event. I’m Ryan Jones, and right now I’m in Los Angeles Union Station, a hub that connects people and places, much like how we aim to connect innovators with opportunities. From this station, journeys begin, and destinations are reached. Over the past few weeks we’ve been on a journey to find the most innovative ideas for JAWS, ZoomText, or Fusion. Today, we’ve arrived at our top three finalists.
So here we are, the boardroom, where ideas meet execution. Tonight, our three finalists will present their groundbreaking ideas to our panel of expert judges and you, the audience, to decide who will be the winner of the Next Big Thing. So whether you’re joining us from the bustling hub like Union Station or the comfort of your own living room, sit back and relax and prepare to be inspired. And now, without further ado, let’s get going.
MATT: Welcome, everybody. My name is Matt Ater, and I want to welcome everybody to the Next Big Thing event, celebrating Sharkvember here at Freedom Scientific and around the world. One of these contestants is going to win a $1,000 Amazon gift card, plus licensing for their JAWS, ZoomText, and Fusion. So pretty exciting stuff here coming up. We’re going to have four judges from Freedom Scientific and the community, plus one vote for the community at large. That’s the folks voting here that are part of the Zoom event. After the finalist presentations, we’re going to share a poll where you guys will do that voting. Then we’re going to have our judges jump out of the room. They’re going to go do their deliberation, and we’re going to have some great entertainment, both some comedy and some playing of some music.
And so now let’s go ahead and get started. First thing I want to do is thank the judges who participated in the first round. Let’s think about this. This was a very complex event. We started receiving entries back in early September, and the people who submitted their entries, they put in videos, and they spent lots of good time doing that great stuff. And then we need to have people look at those videos and determine which of the three would make it to the finals. And those judges from the Freedom Scientific team were Carl Wise, Director of Software Engineering; Roxana Fischer, Product Manager of Software; Brett Lewis, Senior Software Engineer; and Sriram Ramanathan, who is part of our Innovation and Cloud Director.
So before we meet our finalists, I want to take a minute to thank one of our folks who submitted something, and we saw this as something that we wanted to recognize because we thought it was really cool what they had come up with, and thought it was really important for us to acknowledge it. Lydia Olmsted will do the presentation, and we’ll listen to what her suggestion was.
LYDIA: Hi, I’m Lydia. And my idea of the Next Big Thing is to have the braille markings used for formatting and Unified English Braille, or UEB Braille code, incorporated into the software. It would really help JAWS users such as myself, who are deafblind, but also just blind. Right now, there is a feature in the software where the synthesized speech will indicate with pitch changes formatting in the document, such as bold, underline, italics, et cetera. However, when reading a document with a braille display, when the different braille markings are checked in the JAWS Settings Center, it currently shows up with dots 7 and 8 under the words, regardless of bold, underline, et cetera.
So my thought was to have the Unified English Braille, or UEB, formatting incorporated so that it would be easier to distinguish which part is bold, which part is italics. And I think this would be very useful for students because a lot of times in college it’s required that you incorporate certain formatting. It would be very helpful so that students could proofread documents, correctly cite sources of material, et cetera. Additionally, a lot of times in work, a supervisor or colleague may require you to proofread a document. And they might say, you know, please proofread the passage that’s in bold.
But let’s say there’s a bold and an underlined passage. If you’re reading in braille, you would not know which one is which. And as my mom Julie said, that’s where I would have to ask for support. And so it would be very helpful to have those markings used in UEB so that I could independently identify which section is bold, italic, et cetera. Additionally, even though this can be done with speech, that is not particularly accessible to individuals such as myself who are deafblind, who cannot necessarily identify those pitch changes. And so I really thank you for your time considering my idea for the Next Big Thing. And I hope that this feature will be integrated into JAWS soon. Thank you.
MATT: And thank you, Lydia. What a wonderful idea. And as somebody who’s spent many years working with people using JAWS and braille, it’s a wonderful idea. And it’s still possible that we’ll see some of these features that have been submitted show up in the product at some point.
Next, what we want to do is jump in and take a look at who are our finalists: Mandy Van Cleave, John McKeown, and Abby Duffy. We’re going to jump in and see them in just a few minutes and get a chance to hear from them and then also have our judges tonight ask them some questions. So let’s take a look at who those judges are.
ELIZABETH: My name is Elizabeth Whitaker, and I am the manager of the training and documentation team at Vispero. What I’m looking for in the winning idea of this contest is a feature that is going to solve a problem, something that’s going to help people overcome a challenge, or add functionality to the software. I’d like to thank everyone for submitting your ideas. We value your feedback, and this is a celebration of creativity and community. So thank you for your creativity and for celebrating with us.
JEREMIAH: My name is Jeremiah Rogers, and I work for Vispero’s accessibility group, TPGi, managing a team of accessibility experts. I’m looking for ideas which give screen reader users abilities or efficiencies they have never had or have not had in a long time. I’m looking for something that, when implemented, will cause other screen reader users to say, “I want to acquire JAWS.” The Vispero staff represented here brings years of experience building these tools, but we do not have your experience. We may not have your insight. By being chosen as a Next Big Thing finalist, you have already proven yourself to be as good as the best and better than the rest. I look forward to your presentation empowering me to see the power in your Next Big Thing.
GLEN: Hi, I’m Glen Gordon, software fellow at Vispero, and I’ve been working on JAWS since 1994. What I’m looking for from our finalists is one of two things: either a feature that’s going to impact most people on a regular basis, or something that impacts a small number of users, but impacts them so profoundly that the types of things that they’re able to do based on their chosen profession is really increased. And for those of you who have submitted entries, I just want to thank you all because it’s you giving us ideas and pushing us to strive for the best that really causes our software to improve over time.
CAMPBELL: I’m Campbell Rutherford. I’m studying applied mathematics at Harvard, and I was a Student of the Month for Freedom Scientific in 2021. I’m looking for something creative that we haven’t seen before, but that also is extremely practical. Thank you to all of our finalists and to all the contestants in general for being willing to put your ideas out there. I know that can be a bit intimidating, but I admire the fact that you want to help and empower the community of Vispero product users, that your ideas will go a long way in increasing access for everyone.
MATT: Thank you, judges. We’re really looking forward to having your participation tonight. I want to invite Mandy Van Cleave to join us here on the Zoom. And go ahead and tell us a little bit about Paragraph Preview.
MANDY: Good evening. My name is Mandy Van Cleave, and the feature I submitted is called Paragraph Preview. Right now, we do have a helpful search tool in Windows where you can search file names and also the contents of your documents. If you have gone to Folder Options, the Search tab, and if you have enabled Always Search File Names and Contents, then you can search for keywords or phrases that could be contained in the file names or documents. Now, if you’re at a loss for the keywords, or when you encounter poorly named files or a naming convention that’s really less than helpful, you might like the Paragraph Preview feature. And as you’re moving through your folders with documents with the proposed feature Paragraph Preview, you could review the first paragraph or the last paragraph in the document, for example. And that might give you a better clue than the file name did on what that file is about.
And this doesn’t necessarily have to be limited to the first and last paragraphs, but some parts of the document where there’s information that could differentiate one file from similar files. And maybe this could also be helpful with email messages. And I hope that the Paragraph Preview feature could increase your productivity and make finding the appropriate file faster so that you can get the job done and get on with the rest of your day. Have your screen reader work harder for you with Paragraph Preview.
MATT: Thank you, Mandy. Great to have you here again. And let’s bring our judges out, and they’re going to ask you some questions and maybe provide some thoughts as well. So let’s start with Liz. Liz Whitaker, can you join us, please?
ELIZABETH: Absolutely. Thank you, Matt. And thank you, Mandy, for your submission. So I do have a question. How do you see this feature working for file types other than text? Maybe, for example, Excel or another file type?
MANDY: Well, Elizabeth, that’s a good question. I thought that would be on the developers’ end of things, that they could figure that part out. But I know, you know, we have the OCR tools and the Picture Smart tools that are available from your context menu in the File Explorer. Maybe there could be some way to drill into those documents with something like a virtual viewer without having to open the document and have it load fully. That sounds really great for something like an Excel file.
ELIZABETH: Okay. All right. Well, thank you. And thank you again for your submission.
MANDY: Thank you.
MATT: Thank you, Liz. And let’s go ahead and invite Jeremiah in. Jeremiah, you’re up.
JEREMIAH: Hey, Mandy. I don’t have any question, but I want to thank you very much for the idea that you submitted and the very obvious thought that you put into how to present it to us all. So very best of luck to you. And thank you very much.
MANDY: Thank you, Jeremiah.
MATT: Glen, I think you’re up next, please.
GLEN: Hey, Mandy. Nice to talk to you.
MANDY: Hello, Glen.
GLEN: Was there an experience in your life that caused you to come up with this idea? Is this something that you’ve been longing for?
MANDY: You know what, we get all kinds of interesting file names that I run across from day to day. You’re on a server with shared files or documents that you can’t rename in your own words that would make the document more descriptive to you. So when I run into that, it’s like, okay, I wish I had another way to kind of tell the difference between this document and another one, besides the date and other fields that are available for files.
GLEN: And it occurred to me that sighted folks have had this for a long time with things like the Preview Pane, and you’re just trying to bring a little bit of parity to that.
MANDY: Yeah, exactly. There is a visual preview. And if somebody might visually recognize the shape of the lines of their document, they might find something quicker. Yeah, with that being a speech feature, I think that would, like you said, level the playing field a little bit for users, speech users.
GLEN: The thing that I would want it for is for favorites that I’ve saved, and I have no idea what the page names are anymore. I want to hear the first paragraph of the page for URLs.
MANDY: Okay. I like that.
GLEN: So thank you. Thank you very much for this idea.
MANDY: All those bookmarks that are called Welcome Home.
GLEN: Exactly.
MATT: Excellent. Thanks, Glen. And Campbell, our guest judge.
CAMPBELL: All right. Hi, Mandy. I’m really excited to hear from you today. I think in your one of your introductions, you mentioned that you’re an access technology specialist. So I suspect that you’ve seen lots of things that could be improved, you know, being an instructor. So out of all of the things that you may have seen over the years that could have been improved with JAWS, what made you choose this particular feature? I know it’s a similar question to what Glen asked, but I just wonder what gave this feature priority for you?
MANDY: It was difficult to think about something that wasn’t already available. And some days I feel like, wow, I’ve barely scratched the surface. And here I learn more every day about the software and the capabilities of all that. And I feel like I really just sat and thought about it. And I had passed along a lot of other feedback for other feature improvements in the past. So I thought, well, now I need to think of something that doesn’t already exist and that could boost productivity and efficiency in a user. Just getting through those files that have similar names, and just numbers, and things that aren’t as helpful, then I feel like that’s one thing that could slow somebody down quite a bit.
CAMPBELL: Yeah, for sure. Sorry.
MANDY: Productivity is very important.
CAMPBELL: Yeah, definitely. I think in your intro video, the file name with a big long string of numbers was the perfect example. I think we’ve all seen that before. So, yeah, I am excited for this idea to be implemented.
MANDY: Thank you.
MATT: Excellent. And I want to thank Mandy and thank our judges for that submission and questions. So thank you, Mandy. Up next we’re going to have John McKeown. And John is going to talk to us about how you can exercise JAWS in the JAWS Gym. And, John, if you’re ready, let’s have you take it away.
JOHN: Thank you, Matt. I just want to say thank you for this opportunity. My name is John McKeown. I’m coming to you from Virginia. And thanks for riding along as I introduce you to the Next Big Thing, which is the JAWS Gym. The JAWS Gym, think of it as trivia, but this time it actually has a purpose. The JAWS Gym is just a way to strengthen what you already know about JAWS, or perhaps learn some new things. So think of it as a cross between “Jeopardy,” the TV quiz show, asking you keyboard command trivia, and couple that with your own JAWS personal trainer. And the big deal here is that we’re going to try to unlock some of that value that’s already in JAWS that we might have missed.
So what the idea here is you’d be able to go in, maybe with a layered keystroke, and hit G for Gym, and you would be presented with different trivia questions based on what keyboard command does this, and you would respond with that particular keyboard command. And I’m sure that there are tons of JAWS commands that I still don’t know. I’ve been using JAWS since version 3, and I’m always surprised at how deep the product is. So first thing is it’s going to help me learn new features or features of JAWS that are already in there that I just never uncovered.
The second benefit is it’s going to help me remember those commands. So repetition at the gym is the key to getting in shape; right? But you’ve got to stay in shape. So if we can keep using those commands over and over again, you know, that’s a great way for it to be top of mind. So when I need it, I know it. I don’t have to go looking for it. So I think I’ve probably developed some bad habits over the years and just I’m doing things the hard way when, in fact, if I just knew, oh, wait a minute, there’s a keyboard command to read the clipboard to me; or whoa, I can actually check the battery status of my laptop with a simple JAWS keyboard command. Repetition’s the key.
Third thing, let’s put the fun back in JAWS fundamentals. So the key to that is to gamify the experience. Maybe there would be reminders that would, you know, say, hey, John, you’ve been at the gym five days in a row. Can you make it six? Let’s earn some badges. Again, gamify the experience. Maybe there would be leaderboards. Maybe there could be JAWS keyboard trivia tournaments. Maybe it would be a great feature in the classroom for a teacher to help somebody learn JAWS. And finally, maybe it’s a good onramp to get that JAWS certification that you’ve been thinking about. So three benefits: learn JAWS, remember JAWS, and have fun while doing it. Again, I’d like to thank everybody for putting this together, and thanks for riding along with me.
MATT: Excellent, John, and I want to thank you for your submission. And before we jump into the judges, I’ll say this, and Liz probably remembers this day. We were in the Clubhouse event, and it was probably – well, Clubhouse, when we got big into it was 2020. And we did a “Name That Keystroke” event. And we challenged four people up onstage, and we get to ask them, “What keystroke would you use to do what?” And it’s funny because, you know, you think you know them all when you even work inside the company, or you’ve been using it for, you know, 30 years. You think you know it, and it’s amazing how much is back there in the recesses of the brain and stuff like that.
JOHN: Sure.
MATT: Yeah, yeah, exactly. So thank you, John. And what I’m going to do is invite up some of our judges to ask you a couple questions and comment. So, Elizabeth, you’re up first.
ELIZABETH: All right. Thanks, Matt. Thank you, John. So when you were going through your presentation, I was really thinking about that Clubhouse event that Matt just mentioned. And I thought about, you know, turning this around, too, and thinking, “What if you know a keyboard command, but you don’t know what it does?” So I can see there would be a lot of different ways to apply this. I was just wondering if there was a particular command that might have inspired this idea or a situation or experience that you had when you were trying to learn something or remember something.
JOHN: I think the thing that probably kicked this off for me was when our company started using Microsoft Teams, and that there’s a lot of interesting keyboard commands in there. And that was really the impetus was I’d hit INSERT+W within Teams, and I’d find the particular command that I needed, and I would remember it and retain it. But the repetition, I would, for example, I might, you know, have remembered that, you know, CTRL+F – CTRL+1 gives me the activity. Well, that particular command, if I don’t use it every day, then I found, like, “Ah, I know that there’s a command to do that. Let me go look it up again.” So over time I’ve gotten certainly more comfortable with Teams, but it’s just because of the repetition. So to answer your question, I think what kicked it off for me was just the migration to Teams.
ELIZABETH: Okay. Well, thank you very much for your submission.
JOHN: Thank you.
MATT: Thanks, Liz. And next, we’re going to bring Jeremiah back up onstage to talk to John. Jeremiah?
JEREMIAH: John, thank you very much for your submission, and I really like the obvious amount of thought that you’ve put into it. I’m curious whether you feel like this feature should be sort of limited to JAWS commands. Should it extend out to Windows or application commands? Or should it even go further and do something like any other sorts of key value pairs?
JOHN: That’s a very interesting question, Jeremiah. So I really try to remember as many of these hotkeys as I can because I think, if I have a good command on the hotkeys, I think I can be more productive than some of my sighted users. So I think that the place that this occurred to me, not that long ago, was some of the keyboard shortcuts that are available in the Microsoft Office suite. So, for example, if I want to do autofit within Excel, I can hit ALT+H O I. Well, that’s a pretty specific command. Well, there’s a ton of them, and I’m not sure I know them all. So yes, I would love it if this thing could be extended, you know, beyond JAWS/ZoomText/Fusion, but across all of the Microsoft suite and even, as you said, into the Windows ecosystem with just hotkeys that would help me be better, not just within JAWS, but across as many apps and the OS, as well.
JEREMIAH: All right. Thank you, John.
JOHN: Thank you very much, Jeremiah.
MATT: Thank you, Jeremiah. And before, let’s see, we’re going to jump on here, Glen, if you have any comments, questions for John.
GLEN: So John, I like this idea. My question really relates to how do you get the people who most need to use it to want to use it?
JOHN: So I think the fun here is how do we gamify this so that we can really just, you know, add a lot of fun with metrics to encourage them. So, for example, and we may be able to slice and dice this up into – let’s start out with some of the, you know, beginner, more, you know, just your everyday meat-and-potatoes kind of JAWS hotkeys. And that would be appropriate for somebody that’s just getting started in JAWS. You want them to win at the JAWS Gym. So, let’s not give them too much to do, too fast. And once they get comfortable with that, maybe we could step up their workout a little bit and get into some more intermediate questions.
So it would take a pretty big, you know, metadata list with, is this JAWS command – what apps is it good for? What version of those apps? And what’s the level of difficulty? Plus, what keyboard does this support? But the idea would be to help somebody that needs it most just wade into it and help them be successful pretty early on to encourage them.
GLEN: That sounds really good. And I’m wondering if we could, you know, get like an ElevenLabs voiceprint of Matt and then turn him into this really assertive virtual host.
MATT: I don’t know about that. That just sounds terrible.
GLEN: Yes. Well, thank you, John. This sounds good, actually.
JOHN: Thank you, Glen.
MATT: And we have Campbell up next to share her thoughts and ask some questions for John. Campbell?
CAMPBELL: So I really like the implications that this idea has, especially for students because I know that, in my own experience, I had to spend a lot of time utilizing the free resources that were already available because it was really hard to get assistive technology training in high school. And that’s a pretty common experience.
So my question is, do you see a good way to maybe integrate the existing tutorials into the JAWS Gym if people should need extra instruction? Or maybe even integrate the JAWS Gym into some of the existing tutorials, like with Surf’s Up? If you come to the end of a section, maybe be like, oh, do you want to, you know, test your knowledge with the JAWS Gym or something like that? Do you think that would be something that would be useful?
JOHN: Yes, excellent. I love this one. So, I think it’s possible to go in both directions. So as you suggested, you know, if you get to the end of the Surf’s Up session and, you know, you want to really challenge yourself, you know, I’m not sure of the technology behind this. But I don’t know if you could do this somehow with JavaScript hosted on the website, or if it would just fire up a link that would actually let you jump right into a session right on your own local machine. But it would work that direction, but maybe it would take the other way. That is, if you said, you know, hey, I’m trying to get better in Chrome. I’ve been through my questions today. Well, hey, you know, here’s a link to get you to Surf’s Up to just the right section. So you could kind of have this bidirectional, you know, feature that would let you get in and out of the JAWS Gym, you know, from one of the web resources.
And I’ve often wondered, if you took all of these different sources, you know, from all the different tutorials, it makes me wonder, how many JAWS hotkeys are there, like, in total? And I thought, well, you know, as you said, I think students are very good at learning this stuff. And what I’m afraid of is that, over time, there’s going to be some nine-year-old kid somewhere that knows every JAWS command in the world. They figured it out using the JAWS Gym. And I think, you know, we could all learn a lot from them because they’ve obviously put in their time, put in their homework to get really, really good at JAWS.
CAMPBELL: Yeah, I think that would be excellent. I love this.
MATT: Thank you, Campbell. And thank you, John. Really appreciate your submission. And I’m thinking about this because I feel like this reminds me a lot of the exercise challenges that everybody does with step counters and, you know, your Fitbit counters. And I remember the time when I would get those badges for things like Climbing Everest because I had walked so many steps over my life. And so it reminded me a lot of that. So great job on that recommendation.
Up next we have Abby Duffy. And Abby, welcome to the stage. You’re going to talk to us a little bit about AI image recognition. So thank you, Abby. You’re up.
ABBY: Hello. So my name’s Abby. And my entire idea surrounds a barrier to access that I face every day, and a lot of other people I assume do, as well. So I am in college. I do a lot of, like, homework, and a lot of things on the web, and a lot of different things along those sorts. So my entire concept surrounds the images that are out there that don’t have any alt text, all right, whether it’s an image that’s generated by using a website or an image that’s just been there and no one’s chosen to spend the time to make the alt text. My idea is using AI somehow within JAWS to do some sort of surface level recognition of what that image is.
I, because I am in college, I have in my psychology classes for instance, I sometimes have to do these little, like, labs, and then we get most of our data conveyed in charts. And those charts are kind of why I had the inspiration to do this because they don’t have any alt text, and they don’t have any descriptions. So I had this entire barrier to access in my classwork that also is seen with other things like on websites and on sometimes even Word documents, if people don’t choose to use alt text. Where if you have an image of some sort that does not have any alt text that someone chooses to put in, there’s no way that I would be able to know what it is. It just shows up as like an image or a JPG or just says a bunch of letters and numbers. We’ve all experienced it, like, while surfing the web.
So if somehow we were able to utilize AI within JAWS to recognize diagrams or images to some extent, it doesn’t have to be perfect, but even just to the extent of saying a chart that’s a line facing upwards, or a picture of a cat. Just like, not in-depth descriptions, but just something to give some sort of idea of what’s happening for those times in life when you won’t have alt text. That would help me personally so greatly because – and I know it would help a lot of other people – because then I would at least be able to understand what I’m looking at. And, like, if the world was perfect, everyone would put alt text on their images, and everything would be fine. But that’s not how it is. There’s a lot of images that we can’t see or know what they are as JAWS and screen reader users. So if JAWS was somehow able to identify those images to some extent, in college or in, like, leisure on the Internet, or people who just are living their lives and want to know what stuff is would have a better time of it. So that was kind of my idea.
MATT: Thank you, Abby. And it reminds me a lot of, I sent a picture earlier today to the group here. And I – it was a picture of a shark fin. And I went ahead and had Be My Eyes do the description of it, and sent it off to them, making them think that I actually saw a shark in the ocean. I don’t think they believed me. But let’s bring Liz up onstage to ask some questions. Liz, you’re up first.
ELIZABETH: All right. Thanks, Matt. I totally believed you, by the way. So Abby, I really like this feature. I know that, you know, there are a lot of different ways that we use images. One of the things I was thinking about while listening to your description or your presentation was how we can also use it to select images. For example, if you’re writing an article, and you need to actually add an image, you know, different ways we could use it to select an image that we might want to add. So I didn’t really have a question, but I just wanted to comment and say that’s, you know, a really great feature, and thank you very much for your submission.
ABBY: 100%, thank you so much.
MATT: Thank you, Liz. And Jeremiah, you’re up next.
JEREMIAH: Hey, Abby, thank you very much for the time and effort that you put into your submission. You’ve clearly thought about it a lot and experienced many situations just like it, as have I. My question for you is, do you have any concern about being able to tell which images are generated by JAWS versus those generated by the content provider? And do you have any ideas how we might think about that as we consider the feature?
ABBY: Well, with this type of situation, my thought process would be that, either depending on how functional the descriptions are, if they were wonderful, then a lot of times maybe it wouldn’t necessarily matter. But an idea I had that I didn’t necessarily include would be like to maybe say “JAWS description” and then start saying the description. Or even like a brief, like click or something so you know it’s a JAWS description and not an actual description that was provided by the provider because often if it is provided by the person who created the website or created the feature, it will always be better because they know what they’re, like, I’m seeing. But AI has a lot of functionality in the world today. So if the JAWS could utilize AI and just have some sort of, like, noise beforehand, or say “JAWS description,” that’s how I think maybe you could tell the difference. Because then you could still get the benefits from the AI-generated description, while not assuming it’s a human-generated description.
JEREMIAH: Thank you very much, and good luck to you.
MATT: Thank you, Jeremiah. And Glen, you are up next for talking to Abby.
GLEN: I’m assuming that one of the things that drove you to suggest this was wanting descriptions of more complex images; is that right?
ABBY: Yes.
GLEN: Do you see this as being just descriptive, or do you see it as being interactive, as well?
ABBY: I think it could be either way. Or at the end of the day it would be up to, like, what the developer saw as their capabilities in the situation because I am fully aware that even, like, human-made descriptions of complicated images are not always effective, like, in comparison to, like, someone looking with their eyes and listening. I guess it’s not always a one-to-one comparison. This is more, my thought is, of the situation where there’s nothing to describe at all. Having even something, to have a description of some sort is better than nothing because then you know at least something of what’s happening.
If it could be interactive, that would obviously be better because then you could, like, ask, like if it was just like a ChatGPT type deal, once it has its generated description, if you did need more information. If you could maybe even ask for more specific detail, that would be greater than it just being auto generated. Auto-generated descriptions, though, would be still better than having nothing. So I think it all depends at the end of the day on what the capabilities of the processing power and of the developers to make it would be. But if it could be interactive, as well, I would personally enjoy it.
GLEN: Excellent. And I assume that the reason you want it in JAWS is because JAWS has easy access to the pictures.
ABBY: Yes.
GLEN: You don’t have to do something special.
ABBY: 100%. Because you could take an image and cut it out of a website now and paste it into some sort of AI of your choice, and it would give a description. But that would take up an extra five minutes or an extra couple minutes that, if you have a bunch of images to go through would not be as functional. But if JAWS is already there and already saw the images, then it would be like almost instantaneous, if it could have some sort of AI feature embedded.
GLEN: Perfect. Thank you very much.
ABBY: Yes.
MATT: Thank you, Glen. And if we could have Campbell come back up and talk to Abby, please.
CAMPBELL: I see this as being very practical. As a fellow college student I run into this all the time. So I get, you know, why it’s on your mind. My question is just sort of a clarification question. I know that we already have Picture Smart with JAWS, which is a layered command. So do you see this as something that’s just there upfront, the text is there upfront? Or do you see it as an enhancement to Picture Smart?
ABBY: Yeah, my dream would be there would be an – like just kind of there automatically for images. Like if you scrolled over an image, it would just kind of give your automatic description. If it was just an addition to Picture Smart, making that maybe more, like, in-depth would be nice. But my hope would be that it would be kind of there automatically.
CAMPBELL: Yeah, that makes a lot of sense. And I think that would be good, as well.
MATT: Excellent. Thank you, Campbell; and thank you, Abby. So next what we have going on is we’re going to get ready for you guys. All of our judges are being shipped off. We’ve sent them off to another room. They’ve gone off. And by the way, for image description purposes, I’ll go ahead and let all of you know that all of them had backdrops of a boardroom. So they’re heading off to the boardroom now. My backdrop, just so you guys know, is some swimming sharks. It’s actually a video running in the background.
So what do we have coming up next? We are going to let you guys – you, you, the whole group here in Zoom land – going to be doing voting on which of these three you would pick in your poll. So I’m going to bring up Tino now to describe the poll and provide guidance on how to do it. And then we’ll give you a chance to go through the poll. And then we’ll have some entertainment before our judges come back. Tino, you’re up.
TINO: Thank you, Matt. So I’m going to launch this poll right after I’m done explaining the instructions. Select the idea that you think is the Next Big Thing. You will receive a notification once the poll is launched in Zoom, and the question will be displayed. JAWS will read this notification. Press the tab key to navigate through the contents, which include the name of the poll, the questions, and the answers. Radio buttons are used for the choices today. Press the UP and DOWN key or DOWN ARROW key to navigate to and select a radio button. Tab to and press ENTER on the Submit button to submit your answers. Then press ESCAPE to navigate away from the poll and place focus in the Zoom webinar window. Should this occur accidentally, you can press F6 to return to the questions. And now I’m going to launch the poll.
MATT: Thank you, Tino. And we’ll give you a few minutes to do that. And so probably give you about two minutes to do that. And then we’ll go ahead and start our entertainment.
UNNAMED ANNOUNCER: Summer is just around the corner, and you’re probably making plans now for festive summer parties around your pool. To make those parties the talk of the neighborhood, plan now to have a shark in your pool. Imagine the surprise on the faces of your neighbors frolicking in the water when they see that gray fin slicing the surface around them. Yes, these little fellows will grow to 200 pounds and will be 12 feet long. Shipped to you from the Texas Gulf Coast, you have your choice of blue or hammer’s head. They’re also excellent watchdogs, if you’re bothered by mischievous neighbor children who climb over your fence to steal a midnight swim in your pool. But they’re really for good, clean fun. So for the most riotous pool parties ever, slip a shark into your pool. Address your inquiries to Shark, Box 6726, Dallas. You do have a pool, don’t you?
MATT: Okay. So next we’re going to have Karl Seitz and Holly Connor do some entertainment for you. It’ll be about 10 minutes, and then we’re going to jump back in and hear from the judges to see where we are in terms of the selection of the Next Big Thing.
AUDIO DESCRIPTION: Former Freedom Scientific Student of the Month, Karl Seitz. Karl walks onstage and turns to face away from the crowd.
KARL: Hi, I’m Karl. Just kidding. Because I’m funny. I’m Karl. I have a cane. It’s a good way to clear my path when I’m walking, and I can make people do as I say with it, like laugh. I say laugh! If you’re bad, I can also use it to give a good hard beating, like this. I also have a superpower. I can read in the dark with my fingers. Sucks for you guys that can only read in the light. Well, of course. I have a buddy dog from Guide Dogs for the Blind. A buddy dog is basically a dog that failed his intensive training. Therefore my dog Horton is a buddy dog. Da da da da da. Why did he not pass? Because he was too sensitive. So now he just gets to cuddle with me all day long. He barks at our pool floaties, he barks at the cat, and he even barks at his shadow. Told you he’s sensitive.
I am also sensitive. I can tap into my emotions because I am an actor. You will see me on “This Is Us,” “Tab Time,” and multiple Target commercials. Whenever I have to practice for an audition, I just practice with Horton because we are two sensitive souls. I also tell him my favorite jokes, like this one. Why did the dragon not come to work the next day?
AUDIENCE: Why?
KARL: He got fired. Why did the student eat his homework?
AUDIENCE: Why?
KARL: Because it was a piece of cake. And my favorite knock-knock joke. Knock-knock.
AUDIENCE: Who’s there?
KARL: Mikey.
AUDIENCE: Mikey who?
KARL: My key won’t fit in the lock. Can you please let me in? See, I am an actor, comedian, and did I also mention I’m a surfer? Because I am. I learned in Hawaii. First step, cobra stance. Step two, foot up. Step three, stand like the wind. Step four, wipeout. Which is probably the most fun part, by the way. Well, I’ve got an important audition to practice for. Time to tap into my sensitive side with Horton. I am Karl. Thank you and good night.
AUDIENCE: [Applause]
AUDIO DESCRIPTION: Former Freedom Scientific Student of the Month, Holly Connor. Holly sings while a woman plays piano.
AUDIO DESCRIPTION: Holly sings and plays piano with a jazz band.
AUDIO DESCRIPTION: Holly sings with a choir.
AUDIENCE: [Applause]
MATT: What an amazing opportunity to be able to showcase some of our Student of the Month winners. I mean, what an amazing job. I mean, we heard two there today; plus we also heard from Campbell, who’s one of our Student of the Month program winners, as well; and I think Abby was, as well. So how cool is that, that we get all these folks involved in the Next Big Thing event. And I think our judges should be back soon. What an exciting time, where it’s like – did I hear Ryan?
RYAN: Yeah, the judges, we are back.
MATT: Are you guys back?
RYAN: We are back.
MATT: Was it – was there any – I feel like Shark Tank here. Was there any – I don’t know if we’re allowed to say that. We didn’t name it that, so I think we could just say it feels like, especially since there’s sharks swimming behind me. Was it – was a little arm wrestling? Any –
RYAN: The judges were very cordial. And actually, the judges found that this was way more difficult than they even thought that it would be. That was a comment that I heard. Because the quality of the ideas, the quality of your presentations was topnotch. And so they had a hard time deciding. But we do have a decision. But it was – actually, it was extraordinarily close. It was as close as voting as it could possibly be.
MATT: And we even had – and we had a mix of voting from the audience, as well, that we did through the poll, and that seemed to potentially sway some stuff, as well.
RYAN: You know, the judges kind of had their minds made up. I don’t think that the audience – the audience vote had an impact on the overall score. Absolutely.
MATT: Sure, 25, 25%, 20%.
RYAN: Everybody had 20% weight, the four judges plus the audience. Yeah.
MATT: Good. Well, excellent. Well, let’s hear – let’s see. Ryan, are you ready?
RYAN: I’m ready. Let’s talk about this. So I want to give the results from the – and we’re only announcing the first place winner. So the first place winner for the audience score was Abby Duffy. And Abby won the audience vote for her idea of AI image description. So congratulations for winning the audience score. So now the final total after all the tallies, after all the work – and again, congratulations to each of you because I’ve – I couldn’t have envisioned a closer tally at the end of the day.
But the winner of the 2023 inaugural Next Big Thing event is Abby Duffy with AI image descriptions. So Abby, congratulations. John, Mandy, it was phenomenal. This was a great event. Thank you for participating. Thank you for being with us. Abby, congratulations. Do you want to say anything, make any comments?
ABBY: Just thank you. I’m very excited. I’m glad that my idea was something that other people also thought was a good idea. So just thank you.
MATT: That’s very cool, Ryan. Very cool. Abby, congratulations. I think this is, as mentioned, a wonderful event. And it’s great to see participation from so many people, including the audience. I mean, Ryan, how many – Tino maybe can pop in and at least provide us. How many people attended virtually here on the Zoom?
RYAN: Yeah, I saw 150 plus at one point. I don’t know where we’re at right now. But we had a very good attendance. We’ll be streaming this on YouTube later or rebroadcasting this video. So it’s been a great turnout. Matt, great job. Thank you to the judges. Thank you to – we had a whole team putting this event on that you never even heard from, Cori and Tino, Rachel and Shelly. So there’s a whole crew producing and putting this event on who have done a wonderful job tonight.
MATT: Yeah, it was amazing. Great video work. Great audio work. Everything just turned out great. And I get to expense my vacation since I got to do it from here; right, Ryan? No? No?
RYAN: Yeah, we may check that again later. But I don’t know.
MATT: No, I don’t think that’s going to happen. But other things coming up this month. We’ve got a webinar later this week. And then at the end of the month, you have an event on the 30th; correct?
RYAN: FSOpenLine will be November 30th, Sharkvember edition, as we get ready for the holidays. So, yeah, we have a lot of things going on this month for Sharkvember. This was obviously one of the key pieces of it, but quite a lot going on.
MATT: And for those who don’t know that, if you go back to the website for the Next Big Thing, you will find a Keystroke for the Day, right, each day?
RYAN: Every day.
MATT: Every day.
RYAN: There’s a keystroke. And if you sign up for our blog, we’re posting on the blog every week a recap of all. And we’re not even leaving out the weekends, Matt. I mean, the team, they’re putting in keystrokes for Saturday and Sunday. We only work five days a week, supposedly. But they’ve got stuff coming up on the weekend. So well, well done.
MATT: That’s awesome, yeah. Great job to everybody. Thank you, everybody, for the coordination, the event. And congratulations, Abby. It’s so wonderful to have one of our Students of the Month be the first winner of this program. And I will send it back to you, Tino. Thank you.