GLEN GORDON:
On FSCast 247, I say goodbye as host of the podcast and turn things over to Oleg Shevkun, but not before I have a chance to ask him about growing up in the Soviet Union, studying in the United States and coming to work for us at Freedom Scientific. Then Oleg and Mohammed Laachir talk about picture-smart improvements in the July update and what's coming with AI in the future.
This is a bittersweet edition of FSCast for me because I've been hosting it since the beginning of the pandemic. I think I started in April of 2020 and have gotten to talk to some great and interesting people, folks who I couldn't have buttonholed for an hour or so on the phone to have asked them a variety of questions if I weren't hosting FSCast, and so that was a great pleasure.
And I've enjoyed hearing from so many of you that during the preceding 26 years that I've been working on JAWS, I have not heard from. And so having these added connections has been a really fun thing. But the truth of the matter is that I am an introvert and the thing that I love doing most is working on JAWS and improving JAWS for those of you who use it. And I want to spend more of my time, all of my work related time doing that rather than hosting FSCast moving forward.
And one of the reasons I feel comfortable in transitioning things over is because we have the perfect person who knows not only the technology of how to put together an episode, but he knows our software well since he works for us, he understands the concept of broadcasting and putting together things that flow well, and he asks really good questions, all of which combine to make him the perfect next host for FSCast. And I speak of none other than Oleg Shevkun. And as my final duty as the host of this podcast, I have asked Oleg if I can ask him a whole bunch of questions about him. So we'll be doing that for the first part of the episode, and then he'll be taking over for the second half.
GLEN:
Oleg, welcome to the podcast.
OLEG SHEVKUN:
Well, thank you, Glen, and thank you for having me here. And in fact, that bittersweet moment, you mentioned the April 2020 episode, I remember that, we were all locked down and I was listening to that episode with the presidents of NFB and ACB on how those associations of the blind were coping with the pandemic, and it was amazing. I was really, really excited to hear that coverage because it broadens your horizons, takes off the walls. It was amazing that you mention it right now.
GLEN:
You were born and raised in the Soviet Union, though people hearing you from your accent would probably not necessarily know that. You were born when?
OLEG:
Yeah, I was born in 1969, in what at that time was Leningrad, USSR, and now it's St. Petersburg of course. I was an avid radio listener. Officially, we had only three or four radio stations. Everything else had to come from probably another city in the same country or it had to come from overseas. But I started listening to those three or four radio stations knowing all the shows, the announcers, the presenters, and everything. So I got that radio bug. Can you say that? Is that too much to say?
GLEN:
Yeah, no, exactly. So many of us have had that radio bug.
OLEG:
The other bug I got at that time in Russia was the telephone bug. And probably that was troubling to some people—calling some phone numbers that I shouldn't be probably calling and figuring out some tricks that I probably shouldn't be using, that kind of stuff. And then actually after spending some time in the States, I got to meet some people both blind and sighted who had that phone bug as well.
And then another passion was talking books and anything recorded. I mean, those tape recorders, at that time they were tapes, like reels. You'd remember those huge reels that you had to put on, not just listening to those recordings, but I want to cut out a certain word or a certain phrase and then just use some scotch tape to glue this together and to make sure it sounds good.
Or I was playing the piano, and I had a pretty bad mic. I realized it was bad, but that's all I had. And then the idea was to find out the best spot to record the piano with that mic. And one day I moved the piano to the middle of the room. I was probably like 12 or 13. And when my mom asked me why, I said, well, because I was trying to find a spot and in the middle of the room, if I put the microphone like this, it sounds better when recorded. So that's pretty crazy, but that's part of my childhood.
GLEN:
You were born blind or at least low vision, right? I assume your parents were not all that prepared. Did they ever talk to you about this? Do you have any recollections of those early years and how they helped move you forward?
OLEG:
Actually, I should give credit to my parents because... especially to my dad, because he prepared me or he started preparing me for life as a blind person or actually a visually impaired person. I do have some eyesight, which was like 2% of what it would normally be, but he was really good at that. Plus I had some good teachers. At age five, I started learning, and my piano teacher was blind and he said, "Along with teaching you music, I must teach you braille, because you're not going to learn music without braille."
So at age five or six, he started teaching me music. He started teaching me braille. And my dad is the kind of a person who wants his son to excel. Sometimes it can be a blessing, sometimes not as much. So when I was six, he needed to find something for me to read, and he took out a book from the library for the blind, which was a book of articles about music and musicians that was written for adult audience, that was not a children's book. And I remember reading out to my father, reading aloud to my father an article about Frederic Chopin in Poland in the early 19th century and the balls of the ballrooms of Warsaw. So it was a bit of a torment, and yet it was a challenge and a nice one at that.
GLEN:
Growing up, did you have any choice but to go to a residential school?
OLEG:
No.
GLEN:
Was that the option at the time?
OLEG:
The only choice was to go to a residential school for the blind. We didn't have mainstreaming, we didn't have the education of blind people with everybody else. So yeah, the only choice was going to a specialized school.
GLEN:
And looking back on it now and what you remember, what was the quality of the education?
OLEG:
Superb but isolating. So we spent six days per week in that boarding school. That was for those of us who were actually living in Moscow. And you are in a small group of blind people, well, relatively small, like classes of 8 to 10 people in the whole school of about 200 kids. The education was good in that we knew our math well, we knew our geography well. We did not know how to order some food at a cafe or a restaurant. That's what was never taught because we were isolated. I did not know, I did not understand how sighted people communicate with their body language, like with their eyes, with their gestures. I assumed that if a person says something and I cannot hear anything otherwise in his voice, I assumed that was what they meant.
And when I became a university student, I realized there's a pretty nasty thing that is going sometimes, you talk to the person or the person talks to you and then the person is showing some signs, some body language to somebody else who is besides him, and you are cut out of that conversation. Imagine it or not, believe it or not, until age 18 or 19, I had no idea that could even be happening. So brilliant education, but the problem was we were cut out from the society in general. We were not really mingling with kids our age.
GLEN:
Did you have role models available to you growing up?
OLEG:
Well, absolutely. First of all, we had some fantastic teachers, some people who really, really meant a lot to us, like our history teacher, our music teacher, and these people were blind, and these people knew how to cope with blindness and how to excel in that. In some of them we saw independence we tried to achieve. That was important to me.
Then there was something unusual about me. I got involved in what is pretty much defunct right now, but was pretty popular, especially in the blindness community in my country, in the '70s and the '80s. It was the international language called Esperanto. So I was involved in that at age 10, and at age 12 and 13, I started corresponding with people in other parts of the world, in other countries in Esperanto, and realizing that life of blind people in other countries is very, very much different. I remember receiving letters in braille from a gentleman in France named Raymond Gonin, and I would smell those letters because they had this wonderful... I didn't know what it was, then I realized what it was. He was using perfumes. And the letters actually carried that scent over the borders.
GLEN:
I was thinking in the days of the Iron Curtain and the Cold War that you were isolated from the rest of the world.
OLEG:
You were as isolated as you chose to be. So right now in my country, you are not allowed to use Facebook or Instagram or many other services or view many other news sources. But there's VPN. If you install VPN, well, some of them are blocked, some of them will not work, but others will. So if you're persistent, you're not that isolated. That was pretty much the same in those days. If you are persistent, if you knew what you wanted, if you wanted to break out of that cage, if you were patient, yeah, you had to wait for a response to your letter for maybe two months. It would take one month for your letter to travel to the States and one month to get back, for the response to get back, if it got back at all. But you could do it, and that's what mattered.
GLEN:
What was the transition like, going to university from previously being in a school with all blind kids?
OLEG:
It's like if the walls have fallen down. It's the world getting broader, it's you realizing a couple of things. First, you realize how many things you actually know and you are able to do them that others can't. But then realizing how often people misunderstand you and have false preconceptions about you.
I'll give you just one example. My first class in Latin, and there were about eight of us in that language group. And five minutes before the professor shows up, everybody's kind of concerned, who has done the homework, who has translated those Latin sentences or those sentences from Russian to Latin? It turns none of the students have done this. And I say, "Hey, I've done the homework." And everybody goes, "Can I copy it from you? May I just please, may I?" And I go, "Yeah, but I've done it in braille." And everybody goes, "Hey, can you just dictate this? Just say it out loud and we'll write it down." And I go, "Sure, why not?"
So in those five minutes, I quickly dictated that homework. The professor walks in, and he starts asking us to read the homework, and he goes around the circle, in circle around the room and he asks each person to read a sentence, and he passes me by. And I thought, "Well, that's something strange. Why is he doing this?" So he goes in the second circle and the second time he passes me by. And I raise my hand and I say, "Excuse me, but why aren't you asking me to read? I'd like to do this as well." He goes, "Have you done this one as well?" And everybody just starts laughing because yeah, you know why.
GLEN:
They all got it from you.
OLEG:
Right. But actually that helped me to build relationship with people in the group.
GLEN:
When did you first travel to the West?
OLEG:
1990, to Eugene, Oregon in the United States of America. We had an exchange of students with disabilities. There was a large group of students who came from the USSR to the States, and then next year a group that came from the States to the USSR. There was and still is an organization called Mobility International USA that promoted those exchanges. You've got to understand that the world was opening up and we all were trying to build bridges, and they brought us, Soviet students from different parts of that huge country to the United States to stay with real families, normal American families, spend a few days, like a couple of weeks in Eugene, Oregon, and then in New York, New York. That was an earth-shattering experience.
GLEN:
What were the things about American society that were most surprising to you?
OLEG:
The role of a personality or a person rather than the masses or the collective. To me, that was a first shock. The responsibility, as in the rights, the ability to feel the fresh air and to breathe in the fresh air and to have fun, just get together in the streets and play music or lie on the grass without somebody telling you to keep away from the grass, although there are lots of grasses where you would want to keep away from.
And the other thing was blindness technology, because we were invited into the home of a blind lady, I don't even remember her name, and she gave me that little box. And I was the only blind person in the group who knew some English and quite good English because I was learning English. And she gave me that little box with seven braille keys and the speech synthesizer built in and a box that you turn it on and it goes: “Braille ‘n Speak ready”. When I started typing in and editing and thinking how it could transform my life. So that was July and August 1990. That was a huge experience.
GLEN:
How long between you seeing the Braille ‘n Speak and you getting one for yourself?
OLEG:
Five months.
GLEN:
How did that come to be?
OLEG:
I had some friends, some Christian friends from the United States who wanted to give me a good present and they wanted to give me an Apple computer at the time. And they were showing me, "Look, I can blow up the font, I can blow up the letters are going to be huge. You're going to be able to read it." And I saw this and I said, "Nah, I want a Braille `n Speak." And they say, "But the Braille `n Speak is a strange technology, and an Apple is a full computer." And I said, "You don't understand. After using those large letters for five minutes, I'm going to have a headache and aching eyes, I want a Braille ‘n Speak." So they brought me one.
GLEN:
Did the Braille ‘n Speak change how you did your work?
OLEG:
Let me tell you something I did. I started taking lecture notes, especially for English classes, and I found a colleague that had a printer and I started taking those notes. And then when somebody missed class or was just too lazy or didn't want to take notes, I would print out those notes from Braille ‘n Speak and actually sell to that person. Very few of those, but yes, it changed life.
GLEN:
And speaking of that, how much did economic conditions change for you and your family practically after Perestroika?
OLEG:
I was not your typical Russian because I had already connections to friends in the States and other countries. So right after going to university, I went on to Dallas Theological Seminary in the States to study theology. And before that, I got married. We were having a hard time, but it was not as hard as many others in our country. So for us, the '90s was the time of boom or at least of a huge improvement. While for most people in the country, the '90s were the time when you had to think of what you were going to eat the next morning.
GLEN:
How long were you in the US?
OLEG:
Four years.
GLEN:
That I assume further changed your perceptions of the world.
OLEG:
Yeah, getting introduced to NFB philosophy, the works of Kenneth Jernigan and blindness not being a disability or definitely not a disease, but rather, a characteristic. Yet I had grown up with a totally different perception, thinking of blindness as a disease! So that was not just a theological training for me, but it was also a training about philosophy of blindness.
GLEN:
What caused you not to pursue some role as a religious scholar or a minister after graduating from seminary?
OLEG:
But Glen, I did! And I still pursue that. I served as a pastor for five and a half years after coming back from seminary and also did Christian preaching and lectures and meeting with young peoples, with student ministries and so on. And after that, I went more into the world of business, but I still kept my Christian involvement, involvement in the church and Christian broadcasting. So even now I work for Vispero, but I also do a weekly radio program, or actually now it's an internet program on the Bible, going through the Bible in five years, in Russian. So it's not just switching, it's rather adding things.
GLEN:
The only reason I said things as I did is because you are so entrenched in technology, and so I just assumed you immediately started working in technology. How did that all come about?
OLEG:
Well, first of all, when I was in seminary, I got my first demo of JAWS, which was JAWS 2.0. And I wrote at that time to Henter-Joyce. I said, "I'm from Russia. I'm a student here. I tried JAWS, would love to use it, but there's no way for me pay the whole price now. Is there a way to do it in installments?" And the response came from Ted Henter who said, "Well, here's your JAWS license." I say, "How about installments?" And he said, "Don't worry about this." And then he goes, "Would you like to start working for us as a scripter?" And I go, "I know nothing about scripting, so thank you, no."
So at the same time, I was using my Braille `n Speak and then a braille light with a braille display and was in touch with Blazie Engineering. And they invited me after some time to start beta testing for Blazie and also to do a Russian localization of Blazie products. So getting introduced to JAWS and then getting introduced to the Blazie beta testing, I think these were the two initial steps. And I thought I can contribute something not just as a user, but as somebody who understands the needs in another region, and then turns out not just in another region, in finding bugs and finding things.
GLEN:
You worked for us as a product specialist for a while, and that was, what, in what had become Russia by that point?
OLEG:
Yeah. In 2002 to 2011, we really needed to promote the products, and they could fill a great need that we had in Russia at the time. Yeah.
GLEN:
What prompted you and your wife to move from Russia to Germany a few years back?
OLEG:
Well, as you know, in February 2022, Russia attacked Ukraine. This was an unprovoked attack. And a few years before that, in 2014, Russia annexed Crimea from Ukraine, that went against everything that I was taught, that went against anything I believed, and along with other people, I just started saying, and publicly so, I started saying, no, that's wrong. That shouldn't be happening.
But for eight years, it seemed that the situation could be changed. It seemed that the Russian people would finally realize that something badly wrong, terribly wrong is being done. Yet February 2022 showed that it could go worse and it's now getting worse. And my wife and I made a choice. Let me put it this way. We could not change the country. We could be fined, we could be jailed or anything, but we could not change the country from inside. So we made two choices. One of them was to see if we could live in a free country, but second, living in that free country to reach into Russia, to keep our contact with our people to show what life is like in the rest of the world and how Russia is not just going back to the Soviet era, but it's really, really far worse. It's going there and is trying to get the huge part of the world into that Soviet era.
So when we came to Germany, the requirement actually was to find a job. And I contacted my colleagues and friends at now Vispero and I was really and happy and overjoyed that Vispero would say, yeah, we can use your services.
GLEN:
What makes you excited about working with Vispero and working with this kind of technology?
OLEG:
To me, technology is a tool, but it's got to work well. That's why being part of developing or testing technology allows me to participate on the team that sharpens that tool. And we have a fantastic team. When something is not exactly the way you would like it to be in JAWS and in other products that we have, it's not because we don't want the best for the products. We want our tools to be sharp. And that's why I really love being involved in assistive technology. I really love working with this fantastic team here at Vispero.
GLEN:
Normally at about this time, I would thank whoever the guest was and say, thanks for joining us, but we live in special times, and so rather than saying thank you for joining us, Oleg, I will say it was great talking to you about you and your life, and now I very happily and proudly turn the FSCast hosting duties over to you.
OLEG:
And thank you, Glen. It's a huge privilege. We here at FSCast have had a fantastic lineup of hosts starting with Jonathan Mosen, then John and Larry Gassman, then Glen Gordon, and I'm really thrilled and excited to be part of this endeavor. One thing I'd like to develop on FSCast is feedback from listeners. So if you folks have anything to suggest or to criticize or you think of an improvement, fscast@vispero.com is the address to write to. And Glen, these four years with you doing FSCast was a fantastic time.
GLEN:
Thank you, thank you very much. And if any of you have been listening for some or all of these four years and have never written in, please take the opportunity to do so in the next month or so. I will be a lurker on the FSCast email address for another month or two, and it would be great to hear from you. And after that, if there's any way I can help, I'm ggordon@vispero.com and I'm not going anywhere.
OLEG:
And now it's time to talk about artificial intelligence. In March, we released Picture Smart AI, which was really well received by our customers. This was then updated in May and most recently in July, but that's only the beginning of the road. To find out more, we've asked Mohammed Laachir, a product owner with Vispero, to join us in our virtual studio. Hello Mohammed, and welcome to FSCast.
MOHAMMED LAACHIR:
Thank you so much, Oleg, and it's very good to be back.
OLEG:
And by the way, welcome back from the good old United States. You've been there for the conventions. How was it?
MOHAMMED:
It was pretty nice. It was a whirlwind round trip. We went to the NFB Convention in Orlando and the ACB Convention a couple of hundred kilometers up to Jacksonville, and it was pretty good. I enjoyed myself, and it was very good to talk to our users, and it was very good to show them what we've been up to for the last couple of months because it has been spectacular.
OLEG:
Is that your first convention?
MOHAMMED:
Yes, these are my first NFB, ACB conventions. I've never been to one before. Of course, I'm based in Europe, so I don't get a chance to go very often. And when I got the chance, I was very happy because it's always good to talk to people who use our products and to show them off what we've been doing and to see what their problems are. I mean, some people just came into the room and it was called the JAWS AI Suite, but they came in and they were like, "Well, I have other questions." And there they were with ZoomText questions, Fusion questions, JAWS questions, and I was very happy to answer those and to see where the bottlenecks in our software lie. That was very good to do.
OLEG:
So somebody comes up to you and says, "Well, here's a problem." And you go ahead and you try to reproduce that and you see, yeah, that's a real problem. What happens then?
MOHAMMED:
So if there is a real error or deficiency in the product, we reproduce it. And once we have very clear steps, so sometimes they're provided by our user, but sometimes you have to coax the steps out of the problem that they're facing because even they don't know exactly how to get the same problem each and every time. Once we have realized that, it goes into our system, our bug tracking system it's called, and we keep track of it. So it gets a number, people can see it, it's right there. And we prioritize the bugs that we get along with new features because of course we have to do both, we have to juggle everything.
So we know about the bugs, we know about the features that we want to build. We know about the maintenance items that we need to complete. I mean, we don't always talk about those, but they're very important too. We need to keep up to date so that we're secure and we can use the latest functionality of other pieces of software that we use. So it is just a huge list of things that need to get done and that get sorted. And we, as product owners, but also developers who know a lot about our software, we'll look at that and try to make sense of what needs to go first and what comes after. So yes, once we know that there is a bug, it's in our system, and we definitely do keep track of it.
OLEG:
The encouraging thing to know is that even when the defect is not immediately fixed, that defect is not lost, it's there, it's being tracked, it just has to be prioritized.
MOHAMMED:
Absolutely. Our bug tracking system does not forget. We might, but our bug tracking system will remind us.
OLEG:
I'll tell you, it reminds us every time we look at it. And yeah, we go through the revision process, reconsideration process and so on.
You mentioned the AI Suite. Why did you choose that name?
MOHAMMED:
Well, because there is a significant breakthrough that has happened, I believe, since November 2022 when OpenAI released ChatGPT, but there's a lot more on the market right now where AI and in particular a certain form of AI called large language models have really upended the forward outlook for technology.
In the beginning, AI was very exciting and it was very cool to see, but it felt a little bit like a gizmo, like a gadget. It's new and it does things that we'd never seen before. Wow. But now it's actually starting to show some real utility. Now people have figured out how to use it to do a certain tasks, and it's starting to make an impact on people. And we believe that when it's done right, AI can increase accessibility tenfold, a hundredfold, maybe even a thousandfold. And we are very excited about this new tech, and we want to show people that we are working with it and on it in order to make it accessible, in order to make that future happen.
And that's why we chose the JAWS AI Suite because it's a technology that we really want to highlight that we believe is going to fundamentally change not only screen reading, but everything that we work with, in the coming years. And so we wanted to show the first glimmers of that because hard as it is to believe what we showed at the convention is the worst form of a large language model you will ever see, it's very impressive.
OLEG:
What do you mean?
MOHAMMED:
Well, it's only going to be developed more, it's only going to increase in quality, it's going to become better and better.
OLEG:
One thing I've discovered about AI for myself, and I think you must have discovered that as a product owner is it's not just about the AI model used, but it's also about how you ask questions. You can formulate your questions in such a way that you get less useful answers or more useful answers. In the beginning when we asked AI to describe some pictures, we were getting some, shall I say, condescending remarks. I'm sorry you're blind and all that kind of stuff. And then when you learn to ask better questions, you get better results. Is that really something that's happening within our product development? Are you learning how to interact with AI to get better information?
MOHAMMED:
Absolutely. So we have something we call the playground, and the playground allows us to ask questions of AI and get answers and see how that works. And we have people whose job it is to… Well, I say people, one person, but she's doing a tremendous job… whose job it is to not only help us with designing, but also help us with prompting. And that's what you're alluding to here. It's called prompting. And yeah, what she does essentially is she tries out different prompts, gets the results and presents them to us, and we can see what works best, what we need to adjust, what we need to change. And the good thing is a lot of those prompts we can design in the background and give to the LLM, give to the AI, along with the question the user asks. So even if the user forgets to tell the AI, I'm blind, I don't need your pity, we can tell the AI that instead and the user can just focus on asking that question.
OLEG:
I've never thought of telling AI in JAWS that I'm blind. It's kind of assumed apparently, and that's exactly what you are talking about. So it knows already.
MOHAMMED:
Exactly.
OLEG:
And it looks like it already knows that there is no other way I can get that picture description. So it's being very, very good at it.
So in March we introduced our Picture Smart AI when AI functionality was added into our Picture Smart feature where you could get the descriptions. In May we introduced the follow-up dialogue where you could ask more questions and so on. In July we introduced something else, which is a preliminary dialogue. That's the one where you basically add the Alt key to your Picture Smart commands to receive additional functionality like pressing INSERT+SPACE P, followed by ALT+S to do the entire screen or INSERT+SPACE P, followed ALT+W, to do current window. Talk to us about this.
MOHAMMED:
Well, so with Picture Smart AI, you send a picture up and it describes it, and people told us that's all well and good. And of course we tried that stuff too, and we thought the same thing is it's all well and good, but sometimes you already know what a picture is. A heading will say, this is a graph of such and such a thing, and you go and do Picture Smart AI on that and it tells you it's a graph of such and such a thing, and you're like, "Well, great, I already knew that." So instead, what you would like to do in that situation is ask a question, a specific question about that graph. So for example, where do two lines intersect or where does the line intersect the X-axis, right? All those questions you want to ask-
OLEG:
Before submitting the graph.
MOHAMMED:
Exactly, because otherwise you have to read through the description, click the link to ask more questions, ask your question. Now all of that is a productivity drain. You don't want that. So if you want to have an answer about an image, and you already know what the image is, you just ask a question and get the answer without all the preliminary description stuff.
OLEG:
I'm going to ask you about a usage scenario in a moment, but I'll tell you about something that I discovered a few weeks ago. I would love that to have been available many years ago. It was not then, but now it is. So I loaded a linguistic map of Asia, that's basically the map of Asia with a clear indication of where each language is spoken. And I gave a prompt to AI in the preliminary dialogue. I said, "Describe this map, reading it as if you were reading a book, dividing it into imaginary lines from left to right, top to bottom, and tell me which languages are spoken in each part of Asia." And the description I got, it basically told me, yes, this map can be divided into this many lines. However, some of those languages will not fit into a single line, but I'm going to do the best that is possible. And it gave me a very clear and easy to read, easy to listen to description of the linguistic map of Asia. If I had that many years ago in my language exams, that would've been fantastic.
Are there any other usage scenarios for the preliminary dialogue that you could come up with or that you've seen or you've heard from people where they ask the question and that immediately helps them to understand something that was beyond their reach?
MOHAMMED:
Oh, yeah. So if you have, for example, a page open on any web shop of a coffee machine and you know that the image is a coffee machine, but you want to know if it has physical buttons, now you can just ask the question and you get the answer easily, almost like interrogating the image yourself and just looking at the things that you want to look at.
The same goes for me. I'm making quite a few PowerPoint presentations as part of my job, and I will sometimes, when I know that I've put a lot on a slide, I will try to use Picture Smart AI and I will succeed, of course. I will use Picture Smart AI in order to tell me whether everything fits in the slide and whether the text is still readable. And it will tell you that. It will say, yeah, the text is readable, or no, the text is not readable, you have to maybe remove some text from the slide and make the font size a little bit bigger.
So all of that is available to you right now, and there is more. Anything that you know of, that you know the general gist of the picture, but you just need to have one single detail. You can now use this and be just a little bit faster than the May update Picture Smart would let you be where you could also ask questions.
OLEG:
You mentioned a use case where you are in PowerPoint, you're making a presentation, so can we be a step better than what you've just described? Can I be in PowerPoint and just tell AI once, "Hey, AI, I'm making a presentation, I want you just to tell me if a text fits or the graphics are in place so that I don't have to repeat that background information with each next use of the smart AI feature"? Can we do that?
MOHAMMED:
At the moment it's not possible in JAWS, but this is something that we always think about – new ways to improve the feature, and this is one of them. So there are ideas floating around in the company and in the team to do exactly what you want. It's not there yet, but I would say stay tuned because as time progresses, this feature is only going to get better.
OLEG:
Context is a huge thing. And with Picture Smart AI, we have multiple commands, and you always have to remember which one to use. It can be a file, it can be a clipboard contents, it can be a control or anything. But we've also introduced a new command and you just press insert space or rather JAWS key space P for Picture Smart, followed by the enter key. So you have to remember the enter key, and that's all. Talk to me about that command and how context works there.
MOHAMMED:
So just so you know, this was a feature thought up by one of our developers. We talked a lot about complexity, and he came to me once and he said, "You know, wouldn't it be great if I could just have a command that does whatever I need without me having to remember all the keystrokes?" And he said, "You know, I've got a technical idea on how to do this, so maybe we should." And I said, "Great, go for it." And we came up with the keystroke.
This guy is sighted by the way, but this guy is a user interface genius. He builds very, very good user interfaces. He thinks about usage a lot and how people can use the product. And he came up with this feature and we built it, and it's remarkable.
So it's only going to get better from here, but what it can currently do is it can determine based on the app that you're in, and based on the thing that has focus, it can determine which command to use. So let's say I'm in File Explorer, I'm on a file. It says to me, car.jpg and I want to look at that file. I'll just press INSERT+SPACE P ENTER, and it knows to use the F command for file.
Now if I then move on to a webpage, I'm on a website, news website and I see a picture that's part of a news story, I press the INSERT+SPACE P ENTER, and because it knows that I'm on the internet, it knows that I'm on an image, it knows to use the C command. And this is just magical because you now no longer have to think about context yourself. The screen reader does it for you, and we have some more ideas on how to improve it, and if people have ideas on how to improve it, softwaresuggestions@vispero.com is an email address that you can send your emails to. That's something we all read. And we'd like to know real-world use cases where the INSERT+SPACE P ENTER or JAWSKEY+SPACE P ENTER would be nice to have and would work for you, because as we build out the context awareness of JAWS, it's only going to get better again. I mean, I feel like a broken automaton, but it's only going to get better, it's the actual truth.
OLEG:
You mentioned something else that was pretty interesting. So you're on a website and you have an image, and in the past what happened sometimes is, your image is not entirely in view. The zoom was set incorrectly or you haven't scrolled to that image so that it fills the entire area, something is cut off and so on. And you could get some really interesting results, to put it that way. In our July updates, we have improved that, we have done something or provide something that will help our users in that regard. Talk to me about this one.
MOHAMMED:
Yeah. So in this case, information is key. You really want to have the information in order to take steps to make this problem go away or at least make this problem less of a problem or decide that it isn't a problem and that you were only interested in what's in view anyway. So what we do is we tell you on top of the viewer that comes up, that gives you the description, we tell you, "Hey, we noticed that your image is cropped a little bit. Maybe your window is not maximized. Try to maximize your window," and that way you know to scroll or to maximize your window in order to get the full image in view.
Or you may look at the descriptions and say, "I got everything that I needed out of this. I don't care that the last two rows of pixels is cropped. I will move on from here." But at least you knew, you know that the problem is there. And so that's what we've tried to do here. We give you just a little bit of a warning in one line that you can expand, and then you'll get more explanation as to what we've detected.
OLEG:
Yeah, and that's actually helpful because again, it gives you more control and it gives you more understanding of what's happening.
And another thing that happened in our July update is that the new Picture Smart AI feature is now also available in Fusion, so that our low vision users can benefit from that. Initially I thought, "Well, why in the world would a low vision user need that?" And then I realized there are plenty of situations why that may be helpful. So once again, if you're a Fusion user and you would like to check out our new Picture Smart AI functionality, then all you need is the current Fusion license and the July 2024 update.
Now to our AI Suite at the conventions, there's a rumor going on that some JAWS features were shown off there.
MOHAMMED:
So yes, we did show off a new feature that we are planning to release somewhere later this year, end of this year or this fall, that will help people actually find out how to do things on the computer. So often if you forget how to do something because you do it once a month, twice a month or once every two months or even less often than that, you have to go google how to do this, and then you come to some sort of documentation that either Microsoft or whoever else, Google, has written, and it's all mouse-centric, click on this icon and click there and click here, and now you are thinking, “How do I translate it"?
OLEG:
Mohammed, a huge, terrible secret, that's how I'm using Reaper these days. I mean, how do I do this thing? How do I use envelopes? Let's google that! What if there was a command or a wizard or anything that just answered my question, boom, and there's the answer.
MOHAMMED:
And answered it in a JAWS-centric way.
OLEG:
And correctly too. That helps.
MOHAMMED:
Correctly, yeah, yeah. Answered it in a JAWS-centric way with keystrokes instead of mouse clicks and just telling you how to get something done. Well, I think what's in the center of that rumor is maybe something that will help you do that.
OLEG:
So how did it look in the presentations?
MOHAMMED:
Well, it is essentially very simple, right? So you type in a question and you get an answer, and the answer tells you how to get a certain thing done. It's based on AI, and so it talks to you in natural language and it tells you what to do. That's all. That's really all. It feels and seems so simple, but up until now it has been impossible.
OLEG:
What was the response at the conventions?
MOHAMMED:
Oh, the major response for both the new feature that we showed off and Picture Smart AI was absolute wonder. And it transported me back to the first time that I used this functionality for both Picture Smart, but also the new feature. And it is just amazing to see people light up as they get those descriptions, people who've never been able to see and now can connect to how things look, people who have been able to see and now can connect to their past, knowing what's on a picture, knowing something that they could have seen before but can't see anymore, and just getting that information again, and just the happiness, the delight, and the excitement, the absolute...
There was this lady who is color-blind and low vision, and Picture Smart AI was able to tell her all the colors that were in a picture, the colors that were in a graph, and she was just so happy, so excited. And there's of course more functionality in JAWS that will allow you to read colors to show that off. And she got happier and happier as we moved along. And she was... I mean, that just stayed with me. This stuff solves real problems for people and it brings back so many memories and it showed, it really showed.
OLEG:
You're making this a bit personal, so I will pursue that. I mean, you were losing your eyesight gradually, right? So you could identify with a person who could see a little, say, 10 years ago, five years ago, now he's losing it and now he's trying or she's trying to use AI and getting back some of what has been lost. So am I reading this correctly? This is a bit personal for you, it's not just business related.
MOHAMMED:
Oh, this is absolutely personal for me. I had the same reaction to this. I was able to see pictures not very well, but I was able to see them, I was able to see colors, I was able to see what I got out of the fridge and all that stuff, and now that has all gone. But with AI, it's all coming back, and it feels great. It feels so good to be able to almost communicate with your past and get an ability back that you had before.
But it's also very nice to connect with people who have never been able to see, but can now get information that they haven't been able to get before because it was so visual. And the AI will explain everything the way it's laid out visually. It's like sitting next to an extremely patient sighted person who is very eloquent and willing to explain every single detail to you.
And that is just to see that with totally blind people who've been blind from birth and have never been able to see, and with people like me who have lost their vision, even though it was not very good to begin with in my case, is just... I mean, it opens worlds. I don't know how else to say it. It opens worlds, and it is fantastic to see how that is impacting people. And it is fantastic to be impacted by it, because I'm not divorced from it. I use it all the time, and it has opened worlds for me as well.
OLEG:
Two more things I wanted to mention here. One is, to use Picture Smart AI with JAWS or Fusion, you don't have to make any additional payments or set up any tokens or do anything like that. If you've got a JAWS license or a Fusion license, that means you've got access to the latest AI technology right there.
And secondly, if English is not your native language or English is not your preferred language, there are ways to use that in another language. Well, if your JAWS interface is in a language other than English, then AI will work in that JAWS interface language. So if it's in Dutch, it's going to be in Dutch and so on.
But even if your language is not supported by JAWS interface at this time, go to Settings Center and look for Picture Smart Language, and you can set it right there. There's a good selection of languages, and it's very likely that your preferred language is among these, so your JAWS interface may be in English, but your Picture Smart AI may be in some other language that it's better for you to use. Now that's really fantastic, and I'm really excited as an international user of this multi-language support we have.
Now sometimes there are errors, sometimes there are problems. We're aware of them, we're dealing with them, we're interested in hearing about them, but the functionality is there.
MOHAMMED:
Absolutely. And if you feel like one language needs to be added given the AI model supports that language, then yes, please tell us about them, and we will try to add as many languages as possible to that combo box.
Also, one thing that you might want to keep in mind is that we are using the version of these services that excludes your data from being trained on. So nobody's saving your data, not us, not the AI providers. And that means that when you send us a picture, you can be absolutely certain that it's absolutely safe. We do not store it, we do not share it except for processing, but nobody may store our data. That's in the contracts that we have with these AI providers.
OLEG:
That's really encouraging and reassuring to know. And in a couple of months, probably if all goes well, we'll have our FSCast about JAWS 2025 features. And I'm looking forward to hearing more.
Thank you, Mohammed, and it's been really, really good talking to you on this FSCast episode for July 2024.
MOHAMMED:
Thank you, Oleg. It's always very good to be here and looking forward to showing more off later.
OLEG:
This was Mohammed Laachir talking about AI technologies in Vispero products, and this brings us to the end of our FSCast episode 247 for July 2024. Please keep your feedback coming. By the way, next month we're planning to talk about reading and writing math with Vispero products. So if you have any math accessibility questions, it's high time to send them in. The address is fscast@vispero.com, and I do hope to start the next episode with Listener Mail, so please keep those messages coming in, will you? But for now, this is Oleg Shevkun and our entire team at Vispero wishing you all the best.