In the constantly shifting sands of technology, true revolutions are rare. Yet, as I reflect on my 25-year journey as a JAWS user and my role at the helm of Freedom Scientific’s software team over the past year and a half, I am thrilled to share a development that marks a genuine leap forward: the introduction of Picture Smart AI. This isn’t just another step; it’s a giant leap, transforming accessibility for those of us who are blind or have low vision.
The Dawn of a New Era with AI
Early in 2023, the potential of AI to revolutionize accessibility became unmistakably clear. It promised a future where some of the digital barriers that have long stood firm could finally be dismantled. Our commitment at Freedom Scientific has always been to pioneer advancements that break down these barriers. We started the AI journey by introducing the Face in View feature in October which provides feedback to help someone get centered and aligned in the camera feed for online video calls.
Project Horizon: The Challenge That Became a Milestone
Back in the Summer we started to realize the potential of how AI technology could break down one of the biggest barriers we have always faced. Detailed and dynamic descriptions of images and graphics has always seemed to be this elusive dream; it’s been getting closer over the years but was always just over the horizon. To me, it is one of the last frontiers to conquer in traditional screen reading. When we realized we could take a sledgehammer to this barrier, we knew we had to jump. Our excitement was amplified when the winning entry to the Next Big Thing event in November was also the idea of integrating AI powered image descriptions into JAWS. In early December I posed a challenge to our software team. I asked them to integrate this new technology into JAWS, to do it in record time, and to make sure it was available to all our users around the world. The team accepted the challenge and Project Horizon was born. The good news was that we already had the framework built. The initial Picture Smart feature was introduced in 2019 and was powered by early iterations of AI technology. It served its purpose and provided information about images that we would not have otherwise been able to access. However, the older technology pales in comparison to what is available now from companies like OpenAI and Google. Three months and a lot of hard work later and here we are; Picture Smart AI is released and available for all to try out. We chose to first place it in our Early Adopter Program as a signal that we are nowhere near done with it. In fact, we are well on our way in developing the next phase of Picture Smart AI but we wanted to get this out for everyone to start using now.
Picture Smart AI in Action: A Personal Perspective
Picture Smart AI is more than a feature; it’s a vision come to life. Using advanced AI algorithms, it interprets and verbalizes the content of images, bringing them to life for those of us who rely on JAWS to interact with the digital environment. This technology will revolutionize the way we engage with digital content, offering a level of independence and interaction that was previously unattainable.
One way this recently came to life for me was when I discovered that an incorrect company logo was being used in a PowerPoint slide I was reviewing. I used Picture Smart AI to describe the slide and it identified the company logos that were present which alerted me that one of them was for the wrong company. Traditional screen reading could not tell me this because the logo was an image. I’ve always been a pretty good proof reader with JAWS but this was the first time I was able to discover something that was purely graphical information.
Another way this new technology solved a problem for me is when I was looking to purchase a new coffee machine. I was reviewing product pages on a popular shopping site that rhymes with Hamazon and wanted to ensure the one I purchased would have buttons and not touch screen controls. Normally I would need to dig through dozens of reviews hoping to find someone mention this kind of info. This time though I reviewed the descriptions of several of the product photos and got quick confirmation that the machine I was interested in did in fact have physical buttons.
Each day we are discovering new ways to benefit from Picture Smart AI and I love hearing about how others are using it to transform their lives. I ask everyone to please share with us and the community at large how you find it useful. Together we will explore this new world that has opened up.
Sometimes the picture is distorted
One thing I need to be clear about is that AI powered image descriptions are not perfect. There are many factors involved but the bottom line is that sometimes the description has errors or may even be mostly incorrect. The term for this is “hallucinations” and it happens when the AI makes assumptions or misunderstands what is in the picture. I have no doubt hallucinations will become less frequent as time goes on, in fact, we are already seeing less of it, but it will probably be something we will always have to deal with. One way we are addressing this is by providing descriptions from two independent services so you can compare results and look for discrepancies. Learn more about this by reading my post about AI image description hallucinations.
Looking Ahead with Hope and Excitement
As we look to the future, I am filled with excitement for the possibilities that lie ahead, not just for Picture Smart AI, but for the entire field of accessibility technology.
I invite you to join us in this journey, to share your experiences, challenges, and successes. Your insights are invaluable as we continue to evolve and enhance Picture Smart AI. Together, we can continue to break down barriers and open up a world of possibilities for visually impaired users around the globe. Read more and learn how to use Picture Smart AI at the Picture Smart AI feature page.