Many tech companies have come up with screen reader software for visually impaired people to make it easy for them to use social media platforms. However, until now there was no technology available to translate memes that don’t have alternative text or alt text to describe the meme.
Now, researchers from Carnegie Mellon University have invented a technology that identifies memes and adds descriptive alt text to make it accessible for visually impaired people.
According to Jeff Bigham, associate professor in CMU’s Human-Computer Interaction Institute (HCII), “Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention.”
Usually, it’s not possible for meme creators to add alt text on social media where most of the memes are found. Twitter, however, allows users to add alt text the images they are uploading but the feature is difficult to find and most users often ignore it.
A study conducted by CMU researchers showed that only 0.1% of the one million photo tweets had alt text.
The new technology uses basic computer vision techniques for identifying the meme template and OCR techniques are deployed for reading the overlaid text on memes. Researchers are trying to standardize a template for describing a meme and the overlaid text can be added according to a particular meme.
However, converting a meme into accessible text for visually impaired is still a challenge.
As Cole Gleason, a Ph.D. student at HCII says, “It depended on the meme if the humor translated. Some of the visuals are more nuanced”
Researchers have also created a platform for converting memes into audio format. The platform allows users to drag and drop audio files into meme templates. The system is also capable of translating existing memes into audio memes to convey the message through sound effects.
Gleason says, “One of the reasons we tried the audio memes was because we thought alt text would kill the joke, but people still preferred the text because they’re so used to it.”
Now that researchers have developed the technology, the next step is to integrate it into various meme generator websites and social media platforms.
Researchers are thinking about speaking to Twitter for adding a new feature to make it easy for adding alt text. CMU researchers are also working on developing a browser extension for Twitter to add alt text for every image.
In another related project, researchers are finding a means to integrate alt text into the metadata uploaded with the image. It would append the alt text with the image regardless of the platform it is uploaded on.
Also Read: Lenovo’s Leez Single Board Computer Is Here To Rival Raspberry Pi Boards
