BeMyVoice

This product aims to help people with disabilities like deaf/dumb people. We aim to make Sign language translation through Machine learning inorder to make their lives better.

Our system aims to collect data of different sign languages from around the world. The data that is being collected will be converted into datasets for converting them into Deep-learning models for translation to local human language.

So we are building an android application using JAVA, XML, and firebase. The application will collect the data and upload it into firebase. While the process of data collection will happen we have also built various other features in our app that will help people with hearing and speech disabilities to communicate. These are:

1. Text-to-speech conversion, using Google's Wavenet voices. These are not usual robotic voices, but humanlike voices. We are using google cloud text-to-speech API.
2. We have added emotion settings by manipulating the pitch and speed the user can customize their voices In neutral, angry, sad, happy, shocked, and sick modes.
3. We have a speech-to-text conversion. For enabling two conversations for the deaf community.
4. We have a Google Translate feature as well for creating an environment for international conversation while they are traveling.
5. Lastly, we have added a feature to save repetitive texts that they use in their daily lives. This will help them to convert text to speech faster way.

Problem-

• Deaf is a disability that impairs their hearing and makes them unable to hear

• Mute is a disability that makes them unable to speak.  

Current Solution:-

• Services of Sign language interpreter- Which can be costly.

• Carrying notepads when they want to Communicate.

Proposed Solution:-

• Sign Language Recognition System:-The system aims to recognize the sign language and translate it to the local language via text or speech.

  However, when we tried to find datasets for Sign Language Recognition System, we couldn’t find datasets with actual words. We could only find datasets of alphabets of different sign languages.

  To have a proper Sign Language Recognition system, we will be collecting data from all around the world in different languages of actual words rather than just the alphabets.

  This will help us in creating datasets for many languages. These datasets in the future could be used by researchers from all around the world in coming up with the best Sign Language Recognition System which may support a cross-language interpretation.

According to DownToEarth over five percent of the global population — 432 million adults and 34 million children — have disabling hearing loss. Till 2020, however, only 17 percent of those who required hearing aid, actually used one.

One in every four individuals, or around 2.5 billion people, across the world, will experience mild-to-profound hearing loss by 2050, said World Health Organization (WHO) in a new report.

Our Solution is going to serve the specially-abled population of the world, they could be deaf, mute, hard of hearing, or even may have the problem of stuttering. 

Here are some stats:

According to DownToEarth over five percent of the global population — 432 million adults and 34 million children — have disabling hearing loss. Till 2020, however, only 17 percent of those who required hearing aid, actually used one.

One in every four individuals, or around 2.5 billion people, across the world, will experience mild-to-profound hearing loss by 2050, said World Health Organization (WHO) in a new report.

According to WHO, It is estimated that at least 700 million people will suffer from disabling hearing loss and require ear and hearing care

WHO defines disabling hearing loss as greater than 40 decibels (dB) loss in adults and greater than 30 dB in children.

WHO states the reason for hearing disabilities in the future will be:

• Exposure to excessive noise

• Genetic causes

• complications at birth

• certain infectious diseases

• chronic ear infections

• use of particular drugs 

• Aging. 

So we are not just preparing for the Present. But the Future.

For now, while we are simply collecting Data the other features mentioned in my solution like, Text-to-speech conversion will help people who are mute/have a speech impairment. The speech-to-text feature will serve the people who are deaf/hard of hearing. And the translation feature will help these specially-abled people to communicate while traveling

In order to understand the needs of the population that are suffering from deaf/mute disability, I have gone and done conducted one on one interviews with a few deaf/mute people in my locality and noted the important points such as:

• What difficulties that they face when they try to communicate with people who don't know sign language in their local vicinity.
• What difficulties do they face while traveling locally and internationally?
• How do they manage when they do not have a sign language interpreter.
• Whether or not have they used a text-to-voice converter?
• How do they communicate when they meet a foreigner who is also deaf/mute but does not know their local sign language.

The features in my app try to address these points based on their answers.

We are trying to solve the communication problem faced by 5% of the world population just because they are specially-abled. In doing so we are opening a wide range of opportunities for them which otherwise would have been not open or would have been very difficult to overcome with the communication barrier.

The possibilities that our project will be opening in the future are:

• DL models: Deep Learning models can be used to convert sign language to normal language. 
• Universal Learning Portal: Most Videos collected from all sign language datasets will be available here. The most accurate among those videos would help in creating a learning platform for people around the World.
• Sign Language to text API: After successful model generation, we will deploy it on the cloud to create our own API which can be used as a service on different Platforms. (One fine example would be like booking trains/flights or even ordering food at a restaurant/drive-by by signing at the AI-controlled camera.)
• AR messages model: The video data collected can be used to develop an augmented reality model which can enact sentences in sign  Languages. (This could be used to send messages by the users to their loved ones in an amazing new way)

We have selected this stage of development because we are still under the testing stage of the prototype. Once we have completely conducted the beta stage we will be releasing the app for testing we will conduct a gamma test by releasing the app on the play store.

Our achievements are as follows:

• During our alpha testing phase, we have participated in the Hack-To-Enable hackathon on HackerEarth. In this hackathon, their aim was to help and uplift a broad spectrum of people with disabilities - from the people with visual/speech/hearing impairments to amputees and wheelchair users, and the like. We were one of the winners in it. here is the link for reference- Hack-To-Enable. And the certificate.

• We had also participated 3M-CII Young Innovators Challenge Awards. We had been one of the semi-finalists in the competition. Even though we didn't actually win any prize they have actually helped us a lot in designing a revenue model for the project which would make it a lot easier for us to get investors in the future. Here's a pic of the certificate.

We are mainly using JAVA/XML to build the application front end and in the backend, we are using Firebase/Firestore also we are using Google cloud projects API for a few features of the app. In the future, we are planning to use the GCP ML kit for making and deploying the sign language to the human language model once the data is collected in sufficient amounts. Currently, though it's an iceboxed feature.

Currently, we are still in the testing phase, but we plan to serve the entire population of deaf/mute in the world. That is almost a little more than 5% of the world population. But by the end of the next year, we are planning to serve at least the Indian deaf/mute community and then eventually, we would expand to other countries as well.

Our impact goal for next year is to at least collect data entirely for one Sign language that could be used to actively make one model for translating sign language, We will be focussing on Indian sign language for this. Since it will be easier for us to gather the data in our native country. For achieving this goal we will focus on visiting relevant communities like deaf/mute schools, or places where Indian sign language is taught. We will also be reaching out to the deaf/mute communities in different states of the country to maximize the contribution in our dataset and by doing so we will have increased the variable parameters that will help us in creating a more accurate model. 

We are measuring our progress by setting milestones.

Our milestones are as follows:

• Design the app prototype
• Create an MVP
• launch alpha test
• Improve the app based on alpha test 
• Launch beta test  by approaching different deaf/mute communities in the locality (We are currently in this stage)
• Improve app based on Beta test
• Launch the app in test mode in Playstore
• Improve the app based on Playstore feedback
• Launch the app

Sustainable goals that align with our solution are Sustainable cities and communities. Since we are trying to create a better mode for the deaf/mute communities to communicate with locally and internationally.

The biggest barrier that exists for us to accomplish our next year's goal is the lack of financial investment, we are currently working on our own without any investment. This lack of investment hinders our marketing which results in reaching fewer people who could actually use the application. Also with an increasing list of users, we will require more funds to maintain the server/API cost. Even though we have a revenue system set up for the app but we will still need some funding that will back us up in reaching our goals faster.

Prof. Sourav Mitra, who is mentoring our project is the professor under whom we have started to implement the project in our 4th year. He had mentored a lot of students throughout his career as the Professor of Information and Technology in Netaji Subhash Engineering College. 

Faiz Alam i.e. is me, I have experience working as a freelance android developer, UI/UX designer, and Video editor/animator for a little over 6 years. Here is the link to my Fiverr profile. My experience in android development and UI/UX designing help me to develop the entire application under the guidance of Professor Sourav Mitra. 

Aditi, who is a part-time tester of the application, is a Graduate of Engineering from Netaji Subhash Engineering College as well in the field of ECE. She has been continuously helping us test each version of the app while we are developing it.

Currently, we have not partnered with any organization, but we will soon be approaching different NGOs and communities that are helping deaf/mute people of their localities to get their valuable help in diversifying and filling our video dictionary database for Indian Sign language.

NA

We are qualified for this because our solution is focusing on creating a platform for the disabled to have two-way communication with everyone around them. Which otherwise is a very difficult thing if the people around don't know sign language. In the future, we are planning to create a cross-language translation model as well which will help them in communicating with everyone from around the world. So if we are given the prize it will help us reach our future goals without worrying about the monetary drawbacks that innovators usually have to face while developing/advancing their projects.