NeuroSight

My grandfather was a Parkinson's Disease patient, and when he passed away, I was in the next room unaware. It isn't uncommon for patients suffering from similar neurological disorders to be unable to communicate with their loved ones and caregivers let alone do anything themselves. Caregivers need to predict the needs and wants of paralyzed patients, which can cause inadequate burdens, complications and  uncertainties. One of the optimum solutions for this scenario is a Brain-Computer Interface (BCI) which reads the brain activity of the patient to spell out what they are thinking into words and more recently via generative models into images. Such solutions can alleviate the distress of the patients, their family, friends and caregivers exponentially. Reading the human mind has been a part of science fiction stories for many years. But, with the advent of current technology, it is not only become possible, but also necessary. Such Brain-Computer Interface (BCI) enable patients suffering from paralysis or neurological disorders having speech impediments and severe disabilities to interact directly with computing devices and in turn auxiliary output devices such as speakers, through their brain signals. However, the systems presently in use are inconveniently bulky, have a high latency between input and output, and can be limitedly operated only by professionals. Scientists are trying to find new avenues of reading brain waves for domestic applications, however current solutions are  highly surgical in nature, expensive and again limitedly operated by professionals. Successful and largely documented attempts at reconstructing text and/or images from the brainwave oscillation data using non-invasive techniques have used Functional magnetic resonance imaging (fMRI) data to facilitate their brain decoding, because of its large spatial resolution. But, fMRI imaging itself utilizes bulky instruments, which are expensive and has a substantially large processing time from input to output. Even though such attempts have been highly successful, they are limited in their capabilities to laboratory processing and obtaining results, which not feasible for individual users commercially.

We are currently developing a working prototype for a ingle end-to-end architecture to examine Electroencephalography (EEG) and Magnetoencephalography (MEG) data, which is comparatively easier, cheaper and smaller but has lower spatial and temporal resolution as compared to fMRI to be applied while reconstructing multimedia. The data is collected by non-invasive sensors on the scalp of the user in a low-cost portable headset, which parses it through a deep-learning autoencoder framework stacked with a generative method, which learns a more compacted representation of EEG and MEG data with visual imagery (VI) and eliminated background activity and employs it to generate, from the semantic characteristics in the spatial features of the data with the visual stimuli to provide image reconstruction evoking from specific brain responses and attempts to reconstruct the stimulated words to mimic natural speech through open vocabulary Sequence-To-Sequence decoding and zero-shot sentence sentiment classification, allowing the decoding of phrases which are previously absent from the training set. The device is paired with a companion app on user’s smartphone for real-time processing and text or image generation for the use of their caregivers in order to better understand what are the needs of the patients, their cause of irritation and provide a means of general communication. The text generation can be, therefore, performed for any translated foreign language spoken by the user. This process can be further utilized to study subconscious responses and help in understanding conditions such as post-traumatic stress disorder (PTSD) and provide alerts during a panic attack, after training a machine learning model to classify certain trigger responses. The study also tries to provide more portable solutions to higher spatial resolution imaging techniques to provide alternatives for fMRI using fNIRS to be used in user-wearable headsets on a daily basis for the same process. The research has footing in previously proven data and currently trying to integrate previous studies with this new method. This procedure proves to better cost-effective as it leverages hardware component which are already in supply from current stakeholders in the industry, but are being used in different avenues. This innovation could further be utilized in diagnosing disorder in their early stages from the collected brainwave data even for seeming healthy individuals.

Our solution directly serves the patients suffering from ailments such as Parkinson's, Stroke, Paralysis, Epilepsy, Palsy and other neurological disorders and at-risk people who are unable to even communicate their basic needs and necessities to their caregivers and are unable to even talk with their loved ones. The proposed solution is cost-effective in nature to produce and easy to be operated by even non-technically aligned users. The wearable device uses non-invasive sensors and imaging to collect data which is in non way proven way harmful to the users in contrast the currently developing devices. The sensors and other hardware are already in production for sectors such as entertainment by industry stakeholders and can cheaply sourced. This alleviates the disparity that would be forced upon users and patients which are not financially well-off. This technology could provide a step into a brighter and more accessible future for all by enabling any affected patient or distressed individual to avail the need and use their taken-away ability of communicating; providing a voice to the otherwise voiceless individuals. The research for this architecture has been going on for some time and has been already submitted to avenues such as S.T. Yau High School Science Award (Asia), Columbia Junior Science Journal, Stanford Intersect for Computer Science research. We are currently submitted under the Innovation Department of Government of India for an innovation and entrepreneur individual internship to develop innovations and connect with industry leaders and professional researchers for advice. This innovation could even help people in diagnosing and checking for disorder symptoms even when they are not fully formed for macroscopic detection and help in getting healthcare support earlier than dangerous cases.

I spent most of my school years taking care of my grandfather who was suffering from Parkinson's Disease and have experience firsthand the effects, restrictions and limitations of such neurological disorders not only for the patients but also for their family, friends and caregivers. During my routines I came to know about how my grandfather couldn't properly convey what he wanted to in his mind and quietly tolerated his ache. So, I am personally attached to this project and intend use all of my determination and passion in making this idea into a common reality. I am a deeply resolute person, and I fear most that after my death my life amounted to nothing substantial , so I want to leave behind a legacy of change for good and for the betterment of society as a whole. Even if one person is able to live a better life because of, my aim would come to fruition. We are currently researching and developing our architecture and consulting professional researchers. In March, during my senior final exams, I was in a road accident and broke my knee so I had to be hospitalized after giving my exams. During my stay at the hospital I got know more about the healthcare system and patients' and caregivers' workings, needs and requirements. My interactions helped develop a more deeper understanding and provide a better map of the future of this project in normal, domestic and hospital environments with experiences from actual caregivers and patients.

I am currently conducting and research on this very architecture, decoding and reconstructive generation. The research has been submitted at different venues and I am currently developing a working prototype to implement the findings. At the current stage, the device shows results and more usability as compared to an idea in consideration. I have worked with EEG wave testing and results on myself only, so, the results provide real-world use cases. The major component is the decoding and generative architecture which is currently under work, other than that the wearable headset and the companion app have deployable prototype designs. At the current rate and further support, I project that the device could be launched as local pilot project under the supervision of professions in upcoming months.

I will not lie, the monetary support could prove to be extremely beneficial in development, implementation, networking and resource accumulation. It could not only help in deploying a real consumer-based product but also help in getting professional support and higher educational opportunities. But, what I am truly excited about is getting an opportunity to meet peers, professionals and network with industry experts at such a highly reputed and influential event as Concordia Annual Summit. I would be inexplicably grateful to be able to attend and connect with people through this opportunity, which itself could provide higher avenues for this project and myself to work with. An I could add credibility to this name event at such a young age for people to take this problem seriously.

Aditya just graduated from his studies as a science student with Computer Science major in a public school. The opportunities and resources are low here for students, as the student body of the nation as a whole is mostly focused on clearing exams to get into a college, which hinders the development of ideas outside of the curriculum, or even the resources to actualize them. Hence, when it comes to prototyping, we need to think outside of the box and utilize the means at our disposal to the best extent, which has led us to some exciting and bizarre scenarios as well. This didn't stop us from losing hope altogether, but slowly build towards realizing our goals. He was also the president of our school's technology club and regularly worked with and mentored other students in the school's innovation society. He also takes up volunteering lectures in basic physics and engineering and aeronautics in partnership with local students of other schools who come from economically weaker sections. This has led him to work and socialize with bright young minds and sharing of stories, problems and ideas. His another idea has been submitted under the innovation department of government of India for an incubation internship and to work with industry professionals for further research and development of innovative ideas.

The currently deployed and researched devices and process are highly expensive, surgically invasive, mostly laboratory based and require professional support for operation. Our use of non-invasive neuroimaging and use of already manufactured hardware from existing industry shareholders provide a more greener opportunity for this project and help in decreasing production cost, manufacturing pollution and ease of availability and use for everyday consumer and patient. Such a low cost wearable device and companion computational app support could provide a solution to a long-lasting and dire problem to the hands of any and every affected patient and caregiver, who can use it even with non-professional experience. Giving a voice to anyone who needs it without leveraging financial or environmental freedom and sustainability. 

Currently I am planning a gap year to explore and develop projects such NeuroSight. I intend to deploy it at a pilot stage with the upcoming month and no later than next year. Furthermore, I am currently connecting with local experts for better development and research and in the upcoming years, expect to expand the manufacturing and availability for a consumer market, at least at a local stage of hospitals and healthcare patients. I fully intend to develop this idea and project further and possibly allow open source access for anyone to develop on this framework on their own, providing a global community support on this idea which isn't hindered by local availability or regional restrictions, providing better opportunities of healthcare and support for all.

I am currently mapping out the different stages of the development of this project based upon the immediate usability by a non-professional or a patient. I am monitoring how the framework is working, its latency rate and how much the model is trying to correctly predict the output by visual rating for images and intent rating for text. The timeframe of this project is dependent and calculated based upon time required by the model to predict and reconstruct the inputs and long-time testing failure rate.

I expect the device to immediately provide the affected patients to communicate efficiently with their caregivers and family. Furthermore, I intend for the device to be able to become a substitute for natural speech for such patients, with instantaneous output generation and common everyday communicative exchange. I believe such a solution could not only provide a new medical tool but also prove to be utilized by anyone in the world, regardless of their financial, educational or regional conditions. It could move us towards a more hopeful version of the future with natural accessibility solutions for anyone and everyone. Not only that, it could provide an influential pivot in this technology which is currently commercially utilized for entertainment, to be used in more medical support patterns.

The core technology include the keywords: Electroencephalography, Magnetoencephalography, Deep Learning, Natural Speech Translation, Neural Networks, Image and Text Generation.

The data is collected by non-invasive sensors on the scalp of the user in a low-cost portable headset, which parses it through a deep-learning autoencoder framework stacked with a generative method, which learns a more compacted representation of EEG and MEG data with visual imagery (VI) and eliminated background activity and employs it to generate, from the semantic characteristics in the spatial features of the data, the visual stimuli and image reconstruction evoking specific brain responses and reconstruct the stimulated words to mimic natural speech through open vocabulary Sequence-To-Sequence decoding and zero-shot sentence sentiment classification, allowing the decoding of phrases absent from the training set. The device is paired with a companion app on user’s smartphone for real-time computational processing and text or image generation for the use of their caregivers. The technology is derived from current research in fMRI data reconstruction to be utilized from low resolution data which can be more easily sourced and produced.

We are currently a one-man work group. But I have received and continue to receive valuable and indispensable advice and support from mentors such as university researchers and professional experts through my internships and other acquaintances I have met along my innovation development journey. More importantly, I stay in touch regularly worth my mentors and teachers with updates and suggestions for this project specifically. I am currently planning a gap year to continue to develop projects like this more efficiently. 

Since the beginning of the year almost. I have been conducting research and prototype development for this project and continued to get support and advice from professionals regarding its optimization, development and future utility. Although this idea is a bit older, I seriously started working and developing it in real world prototype since this year as I got free from my academic responsibilities from school and exams.

Well a good aspect of being desperate as a student for guidance and support is that you don't look for specific people, let alone in this generation denying accessibility to different races, that mentality is rarely found in innovative spaces. Hence, a student community provides you the opportunity to be inherently diverse in people you regular with and for. I work and receive support from many people who live in different countries, come from different backgrounds and follow different faiths and beliefs. Thus, I intent to continue that focus in my bigger plan for this project and life in general. Furthermore, as I have mentioned, I would want this technology to be open sourced for people to build upon, hence anyone could collaborate and help in supporting the cause of universal healthcare opportunities and better diagnosis options for anyone regardless of their background.