Proyecto San Francisco Kids

We address the learning process with blind and visual impaired children between 8 and 12 years old at Lima Peru. We create a raspberry pi computer visual application named "Yupana!" focused on teching numeric skills and a game to get auditive feedback. We worked with children from a school at Lima Peru. We create playground dinamycs to enhance learning objectives and want to show it to the world!!.

Public educational systems not include blind and visual impaired children and are  not prepared to stimulate numerical habilities on this segment of population  in most countries. According to the INEI(peruvian official statistic institute) there are aproximately 800 000 people considered visual impaired on 2018. Visual impair is the second disability on peruvian ranking from 2010-2018. Within this segment 77% are considered not income generators so this affects directly their quality of life. Children with this visual impair at most cities in Peru dont have educational facilities and mostly are from families of a low economic segment. Meanwhile there is a gap of specialized teachers because this children receive their learning on the same teaching system of public schools with children that can see an advance at a different rythm. Most of the educational software applications for children with visual impair run on costly computers or smart phones that are far from be affordable for this segment of the population. 

We design a methodology incorporing low cost raspberry pi devices using computer vision technology to give audible feedback to the studen and helps teacher to improve ther alternatives to stimulate numerical skills with basic math operations to enhance this teaching process. We attached a resume video showing how children inteact on a non invasive way receiving computer feedback. We create and programmed three differente stimulation routines. a)Numerical basic adding operations, b) geometric recognition forms; c) ludic game on board called "treasure find".

Raspberry pi are processor boards that have enough computational power to incorporate computer vision algorithms programmed with python language to recognize circular pieces filling holes (on tactile 3d printed frame, named yupana, which is a native peruvian "quechua" word meaning "counter") when this task is finished a python sequence using openCV open source python package verbalize the result of the task, an the children hears a number on the frame he/she calculate.  Teachers in this process are an important part of the method because create a ludic enviroment and supervised learning tasks. 

Raspberry pi boards are low cost comparison to smartphones or personal computers for blind children. 

Our solution helps visual impair children teachers and parents to get access to new numeric stimulation on children between 8 and 12 years old. Further more, this low cost solution can be scalable because not only can be used at school but it is ready to mount on any desk at family space. Actually most visual impair or applications for blind people are mounted on smartphones technologies which are expensive for this low income segment families. We understand this segment needs because one member of our team is a visual impaired professional and currently serves as a teacher for this students at Lima Peru. In the same venue we have the opportunity to implement it (pre COVID time) at a school with visual impair children taking direct observations on the size and improvements to the initial design. 

The problem on educational software applied on visual impaired children is the need to have internet connectivity, have devices such as smartphones or personal computers. Low income population segment cant afford this effort to improve educational objectives on numeric o tactile feedback tasks. Further more Teachers dont have tech resources to enhace digital experience with children with this kind of disability. Our project addresses this problem with a rapberry pi modular board programmed with python software using computer vision tecnology in a non invasive experience for children but a playing method to increase numerical skills. We highlight teacher role as facilitator and key component on the educational process for children with this visual impair condition.

We start working with this prototype on december 2019 and then COVID arrived so we are routing our initial design to be deploy on a remote way due to logistic restriction because pandemics. 

We were working at the beginning of the project at a visual impair school at Lima Peru, named San Franciso de Asis school. Which is a non private teaching center supervised by Education ministery of Peru.  

We test our initial design with 18 visual impair children between 8 and 12 years odl that dont have any other disabilitie (not verbal, or brain damage).

We expect on 2022 as children get back in person to classes to implement a more broad audience in other public schools

In our solution we are integrating non intrusive technology which helps students to feel more confortable with proposed tasks. In addition with this, we proof this prototype on low cost processor board which is a requirement for families involved in educational process of visual impaired children.

New emerging technologies on fields of computational process and voice recognition are main streams on the field to enhance this innovative methodology.

We expect that our solution can be upgraded to different skills in relation with logical and mathematical habilities. 

Our solution involves hardware and software. On the software side we had programmed on python and visual computing libraries a logic to recognize objetcs and numbers.

On the hardware side our difference on the Yupanas application is we requiere a raspberry board, pi camera, keyboard, a base lamp , and a 3D print base where the studen puts their answer.

But what we feel is an important part of the whole solution is the ludic approach of how children interact with the tactile prototype because is necessary that a person reads the task to the children.

Project resume Visual imparement project

Findlater, L., Stearns, L., Du, R., Oh, U., Ross, D., Chellappa, R., & Froehlich, J. (2015). Supporting Everyday Activities for Persons with Visual Impairments Through Computer Vision-Augmented Touch. En Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 383–384). New York, NY, USA: ACM. https://doi.org/10.1145/270064...

Jeon, H.-M., Nguyen, V. D., & Jeon, J. W. (2018). Real-Time Multi-Digit Recognition System Using Deep Learning on an Embedded System. En Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication (pp. 74:1–74:6). New York, NY, USA: ACM. https://doi.org/10.1145/316454...

Morrison, C., Cutrell, E., Dhareshwar, A., Doherty, K., Thieme, A., & Taylor, A. (2017). Imagining Artificial Intelligence Applications with People with Visual Disabilities using Tactile Ideation. En Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility  - ASSETS ’17 (pp. 81-90). Baltimore, Maryland, USA: ACM Press. https://doi.org/10.1145/313252...

Pincay, G. A. A., & Holguin, J. S. V. (2016). STIMULATION OF THE VISUAL REMAINING OF LOW VISION CHILDREN BY A SOFTWARE PROGRAM AND ITS EFFECTS. Píxel-Bit. Revista de Medios y Educación, (48), 115–134.

Sozim, M. B., Valerio, D. C., Ramos, F. V., & Sabo, P. H. (2017). Device That Assists the Visually Impaired People to Identify People Through Face Recognition. En Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems (pp. 58:1–58:4). New York, NY, USA: ACM. https://doi.org/10.1145/316050...

Stangl, A., Hsu, C.-L., & Yeh, T. (2015). Transcribing Across the Senses: Community Efforts to Create 3D Printable Accessible Tactile Pictures for Young Children with Visual Impairments. En Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 127–137). New York, NY, USA: ACM. https://doi.org/10.1145/270064...

Not physical or hazar exposure at all.

Legal aspect: We have request to children parents a consent authorization to involve their children to being part of the project. And ask them authorization to publicly use their face images on our resume video.

We are on the prototype phase 

Not involving our target audiencie because of pandemic gobern normative of remote education in Peru 

We believe we can create a 2.0 version of our current prototype increasing routines and open access to any interest person or institution.

We are a spin off research group of teachers and researchers from Universidad de Lima. 

Our primary goal is to pass from a prototype version to a final product kit that can be delivered to other peruvian centers and increse target population involved.

We will measure it with the numbers of kits delivered and monitored for two audiences. Fist one by a remote web applications applied for urban institutions which can access to an internet connection and the second audiences are the final familiy users whom can use the stimulation kit that consists on the elements to be used at home.

Our project was financed by the scientific research center of Universidad de Lima( https://www.ulima.edu.pe/en/sc...) and we ground our expansion within Universidad de Lima guidelines (https://www.ulima.edu.pe/en) 

As a researcher  one of my aims is to englobe other experts from the field of education and disabilities in Peru. We are going to participate with our work on the Peruvian National research table organized by the consortium of peruvian universities on December 4, 2021.

We are three permanent staff persons.

Researcher team leader: JOSE ANTONIO TAQUIA

Disabities and Children educationexpert: MARIANA CALLE

Math and tecnology teacher student: KARINA LORA VALDEZ

Mrs Mariana Calle is an a full time visual impaired educational teacher with a strong background on tecnology for children, working at a visual children facility named San Francisco de Asis, focused on attending specifically this target of children. In this directions myself, as a researcher professor at Universidad de Lima (https://cris.ulima.edu.pe/en/p...) , we leade this project and want to continue its deployment. I had more than 20 years of experience working on private companies on manage and deploying tecnologie projects. 

Mrs Mariana Calle, has a personal experience on living as a visual impair person so she really understands children needs. We complement each other on methodology in scientific research approach to design the proposed solution.

Mrs Karina Lora has strong experience working with children and youth students on different technology programs.

One of the highlights on the literature review at the state of the art phase in the very beginning of the project was that the visual impaired segment of population in Peru was out or excluded of the economic activities of the society. We strongly believed that making efforts in the educational phase for children with this disabilitie can improve their life opportunities to become more independent and not to be considered a heavy load to a familiy more over most of low income families considered a problem. As its seen in the First National Review for disabilities in Peru of 2012 (National Institute for Peruvian Statistics, INEI) https://www.inei.gob.pe/media/... which resumes where, what and how peruvian persons with any kind of disabilty life in our country. Its from this study where we obtain that aprox. 53% of visual impair segment are women.

We are beginning on our trip to create and design devices for educational purposes. We strongly believe that receiving feedback its a good way to get better. And for sure what we would like is to win for better sake of our children and increase our social impact.

If we could connect with expert persons and institutions to gain insights to refine and receive suggestions to design a better solution we for sure would appreciate that.

We think we have a creative and innovative solution for a real children with visual impaire needs. If we could socialize what we are doing and getting in touch with more advanced institutions that could mentor us we would be grateful. One aspect on our project is that time assigned for the team is not funded so when pandemic education gets back to school that would slow us to reach more institutions.

Engineering institutions with advance knowledge on artificial intelligence to implement disruptive proposals for overcome disabilities on children in relation of their educational process on realities from south america.

Insitutions like MIT, Research Centers involve on visual impaired children.