Camera tracking training app

Project Name:

Camera tracking training app

One-line project summary:

This module will use motion tracking through a mobile phone camera app to track hands and surgical tools for training.

Pitch your project.

The app will allow for those training to select a technique or speciality, will be provided with a list of tools and suitable materials to practice on. The trainee will then set up the phone above the area and practice the procedure. The trainee would need a mobile phone, a stand (which could be purchased or home made, with specifications provided in-app), and surgical tools. For low and middle income environments, phone recycling and donation schemes could be used to provide hardware.

The app will track motion through recognition of hand joints and surgical tools. This will allow trainees to track progress and ensure they are using the correct tools.

Extra features could include videos of best practice, "ask an expert" chat functions and the option to film a practice session for peer critique.

What is your surgical training module?

This module would be a mobile phone app, which would grow over time. Initially we would start with general surgical techniques (sterile techniques, suturing, incision, draining abscesses) and there would be reading materials, videos and chat functions to learn. The app would allow those training to film themselves practising procedures and we would start by using peer/expert feedback. 

(currently considering obstetrics and laparoscopy as initial options). 

Over time we would work to build an analytical model using annotated images/videos. The software would detect hand/finger/tool movement and manipulation of objects. It would be a skeleton detection app which gives a measure of deviation from a "gold standard" example.

What specific surgical skills are you teaching?

We would start with general surgical techniques, to prove the concept works and develop a minimum viable product. After this we would add on modules for different specialities and procedures (currently we are considering obstetrics and laparoscopy as initial options). We would happily collaborate with other groups and surgical teams to deliver this concept.

Who does your project serve, and in what ways will the project improve their skills?

The initial iterations of the app would be designed for students and over time we would evolve the offering to provide tailored content for surgical specialities. Ideally we would work with a local cohort of medical students and progress the app with them starting with the basic techniques and delivering specialised training over the longer term. The basic training app would be able to be scaled much more widely as the ongoing development happens.

In what city, town, or region is your project team headquartered?

What is your project's stage of development?

Who is on your team?

I am looking to transfer project lead to an interested collaborator, or join another team if necessary as I am not in a LMIC.

Who can join your team?

I am looking for clinicians, tech developers, artists and educators, literally anyone who is interested and wants to make this prototype =)