Sound Feel is a project to bring humans and computers closer
Project Currently In Progress, More Documentation In Coming (Download Project Proposal Below)
thesis_topic__3_.pdf | |
File Size: | 909 kb |
File Type: |
Project Overview
Sound Feel is a project I have been working on since covid started. It was inspired from the need to have a solid hardware and software platform for amputees to control prosthetics, but also has other interesting applications. Sound Feel is a computer brain interface made up of two major parts; Hardware to collect data from the users nervous system, and hardware to send data into the users nervous system.
Data Collection:
The data collection side of this project consists of an ultrasound imagining system strapped to a users forearm. This ultrasound imager takes photos of the users forearm muscles as they move, and sends them over to a image classification system built with Tensor Flow. The image classifier then determines what position the users hands are in, and provides that output as input to a robotic or virtual system.
Examples of robotic systems:
Examples of virtual systems:
Data Feedback:
An arm band with temperature and haptic feedback provides feedback to the user from the robotic or virtual system they are controlling. For robotic systems a custom fingertip sensor is currently in development. A gallery of photos has been prepared below that further illustrates the project and its development state.
Current State of Project Development
Data Collection:
The tensor flow part of the project is still in preliminary development, but results with small data set have shown the imager classifier able to determine between a close and open fist. Next steps are expanding the amount of training data the neural network has to work with and getting live image classification working.
On the hardware side, I have been able to modify an existing ultrasound system to be much more ergonomic to attach to a users forearm.
Data Feedback:
Efforts to machine the arm mounted feedback device have been successful, and now I am focusing on developing the custom PCB to run the feedback system. I made a couple mistakes on my PCB, and have designed a new one. It is currently being fabricated. Its a four layer stackup with the second and third layers being ground and VCC respectively
Sound Feel is a project I have been working on since covid started. It was inspired from the need to have a solid hardware and software platform for amputees to control prosthetics, but also has other interesting applications. Sound Feel is a computer brain interface made up of two major parts; Hardware to collect data from the users nervous system, and hardware to send data into the users nervous system.
Data Collection:
The data collection side of this project consists of an ultrasound imagining system strapped to a users forearm. This ultrasound imager takes photos of the users forearm muscles as they move, and sends them over to a image classification system built with Tensor Flow. The image classifier then determines what position the users hands are in, and provides that output as input to a robotic or virtual system.
Examples of robotic systems:
- Prosthetics
- Robot Arms
- Smart Homes
Examples of virtual systems:
- Finger tracking for VR
- typing without a keyboard
Data Feedback:
An arm band with temperature and haptic feedback provides feedback to the user from the robotic or virtual system they are controlling. For robotic systems a custom fingertip sensor is currently in development. A gallery of photos has been prepared below that further illustrates the project and its development state.
Current State of Project Development
Data Collection:
The tensor flow part of the project is still in preliminary development, but results with small data set have shown the imager classifier able to determine between a close and open fist. Next steps are expanding the amount of training data the neural network has to work with and getting live image classification working.
On the hardware side, I have been able to modify an existing ultrasound system to be much more ergonomic to attach to a users forearm.
Data Feedback:
Efforts to machine the arm mounted feedback device have been successful, and now I am focusing on developing the custom PCB to run the feedback system. I made a couple mistakes on my PCB, and have designed a new one. It is currently being fabricated. Its a four layer stackup with the second and third layers being ground and VCC respectively