Haptics with force sensing (game) iteration 2

For the course project the task was to simulate an intra dural brain tumor removal surgery where the surgeon has to perform 4 stages of surgery:

1. incision on the skin layer
2. incision on the bone layer (involves milling and cutting)
3. incision on the dura matter
4. extraction of brain tumor

Using a gamification-based approach our objective was to simulate each stage and provide haptic feedback for each stage. Additionally, evaluation metrics for each layer were implemented as part of the gamification to record the learning curve of novice surgeons.

A working demo video is shown below.

A. UI Development

For the UI development I completed the following tasks:

1. integrate hapi_fisica library with the standard Haply processing sketch
2. implement split-view to XY plane and Z-axis
3. add cp5 library components to the XY plane
4. implement smoothing for FSR
5. implement flow of the project
6. provide a cutting target for each layer
7. provide an FSR target for each layer
8. implement cutting simulation
9. improving UI performance by code-fixes and and making the animation rendering-efficient

Complete UI design User Interface design components: 1) XY Plane for the main cutting operation 2) FSR bar with force represented as % of maximum force 3) Tool Selection 4) Z-axis windows. The four layers from top to bottom: Skin, Skull, Dura Mater, and the brain tumor. 5) Evaluation: blinking time in seconds 6) Evaluation: overshoot score 7) Current Operation stage display 8) FSR target 9) Cutting target on XY plane 10) Reset Device button to recalibrate Haply avatar

B. Gamification

For the gamification, the idea was to have a game-flow. Additionally, having gamification implemented will allow the user to track their learning curve, for example, if their new score was better than the previous one. So the task was broken down to 4 stages as described earlier, and the user needed to complete one stage to move to the next. The task mostly involved moving the end-effector and simultaneously pressing it while following the trajectory. The following evaluation measures were performed for each stage:

1. Cutting accuracy
2. Overshoot score
3. Time