I am currently completing my PhD in mixed reality BCI. I am working on decoding upper limb motion from neural signals using embodied immersive experimental setups.
The video shared here is a 3 minute project pitch highlighting the experimental setup for user motor training combined with the neural network based decoder that trains on kinematic motion data and EEG brain signals. This has been achieved in a real-time closed loop BCI and is presented in a research software application framework (NeuroDojo).
In terms of architecture, we use Unity for the experimental paradigm and kinematic motion capture combined with Matlab/Simulink for EEG signal acquisition and a python based deep learning framework/decoder. Its all held together with tears and duck tape! But it works. Research papers will be linked below as they are published.
McShane, N., Korik, A., McCreadie, K., Charles, D. K. & Coyle, D., 5 Dec 2022, (Published online) 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE). IEEE Xplore, p. 697-702 6 p. (2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 – Proceedings).
Shane, N. M., McCreadie, K., Charles, D., Korik, A. & Coyle, D., 1 Feb 2024, (Published online) 2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE). IEEE, p. 1039-1044 6 p. (2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2023 – Proceedings).
iMaxSpa was part of a research project (FOREUM) between Ulster University and The Western Health Trust led by Consultant Rheumatologist, Dr Philip Gardiner. I was researcher and software developer on the project.
The iMaxSpa app is a solution for home monitoring of spinal mobility in cases of axial spondylitis. The app used a small bluetooth IMU sensor that could be attached to the spine via adhesive pads. The user could follow a protocol of range of motion exercises to get a mobility score. The app had encrypted login and firebase backend for saving data for clinicians to access. One of the goals of the research was to enable people to participate in their condition monitoring.