No Virtual Substitute for the Real Device

During my ongoing literature review I often discover interesting facts about things I’ve never thought about. Sometimes I can connect these facts with my own observations: The result is mostly a completely new idea why things are as they are. Maybe these ideas are new to you, too. Therefore I’ll share my new science based knowledge with you!

This week: This time, I think about some examples where a virtual simulation of a specific learning content cannot achieve a similar training effect as accurate and sensitive physical interactions are required.

Game-based and simulation-based training applications allow learners to learn and train new knowledge in an engaging environment. This virtual environment not only provides immediate feedback about the correctness of a user’s inputs, but it also visualizes the learning content in a way that is not possible in the real world. Also, by As a result, learners can develop an in-depth understanding of the underlying principles in a highly motivated way.

The training effect can even be increased when the training system implements immersive virtual reality (IVR) by rendering the gameplay to a Head-Mounted Display (HMD). An HMD allows users to visually immerse themselves in a virtual environment by blocking all visual information from the real world surrounding the user. In this way, a learner can experience the feeling of being directly inside of the virtual environment. This feeling of presence can increase the training effects as the knowledge then is presented in a more natural way to the learner.

Although training applications can simulate any knowledge and allow for a distant knowledge training, some learning contents still need the right hardware to provide haptic feedback for physical training. For instance, it is possible to present trainees large and complex machines in IVR to allow them to inspect their structure and learn about maintenance procedures even though they are just in a classroom. However, training the physical skills to actually disassemble and reassemble such a machine requires haptic feedback as learners need to know how to utilize the required tools correctly.

This problem also applies to other learning contents that require sensitive physical interactions. Recently, a friend and I were playing a mobile piano game requiring us to touch the touchscreen in the right moment and with the right amount of fingers to get the rhythm and keystrokes right. Thus, this game only allows for a rhythm training but not for an actual piano training.

We also tried a VR piano training game that was developed by a group of students who attended one of my seminars. While this VR game allows for a playful interaction with a virtual clavier, it still lacks haptic feedback as it is played using the HTC Vive controllers. However, the virtual environment has the potential to highlight the correct keys in order to guide the user and to explain the instrument. Hence, a player can only learn which of the keys has to be pressed in order to get a specific note but cannot practice sensitive physical interactions. Using a real clavier to interact with the training application would be the best solution, but then the virtual environment would be no longer needed.

As a result of this, training simulations allow for a good declarative knowledge training. However, when the learning content requires the physical interaction with a specific device, it becomes very complicated to achieve a good training environment due to the lack of a good haptic feedback that could create a substitute for the real device.