Skip to main content

Advertisement

SpringerLink
Log in

MemHolo: mixed reality experiences for subjects with Alzheimer’s disease

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

HoloLens is the most recent and advanced forms of wearable Mixed Reality (MR) technology. It enables the user wearing a head-mounted device to experience 3D holographic objects “inside” the visualization of the real environment where he or she is located. Existing HoloLens applications have been developed in domains such as data visualization, entertainment, industrial training, education, and tourism, but the use of this technology in the arena of mental health is largely unexplored. The paper presents a HoloLens-based system called MemHolo that addresses persons with mild Alzheimer’s Disease (AD). AD is associated to a chronic progressive neurodegenerative process that severely affects cognitive functioning (especially memory) and some motor functions. MemHolo is intended to be used as a cognitive training tool to practice short-term and spatial memory in a safe and controlled virtual environment, and to mitigate the effects of mental decline. The paper discusses the design process of MemHolo, and describes three evaluation studies on progressive prototypes. To our knowledge, MemHolo is the first HoloLens application designed natively for persons with AD. Our empirical work sheds a light on how these people experience HoloLens applications, highlights some challenges and potential benefits of using MR technology in the AD arena, and may pave the ground towards new forms of treatment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Abe M, Yoshimura T, Koizumi S, Hasegawa N, Osaki T, Yasukawa N, Koba K, Moriya K, Sakai T (2005) Virtual forest: design and evaluation of a walk-through system for forest education. J For Res 10:189–197. https://doi.org/10.1007/s10310-004-0131-x

    Article  Google Scholar 

  2. Alzheimer Association (2016) 2016 Alzheimer’s disease facts and figures. Alzheimers Dement (12):1–80. https://doi.org/10.1016/j.jalz.2016.03.001

  3. Alzheimer’s Disease Fact Sheet. https://www.nia.nih.gov/health/alzheimers-disease-fact-sheet. Accessed 3 May 2018

  4. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, Arlington

    Book  Google Scholar 

  5. Aruanno B, Garzotto F, Torelli E, Vona F (2018) HoloLearn - wearable mixed reality for people with neurodevelopmental disorders (NDD). In: ASSETS 2018: The 20th International ACM SIGACCESS Conference of Computers and Accessibility

  6. Astur RS, Taylor LB, Mamelak AN, Philpott L, Sutherland RJ (2002) Humans with hippocampus damage display severe spatial memory impairments in a virtual Morris water task. Behav Brain Res 132:77–84. https://doi.org/10.1016/S0166-4328(01)00399-0

    Article  Google Scholar 

  7. Blaschke CM, Freddolino PP, Mullen EE (2009) Ageing and technology: a review of the research literature. Br J Soc Work 39:641–656. https://doi.org/10.1093/bjsw/bcp025

    Article  Google Scholar 

  8. Brooke J (1996) SUS - a quick and dirty usability scale. Usability Eval Ind 189:4–7. https://doi.org/10.1002/hbm.20701

    Google Scholar 

  9. Burgess N (2006) Spatial memory: how egocentric and allocentric combine. Trends Cogn Sci. https://doi.org/10.1016/j.tics.2006.10.005

  10. Byagowi A, Mohaddes D, Moussavi Z (2014) Design and application of a novel virtual reality navigational technology (VRNchair). J Exp Neurosci 8:7–14. https://doi.org/10.4137/JEn.s13448

    Article  Google Scholar 

  11. Cushman LA, Stein K, Duffy CJ (2008) Detecting navigational deficits in cognitive aging and Alzheimer disease using virtual reality. Neurology 71:888–895. https://doi.org/10.1212/01.wnl.0000326262.67613.fe

    Article  Google Scholar 

  12. Design - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/design. Accessed 3 May 2018

  13. Folstein MF, Folstein SE, McHugh PR (1975) Mini-mental state. J Psychiatr Res 12:189–198. https://doi.org/10.1016/0022-3956(75)90026-6

    Article  Google Scholar 

  14. Garzotto F, Gelsomini M, Clasadonte F, Montesano D, Occhiuto D (2016) Wearable Immersive Storytelling for Disabled Children. In: Proceedings of the International Working Conference on Advanced Visual Interfaces - AVI ‘16. p 196–203

  15. Garzotto F, Gelsomini M, Occhiuto D, Matarazzo V, Messina N (2017) Wearable immersive virtual reality for children with disability: a case study. In: Proceedings of the 2017 Conference on Interaction Design and Children - IDC ‘17

  16. Gaze - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/gaze. Accessed 3 May 2018

  17. Gestures - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/gestures. Accessed 3 May 2018

  18. GitHub - Microsoft/MixedRealityToolkit-Unity. https://github.com/Microsoft/MixedRealityToolkit-Unity. Accessed 7 May 2018

  19. HoloLens hardware details - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/hololens-hardware-details. Accessed 3 May 2018

  20. Iancu I, Olmer A (2006) The minimental state examination - an up-to-date review. Harefuah 145:687–690

    Google Scholar 

  21. Klinger E, Chemin I, Lebreton S, Marié RM (2004) A virtual supermarket to assess cognitive planning. Cyberpsychol Behav 7:292–293

    Google Scholar 

  22. Lee JY, Kho S, Bin YH, Park S, Choi JS, Kwon JS, Cha KR, Jung HY (2014) Spatial memory impairments in amnestic mild cognitive impairment in a virtual radial arm maze. Neuropsychiatr Dis Treat 10:653–660. https://doi.org/10.2147/ndt.s58185

    Article  Google Scholar 

  23. Milgram P, Kishino F (1994) Taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77–D:1321–1329

    Google Scholar 

  24. Mitchell AJ (2009) A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res 43:411–431. https://doi.org/10.1016/j.jpsychires.2008.04.014

    Article  Google Scholar 

  25. Mungas D (1991) In-office mental status testing: a practical guide. Geriatrics 46:54–58 63, 66

    Google Scholar 

  26. NHS (2015) How is dementia treated? In: NHS. http://www.nhs.uk/conditions/dementia-guide/pages/dementia-treatment.aspx

  27. Ohyama S, Nishiike S, Watanabe H, Matsuoka K, Akizuki H, Takeda N, Harada T (2007) Autonomic responses during motion sickness induced by virtual reality. Auris Nasus Larynx 34:303–306. https://doi.org/10.1016/j.anl.2007.01.002

    Article  Google Scholar 

  28. Prince M, Comas-Herrera A, Knapp M, Guerchet M, Karagiannidou M (2016) World Alzheimer report 2016 improving healthcare for people living with dementia. Coverage, quality and costs now and in the future. Alzheimer’s Dis Int:1–140

  29. Sitzer DI, Twamley EW, Jeste DV (2006) Cognitive training in Alzheimer’s disease: a meta-analysis of the literature. Acta Psychiatr Scand 114:75–90

    Article  Google Scholar 

  30. Spatial mapping - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/spatial-mapping. Accessed 3 May 2018

  31. Tamura H, Yamamoto H, Katayama A (2001) Mixed reality: future dreams seen at the border between real and virtual worlds. IEEE Comput Graph Appl 21:64–70. https://doi.org/10.1109/38.963462

    Article  Google Scholar 

  32. Using the Windows Device Portal - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/using-the-windows-device-portal. Accessed 7 May 2018

  33. Voice input - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/voice-input. Accessed 3 May 2018

  34. Werner P, Rabinowitz S, Klinger E, Korczyn AD, Josman N (2009) Use of the virtual action planning supermarket for the diagnosis of mild cognitive impairment. Dement Geriatr Cogn Disord 27:301–309. https://doi.org/10.1159/000204915

    Article  Google Scholar 

  35. What is mixed reality? - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/mixed-reality. Accessed 3 May 2018

Download references

Acknowledgments

We thank Dr. Valoriani and Dr. Oliveto from Sixth Element for their collaboration in the initial phase of our research. We are grateful Dr. Olivieri and his team at the Gerontological Division at the Hospital in Palermo for their contribution to MemHolo design process. Special thanks go to the staff and guests of the Filo di Arianna Center for their help during the evaluation process.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Politecnico di Milano, Milan, Italy

    Beatrice Aruanno

  2. Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

    Franca Garzotto

Authors
  1. Beatrice Aruanno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Franca Garzotto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Beatrice Aruanno.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aruanno, B., Garzotto, F. MemHolo: mixed reality experiences for subjects with Alzheimer’s disease. Multimed Tools Appl 78, 13517–13537 (2019). https://doi.org/10.1007/s11042-018-7089-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-7089-8

Keywords

Search

Navigation