Application Of Augmented Reality In Medical Education: Study Of Human Anatomy
Medical education is a course that integrates knowledge of Human Anatomy. Augmented Reality (AR)-assisted instruction has infrequently been applied in understanding the function and working of human body parts. With help of Augmented Reality the aspiring students who pursue medical can learn and practice medical in an interactive manner and interested in learning. However, it neither involves interactive practice nor embodies both textbook learning and the practice of medical learning skills simultaneously. As a complement to video-assisted instruction, AR can apply virtual messages to learning objects so that 3dimensional models can be superimposed into textbooks, which allows learners to read books while operating 3D character models. To verify the effects of learning outcomes and learning motivation with AR-assisted instruction and the effects of different difficulty levels on instruction materials, two experimental studies were implemented. The findings indicate that AR-assisted instruction is more effective than video-assisted instruction, and the effects are better for more difficult learning.
Index Terms—Augmented Reality, Marker-based AR, Rendering, Vuforia.
Augmented reality is a technology within which we can see the objects in physical world virtually, thus providing a composite view. It gathers a wide variety of user experiences. We are going to develop a system with augmented reality that lets user to try on Virtual 3D object in user’s real human organ structure in real world. The main purpose of this project is to develop an application for Medical education. By using this application it will be convenient for the user to virtually learning purpose. AR applications that have been presented to students have been proven to stimulate learners and enhance the learning experience; however, the apps are not as widely accepted as they perhaps should be. AR has great potential in education, especially in medical education, given its ability to integrate virtual objects within users’ physical reality, making simulations more realistic and immersive.  For each target object, we have its Blender 3D model plus a small set of reference images we will call “keyframes” of the object. We try to match each input frame against only a subset of the keyframes to keep processing at frame-rate, and track features lying on the visible objects over consecutive frames. It adds information and meaning to a real object or place.  Augmented reality is characterized by the incorporation of artificial or virtual elements into the physical world as shown by the live feed of the camera, in real-time.
Augmented reality (AR) is a field of computer research which deals with the combination of real world and computer generated data. Augmented reality (AR) refers to computer displays that add virtual information to a user’s sensory perceptions. It is a method for visual improvement or enrichment of the surrounding environment by overlaying spatially aligned computer-generated information onto a human’s view (eyes) Augmented Reality (AR) was introduced as the opposite of virtual reality: instead of immersing the user into a synthesized, purely informational environment, the goal of AR is to augment the real world with information handling capabilities.
II. Augmented Reality
A. AR-assisted learning
The AR-assisted learning system for medical education class is called AR-PEclass. AR-PEclass learners need to learn using the instructional materials in the book. After starting the AR-PEclasssoftware, learners only need to align the camera lens of a mobile carrier (such as a mobile phone) with the human anatomy picture in the book.  The carrier screen automatically displays a 3D character model, and four functions to operate the 3D character model: (1) zoom in and out (change the size of the 3D model), (2) left and right rotation (change the 3D model viewing angle), (3) direction key (adjust the position of the 3D model in the screen), and (4) the action essentials (provide learners with operating instructions regarding human anatomy). Learners follow the teacher’s instructions, observe the human anatomy changes presented by the 3D character model, and then answer the questions on the study sheet (or worksheet) to complete the learning of each human anatomy in the book. Each function of the AR-PE class software will be mapped for the main learning purpose. Learners learn human anatomy by using the operation function of the 3D character model. For example, the function of zooming in and out may help learners observe human anatomy locally or in detail, and the instructions on the action essentials may help deepen learners’ memory of the learning content.
B. Marker-based AR
Markers work by having software recognise a particular pattern, such as a barcode or symbol, when a camera points at it, and overlaying a digital image at that point on the screen. If the image is three-dimensional or animated, the effect is of a digital experience unfolding on the surface upon which the pattern is printed.
III. Literature Survey Physical education is a course that integrates knowledge of sports with skill drilling. Augmented Reality (AR)assisted instruction has infrequently been applied in sport skill drilling. Video-assisted instruction has frequently applied to physical sports; however, it neither involves interactive practice nor embodies both textbook learning and the practice of sporting skills simultaneously. As a complement to video assisted instruction, AR can apply virtual messages to learning objects so that 3dimensional models can be superimposed into textbooks, which allows learners to read books while operating 3D character models.  To verify the eects of learning outcomes, motor skills, and learning motivation with AR-assisted instruction and the eects of dierentdiculty levels on instruction materials, two experimental studies were implemented.  Augmented reality, while not necessarily a new technology, is becoming more well-known and gaining some momentum in medical education through Google Glass and Microsoft’s HoloLens. Not only can augmented reality aid in student education, but it also can impact patient care through its ability to enhance medical training. Medical libraries can partake in this new endeavor by being aware of applications in augmented reality that can benefit students and educators.  In this paper, the authors show that augmented reality technology has a positive impact on the motivation of middleschool students. The Instructional Materials Motivation Survey(IMMS) based on the AR CS motivation model was used to gather information; it considers four motivational factors: attention, relevance, confidence, and satisfaction. Motivational factors of attention and satisfaction in an augmented-realitybased learning environment were better rated than those obtained in a slides-based learning environment.
IV. Existing System
AR can apply virtual messages to learning objects so that 3 dimensional models can be superimposed into textbooks, which allows learners to read books while operating 3D character models. To verify the eects of learning outcomes and learning motivation with AR-assisted instruction and the eects of dierentdiculty levels on instruction materials, two experimental studies were implemented. The 3D character model zoom in and out 3D model and left and right rotation 3D model viewing angle. The object can be viewed only as a plan 3D character model. Only the outer surface characteristics can be explored using this system.
V. Proposed System
In our proposed system, Interior parts of the model can also be explored and user can view and explore the working and its function virtually. The problem of viewpoint tracking and virtual object interaction. We also added audio functionality to the animation or the user to experience and feel that there are immersed into the space. Our system will enable the teadious process of learning into a more creative and engaging process and make the learners more curious about organs functionality. However, it neither involves interactive practice nor embodies both textbook learning and the practice of medical learning skills simultaneously. To verify the effects of learning outcomes and learning motivation with AR-assisted instruction and the eects of dierentdiculty levels on instruction materials, two experimental studies were implemented.
A. Object Detection
This term refers to an ability to identify the form and shape of different objects and their position in space caught by the device’s camera.Vuforia Plugin is used to detect marker from mobile camera as mobiles have very limited. So, a Unity3d plug in vuforia is used. It is Used to set a image as marker which can then be detected by mobile camera at the run time.
Vuforia is an augmented reality software development kit (SDK) for mobile devices that enables the creation of augmented reality applications. This image registration capability enables developers to position and orient virtual objects, such as 3D models and other media, in relation to real world objects when they are viewed through the camera of a mobile device.
C. Vuforia License
Vuforia License Manager. The License Manager provides you with the tools and information you need to create and manage your licenses. Whether you are developing or deploying an app, you need a license key. The exception to this is Unity developers that do not need to work with targets.
D. Vuforia Target manager
The Vuforia Target Manager is a web-based tool that enables you to create and manage target databases online. You can also manage the assignment of databases to license keys using the Target Manager.
E. Supported Targets
The Target Manager supports both image based targets and VuMarks. Target images for Image Targets, Multi-Targets, and Cylinder Targets must be either 8 or 24 bit PNG or JPG files. JPG files must be RGB or greyscale. Images with the alpha (transparent) channel are not accepted. The maximum image file size is 2.25 MB. VuMarks must be SVG files created in Adobe Illustrator using the VuMark Designer tool. The Target Manager also supports Object Data (*.od) files produced by the Vuforia Object Scanner. Object Data files are used for Object Recognition.
Capturing the live feed via AR camera and the image capturing module will extract the frames from the feed. The Image Processing module access the frames as resources and process frame by frame dynamically. The marker tracking module gets the coordination points from the user through the display touch. Virtual object loaded from the database is rendered into visual 3d format which can be viewed in the rendered module. Further the 3d model can be viewed in the display screen and can be modified with the users preferred orientation. The user can interact with the 3D model with the help of the virtual button displayed along with the 3D model by selecting the button with hand gestures. Then the functional of the body organ is displayed in a video motion. The interior of the 3D models can be viewed by zooming in by using touch gestures on the display screen.
A real-world live video is feed as an input from the laptop camera to the Camera module. Displaying this live feed from the laptop camera is the reality in augmented reality. This live video stream is given as an input to the Image Capturing Module.
I. Image Capturing Module
The input to Image Capturing Module is the live video feed from the camera of a mobile device. This module analyses the camera feed, by analyzing each frame in the video. This module generates binary images i.e. a digital image that has only two possible values for each pixel.
J. Image Processing Module
Inputs to Image Processing Module are the binary images from Image Capturing Module. These binary images are processed using an image processing technique to detect the AR Marker. Detection of AR Marker is essential to determine the position, where to place the virtual object. Once the AR Marker is detected, its location is provided as an input to the Tracking Module.
K. Rendering Module
There are 2 inputs to Rendering Module. First is the calculate pose from the Tracking Module and other is the Virtual Object to be augmented. The Rendering Module combines the original image and the virtual components using the calculated pose and renders the augmented image on the display screen of the mobile device.
L. Virtual Button
Virtual buttons provide a useful mechanism for making image based targets interactive. The size and placement of Virtual buttons must be considered when designing an experience that uses them. There are several factors that will affect the responsiveness and usability of Virtual buttons.
- The length and width of the button
- The area of the target that it covers
- The placement of the button in relation to the both the border of the image, and other buttons on the target
Blender is only one of many 3D graphics applications. Before we can take a look at Blender and its alternatives, however, let’s talk about 3D computer graphics in general. Video Games most modern games, from iPhone to Xbox, rely completely on 3D characters interacting with 3D environments. Medicine Anatomically correct 3D models help medical practitioners create surgical plans and simulate organs. Scanned data, such as MRI voxel data, allows doctors to take 3-dimensional snapshots of a patient’s brain or organs.
Animation: Creating 3D objects is fun, but it’s even more awesome to see them move. Animation is what makes our creations look alive. 3D animation, we only need to set the important frames (called keyframes), and the software will automatically calculate the in-betweens.
Blender file format: You may want to utilize files that either came from other 2D or 3D software, or you may want to use the things you have made in Blender and edit them in other software. Luckily, Blender offers a wide range of file formats (e.g. OBJ, FBX, 3DS, PLY, STL, etc.) that can be used to import and export.
Unity 3D is a cross-platform game engine with a built-in IDE developed by Unity Technologies. It is generally used to develop video games for computer platforms such as web and desktop, consoles and mobile devices, and is applied by several million developers in the world. Unity is primarily used to create mobile and web games, but there are various games to be developed for PC.
Main editor window: Download Unity from official website and install it. When the user runs it for the first time, he or she will be required to register the product by following the prompts. Registration can be done fast online even if the machine which the user has installed it on is not connected to the Internet. To open with Unity, the user always starts with a project. According to the operating instructions, the Main Editor Window would be opened.
Project window: The Project Window mainly stores all asset documents which contain game scripts, prefabs, models, animation models, fonts, physical materials, and GUI skins. The designer can create or delete all material related to the project in the Project Window the Project view includes all the assets would be used for the current project, as well as all scenes or levels are available for the completed game or application.
This paper proposes a marker based augmented reality application using windows operating system which will help to combine virtual objects with the real environment facilitating various applications as mentioned in this paper.a 3D model of human anatomy using a blender software and create database for vuforia then buy a license for activation key and a target manager to storage a image for marker-based AR.The animation or the user to experience and feel that there are immersed into the space. Interior parts of the model can also be explored in the AR. The creative and engaging process and make the learners more curious about organs functionality.
- Kuo-En Chang, Jia Zhang, Yang-Sheng Huang, Tzu-Chien Liu YaoTing Sung “Applying augmented reality in physical education on motor skills learning” Interactive Learning Environments,2019.
- Xiaodong Wei, DongdongWeng, Yue Liu, YongtianWang“Teaching based on augmented reality for a technical creative design course” Computers Education,2014
- Jennifer Herron “Augmented Reality in Medical Education and Training” Journal of Electronic Resources in Medical Libraries,2016.
- Angelo Di Serio, Maria Blanca Ibanez, Carlos Delgado Kloos“Impact of an augmented reality system on students’ motivation for a visual art course” Computers Education,2012.
- SnehalMangale, Nabil Phansopkar, SafwaanMujawar, Neeraj Singh “Virtual Furniture Using Augmented Reality” IOSR Journal of Computer Engineering,2016.
- Mart´ın-Gutierrez, J., Fabiani, P., Benesova, W., Meneses, M. D., Mora,´ C. E. “Augmented reality to promote collaborative and autonomous learning in higher education” Computers in Human Behavior, 51, 752–761, 2015.
- Mu, X ”Towardseective video annotation: An approach to automatically link notes with video content ”Computers Education, 55(4), 1752–1763,2010.
- Orman, E. K., Price, H. E., Russell, C. R. “Feasibility of using an augmented immersive virtual reality learning environment to enhance music conducting skills” Journal of Music Teacher Education, 27(1), 24–35,2017.