Nevertheless, we should also be aware that there are a series of obstacles for its inclusion to teaching activities: it is a novel technology, the lack of implementation of educational experiences/experiments, the lack of resources and learning objects produced in AR, the lack of teachers’ training, the need for the students to have positive attitudes for its addition into educational practice, the lack of educational experiences in the development of AR learning objects, the size of the viewing screen, the lack of conceptual frameworks to rely on for searching for innovative educational practices for the application of AR, the need to create support centers for teachers to facilitate the production of AR learning objects and the maintenance of the servers, the need for the students to have basic technology for its viewing, and the overload of information [ 4 31 ].

With regard to the significant way of joining the educational field, the study in this regard indicates few aspects, although the research conducted is limited but increasing [ 31 ], with this possibly being one of the great problems it has for its addition to teaching. For example, it has been shown that the students have a favorable attitude towards it, and its use increases the motivation towards learning [ 32 34 ], favors the creation of a constructivist context of training [ 31 ], promotes an active learning environment [ 7 35 ], which reduces the cognitive load the students are subjected to in training tasks [ 7 36 ], the students show a high degree of acceptance and positive attitudes towards the technology [ 37 40 ] and it awakens a high degree of satisfaction in the students and the teachers [ 39 43 ]. Also, it improves the spatial ability and orientation of the students [ 44 45 ], and its use improves academic performance [ 46 ]. It should be noted that experiments are being conducted for its inclusion into e-learning activities [ 31 ], as well as experiments in which the students become producers of learning objects [ 47 49 ].

Its use in training or teaching [ 16 29 ] has allowed different possibilities, such as: (a) erase that content which could make it difficult to obtain relevant information for the student to use; (b) generate content that makes the information more understandable to the student; (c) give the student the possibility to observe the object from different points of view and different angles; (d) facilitate the generation of ubiquitous learning for the student; (e) the student is immersed in an “artificial” environment, such as simulators or laboratories; (f) an enriched printed material through different resources; (g) improve the way in which students learn through motivation; (h) learning objects that generate “prosumers” students instead of “consumers” of information. The potential offered by this technology is increased with its incorporation through different disciplines (education, medicine, architecture…) as well as the different educational levels in which they can be used (children, primary, secondary, university, university)) [ 5 30 ]. In any case, it should be noted that the educational level in which there are more experiences and studies is often university.

As for the differences with Virtual Reality (VR) and Mixed Reality (MR), one of them is that AR is closer to the real world environment, while virtual reality is placed in one of the most distant points of the context, while the “augmented Virtual Reality”, or “Mixed Reality”, is placed halfway between them, since it incorporates elements of AR and VR. On the other hand, if we do not move away from the real-world context when utilizing AR, on the contrary, in VR, the subject is found in an immersive technological context that is separate from the physical reality [ 3 15 ]. With the former, the mixing of physical reality with virtual reality is sought after, while for the latter, it offers the users the experience of an alternative world of immersion, simulated with a computer, where different types of sensorial experiences occur, where the subject can interact with the environment as if he or she was in VR [ 5 ].

There is no doubt that it has great relevance for the educational field at all levels, which is established by all those characteristics that make it significant and different from other previous resources. The possibilities of interaction with the environment that it offers are highly important, especially if it is taken into account that it offers a great ease of use, and all this adds to the user (in this case, the student) additional information that he hadn´t had to his provision previously. In addition, it allows to integrate different layers of information and different types of formats (URL, videos or texts) [ 5 ]. Also, the devices utilized for its viewing, such as Smartphones, are easily available to the university students, and this technology has been highly accepted by them [ 14 ].

How the combination is established and the integration of realities is due to several factors. Among them, physical and digital realities are mixed through different resources, such as images, different three-dimensional objects, the incorporation of QR codes, so-called thermal signatures, or GPS coordinates [ 5 ]. On the other hand, the AR systems can be classified according to its location or according to the images used. “The AR systems based on location use the data on the position of the mobile devices, determined by the Global Positioning System (GPS) or WiFi-based Positioning systems. By contrast, image-based AR focuses on image recognition techniques utilized to determine the position of physical objects in the real environment for the appropriate localization of the virtual content related to these objects” [ 13 ].

AR offers the possibility of mixing and combining two environments: the physical and the digital, and all this in real time, through the use of emerging and easily accessible technologies, such as smartphones or tablets. All this makes possible the generation of another reality, a new reality [ 11 ] “Augmented reality introduces the knowledge that the student must learn into his or her real-world environment, in a visible way” [ 12 ].

Augmented Reality (AR) is one of the emerging technologies that have rapidly been incorporated into the education sphere due to the diverse possibilities it offers [ 1 8 ]. Among other reasons, this is due to the ease of accessing information offered by this tool, as it is normally accessed thanks to mobile devices, with these technological resources being highly present with students in the Ibero-American context [ 9 10 ].

The document presented investigates both an original and novel field that contributes, exponentially, to the scope of the sustainability of this magazine, especially considering that Augmented Reality in the educational field has a perfect place in the present monograph “Online and Ubiquitous Training, Mobile Technology in Education and Sustainability”.

For the analysis of academic performance, a multiple choice test was created, comprised of 15 items, which agglomerated information from the categories application (4 items), remember (6 items), and understand (5 items) from Bloom’s taxonomy. The items of the pre-test and the post-test were identical, although the order of presentation was changed.

The resulting values allow us to indicate that these instruments possess a high level of reliability, in their entirety as well as in their different dimensions [ 75 76 ].

For the TAM instrument, as well as the IMMS, an index of reliability was obtained with Cronbach’s Alpha, as this test is the most appropriate for Likert-type instruments [ 75 ]. Table 4 shows the values for each instrument as well as the different dimensions that comprised them.

Two multiple choice tests were created to measure students’ academic performance through the use of different learning objects with AR, a pre-test and a post-test, consisting of 15 questions that were configured around the three Bloom’s first categories of taxonomy, which are: apply, remember, and understand, and where the items of the two tests were the same, simply modifying their order of presentation.

As shown by Loorbach, Peters, Karreman, and Steehouder [ 67 ], the model formulated by Keller: “has been used countless of times to apply motivation strategies for teaching materials and to test their effects. Although the model was originally designed to influence the student’s motivation in a classic learning environment, with the personal interaction between the professor and the students, it has also been applied and thoroughly tested, such as for example in computer-aided teaching and virtual and distance education”. More specifically, the model has been used to understand the degree of motivation that different technology formats created in the students: videogames [ 68 ], the addition of video to e-learning training sessions [ 69 ], audio podcast [ 70 ], the MOOC [ 71 ], or AR [ 72 74 ].

As for the IMMS by Keller, it should be underlined that it is composed by 35 items which collect the information from four dimensions: attention (12 items), confidence (9 items), relevance (9 items), and satisfaction (5 items). The instrument has a Likert-type construction with seven degrees of response (1 = Extremely unlikely/in disagreement to 7 = Extremely likely/in agreement).

The instrument that was used was configured by 15 questions (four for the perceived utility category, three for perceived ease of use, three for perceived enjoyment, three for attitude towards use, and two for intention to use), Likert-type where there were seven valid response options ranging from “Extremely Unlikely/Disagree” (1) to “Extremely Probable/Agree” (7).

The application of the TAM model to AR will essentially indicate if the student, when utilizing it in teaching-learning situations, will consider its use to be relatively easy and if it will contribute benefits, and if an increase in the intent to use will be produced, which will signify that the student will utilize it in the future.

For example, there are different contexts of use: mandatory and voluntary. There is an extensive review of whether or not the concept of attitude itself should be incorporated into the model as it creates confusion with the theory of planned behavior. In educational research, the data obtained through tests such as self-reports, generate in the researcher some limitations, especially in the relationship established between the intention to use and its actual use, the latter being able to be conditioned by variables of another type. Without forgetting that the limitations also include the difficulty that exists to obtain measures that are objective to establish the degree of acceptance of a technology and its conceptual simplicity with the model.

In any case, it should not be ignored that like other educational models, it also has certain restrictions [ 35 66 ] that should be assumed: the context in which the technology is incorporated and used will determine, to a large extent, the results obtained in the study, which is evident taking into account the distinctive characteristics of the subjects participating in it

There is a large number of research studies carried out on this subject in the educational field related to RA, as well as a high number of meta-analyzes of the different educational models which allow to verify that they offer validity and solidity to how the intention of developing use of different technological environments in subjects. Recently, this has also been confirmed when analyzed with the structural equation models [ 55 65 ].

The TAM instrument seeks to obtain information from four dimensions: perceived usefulness (PU), perceived ease-of-use (PEU), perceived enjoyment (PE), attitude towards use (AU), and intent to use (IU). The proposal by Davis suggests that the pre-disposition an individual has towards the use of any technology is determined by the attitude towards it, which is conditioned by the “perceived ease-of-use”, the “perceived usefulness” and “perceived enjoyment”. Its starting point is that the beliefs and standards of each person are responsible for a specific behavior.

Three instruments were used for the research study: the “Technology Acceptance Model” (TAM), formulated by Davis [ 53 ], and utilized for analyzing the degree of acceptance of the AR technology by the students; the “Instructional Material Motivational Survey” (IMMS) created by Keller [ 54 ] for the analysis of the motivation created in the students by their participation in the experiment; as well as the realization during the experiment of a multiple choice test, whose purpose was to measure and analyze the student’s academic performance.

It should be mentioned that for each of the AR objects, a guide was created, where the different objects were included within the notes, the students were shown where to download the app, and the complementary bibliography for delving into the subject was included. Figure 3 shows an example of the guide for the subject “Design, production, and evaluation of the ICT used for education”.

The programs worked in Android as well as iOS devices. Table 3 shows the URLs where videos showing their functioning can be viewed, and where the corresponding apps can be downloaded.

The three objects were of different types; the ones from “Education Technology” content (Roles of the educational use of the video) and “Anatomy” could be classified as type II [ 5 ], in an “AR-enriched notes” format, with the first one linking to a video resource, and the second to a 3D object that could be manipulated by the student.

The experiment was conducted with three AR objects that were especially created for the study, and they referred to the content from the Education technology discipline: “ways of using video in the teaching-learning processes”, from Anatomy (Medicine) and the field of Art. For each of these objects, a guide of the program was created, which offered information to the students about how they could access the downloads of the apps to be able to use the learning objects, what possibilities they offered them and where were the bibliographic references that allowed them to expand the subject studied.

As mentioned, the sample was composed by students from the Pedagogy, Education of Children, and Primary Education Degrees from the University of Seville. The samples were non-probabilistic, convenient, or causal [ 51 52 ], and the criteria for choosing it were the ease of access that the students who were part of the study carried out by the researchers presented. Table 1 shows the number of participants in each of the experiments.

the scores obtained in the pre-test and post-test do not present significant differences—with an alpha risk of rejecting the null hypothesis of 0.05—after the participation of the students in the experiment with AR objects.

the scores obtained in the pre-test and post-test do not present significant differences—with an alpha risk of rejecting the null hypothesis of 0.05—after the participation of the students in the experiment with AR objects.

The intention of using the learning objects in AR by students can be modified in a positive and significant way by the attitude towards their use.

The intention of using the learning objects in AR by students can be modified in a positive and significant way by the attitude towards their use.

The way in which students verify the perceived ease of use in AR is positively and significantly related to the perceived utility of the use of learning objects in AR.

The way in which students verify the perceived ease of use in AR is positively and significantly related to the perceived utility of the use of learning objects in AR.

The way in which students verify the perceived ease of use in AR is positively and significantly related to the perception of enjoyment through the use of learning objects in AR.

The way in which students verify the perceived ease of use in AR is positively and significantly related to the perception of enjoyment through the use of learning objects in AR.

In the study carried out, different AR objects were produced, which were used in the research, and which responded to three different contents from the courses “Education Technology” and “Information and Communication Technologies (ICT) applied to Education”: manners of using video in teaching, design, and production of ICT for teaching, and educational uses of the Web 2.0 tools. The courses were part of the Pedagogy, Education of Children, and Primary School Education Degrees taught in the Faculty of Education at the University of Seville.

Two pre-experimental designs were used in the investigation, the two single-case designs, which analyzed the degree of acceptance of AR technology students had with the learning objects produced, and point out that the experience was developed through different studies. This analysis was conducted with a post-test measurement; and a second one utilized to understand if the degree of acceptance of the AR technology had repercussions on the academic performance of the students. This was conducted for the analysis of the academic performance, with pre-test and post-test measurements [ 50 ]. In both designs, a treatment was administered (AR object produced) and a measurement was performed (degree of acceptance of the technology by the students after participating in the experiment, and the academic performance).

To know if the participation in the experience was similar, in the scores from the degree of acceptance of the technology, the motivation, and the performance, with the different objects produced.

Analyze if the degree of acceptance and the dimensions that comprise it, through the TAM model by Davis, and the motivation and dimensions in agreement with the instrument by Keller, had repercussions in the acquisition of knowledge by the students.

To understand the motivation it created in students through the use of the Instructional Materials Motivation Survey (IMMS) by Keller, and the dimensions it comprises (attention, confidence, relevance, and satisfaction), that the use of AR objects promoted in the students who were enrolled in Education courses. (See Table A2

To understand the degree of acceptance of the AR technology measured through the Technology Acceptance Model (TAM) by Davis, and the dimensions that comprise it (perceived usefulness, perceived ease-of-use, attitude towards use, and intention of use) that the use of AR objects fostered in the students enrolled in Education courses.(See Table A1

As the null hypothesis (H0) was rejected, the size of the effect was calculated in order to understand the strength of the differences found between the pre-test and the post-test, and this was obtained with Hedges’ g [ 79 80 ], with the following results obtained:

The results reached allow us to reject H0 with a level of significance of p ≤ 0.01; and as a result, it can be concluded that the student’s participation in the experience allowed them to achieve the goals of the previously-mentioned courses. This was found for all three content blocks.

In other words, it was found that through experience with AR objects, students obtained a high degree of motivation and acceptance of this technology, both increased by their incorporation and use by students in the designed training action. This is indicated by the very high mean values in the total for the instruments, as well as their different dimensions, and this has resulted in a positive relationship with the performance acquired with the three objects produced in AR.

The values obtained show for both types of instruments, according to the studies carried out by Etxeberria & Tejedor [ 78 ], different elements to be taken into account, such as the direct relationship between the performance and motivation variables and the degree of acceptance of the technology, and that this relationship moves between “moderate” and “high” values, which is significant for our study, especially considering that these relationships are significant at≤ 0.01.

Next, the hypotheses that referred to the question of: if the motivation and the degree of acceptance, and the dimensions that comprised the instrument, had a relationship with the performance achieved. Pearson’s correlation was used again, with the results obtained for the IMMS shown on Table 8

As for the academic performance achieved, Table 6 shows the mean scores and the standard deviations of the pre-test and post-test for the three objects, as well as for the overall experiment.

As observed for the overall results of both instruments, and in the different dimensions that comprise them, the scores obtained exceed the value of 4.5. At the same time, it should be pointed out that the general evaluation of both instruments has been very broad, with a score of 5.77 for TAM and 5.49 for IMMS, which out of 7 possible points suggests a high degree of acceptance and motivation of the experiments conducted with the AR objects that were produced for the research study.

The results will be presented first by showing the means and standard deviations obtained in the dimensions of the TAM and IMMS instruments ( Table 5 ), and will finalize with the results on academic performance.

4. Discussion and Conclusions

The experiment was developed with three different types of AR objects as for their design and with the incorporation of different resources, which were especially created for the study. These objects referred to content from the disciplines “Education Technology”, in the Pedagogy Degree and the “ICT applied to education” from the Education of Children and Primary School Education Degrees from the University of Seville, allowing us to obtain conclusions about different aspects.

83,84,85, Firstly, it should be noted that the reliability results of the diagnostic instruments used showed that both Keller’s IMMS and Davis TAM were good predictors for diagnosing variables relevant to our study, such as motivation, confidence, attention, relevance, and satisfaction. All this linked to the interaction of students and the object of AR in the specific case of IMMS. Note that the findings of the present research coincided with the works conducted by different authors [ 82 86 ].

64,87, Regarding the use of the TAM instrument, this research has shown that many of the values obtained coincide with the results of other authors who have researched the same subject and at the same educational levels [ 40 88 ]. Along this line, it should be mentioned that these results were independent of the AR object with which the students who participated in the study interacted with, as well as the course they were enrolled in; this supports the robustness of the results.

Ultimately, it should be pointed out that these were two instruments, and two models of diagnosis of the degree of acceptance and motivation that a technology created. This can also be observed in its recent use in other research studies that analyzed other technologies, i.e., in the case of the TAM, through the different works conducted for videogames [ 89 ], for Moodle [ 90 ], or for virtual training [ 91 ]; and for the IMMS, the study with videogames [ 89 92 ]. Therefore, our work points to the usefulness of both models.

Another of our conclusions indicates that the use of AR objects has resulted in a high degree of acceptance and an acceptable degree of motivation, as indicated by the high mean values reached, as well as the high scores shown in “the intention of use it in the future” and their “satisfaction”. Therefore, it should be concluded that negative attitudes towards its use were not found.

On the other hand, the study does not indicate that these objects are easy to use by the students, as the mean values found in the dimension “perceived ease-of-use” of the TAM clearly exceeded the middle value of 3.5 (4.77, 6.23, and 6.46). These scores increased as the experience progressed, even with the objects increasing in difficulty due to the number and types of objects incorporated.

19,42, As for the degree of acceptance of the technology, as previously mentioned, starting with the TAM model formulated by Davis, the results obtained allowed us to conclude that the students, independently of the courses they were enrolled in, had a high degree of acceptance of AR. This indicates a strong attitude towards its use in training, pointing to a strong intention of use in the future. These results were confirmed with the analysis conducted to contrast the hypothesis with the scores obtained jointly with the two objects produced, just as when it was done for each of them, thus re-enforcing the findings even more. To a certain degree, it should be pointed out that the study confirmed the results found by other authors [ 18 72 ], which suggested that the students have high degrees of satisfaction when they interact with objects produced in AR, also mentioning that they would like to learn with AR in the future. This is reinforced by the perceived ease of use, which means that the participating students enjoy the use of this resource and the usefulness of this tool increases in the same way.

The study conducted shows that the participation in AR experiences increases the motivation of the students, measured through the IMMS and the different dimensions that comprise it: confidence, attention, satisfaction, and relevance. A significant relationship was found between the degree of motivation and the increase in performance. The greater the motivation (confidence, attention, satisfaction, and relevance), the greater the performance or the retention of the information learned. These results have also been achieved in other research works that utilized AR for the training of university students conducted in our context [ 47 ].

As for performance, it should be indicated that the students significantly improved their scores in the knowledge tests that were administered at the start of the experience. The results reached indicate that the use of notes enriched with AR objects can facilitate the student’s learning and the acquisition of knowledge. This result is concordant different authors, in that the construction of books and notes with AR can be greatly useful for training [ 85 93 ]. This opens strong possibilities for observation in the new scenarios of learning, when the students receive complementary information through this technology, which will facilitate their understanding of the content, and will make easier the increase in performance. Thus, this work contributes with references for justifying the incorporation of notes enriched with AR in university training, where, aside from increasing or awakening the motivation and satisfaction of students, it can facilitate the acquisition of content presented with it, with the students not showing difficulties in their management.

These possibilities suggest that they can be useful objects that can be incorporated into emerging technologies such as the “flipped classroom”, where the students work at home with these learning objects, and the in-person sessions are dedicated to resolving doubts and problems and conducting hands-on tasks or for the implementation of “collaborative learning” activities [ 94 95 ].

Therefore, it is necessary to consider that we must forget the novelty effect of technology and think more about its implementation and proper use. It is interesting that its integration in the classroom is within an educational project that always puts the pedagogical before the technological. It is advisable to include with the material some indications to take into account for its use, as well as a brief explanation of the characteristics of the object, without forgetting the importance of training the teacher in both a technical-instrumental and methodological and pedagogical dimension.

Nevertheless, it should be underlined that its use is dependent, on the one hand, on the existence of good internet connections in the education centers, and on the other, that in its design the following be included: principles of accessibility, ease of use, and flexibility for its adaptation to different contexts.