Identify the clarity with which this article states a specific problem to be explored.
Authors Erhel and Jamet identify the problem as the differences within instructions provided impacting the learning and motivation of students. To do this, the authors compared specifically learning instruction to entertainment instruction. Learning instruction is primarily categorized as traditional, direct instruction and entertainment instruction categorized as Digital Game Based Learning, or DGBL. While the problem is identifiable, the introduction is not clear enough in outlining the specific problem in comparing the two in reference to motivation. Providing an explicit explanation of the problem at the end of the introduction would allow for the problem and focus of the research to be easily identifiable and outline the purpose of the rest of the paper.

Comment on the need for this study and its educational significance as it relates to this problem.
The authors highlight the comparison of instruction in relation to entertainment vs. learning environment, laying the groundwork for the study to be a need for education. It does address an issue of how DGBL is administered and if the instructions provided with them can improve or have a benefit in learning. However, I find the arguments made in reference to motivation are not nearly as viable as much of the research alluded to by Erhel and Jamet points to motivation being improved as a result of DGBL. As a result, the evaluation of motivation in this study as a result of the instructions used through DGBL seems redundant and not a value problem worth any true significance of research within this study. The study carries merit in its search for how DGBL are presented and instructed for students and addresses an aspect of DGBL that has primarily not been addressed previously.

Comment on whether the problem is “researchable”? That is, can it be investigated through the collection and analysis of data?
Erhel and Jamet include a study from Schraw and Dennison (1994), a study which altered the instructions in a reading passage when evaluating recall. They found that the specific instructions included often altered the reader’s interest and recall ability when reading for a specific reason or piece of information. Another study from McCrudden, Magliano, and Schraw (2010) identified similar comparisons of relevant instructions and their impact on processing relevant information within a specific text. Both studies found individuals were able to recall and identify relevant information to the instructions provided. Further, both studies effectively collected data throughout the study and show relevant comparisons to that of Erhel and Jamet. While these studies did not research these findings within Digital Game Based Learning environments, their evaluation of various types of instructions is relevant and shown to be researchable.

Critique the author’s conceptual framework.
The author’s conceptual framework for this study is primarily focused on the benefits of motivation for learners during DGBL, benefits of DGBL compared to conventional media, and the use of instruction to improve learning effectiveness. Early on in the article, a quote from Prensky (2001) sets the tone for the study’s framework and focus in regards to DGBL: “one of the medium’s key characteristics is the “coming together” of serious learning and interactive entertainment. In other words, digital learning games can be regarded as an entertainment medium designed to bring about cognitive changes in its players” (Prensky, 2001). This quote sets the focus of the framework as an aim to identify the deep cognitive changes (learning) as a product of DGBL. Erhel and Jamet highlight the positive relationship between intrinsic motivation and performance in digital games (Liu, Horton, Olmanson, & Toprac, 2011). While Erhel and Jamet point to this framework early as the foundation, much of the following literature is inconclusive, with the authors even describing it as “contradictory.” As a result, the authors decide on a value-based approach to determine the benefits of instruction use within digital learning games.

The other primary component of the authors’ conceptual framework refers to instructions within DGBL. The authors describe the methods of instruction as guidance for individuals’ perspective of interaction with the digital learning game they are encountering. The authors provide evidence and research of the various methods including evidence from van den Broek, Lorch, Linderholm, and Gustafson (2001), where students asked to identify details from a magazine through instructions designed towards studying for an assessment, yielded improved recall results compared to those with entertainment. This framework is crucial to the study as it directly links to the purpose of the study: comparing instruction and motivation in DGBL. The authors outline various forms of instruction and provide evidence of what the research indicates about these forms of instruction. It creates a sound framework for the reader to understand the variable being altered within the study and how the authors are considerate of the experiments’ instruction.

How effectively does the author tie the study to relevant theory and prior research? Are all cited references relevant to the problem under investigation?
The literature review, mainly identified as sections 1.1–1.4, describes various elements of the research relevant to the study itself. While it is effective in including research that is relevant to the problem, the literature review is unfocused at times. For example, the authors highlight a variety of research including Hays (2005) and Vogel et. al (2006) in which they found motivational benefits comparing DGBL to conventional learning environments. While the motivational benefits of DGBL are discussed in section 1.1 and are a core part of the conceptual framework of his study, the comparison of the conventional learning environments is not relevant to the issue of instructions in DGBL. With this study focusing on the issue of how instructions are implemented in a DGBL, the comparison with conventional learning environments serves no purpose in addressing the problem highlighted.

Further, the authors mention the comparison of conventional media as compared to educational games and learning experiences by including studies such as Vogel et. al (2006), Hays (2005), Ke (2009), etc. These studies highlighted various differences between conventional media and DGBL, “measuring the learning outcomes of people who play an educational game against the learning outcomes of people who learn through conventional media,” (Erhel & Jamet, 2013). However, the authors highlight these studies are often contradictory and while that is an important distinction, this specific study’s problem is not in media comparison, but in the use of instruction and the comparison of how those are presented to individuals in a digital game-based learning environment. With the problem of this paper being identified in how the instructions are presented, this research is not relevant and does not serve the purpose of the study.

One of the key components of the research later in the paper is the inclusion of feedback in Experiment 2. In the preliminary literature review of this study, feedback within DGBL is not mentioned until Experiment 2. The authors provide relevant research in that section as to the appropriate nature of feedback, knowledge of correct response (KCR) feedback specifically. That said, this concept is missed within the problems and conceptual framework outlined at the beginning of the study. The lack of organization in this section also makes it unclear as to what the focus of the previous research is and how it is relevant to this specific study. Including KCR became a critical component in Experiment 2 but was a limited part of the literature review in this aspect.

Does the literature review conclude with a brief summary of the literature and its implications for the problem investigated?
A brief summary of the literature is not found within the literature review itself. However, a brief summary is found later on in the General Discussion.While it summarizes the use of instructions in learning instruction, referencing back to the literature review and the literature discussed in that section, he also includes the discussion of knowledge of correct response feedback (KCR) methods. The literature review fails to mention any idea of KCR feedback in addition to completely lacking any analysis of feedback research within learning or entertainment instruction. The authors also refer to previous studies and findings through feedback in reference to self-regulation. However, the study is not found in the literature review in addition to any findings discussing the impact and benefits of feedback in DGBL.

This summary primarily focuses on the inclusion of feedback and discussion of KCR feedback specifically and does not directly address the research questions at hand. The research questions focused on the motivational benefit and deep learning outcomes of DGBL through the variable of instructions and this is lost in the authors’ summary. A clearer outline of the summary at the end of the literature review, in addition to the inclusion of feedback as a mechanism for deep learning, would allow the reader to understand the full scope of the research prior and present study.

Evaluate the clarity and appropriateness of the research questions or hypotheses.
The research question for this study is not clearly stated throughout the article. While the intention of the research is somewhat addressed in various parts, an explicit statement of the research question would be beneficial to the reader. However, a statement within the second paragraph of 2. Experiment 1, appears to address the would-be research question of, “…whether the effects of instructions given during the reading phase that have been observed for
text-based learning would also manifest themselves during DGBL” (Erhel & Jamet, 2013).

The authors also do not state the hypotheses of their study explicitly either, though they can be identified through two statements:
1. If the nature of the instructions influences learners’ motivational investment, then an entertainment instruction would improve our participants’ subjective experience (e.g., Vogel et al., 2006), reflected in significantly higher intrinsic motivation scores.
2. In accordance with previous research on the positive effects of motivation on learning in DGBL (Lieberman, 2006), the participants in the entertainment instruction condition would achieve a higher learning outcome.
(Erhel & Jamet, 2013)

Both statements do align with the aims of the first experiment in evaluating motivation and deep learning outcomes in DGBL. However, neither of these hypotheses address the inclusion of feedback in addition to entertainment and learning instructions. A third hypothesis may be added to address this stating: In addition to the positive effects of motivation in game-based learning, the inclusion of feedback in both entertainment and learning instruction would produce increased deep learning outcomes. This hypothesis would address the missing component of feedback that is largely ignored throughout the literature review and Experiment 1.

A crucial error with both the research questions and hypotheses is in their location of the paper. The hypotheses are mentioned within Experiment 1, rather than prior to the experiments in a separate section. This makes the hypotheses challenging to identify in the article and unclear for the reader. These hypotheses also change as a result of Experiment 2 which requires the mention of a third hypothesis, as previously mentioned.

Critique the appropriateness and adequacy of the study’s design in relation to the research questions or hypotheses.
The authors outline directly their choice of using a value-added approach to their study. The authors’ rationale for this approach is in the vein of the “confounding factors including the format and pace of instruction” (Erhel & Jamet, 2013). The authors argue this approach is appropriate in applying the same educational game presented in different formats. In relation to the purpose of this study, this approach is appropriate in that the variable changed across the two groups in both experiment one and two were the instructions provided. The study’s design in this regard is appropriate in addressing the effect of various instruction methods in DGBL.

The study also relies on the use of questionnaires for recall and motivation after the learning experience. Specifically for motivation, interviews may have been a better method of collection in regards to identifying motivational benefits of instruction. Interviews as a method of data collection are often appropriate in studies where the data may be open-ended, individualized, and not standardized (Hoepfl, 1997). Understanding the intrinsic motivation of the task would provide more specific details on the motivation of the individuals. Thus, a mixed-method approach could have allowed for a better understanding and evaluation of the motivation of the participants while completing the DGBL, a key component of the research question.

Critique the adequacy of the study’s sampling methods (e.g., choice of participants) and their implications for generalizability.
The authors of this study use Cluster Random Sampling to identify their participants. The authors recruited participants from several universities in Rennes, allowing for the sampling to be consistent across various pools (schools). This was an appropriate method to allow various pools of students to be accounted for for generalizability. For Experiment 1, 46 participants were included, all from a range of 18-26 years of age, with an average of 2.25 years of schooling experience. According to the author, these participants were then split into two groups of 24 (nine men and fifteen women), for both the learning and entertainment instruction groups respectively. However, this math does not correctly add up to the total number of participants, 46, creating a discrepancy of the number of participants. While this error does not directly impact the implications and generalizability of the sampling, it is worth noting. Further, Experiment 2 uses the same method of Cluster Random Sampling of 44 participants, but does not directly mention how participants were organized into the two groups. While this can be assumed, with variable changes from Experiment 2 (including the inclusion of KCR feedback), the absence of this information calls into question whether the results from Experiment 2 were at all impacted based on the context of the groups (men vs. women, size of the groups, etc.). The implications of this sampling is adequate for students of the college level, as sampled in these experiments. That said, generalizability for other grade levels could be called into question based on the intrinsic motivation that may change for DGBL and younger age groups.

Critique the adequacy of the study’s procedures and materials (e.g., interventions, interview protocols, data collection procedures).
The pre-test at the beginning of the study, while appropriate in nature, lacks depth and items which could lead to a misrepresentation of participants’ prior knowledge. Six total items can allow for random error and an inaccurate representation of prior knowledge. The authors may have wanted to consider emulating the recall questionnaire given after the ASTRA learning session. The procedures within the ASTRA learning environment itself were well-structured. Participants were placed in separate rooms, with an instructor reading and providing the same instructions across the experiment. The recall questionnaire that followed was also an appropriate measure for learning outcomes of the DGBL. By providing various forms of the questions, the authors of the study were able to compare the inferences and paraphrasing of the content and analyze the participants’ understanding. As mentioned previously, the choice of a quantitative questionnaire to gather evidence and measure intrinsic motivation is one that could be improved. Interviews would have allowed for more context and understanding of the driving factors within the ASTRA platform and participant performance. The ASTRA tool is also a point of concern as the authors highlighted the lack of engagement within their limitations. It is a point of concern for this study as when measuring motivation, engagement is a crucial part of measuring motivation, especially intrinsically like this study suggests.

Critique the appropriateness and quality (e.g., reliability, validity) of the measures used.
The appropriateness and quality of the measures used appear valid but do present some concerns. The pre-test for example leaves the reliability of this study open for question. Participants in both Experiment 1 and Experiment 2 were excluded based on their score of 3/6 on the pre-test. A smaller assessment of this size allows for the opportunity for random error to occur and inaccurate representation of results through guessing and perceived ‘luck’ (Drost, 2011). With the limited number of questions on this pre-assessment, the test is subject to random error through guessing and limited questions to properly identify prior knowledge.

The knowledge questionnaire results show similar results in reference to paraphrasing across the two experiments. Despite the inclusion of KCR feedback in Experiment 2, these results were consistent displaying validity across the paraphrase-type questions. However, one major limitation mentioned by the authors was the ASTRA platform and its limited engagement for the participants. Being that motivation was one of the primary factors within this study, the authors acknowledge this may have impacted the motivation data gathered. As a result, the validity of these measures are called into question if there is no difference in motivation based on the instructions provided.

Critique the author’s discussion of the methodological and/or conceptual limitations of the results.
The author’s description of limitations from this study include three main limitations. The first mentioned is the “choice of ASTRA as the experimental material” (Erhel & Jamet, 2013). The authors’ perspective on this limitation is that ASTRA provided limited ‘interactivity’ and may have limited engagement and motivation from the learners. As a limitation to the study, the choice of ASTRA as the DGBL in the experiment presenting limiting engagement in an experiment designed to assess motivation is puzzling. It calls into question the choice of why choose ASTRA? If this limitation was known to have limited interactivity, it is not surprising that the motivation in comparison with instructions would not vary much, as identified in Experiment 1.

The second limitation highlighted from the authors were the quizzes administered accompanied by KCR feedback. The authors describe the limitation as the scores on the quizzes being higher on average, “meaning that the learners seldom received feedback correcting their comprehension errors” (Erhel & Jamet, 2013). The author then also highlights that the results acquired from Experiment 2 show a benefit to the use of KCR feedback despite the feedback not being effective in reducing “superficial processing.” While the comparison to the KCR feedback’s effectiveness is needed, the authors’ explanation of this limitation is not informative for future studies. The authors highlight the need for more ‘taxing quizzes’ but do not describe how to achieve this and do not specify the changes needed to better evaluate the feedback.

The third limitation described by Ehler and Jamet refers to the collection of offline data as opposed to online data which may have allowed the authors access to “log file analysis” (2013). Again, the authors, while acknowledging what they could have changed to address this limitation, are not specific in their explanation. They provide an alternative, but do not directly address how online data may have better supported deep learning processes. An explanation of this with more depth and specificity may allow others in the future to add and expand on this study’s findings, as mentioned by the authors.

How consistent and comprehensive are the author’s conclusions with the reported results?
The results provided in both Experiment 1 and 2 are fairly consistent for the most part. However, the comprehension and representation of findings can be better represented in a few areas. In Experiment 2, the results of the Entertainment group are identified as producing significantly higher results in inferential questions though the deviation is only 1.13. While the increase is noticeable, considering it as a significant increase I found to be a misleading representation of the results. Experiment 1 also fails to provide a table of the findings of the motivational findings, making it difficult for the reader to visually identify the results of the experiment. While the authors acknowledge the results revealed no significant difference, the lack of visual representation leaves the results lacking.

Further, the general conclusion made by Erhel and Jamet can be misleading in correlation to the findings. This is highlighted in their concluding statement, “The usefulness of the present study therefore lies in its demonstration that KCR feedback, coupled with an entertainment instruction, can promote deep cognitive processing during learning, all the while enhancing learners’ motivational investment” (Erhel & Jamet, 2013). This conclusion identifies a motivational benefit to learners in DGBL when providing entertainment instruction in conjunction with KCR feedback. However, Experiment 2’s results point to Learning Instruction consistently showing a non-significant difference or yielding higher results in Performance Goal Avoidance. The conclusion that there is a motivational benefit in entertainment instruction as compared to learning instruction is unfounded in this present study and an inaccurate representation of the findings.

How well did the author relate the results to the study’s theoretical base?
While mentioning the previous literature and their findings when comparing various types of instruction, the authors make little mention of any literature comparative of the results found in Experiment 1. The studies mentioned, Randel et al. (1992) and Vogel et al. (2006), highlight an added benefit of motivation in the use of DGBL. However, both Experiment 1 and 2 do not use any other forms of learning, similar to those studies. This is acknowledged by the authors but carries no relevance when evaluating motivation in relation to types of instruction being used in DGBL. Thus, the only mention of the literature in Experiment 1 is non-applicable to the findings of the corresponding study.

Further, Experiment 2 ties the results found to Sweller et al. (1998), a study which found feedback can be triggered for deep cognitive processes. That said, no other ties to foundational research are made within the discussion of Experiment 2. Another connection to research is made in the General Discussion linking KCR feedback success through inference-type questions to the findings of Leutner (1993) stating, “This result for inferential reasoning was particularly interesting, as it demonstrated that the effect of feedback is not restricted to enabling learners to reproduce the information they have absorbed, as the ASTRA environment had not explicitly provided the answers to the inference-type questions. Our results extend Leutner (1993)’s findings that adding feedback in DGBL enhances memorization.” This connection to the theoretical base is unclear in connecting memorization to inference. While both can exhibit comprehension, a better phrasing of this connection would allow a clearer understanding of the relevance of this study to previous research.

In your view, what is the significance of the study, and what are its primary implications for theory, future research, and practice?
In my view, the study is significant in that it raises questions in the presentation of DGBL and its impact within student motivation and deep learning outcomes. The large skepticism around this study remains identifying the focus and clarity of the study but the framework it outlines as a purpose for research is sound. One of the major themes of this study is in presentation. While the ASTRA platform was identified by the authors as a limitation in engagement, other platforms may benefit specified instruction. Implications for future research are crucial when considering new technological resources that are now available in the decade since this studies’ release. For example, the aforementioned McCrudden, Magliano, and Schraw (2010) study identified the impact on learning from the lens of specified instructions. Though not focused on DGBL, these principles can be linked to the technology based activities and tasks through specified instructions to achieve specific learning goals. This study, while attempting to address a similar concept through learning vs. entertainment instruction, failed to properly address these findings but also opened the door for these conversations and studies to be had. If the platform were more engaging, would the form of instructions change the cognitive processes for the individuals in the learning environment? Is feedback a critical component of DGBL when evaluating deep learning outcomes? Moreno and Valdez (2005) identified feedback as a trigger for deeper learning and this study does attempt to address this aspect of DGBL.

This study lacks long-term implications in its findings but does raise some good questions regarding DGBL’s implementation in its existence and future research of DGBL. How DGBL is implemented and the potential benefits have been a focus of research in the last two decades. For example, a study from Chen et. al (2020) examined the use of competition in DGBL to impact learning outcomes and found a net benefit in learning outcomes in specific subjects such as Math and Science. While this study highlights motivational benefits of DGBL, the research has addressed this thoroughly and this study fails to contribute anything of note to a vast amount of research which identifies motivational benefits and learning outcomes. Tisza et. al (2022) looked at the enjoyment of DGBL in comparison to learning outcomes. While they found that there was no correlation between ‘fun’ and learning outcomes, the results showed a strong impact on students’ perceived learning. These studies alike, Tisza et. al (2022) and Chen et. al (2020), address impacts of DGBL and ideas indirectly and unclearly brought up in this study. I argue that while Erhel and Jamet (2013) do not provide any groundbreaking findings or implications for new research moving forward, its existence is a net positive in providing an avenue and lane which clearly requires more research and study moving forward.

References
Chen, C. H., Shih, C. C., & Law, V. (2020). The effects of competition in digital game-based learning (DGBL): a meta-analysis. Educational Technology Research and Development, 68(4), 1855-1873.

Drost, E. A. (2011). Validity and reliability in social science research. Education Research and perspectives, 38(1), 105-123.

Erhel, S., & Jamet, E. (2013). Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness. Computers & education, 67, 156-167.

Hoepfl, M. C. (1997). Choosing qualitative research: A primer for technology education researchers.

McCrudden, M. T., Magliano, J. P., & Schraw, G. (2010). Exploring how relevance instructions affect personal reading intentions, reading goals and text processing: A mixed methods study. Contemporary Educational Psychology, 35(4), 229-241.

Moreno, R., & Valdez, A. (2005). Cognitive load and learning effects of having students organize pictures and words in multimedia environments: The role of student interactivity and feedback. Educational Technology Research and Development, 53(3), 35-45.

Prensky, M. (2001). Fun, play and games: What makes games engaging. Digital game-based learning, 5(1), 5-31.

Schraw, G., & Dennison, R. S. (1994). The effect of reader purpose on interest and recall. Journal of Reading Behavior, 26(1), 1-18.

Tisza, G., Sharma, K., Papavlasopoulou, S., Markopoulos, P., & Giannakos, M. (2022, June). Understanding fun in learning to code: A multi-modal data approach. In Proceedings of the 21st Annual ACM Interaction Design and Children Conference (pp. 274-287).

van den Broek, P., Lorch, R. F., Linderholm, T., & Gustafson, M. (2001). The effects of readers’ goals on inference generation and memory for texts. Memory & cognition, 29(8), 1081-1087.