Browsing by Author "González, Luis A."

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Exploring the Impact of Generative AI for StandUp Report Recommendations in Software Capstone Project Development
    (2024) Neyem, Hugo Andrés; Sandoval Alcocer, Juan Pablo; Mendoza Rocha, Marcelo Gabriel; Centellas-Claro, Leonardo; González, Luis A.; Paredes Robles, Carlos Daniel
    StandUp Reports play an important role in capstone software engineering courses, facilitating progress tracking, obstacle identification, and team collaboration. However, despite their significance, students often grapple with the challenge of creating StandUp Reports that are clear, concise, and actionable. This paper investigates the impact of the use of generative AI in producing StandUp report recommendations, aiming to assist students in enhancing the quality and effectiveness of their reports. In a semester-long capstone course, 179 students participated in 16 real-world software development projects. They submitted weekly StandUp Reports with the assistance of an AI-powered Slack, which analyzed their initial reports and provided suggestions for enhancing them using both GPT-3.5 and the early access GPT-4 API. After each submitted report, students voluntarily answered a survey about usability and suggestion preference. Furthermore, we conducted a linguistic analysis of the recommendations made by the algorithms to gauge reading ease and comprehension complexity. Our findings indicate that the AI-based recommendation system helped students improve the overall quality of their StandUp Reports throughout the semester. Students expressed a high level of satisfaction with the tool and exhibited a strong willingness to continue using it in the future. The survey reveals that students perceived a slight improvement when using GPT-4 compared to GPT-3.5. Finally, a computational linguistic analysis performed on the recommendations demonstrates that both algorithms significantly improve the alignment between the generated texts and the students' educational level, thereby improving the quality of the original texts.
  • Thumbnail Image
    Item
    Towards an AI Knowledge Assistant for Context-aware Learning Experiences in Software Capstone Project Development
    (2024) Neyem, Hugo Andres; González, Luis A.; Mendoza Rocha, Marcelo Gabriel; Sandoval Alcocer, Juan Pablo; Centellas, Leonardo; Paredes, Carlos
    Software assistants have significantly impacted software development for both practitioners and students, particularly in capstone projects. The effectiveness of these tools varies based on their knowledge sources; assistants with localized, domain-specific knowledge may have limitations, while tools like ChatGPT, using broad datasets, might offer recommendations that do not always match the specific objectives of a capstone course. Addressing a gap in current educational technology, this paper introduces an AI Knowledge Assistant specifically designed to overcome the limitations of existing tools by enhancing the quality and relevance of Large Language Models (LLMs). It achieves this through the innovative integration of contextual knowledge from a local “lessons learned” database tailored to the capstone course. We conducted a study with 150 students using the assistant during their capstone course. Integrated into the Kanban project tracking system, the assistant offered recommendations using different strategies: direct searches in the lessons learned database, direct queries to a GPT model, query enrichment with lessons learned before submission to GPT and LLaMa models, and query enhancement with Stack Overflow data before GPT processing. Survey results underscored a strong preference among students for direct LLM queries and those enriched with local repository insights, highlighting the assistant's practical value. Further, our linguistic analysis conclusively demonstrated that texts generated by the LLM closely mirrored the linguistic standards and topical relevance of university course requirements. This alignment not only fosters a deeper understanding of course content but also significantly enhances the material's applicability to real-world scenarios.