Measuring Costs and Benefits of AI to Support TVET Learning

As part of the ASEAN–UK SAGE programme, SEAMEO VOCTECH has partnered with EdTech Hub to leverage the Hub’s sandbox approach and methodology to strengthen SEA-VET Learning, the online learning component of the SEA-VET.net platform.

In Sprint 1 of our sandbox, in collaboration with SEAMEO VOCTECH, we wanted to learn:

To answer this question, we tested the utility and desirability of an embedded AI chatbot that can support self-paced and asynchronous learning. Having established how learners might realistically use AI to support their learning, we turned our attention to testing the feasibility and viability of integrating an AI chatbot on the SEA-VET Learning platform. In this sprint, we examined:

• Technical and operational requirements
• Cost implications
• Inclusivity and multilingual considerations
• Data protection and maintenance needs

The goal was to familiarise VOCTECH with available AI‑chatbot options and help VOCTECH assess their readiness to integrate an AI chatbot into the platform.

Sprint 2: Assessing AI Integration Options Within VOCTECH’s Capacity

We conducted a workshop with VOCTECH that brought together the SEA-VET Learning Team, representatives from the IT Department, and senior leadership. During the session, we introduced a decision‑making framework that helped VOCTECH evaluate each use case in terms of learner value, desired functionality, and the costs and capacity required for AI integration. The workshop also created space for VOCTECH to discuss which use cases would have a high impact in supporting learners while remaining feasible within VOCTECH’s resources and capacity.

AI Options: Software as a Service (SaaS) vs. API-Powered LLMs

Before the workshop, our team identified two types of AI chatbots that could be appropriate for the SEA VET Learning context. Rather than build a bespoke chatbot from scratch, we explored existing AI chatbot vendors. We found that chatbots originally designed for e-commerce use cases offered relevant features, making them a practical starting point. From our early discussions in Sprint 0 with VOCTECH, we also knew that costs, technical, and operational considerations were key concerns for the team, so we prioritised these factors when sharing this information. 

To support VOCTECH in applying the decision-making framework, we applied these chatbot options to each use case. Seeing the concrete differences in functionality and integration requirements helped the team envision what these AI chatbots could provide and the trade-offs between them.

Choosing High-Impact Use Cases

In Sprint 1, we identified three use cases for using AI to support learning: 

• Lowering access barriers through multilingual and navigational support
• Supporting career guidance and next-step decision-making
• Strengthening learning through prerequisite and just-in-time academic support

Out of these three options, VOCTECH identified multilingual support and career guidance as use cases they were most interested in pursuing. VOCTECH echoed the findings from the user survey responses, noting that communication and learning in English can be challenging for some learners, so translation support and additional contextualisation in local languages would be especially valuable. They also shared that learners will often ask about courses or pathways that come next after completing training, highlighting career guidance as another feature that can support students on the platform. 

Cost Considerations

Our research found there are several factors that drive costs for AI chatbot vendors, so it is important that VOCTECH identify high impact use cases first to provide costing estimates. For instance, for a chatbot for multilingual support, VOCTECH needed to consider whether they wanted code-switching, the ability to use both English and a local language within the same response, and how many languages they wanted to support. Other factors to consider that drive the cost include: 

• Expected usage volume – the number of users and volume of queries. This can inform the most cost-effective pricing model for the platform.
• Feature complexity – simple FAQs are cheaper, while personalised career guidance is expensive
• Multilingual support – as mentioned before, basic translation is low cost, while more contextualised translation with code-switching capabilities increases costs
• Level of integration – standalone chatbots require less resources than LMS/data-integrated systems

Off-the-shelf SaaS chatbots appear to be more cost-effective for embedment into the SEA-VET Learning platform, with its expected number of users, but a detailed consultation with the vendor that customises the features and complexity of the chatbot is necessary to find the right chatbot solution for the identified use cases.

Reflections and Lessons Learnt

Several important lessons emerged during Sprint 2 as we continued exploring how AI could meaningfully support self-paced learning.

1. Aligning technology with operations and strategy

Throughout the sandbox, we worked closely with the SEA-VET Learning operations team. In this workshop for Sprint 2, we intentionally included members of the senior leadership team and the IT department. Bringing these additional perspectives to the workshop provided valuable insights and generated new questions. For instance, the representatives from IT shared additional context about the number of users on the SEA-VET platform. This is crucial information for assessing costs and evaluating which chatbot option is more appropriate for the platform. Having senior leadership present also ensured that these conversations connected to larger organisational priorities and long- term strategy. 

2. Listening to partner needs 

In Sprint 0, we heard from VOCTECH that key concerns they had about adding features to the platform were costs and operational feasibility. This was reinforced in Sprint 2: one participant noted, “In terms of the feasibility and viability, for me, those are some of the more concerning ones. When it comes to desirability, we can all agree about how valuable it is.” Hearing these concerns come up across the sandbox helped us focus the workshop in Sprint 2 and provide VOCTECH with information that would make this idea work in practice. 

3. Designing with intent

One of the central lessons learned from the sandbox and workshop was the importance of approaching AI integration with intent. While there is a lot of hype around the potential of AI to transform learning, its value depends on how well it aligns with learners’ needs, organisational priorities, and intentional implementation. This began in Sprint 1, when we went directly to users to assess if they would value an AI chatbot in the first place. Sprint 2 continued this practice by encouraging VOCTECH to consider concrete questions about feasibility and identify which use cases would be most impactful to support learners on the SEA VET Learning platform. 

4. Customise AI complexity to fit the purpose

Our research on the use cases for AI chatbots for SEA VET, as well as the types of AI chatbots that can be embedded in the platform, shows that there are many decision points in the whole process for decision makers. For instance, the amount of personalisation of the responses from the chatbot can have many different levels – from contextualising each response with the users’ profiles up to their performance in each of the assessments they have taken – that decision makers need to align with their goals for the chatbot. Similarly, the complexity of the chatbot’s responses for career guidance can range from simple, limited FAQ level responses to live, labour market informed responses.

Asking ‘should we?’, not just ‘can we?’

Along with the work we did to test desirability, feasibility and viability of an AI chatbot, we also encouraged VOCTECH to consider the ethical implications of integrating AI into the SEA VET Learning platform. Although ethics does not traditionally appear in the Venn diagram of design thinking, it adds an essential question to the process: ‘Just because we can, should we?’ Asking this question is important no matter the innovation, but especially relevant when AI is involved. Given the ongoing debates about responsible AI use for education, it is critical to pause and assess whether adding AI is the right choice at all, not simply whether it’s possible. 

What’s Next

These sandbox sprints provided foundational work to assess if an AI chatbot was an appropriate choice to encourage and motivate learners for TVET learning on the SEA VET learning platform. These steps were essential to test our critical beliefs and ensure the development of a future prototype or pilot was grounded in evidence rather than based on assumptions about what learners wanted or what VOCTECH was able to implement. 

In our workshop, VOCTECH surfaced possible opportunities for testing an AI chatbot with specific capabilities at a future workshop or exploring the capabilities of a SaaS chatbot. Although Sprint 2 concludes this sandbox, VOCTECH now has clearer options for what the next phase could look like and direction for a possible Sprint 3. This comes at an important moment, as VOCTECH considers broader decisions about the platform’s future direction and what role, if any, AI has in that long-term vision.