💬 *MEGATHREAD*: Say Hello to Role Play! 💬

@Boris035 thank you for the example - is a good starter. Don’t worry i will never copy :)

@Boris035 My assumption about goals is that they will measure how well the student met the goals of the role play, not how much they may have learned. But, I could be wrong, happened before.

@RobertBrown028 Thanks for warning me. You are probably right. However, I'm already an outsider, so...

…I'm perfectly positioned to experiment and do what I deeply believe in ;-)

looks really intresting!

I think @Boris035 is really on to something with his complexity scale. I hope Udemy will read his post.

If we add the availability of data to his gradations then potentially we are walking into to land mines.

[1] If we have lots of data, LLMs are pretty good.
[2] If have limited data, then we are at risk of LLMs hallucinating.
[3] If we want the Udemy role play to prioritise our course material, then potentially we are in the not enough data category (your course v humanities output)
[4] If Udemy role play is going to use "the world library" then our course materials could be considered on the edge and therefore out voted on the "probability of the next word" method.

Last time I asked ChatGPT for comment, this time Claude 4.0, see below:

[Extract]
For effective role-playing that stays true to your course material, you'd need the AI to heavily weight or prioritize your specific content over its general training. This typically requires either:

• Fine-tuning specifically on your course material
• Using retrieval-augmented generation (RAG) to pull from your course content first
• Prompt engineering that strongly emphasizes staying within your course boundaries
• Having enough of your course material in the training mix to make it statistically significant

[ End Extract]

Conclusion
In the words of Dirty Harry, "Do you feel lucky punk?" Are you prepared to let an AI answer on your behalf?

• Consider the complexity levels
• Consider the where you sit in the data pool
• Monitor role plays (Will we get the transcripts? Is our content preferred or not?)
• Take action to fine tune.

In my mind the student comes first. Will this tool help them? Yes it can, based on the above thread, but buyer be aware, you could catch a bullet.

@MarinaT, I've been getting quite a bit of "Connection Interrupted" errors with the new Role Play feature. I've created 30 role plays, and I'd say I get this error message around 20% of the time when testing them.

I also have a student account with the Personal Plan, and it happened to my student account on the third role play I was going through. When this happens, you can't end the role play because the "End role play" button is disabled, and the only way to end it is to refresh your browser. I reported the issue with my student account via the feedback link, but am reporting it here too.

Yes I have had the same problem testing one of my role plays as well.

Hi @AHardin and @Agilemastery,

Thank you so much for reporting this, and I’m sorry you’re running into that issue. So far, we haven’t received any other reports about it. Has it happened again, or was it a one-time occurrence? If you don’t mind reporting it to our Support team, that would really help their investigation. It’s truly appreciated!

Just found out some pretty negative habits of AI characters in my first Role Plays.

In Scenarios where Student is the Newbie in a company and AI Character is the mentor, the AI Character´s statements and recommendation sometimes and partly contradict my course content (or select Option B while I am recommending Option A in the course). So feeding the AI character with the course content is essentiell.

Even worse, I have seen multiple times statements such as "If you wanna know more and dive deeper into that topic, check videos or online courses on Youtube, Coursera or LinkedIn Learning". How on earth is this possible?

A lot of the challenges with role plays seem to have a common theme - trying to get the AI to play the role of a teacher or mentor. I think that is backwards from the envisioned usage.

Role plays are not intended as a teaching tool; they are a reinforcement tool to practice what you have already learned.

For example, your course might teach how to have a difficult conversation with an employee. The AI side of the role play is great at simulating an employee and how they might react to what you say. But, it has no knowledge of your course content. The AI can play the receiving end, not the end that is driving the conversation and demonstrating the goals you are trying to reinforce.

@MarinaT, thanks. It actually happens quite often. I assumed Udemy was aware of it, so I didn't report it from my instructor account. I used the feedback link from my student account to report it, but I'll also report it from my instructor account too.

@WilliamStewart there is another thread where we address the questions you raise - this capability to more specifically reference the course material is in the roadmap https://community.udemy.com/en/discussion/comment/168583#Comment_168583

So far, it looks like there’s no RP tag visible in search results, only on the course landing page. Will there be any visibility boosts or prioritization for courses with roleplays in search results, or does the RP feature not affect search performance at all?

In case I missed the information, please accept my apologies:

Can we as instructors see:

1. How often role plays have been used?
2. How often they have been completed or abandoned?
3. If the learners liked the role play?

If have set up 10+ role plays across multiple courses and need feedback to improve them or continue doing what I'm doing for my other courses.

Thank you :)

Have Role Plays been launched already? I still get the message "Your role play has been queued and will be available once we launch this feature".

Hi @AlexHagmann,

Yes, the Role Play feature went live on May 27. I’m sorry to hear you’re experiencing issues getting it launched. Have you tried clearing your browser’s cookies and cache, then clicking “Publish” again?

If it’s still not working, I recommend reaching out to our Instructor Support team. They’ll be able to take a closer look and help resolve the issue.

Hi @EvrimKanbur,

Thank you so much for your feedback about displaying the Role Play tag in search results, not just on the course landing page. I’ll be sure to share this with our internal team! Currently, there’s a “Role Play” filter available under the Hands-On Practice section within All Filters, which helps learners find courses that include role plays.

As for how the Role Play feature impacts search results and course visibility, I’ll check in with our internal team and follow up with you as soon as I have more information.

Hi @AxelRittershaus,

Those are great points, and I completely understand how having insights into learner engagement with Role Plays (like usage, completion rates, and feedback) would be incredibly valuable. At this time, those analytics aren’t available, but I’ll share your feedback with our internal team and we will keep you updated on any developments!

Thanks again for raising this!

Hi @MarinaT and @AHardin  and @Agilemastery ,

I also have been getting repeated messages saying “Connection was disrupted.” I was testing at peak internet usage time and simply assumed that the issue was my end due to my internet connection being poor.

I published the role play. Now I am wondering whether I should unpublish until this issue is addressed. I will send an email to Support.

Another issue I have experienced is that when I was testing, I simply wrote ‘test’ in the subject line that shows up in the curriculum. When I went to update the subject heading, I was not given the option to do so.

Hi @EvrimKanbur,

I'm following up with an update from the team regarding your question about whether role play is considered in the search ranking: While we don't apply a direct boost to courses that include role play, the increased engagement these activities generate can positively impact a course’s visibility in search results.

If you have any other questions, we're happy to help!

@Paul_Imre @FrankKane if i then understand well

A) Role play cannot be trained on our content (for the limitation explained)

B) Therefore is not really a teaching tool

If the above is true - how would you use effectively outside the “soft skill scenario”?
Any comment is welcome!

Thanks

@OxfordLearnLab With sufficient detail in the description of the AI persona, it may be possible to coerce it beyond soft skills practice.

But if that's your intended usage, I think I'd just wait until the next iteration on this feature. If they include AI access to your course materials or material you explicitly provide to it, it would work a lot better in a "teacher" role as opposed to someone you're practicing soft skills with.

@OxfordLearnLab - you can definitely use role play for more than soft skills scenarios. I'm using it in my computer networking and cyber security courses and it works well. However, to get it to work as intended, you have to be very descriptive and detailed in all aspects of it (scenario, meeting goals, personas), where you test, tweak, and fine tune it.

However, because it's not trained on our course content, it's not perfect and won't always stick to just your course material - even if you define scope boundaries in your prompts. But, if your craft your role play well, it seems to do a good job overall (most of the time).

Are there plans to add API's for roleplays in future . Or an interface where a instructor can see how many courses have role plays and how many doesn't.

We have also created 8 role plays in 8 different courses we are offering. We are looking forward to create more role-plays in courses that we have not yet done. This is an excellent way to give confidence to students based on situations that are relevant in the courses. And we have created situations for emergency situation as well compared with a normal situation. The AI actors are behaving the way they are prompted. But we are now more focused on acquiring learner's feedback over the same. How do they feel about that? Obviously; we need enrollments.

I love Role Play - Amazing tool - I think I have implemented atleast 3 role plays in each of my 20 courses. They are getting really good feedback from my students :)

Loved it! Right away, I was able to create two role-play scenarios for one of my courses and tested with success. It was easy to create and launch it.
Looking forward to creating more for my other courses😊

Hi,

I've added role plays to all the lessons in my business English course. It's a fantastic way to enable learners of English to practise the language in a realistic setting. The AI voices are quite realistic and the role plays themselves are very easy to create. I thought I'd finished adding content to the course some time ago but the AI role plays introduce a whole new dimension.

Dear Udemy Team,

We at LearnersCare LLC are excited about the Role Play feature and how it has enhanced our courses. With nearly 1,000 Role Play sessions across 25 courses, we’ve seen how AI-driven co-instruction boosts engagement, introduces content effectively, and creates structured interview experiences at the end of each section.

This tool has been highly effective in helping learners develop real-world, interactive skills. Its seamless integration and ease of use make it a valuable addition for both instructors and students.

We look forward to contributing to the evolution of Role Play and sharing insights from our experience. Thank you for this fantastic innovation!

Best regards, Team LearnersCare LLC