Hey there instructors,
On January 25th, we had the pleasure of having an AMA-style webinar with our Senior Director, Product Management Jacob Coulter and we wanted to share a recap of what we discussed.
As background on who Jacob Coulter is and what he does here at Udemy.
Jacob joined Udemy in June of 2022 to bring features to the Udemy platform to help you succeed as an instructor! At the moment his area of expertise is the following:
Tooling for instructor content management
Instructor insights (data, reporting, course performance dashboard)
The learner <> Instructor interactions
Here’s what we covered during the Webinar!
For the first portion of the AMA, we had Jacob answer some of the questions that you posted in the webinar invite.
Instructors are going off the platform to deliver coaching, bundling, live meet-ups & certifications. Does Udemy plan to build those into the platform?
-Lawrence Miller, Anup Shukla, Peter Dalmaris
The short answer is that these are not currently on the roadmap. As we inspect the needs of our learners, organization, and instructors, the largest gaps we found are practice or applied to learn. So we are working to improve upon practice tools and practice insights for existing content types. We do plan on taking a look at credentials in Udemy this year with the goal of how can we help learners demonstrate skills acquired by learners. This is a larger initiative but it is on the roadmap.
What tools are we planning to get to help engage our students more? (e.g. interactive videos, drag&drop questions?)
-Marious
Right now we are working on hands-on exercises for a learner to practice a skill whether that’s a quiz, an assessment, a lab, or a coding exercise. This has proven very successful in helping conversion and driving engagement. We received feedback from practice tests about more flexible question types (drag and drop etc). With that feedback, we will be taking a look at those features in Q2 2023.
How are you planning to provide instructors with more actionable insights?
-Jonathan Turner, William Stewart
When we think about insights, we want to provide something actionable. When we provide you with something, it helps you monetize efforts/time, better cater to learners, or take advantage of a new opportunity. We currently have a new beta insight for UB (Udemy Business) instructors and received great feedback. So with our initiatives for insights are aiming to have a higher impact on the instructors.
Follow-up question - Community Champion Lawrence Miller has been a huge advocate to provide year-over-year insights from teaching, will that be coming?
This is on our roadmap to be explored in Q1. We understand that this is valuable for your business and that the current setup is not explaining seasonality. We don’t have details on how this will look, but it is being explored as we speak.
What are the biggest things planned for this year and beyond?
-Samuel Lenk
Practice and Insights - Practice and insights that will maintain a focus for us. Another project we are focused on is what we call internal Supply Gaps and we know you want guidance on where to focus your content efforts. With Supply Gaps we will evaluate how you can best do that. We currently have the market insights tool but we want to expand further to see what the demand for insights from learners and customers is. We don’t have the full answer to this yet but it is on our roadmap.
Skill Attainment - To expand further, we currently have areas of the importance we are looking into. We are looking into skill attainment. How can a learner get various skills on our platform and present that to a manager showcasing that skill? We are also looking into skills-based guidance which means how can we assess the skills needed to learn, what is the goal of a learner, and with that, we can give that information to the instructor community to continue meeting the needs of these learners.
Active learning - We are looking at coding exercises and practice tests but we are looking at other aspects such as labs or practice tests. We are very interested in expanding this further.
Learning habits and motivation -This is a method to guide learners through their content and continue forward. Not a complete roadmap but it is on our radar.
After the pre-selected questions, we opened it up to the audience for live Q&A. Here were some of the questions that were asked:
Metrics for newer initiatives such as labs and personal plans, quizzes, and coding exercises?
Happy to say we have a project on the roadmap in H1 called practice insights. We did lengthy research working with instructors to say what the friction points preventing you from creating content/practice content. A huge feedback piece was the lack of insights. In Q2 this year, we will be working on practice insights. We are figuring out whether it’s best to provide a holistic view or focus on certain content types over others. We are leaning on the instructors to provide those answers but it is coming!
Are there any improvements coming to practice tests?
Absolutely! The practice test is another project we are committed to expanding. There was a variety of feedback we received about the flexibility of Question and answer types. We know that there need to be more granular options to fill the need and we are working on developing that. We also are working on expanding the management of existing practice tests. It’s a project on the roadmap for us this year and we look forward to bringing improvements for you.
What is the next feature you are most excited about that will be moving into beta soon?
There was a reference to ai within the questions. We are looking into AI and the role it can play in supporting platform experiences. We are early on with this but we are starting to look at generative AI playing a part in content creation. It’s never a foolproof plan, but will this support you in your efforts? We are not sure yet, but we are looking at the opportunity to make the platform more scalable.
Will students ever be able to re-do quizzes or homework and will instructors be able to see answers within quizzes?
It’s a common request we receive. In general, across practice types, we are looking at this and exploring it within the insights tool. We are researching the best way to present the elements of a student's submission in quiz/assignment to the instructors.
Is there a plan to add better filtering, richer content, etc to the instructor announcement tool?
Not immediately planned in the roadmap for the first half of this year but we are looking at this subject closely. We are fixing the stability first but we like the idea of scheduling an announcement and it is higher on the list. It’s penciled in to evaluate in the second half of the year.
What is the makeup of your team?
I work with a couple of talented teams in Udemy. We put together product managers, designers, engineers, and user researchers. We have defined workflows and work very closely together to work off those flows. If you have participated in our beta projects, chances are you interact with a variety of those folks. We put research and insights at the forefront of our process and collaboration with our colleagues is critical to success.
Are there any improvements coming to reviews and ratings?
The biggest one is better assessing the changes in those data points between the courses marketplace and UB. The insights feature will allow for that but no immediate plan beyond that.
Are there any plans to have instructors have a live Q&A session with students?
We like this and something we may want to pursue, but we need to answer how we can make this scaleable for the entire instructor community. We haven’t been able to fully answer this yet. What we’re hearing is that there are more gaps with active learning initiatives. We currently have priority for that subject, but it is on our radar.
Will we have the option to do A/B testing on a course landing page?
We haven’t discussed this yet. We currently have a rigorous testing process for the landing page ourselves and provide the most profitable layout to our instructors. Another challenge is that if we provide that to our instructors, not all instructors might get a sample size to make a qualified decision for an A/B test.
Are practice tests just for coding courses?
No. They mimic an industry certification, but we have seen applications for broader use cases. It certainly has non-web developing use cases.
What are Supply Gaps?
A term we use to refer to unmet demand on the platform. So what we are looking at here is the Market Insight Tool to look at what supply exists on the platform today and how it performs, how it’s rated. But it doesn’t yet include insights from the UB customers. Supply gaps are the general theme of what learners are searching for in content that we might not yet have.
We also opened the floor for instructors to provide feedback to Jacob after the AMA session.
To receive the feedback we asked: If Udemy could fix ONE pain point on the platform for you next week, what would it be?
We would love more feedback from the community so if you have any more suggestions, feel free to share them in the comments below!
... View more
•
Community Manager
