I think that in order to protect the instructors from low ratings without any justification, udemy should not accept 1 or 2 stars rating unless the student provides justification about his/her rating.
This will help the instructors know their mistakes and correct them according to the students feedback.
@AhmedMagdyMohamed @FrankKane @Mafer R I think I have the same problem. One of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have several 5+ ratings too, where the users have provided comments describing what they liked about the course. I also have two ratings of 1 star (in 1 of them, the user provided no comments). Yet my ratings are declining. Performance by course attributes is that 97% feel they are learning valuable information, and 85% feel the course is delivering on expectations. It seems the Udemy algorithm has prioritised a few bad reviews over good ones. Disproportionately. In fact, I don't think they are considering the good reviews at all.
You are absolutely right and the algorithm puts higher values on negative reviews than positive ones and each time we say that the community champions will start to accuse us that the positive ratings we have gained are not authentic beware if you received such accusations just neglect them. udemy approves or flags each single review so move on and donot listen for others you will do great if you just focus on what you do.
Udemy (as far as we know) still weights more recent reviews higher, as well as reviews that have written comments in them. How far in the course the student was when the review was written may also be a factor. So if those one-star reviews fit any of those categories, they may be weighted disproportionately.
All any of us can do in this situation is try to drive more reviews to average out the effect of the occasional bad ones. Your courses appear to be quite popular, so the good news is it shouldn't take too long for some more rational reviews to bring things back up.
Hi @FrankKane, this is how it usually is. However, now I don't think so. In my specific case, I got the 1 star review 15 days ago. Since then, I got several 4 and 5 stars. Including those from people who completed more than 90% of the course and gave comments. Yet my ratings are declining/stagnating. Seems that Ahmed has a similar issue
Hm, you're right - that doesn't sound right. The only explanation that would make sense would be if a bunch of 5-star reviews have recently expired from your review average calculation, due to them aging out (3 months I think it is.) But it sounds like your reviews are consistent enough for that to be unlikely.
Hi @FrankKane this has been happening for a few courses since the start of this year. I am at my wit's end, as good ratings don't seem to count anymore. There is no point in contacting Udemy as they give the usual standard responses. Now my courses are going to be thrown off UFB.
@MaclayAcademy23 it is a common issue their algorithm is so brilliant. Only one 1-star rating can ruin your entire course's rating even stupid sellers wont do this to themselves. our courses are the source of income to udemy if they wont correct that, on long run all courses will be low rated and no student will buy a low rated course. Enjoy it Udemy!
I agree with every word you've written and the antidote here is effort. I think in my years here I've once seen an instructor complaint about a bad rating acted upon. For every other complaint there has been the complaint template sent by Udemy staff. I see no real effort o the part of Udemy to act other than the stock standard "true reflection of the market" rhetorical email.
And more often than not admins just ignore these posts in the community. I've seen them escalate into truly horrible and completely unnecessary catastrophes where people have been attacked and clearly very hurt as the community turns on them with no intervention from admins. And as a person with a working history in mental health it's incredibly distressing to watch. In fact this whole drama is unfolding now in another thread in this community and I'm watching it go dangerously close to the way it often seems to go.
There seems to be legitimate effort to apply some spit and polish to the surface of the Udemy community right now. But the whole ongoing debate about the rating system still seems to be a dirty back alley where admins and staff refuse to go.
We'll see if new CEO Greg Coccari decides to shine a light on this shady part of Udemy. But I'm not holding my breath.
here is the other thread https://community.udemy.com/t5/Course-management/The-most-biased-review-system-ever/m-p/118676
Udemy the U.S. Company is doing the most racist behavior in a public thread in Udemy Community for a website that is designed to teach students and help instructors to share their courses and knowledge.
Hi @DamianMcKinnon, thank you for sharing your honest thoughts. We strive to respond to community posts as soon as possible, however, we know that we have instructors from all over the world in the community due to time differences, sometimes we're unable to act on a thread as quickly as we'd like. We're very sorry to hear about your negative experience. Just know that your feedback is very important to us and we'll definitely take this into account so we can work on preventive measures for dealing with these types of issues moving forward.
This is an old topic that refuses to go away. That same stock standard copy and paste email - it's not good enough. I think in all my time in this community I have seen one complaint acted upon. Let's see if the new CEO makes a priority of representing instructor interests. But if I was a betting man I know where I'd lay down my money.
A couple points about reviews, reviewers and reviewees:
1. First, all review systems are based on the subjective biases of those doing the reviews. There is no objective review system. Even when the reviewer is a professional whose job it is to review, they often get it incredibly wrong because of their own biases. For example:
MobyDick, the reviewer wrote: "First, we must ask, does it have to be a whale? While this is a rather delightful, if somewhat esoteric, plot device, we recommend an antagonist with a more popular visage among the younger readers. For instance, could not the Captain be struggling with a depravity towards young, perhaps voluptuous, maidens?"
...of Rudyard Kipling: “...you just don’t know how to use the English language.”
...of D.H. Lawrence and his Lady Chatterley's Lover: “...for your own sake do not publish this book.”
...of Steven King: “We are not interested in science fiction which deals with negative utopias. They do not sell.”
...Louisa May Alcott's Little Women: “Stick To Teaching.”
The Beatles were rejected by numerous record labels and told that there was no market for boy bands with guitars. Too bad for them.
Successful authors, musicians, etc. recognize and accept the subjectivity of ratings/reviews and move on.
2. Locus of control. There is a continuum from internality to externality (see the Rotter Scale) in how we explain or interpret events in our life. The first stage of my career was working in prisons in North Carolina with young inmates. Most of them had one thing in common. They didn't think they were responsible for being there. They all had a story about how the "other guys" got them in trouble. If we could have caught those other guys we would have eliminated crime in North Carolina. They scored very high on the external locus of control scale. They didn't believe in their own responsibility, therefore they didn't process feedback and had great difficulty improving. Working with successful entrepreneurs and executives it was very apparent that they scored high on the internality side of the scale. That is why so many successful entrepreneurs first have failures, acknowledge and learn from those failures. They don't talk about "the other guys", they ask "what should I have done differently?" And, they move on quickly. They take action, doing what they can to determine future outcomes.
Everyone of us have received bad reviews, the issue for instructors is how we react to those reviews. Do you blame the reviewer or the review system (the other guys), or do you accept the inevitable subjectivity and learn from any useful feedback? How this is answered is an obvious distinction between the more and less successful instructors, and the same could be said of authors, musicians, etc.
Very good insight here.
However, the issue stated here isn't really about useful feedback that just happens to be negative. The issue is bad reviews with no justification or actual content to them that states why they gave a bad review. Fake reviews come to mind.
There is no issue with someone not liking my course, but if all they say is " . " or a one word "no" that doesn't do anything for future readers or for me as an instructor to improve.
Hope this helps.
Hi @georgeshahat @RobertBrown028 @AhmedMagdyMohamed @DamianMcKinnon, thank you all for taking the time to share your feedback. We understand how important this is for instructors and will continue to forward your insights to the appropriate teams at Udemy. We want to be transparent and clarify that we currently are not prioritizing updates to the weighted average rating or requiring written reviews, but when we do, your suggestions will certainly be evaluated.
Once again, thank you for your comments. Our team is very grateful for your passion, engagement, and willingness to share your opinions to help improve our product.
Speaking from my own experience, one of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have a few 5+ ratings too. Users have provided comments describing what they liked about the course for many of these positive ratings. I also have two ratings of 1 star (in 1 of them, the user provided no comments). In my specific case, I got that 1-star review more than 15 days ago. Since then, I have gotten a couple of 4 and 5-star reviews, including those from people who completed more than 90% of the course and gave comments.
Yet my ratings are declining. I pursued your Course Reviews FAQ, which clearly stated, "Rather than using a simple mathematical average of all ratings, the course rating calculation gives more weight to reviews from its most engaged students. When assessing a student’s engagement, we consider things like course consumption, the recency of their rating, and the length of their written review".
Since the start of this year, this has not been the experience of many of us instructors, myself included. I don't think the algorithms are considering the 4+ reviews at all. Even @FrankKane opined that the lack of reflection of the relatively recent ratings is rather strange. @AhmedMagdyMohamed has mathematically identified that negative ratings have a much more significant impact than positive ratings. Many instructors have noted and commented on such a phenomenon over the years, and @AhmedMagdyMohamed very kindly looked at the matter quantitatively. So what you state about your rating calculation system seems to be misleading at best. The issue of how Udemy generally handles its reviews and ratings has been raised several times by us instructors. But as @DamianMcKinnon pointed out, Udemy admins continue to ignore our concerns about the rating issue. Now while no one expects Udemy to make its review algorithm public, it is high time Udemy clears the record and tells us if negative ratings are given extra weightage over positive ratings. Setting the record straight on this specific matter will (and acting on individual concerns as needed) will go a long way in reaffirming Udemy's commitments to "positive impact on employees, learners, instructor
Udemy is now a publicly-traded company listed in the USA. I am a researcher in the ESG/SDM domain (MPhil from Oxford University and PhD from Cambridge University). It was most gratifying for me that in 2022, Sustainalytics, a subsidiary of Morningstar(a renowned ESG (environmental, social, and governance) research, ratings, and data organisation), ranked the Udemy first in the Internet Software and Services sub-industry for the second consecutive year (https://about.udemy.com/press-releases/udemy-ranked-first-in-internet-software-and-services-sub-indu...). The press release further noted Udemy's commitment to a positive impact on instructors. The current rating system, the lack of transparency around it and the fact that negative ratings may be disproportionately weighted over positive ratings (which arguably flies in the face of what you state on your own Course reviews page) are doing the exact opposite. There is no such thing as a perfect algorithm. If the actual computations of your rating algorithms weigh negative ratings more than positive ones (thus running contrary to your stated policy), then both instructors and students are being misled. This is not only unethical but could potentially be a case of ESG-washing. At the end of the day, it is not about one course or one instructor. The fundamental issue of instructor welfare hearing us out and ensuring that ESG commitments are met transparently and meaningfully. From now on, I sincerely this will be borne in mind and instructor concerns acted upon.