Low Ratings

cancel
Showing results for 
Search instead for 
Did you mean: 

Low Ratings

I think that in order to protect the instructors from low ratings without any justification, udemy should not accept 1 or 2 stars rating unless the student provides justification about his/her rating.

This will help the instructors know their mistakes and correct them according to the students feedback.

 

61 Replies

Hi @georgeshahat Thank you for sharing this with us. I'll make sure to pass this feedback on to our team. We understand ratings are very important for instructors.

Fernanda Rivera

Community Moderator

The policy team will send the copy paste template and say these are our policies and no changes at the moment

That's what exactly happened 

This is the same answer Bella often gives - we'll pass it on. This topic comes up again and again and often around this time of year. And it gets "passed on". The only result is we will end up chasing our tails over this one all over again in about 6 months when nothing happens and the topic is forgotten..

@DamianMcKinnon 

 

That is because the system is full of bugs and we shall thank god that is still working!

 

Each start of a month, the conversion rates are reset to zeros (all the time) it shall be reset to zeros only for the new month! and they always say "it is just a visual issue" and that is it. 

 

An issue happened in all search 1 day ago and the search was getting almost zero results and it stayed like this for above 30min I guess! 

 

Another issue happened 2 days ago and all instructors received a warning that they donot have permissions to look at student statistics, course engagement or conversion rates and they fixed it after alot of questioning about that glitch! 

 

Some students accidently put a 2 star rating by mistake and I heard that many times here. 

 

All these issues are still on going and it is happening at regular times. 

 

 

Hi. I have the same issue. One of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have several 5+ ratings too, where the users have provided comments describing what they liked about the course. I also have two ratings of 1 star (in 1 of them, the user provided no comments). Yet my ratings are declining, and my course will be removed from UFB in a few weeks.  Performance by course attributes is that 97% feel they are learning valuable information, and 85% feel the course is delivering on your expectations. It seems your algorithm has prioritised a few bad reviews over good ones. The whole point of putting in so much effort and keeping the content fresh if the good feedback is simply not acknowledged by what is essentially computer code?

 

 

Wow  I taught I was the only one noticing this 

Sometimes there's nothing you can do

If you don't have the power to do anything then yes there's nothing you can do. But there are people drawing a wage right now that supposedly have a responsibility to act. And more often than not they copy and paste a stock standard response. And it' not good enough.

That is how you show to public how the system is civilized and deep down there is no credibility 

As I understand consumer review systems, their primary purpose is to provide perspective customers an honest evaluation of a product: Hopefully for the business, the content provider and the customer, information that promotes a purchase.  A dishonest system, one that has a response bias, will soon cause damage to all.

 

A secondary benefit of reviews is feedback for the business and content providers to improve their products. Negative comments are especially helpful for this purpose.  However, review systems are not designed to protect or support defective products. Somehow, the review system must weed out poor products, improve others, and enable customers to purchase the right product. Not an easy task.

 

There are philosophies behind any review system. One is to maximize responses, usually by allowing quick and easy ratings with no comments. This encourages responses of all kinds. Another approach is to mandate written comments. This usually winds up with extremes, high praise and livid negatives, with fewer in-the-middle comments.  The bias of this system is toward the positive.

 

Amazon.com maximizes ratings, supporting customer input but also allowing them to cheat the system with bogus reviews, both positive and negative. A few of my books, for example, received obvious troll one-star reviews with no recourse to remove them.

 

I don’t believe the argument that ratings without comments are a disservice to instructors will work. The ratings are for prospective students. And they work as intended.

 

My suggestion is for the insides of the rating system to be shared with instructors so that a more informed discussion could be held. Perhaps the details must be confidential so it cannot be gamed; yet instructor concerns deserve a more open and informed discussion. Maybe together we can create a stunningly effective system.

 

 

Robert Brown, Ph.D.
Unique insights for personal & business success

Sorry but No! The review system which is provided by Udemy is not working as intended and many reviews are manipulated and their policies are vulnerable to these kinds of fake reviews and they accept that as well which is unethical and not credible. 

 

I have many reviews which are manipulated by haters and may be other competitors too and I provided evidence to them but the policy team is doing nothing but searching for bot activity! Any fake review can be done with a created account and few dollars to pay in a sale day without any need for a bot to do that! Their policies are doing nothing they cannot defend our hard work from dishonest people and fake accounts. 

 

Another thing which I discovered recently a negative paid review is highly weighted than a paid positive review! It is like we love negative reviews more and we donot respect the 5 stars!!!! 

 

Thanks to the review system!!! 

FrankKane
Community Champion Community Champion
Community Champion


@AhmedMagdyMohamed wrote:

 

Another thing which I discovered recently a negative paid review is highly weighted than a paid positive review!


How do you know that?

@FrankKane 

If you got a 1 star review from paid student (not from a free coupon) , your overall course rating would drop almost 0.5 or more and any badge would fade out (in case you had one).  on other hands,  your course got a 5 stars from paid student, your overall course rating would rank up almost 0.02 or 0.03 maximum and that is true because it has happened several times and I am sure of that. 

FrankKane
Community Champion Community Champion
Community Champion

Oh, by "paid review" you mean by a paid student... I misunderstood!

 

That makes sense mathematically though, right? A one star review is much further from your existing average than a 5 star would be. I'm not saying that makes it right; if a review is an outlier perhaps the algorithm should treat it as such.

@FrankKane  but what I heard from support team they say all ratings are weighted with fair but whatever the algorithm was .. getting a recent 1 star from paid student doesnot mean my course should drop almost 0.5! because that way, any hater can just buy your course twice from different accounts he made and ruins your entire course's rating! that is the whole story I am talking about and instead of listening to me I am being attacked!!! 

Finally I found A Community Champion who wants to listen! Thanks @FrankKane !

FrankKane
Community Champion Community Champion
Community Champion

Reviews are a very touchy subject. As instructors we put months of effort into a course, and seeing someone criticize it with a few strokes of their keyboard, perhaps unfairly, can be very very discouraging.

 

Just to explain why it works that way - if you have 20 5-star reviews and get another 5-star review, it won't change your average at all. A 4-star would only change it to 4.95. ((5*20+4)/21) But a 1-star would change it to 4.8. ((5*20+1)/21) 

 

What makes it worse is that recency is also a factor, so a brand new one star review counts more than a 3-month-old 5 star review. We don't know the details of how that works, but it explains how a new 1-star review can drag your score down so dramatically.

 

Everything cuts both ways, though. A new 5-star review also carries more weight than an older 1-star. It's just that if the 5-star is closer to your existing average, it won't move it as much. It's all quite fair on paper, but yes it does open up the possibility of people really torpedo-ing a course with a one-star written review.

 

Udemy has said below they have no plans of changing this any time soon, so we have to learn to deal with it. If you're in this situation, all you can do is see if you're lucky enough for the review to violate some Udemy policy in which case you might get it removed - and if not, try to drive more reviews to dilute the effect of it. Or simply wait it out - after a few months, it won't affect your score.

 

Most of your courses have exceptionally high review scores. We tend to fixate on the negative feedback at the expense of the positive!

@FrankKane  thanks for clarification now I got my answer after a long journey of misery here. However that makes sense why anyone hates me can ruin my ratings easily and for your recent question how I keep my ratings high like some of my courses now because I report each single 1 star or even 3 star and some of them are removed or expired (after 3 months) but they discovered a new way now to ruin my work without any effect from my reports so I started to complain that is the whole story.. in short.. YES I have enemies in the marketplace and I donot know them but all of them have something in common in their ratings .. in their written feedback and in their way in skipping lessons and finishing the course in a short time. 

Thanks for listening!

@FrankKane  that is not a miracle people! I am just keeping good track of my ratings as much as I can by reporting negative reviews, sharing my free coupons (only for real students in private groups) rather than a spammy public post! and it actually goes well but who said all my courses are high! I have 3.9, 4.2, and of that 4.8s were actually around 4.4 but the negative reviews are expired recently.. when negative reviews are expired my ratings get high.. Why you feel it that way guys! 

@AhmedMagdyMohamed @FrankKane @Mafer R I think I have the same problem. One of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have several 5+ ratings too, where the users have provided comments describing what they liked about the course. I also have two ratings of 1 star (in 1 of them, the user provided no comments). Yet my ratings are declining. Performance by course attributes is that 97% feel they are learning valuable information, and 85% feel the course is delivering on expectations. It seems the Udemy algorithm has prioritised a few bad reviews over good ones. Disproportionately. In fact, I don't think they are considering the good reviews at all. 

@MinervaSingh66 

You are absolutely right and the algorithm puts higher values on negative reviews than positive ones and each time we say that the community champions will start to accuse us that the positive ratings we have gained are not authentic beware if you received such accusations just neglect them. udemy approves or flags each single review so move on and donot listen for others you will do great if you just focus on what you do. 

FrankKane
Community Champion Community Champion
Community Champion

Udemy (as far as we know) still weights more recent reviews higher, as well as reviews that have written comments in them. How far in the course the student was when the review was written may also be a factor. So if those one-star reviews fit any of those categories, they may be weighted disproportionately.

 

All any of us can do in this situation is try to drive more reviews to average out the effect of the occasional bad ones. Your courses appear to be quite popular, so the good news is it shouldn't take too long for some more rational reviews to bring things back up.

Hi @FrankKane, this is how it usually is. However, now I don't think so. In my specific case, I got the 1 star review 15 days ago. Since then, I got several 4 and 5 stars. Including those from people who completed more than 90% of the course and gave comments. Yet my ratings are declining/stagnating. Seems that Ahmed has a similar issue 

FrankKane
Community Champion Community Champion
Community Champion

Hm, you're right - that doesn't sound right. The only explanation that would make sense would be if a bunch of 5-star reviews have recently expired from your review average calculation, due to them aging out (3 months I think it is.) But it sounds like your reviews are consistent enough for that to be unlikely.

Hi @FrankKane this has been happening for a few courses since the start of this year. I am at my wit's end, as good ratings don't seem to count anymore. There is no point in contacting Udemy as they give the usual standard responses. Now my courses are going to be thrown off UFB. 

Wow I taught am the only one who noticed bad reviews has more power than the good ones 

@MaclayAcademy23  it is a common issue their algorithm is so brilliant. Only one 1-star rating can ruin your entire course's rating even stupid sellers wont do this to themselves. our courses are the source of income to udemy if they wont correct that, on long run all courses will be low rated and no student will buy a low rated course. Enjoy it Udemy! 

I agree with every word you've written and the antidote here is effort. I think in my years here I've once seen an instructor complaint about a bad rating acted upon. For every other complaint there has been the complaint template sent by Udemy staff. I see no real effort o the part of Udemy to act other than the stock standard "true reflection of the market" rhetorical email.

And more often than not admins just ignore these posts in the community. I've seen them escalate into truly horrible and completely unnecessary catastrophes where people have been attacked and clearly very hurt as the community turns on them with no intervention from admins. And as a person with a working history in mental health it's incredibly distressing to watch. In fact this whole drama is unfolding now in another thread in this community and I'm watching it go dangerously close to the way it often seems to go.

There seems to be legitimate effort to apply some spit and polish to the surface of the Udemy community right now. But the whole ongoing debate about the rating system still seems to be a dirty back alley where admins and staff refuse to go.

We'll see if new CEO Greg Coccari decides to shine a light on this shady part of Udemy. But I'm not holding my breath.

@DamianMcKinnon  

and @georgeshahat 

here is the other thread https://community.udemy.com/t5/Course-management/The-most-biased-review-system-ever/m-p/118676 

 

Udemy the U.S. Company is doing the most racist behavior in a public thread in Udemy Community for a website that is designed to teach students and help instructors to share their courses and knowledge.

@AhmedMagdyMohamed  thank you 

Bella
Community Manager
Community Manager

Hi @DamianMcKinnon, thank you for sharing your honest thoughts. We strive to respond to community posts as soon as possible, however, we know that we have instructors from all over the world in the community due to time differences, sometimes we're unable to act on a thread as quickly as we'd like. We're very sorry to hear about your negative experience. Just know that your feedback is very important to us and we'll definitely take this into account so we can work on preventive measures for dealing with these types of issues moving forward. 

 

Bella Almeida

Udemy Community 

This is an old topic that refuses to go away. That same stock standard copy and paste email - it's not good enough. I think in all my time in this community I have seen one complaint acted upon. Let's see if the new CEO makes a priority of representing instructor interests. But if I was a betting man I know where I'd lay down my money.

A couple points about reviews, reviewers and reviewees:

 

1. First, all review systems are based on the subjective biases of those doing the reviews. There is no objective review system. Even when the reviewer is a professional whose job it is to review, they often get it incredibly wrong because of their own biases. For example:

MobyDick, the reviewer wrote: "First, we must ask, does it have to be a whale? While this is a rather delightful, if somewhat esoteric, plot device, we recommend an antagonist with a more popular visage among the younger readers. For instance, could not the Captain be struggling with a depravity towards young, perhaps voluptuous, maidens?"

 

...of Rudyard Kipling: “...you just don’t know how to use the English language.”

...of D.H. Lawrence and his Lady Chatterley's Lover: “...for your own sake do not publish this book.”

...of Steven King: “We are not interested in science fiction which deals with negative utopias. They do not sell.”

...Louisa May Alcott's Little Women: “Stick To Teaching.”

 

The Beatles were rejected by numerous record labels and told that there was no market for boy bands with guitars. Too bad for them. 

 

Successful authors, musicians, etc. recognize and accept the subjectivity of ratings/reviews and move on. 

 

2. Locus of control. There is a continuum from internality to externality (see the Rotter Scale) in how we explain or interpret events in our life. The first stage of my career was working in prisons in North Carolina with young inmates. Most of them had one thing in common. They didn't think they were responsible for being there. They all had a story about how the "other guys" got them in trouble. If we could have caught those other guys we would have eliminated crime in North Carolina. They scored very high on the external locus of control scale. They didn't believe in their own responsibility, therefore they didn't process feedback and had great difficulty improving. Working with successful entrepreneurs and executives it was very apparent that they scored high on the internality side of the scale. That is why so many successful entrepreneurs first have failures, acknowledge and learn from those failures. They don't talk about "the other guys", they ask "what should I have done differently?" And, they move on quickly. They take action, doing what they can to determine future outcomes.

 

Everyone of us have received bad reviews, the issue for instructors is how we react to those reviews. Do you blame the reviewer or the review system (the other guys), or do you accept the inevitable subjectivity and learn from any useful feedback? How this is answered is an obvious distinction between the more and less successful instructors, and the same could be said of authors, musicians, etc. 

 

Lawrence M. Miller
Author/Instructor

This reminds me of stories by Grand Father. No Offence! 

I am sure your grandfather was a very wise man. You should listen to him.

 

Lawrence M. Miller
Author/Instructor

@LawrenceMMiller  Thanks Man, Enjoy your day, week, month, year and the rest of your career !!!

Hello Lawrence 

Very good insight here. 

However, the issue stated here isn't really about useful feedback that just happens to be negative. The issue is bad reviews with no justification or actual content to them that states why they gave a bad review.  Fake reviews come to mind. 
There is no issue with someone not liking my course, but if all they say is " . " or a one word "no" that doesn't do anything for future readers or for me as an instructor to improve. 

Hope this helps. 

Danny 

Totally agree with you. They even don't reply when I ask them about the reason behind the low rating. 

Bella
Community Manager
Community Manager

Hi @georgeshahat @RobertBrown028 @AhmedMagdyMohamed @DamianMcKinnon, thank you all for taking the time to share your feedback. We understand how important this is for instructors and will continue to forward your insights to the appropriate teams at Udemy. We want to be transparent and clarify that we currently are not prioritizing updates to the weighted average rating or requiring written reviews, but when we do, your suggestions will certainly be evaluated.


At this time, I recommend reading our Course Reviews FAQ for more details about how the reviews system work, and reach out to policy@udemy.com if you suspect that there are any inconsistencies. 


Once again, thank you for your comments. Our team is very grateful for your passion, engagement, and willingness to share your opinions to help improve our product.

 

Bella Almeida

Udemy Community 

Hi @Bella  , I am also struggling with Udemy's rating algorithm, and I agree with  @AhmedMagdyMohamed  that negative ratings seem to be disproportionately weighted over positive ratings and reviews. 

 

Speaking from my own experience, one of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have a few 5+ ratings too. Users have provided comments describing what they liked about the course for many of these positive ratings. I also have two ratings of 1 star (in 1 of them, the user provided no comments). In my specific case, I got that 1-star review more than 15 days ago. Since then, I have gotten a couple of 4 and 5-star reviews, including those from people who completed more than 90% of the course and gave comments.

 

Yet my ratings are declining. I pursued your Course Reviews FAQ, which clearly stated, "Rather than using a simple mathematical average of all ratings, the course rating calculation gives more weight to reviews from its most engaged students. When assessing a student’s engagement, we consider things like course consumption, the recency of their rating, and the length of their written review".
 
Since the start of this year, this has not been the experience of many of us instructors, myself included.  I don't think the algorithms are considering the 4+ reviews at all. Even @FrankKane   opined that the lack of reflection of the relatively recent ratings is rather strange. @AhmedMagdyMohamed   has mathematically identified that negative ratings have a much more significant impact than positive ratings. Many instructors have noted and commented on such a phenomenon over the years, and @AhmedMagdyMohamed  very kindly looked at the matter quantitatively. So what you state about your rating calculation system seems to be misleading at best. The issue of how Udemy generally handles its reviews and ratings has been raised several times by us instructors. But as @DamianMcKinnon pointed out, Udemy admins continue to ignore our concerns about the rating issue. Now while no one expects Udemy to make its review algorithm public, it is high time Udemy clears the record and tells us if negative ratings are given extra weightage over positive ratings. Setting the record straight on this specific matter will (and acting on individual concerns as needed) will go a long way in reaffirming Udemy's commitments to "positive impact on employees, learners, instructor

 

Udemy is now a publicly-traded company listed in the USA. I am a researcher in the ESG/SDM domain (MPhil from Oxford University and PhD from Cambridge University).  It was most gratifying for me that in 2022, Sustainalytics, a subsidiary of Morningstar(a renowned ESG (environmental, social, and governance) research, ratings, and data organisation), ranked the Udemy first in the Internet Software and Services sub-industry for the second consecutive year (https://about.udemy.com/press-releases/udemy-ranked-first-in-internet-software-and-services-sub-indu...). The press release further noted Udemy's commitment to a positive impact on instructors. The current rating system, the lack of transparency around it and the fact that negative ratings may be disproportionately weighted over positive ratings (which arguably flies in the face of what you state on your own Course reviews page) are doing the exact opposite. There is no such thing as a perfect algorithm. If the actual computations of your rating algorithms weigh negative ratings more than positive ones (thus running contrary to your stated policy), then both instructors and students are being misled. This is not only unethical but could potentially be a case of ESG-washing. At the end of the day, it is not about one course or one instructor. The fundamental issue of instructor welfare hearing us out and ensuring that ESG commitments are met transparently and meaningfully. From now on, I sincerely this will be borne in mind and instructor concerns acted upon.

@MinervaSingh66 

I doubt that the udemy themselves know anything about the rating algorithm. I opened that case several times with the support team and their replies contradicted each other. Some of them say that reviews with free coupons are the same as reviews without coupons and we all know that is not right. The reviews for free coupons are lower in weight and the support member said the wrong answer. Maybe it is a single fault mistake, but I raised my concern several times because that rating system encourages sick-heads and poor minds and jealous people to just ruin your work with a 1 star review! But no one is listening. You will keep struggling and you will get no answer from the platform.

We have to try. Remember, at the end of the day, Udemy is now a publicly listed company. We are right in seeking transparency and consideration for instructor welfare. That is the whole point of ESG ratings. 

I completely agree with this....not to prevent me from getting lower ratings, but to prevent all the accidental 1-2 star ratings. I've reached out to SO MANY STUDENTS about low ratings, and every single one of them responded that the rating was an accident (or they don't know how it happened), and they updated it.

Very true and very logical dear. This has been brought to notice of Udemy earlier too but for some reason it does not fit into their policies 

GuilhermeMP
Community Champion Community Champion
Community Champion

As stated before,every review has a degree of subjectivity, and this happens for all instructors

FrankKane
Community Champion Community Champion
Community Champion

Wow @AhmedMagdyMohamed way to go with throwing out accusations of racism, and then following it up with ageist remarks. That sort of behavior isn't acceptable here, or anywhere else. Perhaps you could offer some more constructive feedback, such as how you earned ratings of 4.9 - 5.0 on some of your courses. We must have a lot to learn from you about how to create some of the best content on the platform, and how to motivate students to write positive reviews. Please share your tips with us!

 

Look, EVERY review system is fatally flawed. They are ALL subject to gaming and the biases of the reviewers. But like it or not, students - which are Udemy's customers, and our customers - expect them as a tool in making a buying decision. If I were in charge, I'd eliminate reviews entirely in favor of some objective engagement-based metric from the behavior of paid students. But I'm not, and reviews are here to stay.

 

Bad reviews hurt my feelings too. That's why I don't look at reviews at all, or respond to them. Only if I notice a course is doing noticeably worse than my others will I dig into reviews looking for actionable feedback on what's wrong with it. And if there is some, I act on it. Otherwise, I accept that reviews are for students more than they are for me, and that the system is about as good as it can reasonably get.

 

It's been awhile since Udemy talked about how the review system works, but there are quite a few nuances that help us out you might not know about. The last we heard, written reviews carry more weight in your review average than reviews with no text - so those one-star reviews with no text may not have useful information for you, but they also don't count much. And reviews don't count at all after a few months, so even if you are hammered with unfair one-stars, they will eventually go away and let you start with a clean slate again. I think Udemy thought this all through a lot better than some of the huge tech companies, like Amazon.

 

Suggesting mandatory written comments on one and two star reviews is not an unreasonable idea. But it would bias reviews toward the positive, as it would introduce friction to negative reviews, and make the reviews less helpful to students as a result. "OK then, require comments on all reviews" you may say - but there are just too many courses on the marketplace, and reviews are spread too thinly as a result. Udemy needs to drive all the reviews they can get, in order to get a meaningful measure of every course's quality from learners. I appreciate Udemy's stance on this, and it's really an impossible situation.

 

Anyhow, my advice as always is to focus on what you can control. If your course is getting hammered by bad reviews, there's probably a reason why. Make your course better. Stop offering it for free and making it easy for trolls to access. And if you really think there is a concerted attack against your course from competitors - find evidence of it, and present it to the policy team. Never in the history of this community have I seen anyone actually have evidence of an attack from competing instructors, though. More often it goes back to Larry's point of how we want to blame someone else for bad things. Larry's a very wise man by the way. Y'all should listen.

@FrankKane  Actually all the champions here deserve to be treated the same you treat us if we ask for something you donot like.. So if you see it is racist to say "old man" then let it be! and I donot care at all!

I have been attacked, accused of really bad things I never did and your community let it go so I will let it go the same you let your attacks go. Good Luck with your day, week, month and year and your career. 

FrankKane
Community Champion Community Champion
Community Champion

Perhaps it's a language or cultural barrier thing, but in the US dismissing someone as an "old man" is considered insulting.

cancel
Showing results for 
Search instead for 
Did you mean: 
Stories and inspiration
Pull up a seat and chat with your fellow instructors. This is a great place to get to know each other, swap stories, and just relax.