Low Ratings

2

Comments

  • @FrankKane
    but what I heard from support team they say all ratings are weighted with fair but whatever the algorithm was .. getting a recent 1 star from paid student doesnot mean my course should drop almost 0.5! because that way, any hater can just buy your course twice from different accounts he made and ruins your entire course's rating! that is the whole story I am talking about and instead of listening to me I am being attacked!!!

  • Finally I found A Community Champion who wants to listen! Thanks @FrankKane
    !

  • FrankKane
    FrankKane Posts: 1,859 rolemodel rank

    Reviews are a very touchy subject. As instructors we put months of effort into a course, and seeing someone criticize it with a few strokes of their keyboard, perhaps unfairly, can be very very discouraging.

    Just to explain why it works that way - if you have 20 5-star reviews and get another 5-star review, it won't change your average at all. A 4-star would only change it to 4.95. ((5*20+4)/21) But a 1-star would change it to 4.8. ((5*20+1)/21)

    What makes it worse is that recency is also a factor, so a brand new one star review counts more than a 3-month-old 5 star review. We don't know the details of how that works, but it explains how a new 1-star review can drag your score down so dramatically.

    Everything cuts both ways, though. A new 5-star review also carries more weight than an older 1-star. It's just that if the 5-star is closer to your existing average, it won't move it as much. It's all quite fair on paper, but yes it does open up the possibility of people really torpedo-ing a course with a one-star written review.

    Udemy has said below they have no plans of changing this any time soon, so we have to learn to deal with it. If you're in this situation, all you can do is see if you're lucky enough for the review to violate some Udemy policy in which case you might get it removed - and if not, try to drive more reviews to dilute the effect of it. Or simply wait it out - after a few months, it won't affect your score.

    Most of your courses have exceptionally high review scores. We tend to fixate on the negative feedback at the expense of the positive!

  • FrankKane
    FrankKane Posts: 1,859 rolemodel rank

    Perhaps it's a language or cultural barrier thing, but in the US dismissing someone as an "old man" is considered insulting.

  • @FrankKane
    thanks for clarification now I got my answer after a long journey of misery here. However that makes sense why anyone hates me can ruin my ratings easily and for your recent question how I keep my ratings high like some of my courses now because I report each single 1 star or even 3 star and some of them are removed or expired (after 3 months) but they discovered a new way now to ruin my work without any effect from my reports so I started to complain that is the whole story.. in short.. YES I have enemies in the marketplace and I donot know them but all of them have something in common in their ratings .. in their written feedback and in their way in skipping lessons and finishing the course in a short time.

    Thanks for listening!

  • @FrankKane
    I am sorry I donot mean that but Lawrence accused me and said alot against me in an archived thread I am saying " old man " because I donot know what to call him after all his accusations to me . I was just asking for something!!!

  • @FrankKane
    that is not a miracle people! I am just keeping good track of my ratings as much as I can by reporting negative reviews, sharing my free coupons (only for real students in private groups) rather than a spammy public post! and it actually goes well but who said all my courses are high! I have 3.9, 4.2, and of that 4.8s were actually around 4.4 but the negative reviews are expired recently.. when negative reviews are expired my ratings get high.. Why you feel it that way guys!

  • Absolutely true

  • Hum, this makes sense, for multiple reasons:

    1. Participants are encouraged to reflect on their experience, instead of just suddenly giving a certain rating.

    2. It discourages mean instructors who have competing topics from manipulating ratings. Though, I would imagine this doesn't happen as it would be super obvious!!

  • Here in Brazil too

  • R-Sila
    R-Sila Posts: 4 researcher rank

    fair enough, agree.

  • This is the same answer Bella often gives - we'll pass it on. This topic comes up again and again and often around this time of year. And it gets "passed on". The only result is we will end up chasing our tails over this one all over again in about 6 months when nothing happens and the topic is forgotten..

  • @DamianMcKinnon

    That is because the system is full of bugs and we shall thank god that is still working!

    Each start of a month, the conversion rates are reset to zeros (all the time) it shall be reset to zeros only for the new month! and they always say "it is just a visual issue" and that is it.

    An issue happened in all search 1 day ago and the search was getting almost zero results and it stayed like this for above 30min I guess!

    Another issue happened 2 days ago and all instructors received a warning that they donot have permissions to look at student statistics, course engagement or conversion rates and they fixed it after alot of questioning about that glitch!

    Some students accidently put a 2 star rating by mistake and I heard that many times here.

    All these issues are still on going and it is happening at regular times.

  • An excellent idea; it would be interesting to see the reasons, and check how many are able to justify their rating! In 20 years of lecturing, I have found that students are at times impulsive and emotional in regard to rating lecturers and educators.

  • @CarolCharma086
    No one ever needs to "justify" their ratings. All ratings, whether of courses, lectures, hotels, restaurants, movies, or books, are ALL an emotional response. Every public speaker knows, or should know, that it is not the intellectual content of a presentation, but how the presentation makes the audience feel that is what will be remembered.

    Much of this discussion is aimed at changing the rating system to alter how students rate, to make them more rational, or justify their ratings. Forget it! You can't do it. No one has ever done it. Instead, the instructor should recognize how their presentation makes the student feel and causes them to want to give a good rating. You have to change how you present, not how the student rates. One you have control of, the other you can only complain about. One is useful, the other is useless.

  • Hi. I have the same issue. One of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have several 5+ ratings too, where the users have provided comments describing what they liked about the course. I also have two ratings of 1 star (in 1 of them, the user provided no comments). Yet my ratings are declining, and my course will be removed from UFB in a few weeks. Performance by course attributes is that 97% feel they are learning valuable information, and 85% feel the course is delivering on your expectations. It seems your algorithm has prioritised a few bad reviews over good ones. The whole point of putting in so much effort and keeping the content fresh if the good feedback is simply not acknowledged by what is essentially computer code?

  • @AhmedMagdyMohamed
    @FrankKane
    @Mafer R
    I think I have the same problem. One of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have several 5+ ratings too, where the users have provided comments describing what they liked about the course. I also have two ratings of 1 star (in 1 of them, the user provided no comments). Yet my ratings are declining. Performance by course attributes is that 97% feel they are learning valuable information, and 85% feel the course is delivering on expectations. It seems the Udemy algorithm has prioritised a few bad reviews over good ones. Disproportionately. In fact, I don't think they are considering the good reviews at all.

  • @MinervaSingh66

    You are absolutely right and the algorithm puts higher values on negative reviews than positive ones and each time we say that the community champions will start to accuse us that the positive ratings we have gained are not authentic beware if you received such accusations just neglect them. udemy approves or flags each single review so move on and donot listen for others you will do great if you just focus on what you do.

  • FrankKane
    FrankKane Posts: 1,859 rolemodel rank

    Udemy (as far as we know) still weights more recent reviews higher, as well as reviews that have written comments in them. How far in the course the student was when the review was written may also be a factor. So if those one-star reviews fit any of those categories, they may be weighted disproportionately.

    All any of us can do in this situation is try to drive more reviews to average out the effect of the occasional bad ones. Your courses appear to be quite popular, so the good news is it shouldn't take too long for some more rational reviews to bring things back up.

  • Hi @FrankKane
    , this is how it usually is. However, now I don't think so. In my specific case, I got the 1 star review 15 days ago. Since then, I got several 4 and 5 stars. Including those from people who completed more than 90% of the course and gave comments. Yet my ratings are declining/stagnating. Seems that Ahmed has a similar issue

  • FrankKane
    FrankKane Posts: 1,859 rolemodel rank

    Hm, you're right - that doesn't sound right. The only explanation that would make sense would be if a bunch of 5-star reviews have recently expired from your review average calculation, due to them aging out (3 months I think it is.) But it sounds like your reviews are consistent enough for that to be unlikely.

  • Hi @FrankKane
    this has been happening for a few courses since the start of this year. I am at my wit's end, as good ratings don't seem to count anymore. There is no point in contacting Udemy as they give the usual standard responses. Now my courses are going to be thrown off UFB.

  • Hi @Bella
    , I am also struggling with Udemy's rating algorithm, and I agree with @AhmedMagdyMohamed
    that negative ratings seem to be disproportionately weighted over positive ratings and reviews.

    Speaking from my own experience, one of my UFB courses is quite popular with business clients (most of my students are from UFB), and I updated it around November 2022. I have received several excellent ratings. Most of my ratings over the past month have been 4+. Indeed I have a few 5+ ratings too. Users have provided comments describing what they liked about the course for many of these positive ratings. I also have two ratings of 1 star (in 1 of them, the user provided no comments). In my specific case, I got that 1-star review more than 15 days ago. Since then, I have gotten a couple of 4 and 5-star reviews, including those from people who completed more than 90% of the course and gave comments.

    Yet my ratings are declining. I pursued your Course Reviews FAQ, which clearly stated, "Rather than using a simple mathematical average of all ratings, the course rating calculation gives more weight to reviews from its most engaged students. When assessing a student’s engagement, we consider things like course consumption, the recency of their rating, and the length of their written review".

    Since the start of this year, this has not been the experience of many of us instructors, myself included. I don't think the algorithms are considering the 4+ reviews at all. Even @FrankKane
    opined that the lack of reflection of the relatively recent ratings is rather strange. @AhmedMagdyMohamed
    has mathematically identified that negative ratings have a much more significant impact than positive ratings. Many instructors have noted and commented on such a phenomenon over the years, and @AhmedMagdyMohamed
    very kindly looked at the matter quantitatively. So what you state about your rating calculation system seems to be misleading at best. The issue of how Udemy generally handles its reviews and ratings has been raised several times by us instructors. But as @DamianMcKinnon pointed out, Udemy admins continue to ignore our concerns about the rating issue. Now while no one expects Udemy to make its review algorithm public, it is high time Udemy clears the record and tells us if negative ratings are given extra weightage over positive ratings. Setting the record straight on this specific matter will (and acting on individual concerns as needed) will go a long way in reaffirming Udemy's commitments to "positive impact on employees, learners, instructor

    Udemy is now a publicly-traded company listed in the USA. I am a researcher in the ESG/SDM domain (MPhil from Oxford University and PhD from Cambridge University). It was most gratifying for me that in 2022, Sustainalytics, a subsidiary of Morningstar(a renowned ESG (environmental, social, and governance) research, ratings, and data organisation), ranked the Udemy first in the Internet Software and Services sub-industry for the second consecutive year (https://about.udemy.com/press-releases/udemy-ranked-first-in-internet-software-and-services-sub-industry-by-sustainalytics/). The press release further noted Udemy's commitment to a positive impact on instructors. The current rating system, the lack of transparency around it and the fact that negative ratings may be disproportionately weighted over positive ratings (which arguably flies in the face of what you state on your own Course reviews page) are doing the exact opposite. There is no such thing as a perfect algorithm. If the actual computations of your rating algorithms weigh negative ratings more than positive ones (thus running contrary to your stated policy), then both instructors and students are being misled. This is not only unethical but could potentially be a case of ESG-washing. At the end of the day, it is not about one course or one instructor. The fundamental issue of instructor welfare hearing us out and ensuring that ESG commitments are met transparently and meaningfully. From now on, I sincerely this will be borne in mind and instructor concerns acted upon.

  • @MinervaSingh66

    I doubt that the udemy themselves know anything about the rating algorithm. I opened that case several times with the support team and their replies contradicted each other. Some of them say that reviews with free coupons are the same as reviews without coupons and we all know that is not right. The reviews for free coupons are lower in weight and the support member said the wrong answer. Maybe it is a single fault mistake, but I raised my concern several times because that rating system encourages sick-heads and poor minds and jealous people to just ruin your work with a 1 star review! But no one is listening. You will keep struggling and you will get no answer from the platform.

  • We have to try. Remember, at the end of the day, Udemy is now a publicly listed company. We are right in seeking transparency and consideration for instructor welfare. That is the whole point of ESG ratings.

  • Wow I taught I was the only one noticing this

  • Wow I taught am the only one who noticed bad reviews has more power than the good ones

  • @MaclayAcademy23
    it is a common issue their algorithm is so brilliant. Only one 1-star rating can ruin your entire course's rating even stupid sellers wont do this to themselves. our courses are the source of income to udemy if they wont correct that, on long run all courses will be low rated and no student will buy a low rated course. Enjoy it Udemy!

  • DavidFord695
    DavidFord695 Posts: 1 observer rank

    Hello everyone,

    I've been following this thread with great interest and appreciate all the insights shared. I want to bring up an issue that I believe affects the integrity of our rating system: the potential for personal grievances to skew reviews. While most feedback is genuine and constructive, there are instances where individuals from our past, perhaps due to personal disagreements, might use the review system to express their dissatisfaction, affecting our professional reputation and the perceived quality of our courses.

    I believe it’s crucial for platforms like UDEMY to have mechanisms that can filter out reviews driven by personal biases unrelated to the course content or delivery. This is not just about protecting instructors but about ensuring that prospective students receive accurate and fair representations of our courses. Transparency about why a course was rated poorly can often clarify whether the feedback is content-related or personally motivated.

    Thank you for considering this perspective. I'm looking forward to seeing how we can work together to enhance the reliability of your review system.

  • MinervaSingh66
    MinervaSingh66 Posts: 175 storyteller rank

    I agree.