Does anybody review challenge questions?

Question

I have found too many wrong questions related to C programming. I will mention the ones that blew my mind.

Example:
char str1[] = "solo";
char str2[] = "learn";
strcpy( str1, str2);
strcmp(str1, str2);

str1 can store 4 bytes and str2 5 bytes, excluding the terminating null byte. strcpy() executes without error because str1 can store all 5 bytes of "learn" but then str1 is not a valid string as it is not null terminated.

A similar error in another question included strcat() which was used to append str2 to str1. There is no need to mention again there is not enough space.

The final problem was in printf() calls like the following:
printf("%d %d %d\n", a++, --a, a);

where variable 'a' is an int. The order in which the calculation of the parameters is conducted is implementation defined.
For a = 5 and gcc as the system compiler, the output is:
"4 5 5". Such operations should be done before calling printf.

Good C programmers may lose challenges due to so many mistakes.

Answer 1

Challenge questions should definitely be reviewed (in all languages) and there should be a way to report questions that are just plain wrong. Several questions for python are valid for python 2, but not for python 3 (only python 3 is taught in Sololearn), others are wrong for both python 2 and python 3.

Answer 2

George Zach, as it is, SoloLearn team seems to be still reviewing all the material themselves.

Also you usually wait months for an answer, for lessons and assignments even a year.

We can only guess what's going on; but I suppose they just don't really have the time; so when they get around to doing a few, it happens in a hurry and errors occur.

It's about time they started a separate sort of voluntary mod team to review the material, which will then hopefully happen more quickly and more accurately.

Btw: I recently encountered a C riddle where you had to make assumptions about a faulty array access - undefined behaviour!

Yeah, something should be done...

Answer 3

Yes people review them and you can too. And in fact even someone who knows nothing about a language can review questions in that language! This is one of the flaws in the system.

Answer 4

~ swim ~
I have already reported them using the answers view after the challenge has ended. I am a teaching assistant in systems programming using C in a university and I created Sololearn for fun, not to learn. I found too many wrong questions that many times lead me to losing. In my opinion I would like a more strict approval process. Those many mistakes are not accidental.

Answer 5

George Zach

I wouldn't toss the baby out with the bath water on this, but yes, adding an explanation field (in conjunction with Like / Dislike) would help immensely, much like the options you get after you View Correct Answers from a challenge/quiz match and you report the question. Having that available from the beginning of the process would be helpful.

Verified experts would be good for auditing question accuracy, but I do think that non-experts can help with identifying spam and learn from reviewing submissions.

SoloLearn wants to be a community driven learning app, so it would certainly behoove them to get the tooling down so that the free labor is being adequately empowered to make their investors money. 😁

At any rate, it's my impression that SoloLearn reviews all submissions eventually as they are also responsible for putting together feedback for rejected questions. Granted, you may question their qualifications as well, since their feedback doesn't always make sense.

Who's pointing fingers? 🙂

Answer 6

HonFu I agree. Volunteers can solve the problem. The mistakes I mentioned are very simple. They don't contain a difficult program flow that may lead to mistakes and requires a lot of time to verify.

Answer 7

I would encourage you to contribute toward the quiz submission review process (as Sonic mentioned).

Vertical ellipses
> Quiz Factory
> Rate Submissions

Then we can look up prior threads suggesting improvements to the quiz review process and expand on them as we see fit.

I hope this helps!

Answer 8

HonFu

Hopefully I wasn't implying that lack of participation is the sole reason. It's only one of many bottlenecks, but if someone knowledgeable enough about a language is willing to be part of the solution and not just ponder the problem (which also has some merit, but only gets us so far), I'm definitely going to point out how they can help and encourage that. 🙂

I think I've gone over several bottlenecks in a different thread, but I can't seem to find it at the moment. (Maybe it got removed, since this subject is discussed often.) Even after the Community reviews the submissions, SoloLearn has to actually add them to the pool of active challenge questions, (or reject them with feedback, as the case may be,) which isn't something we control. But maybe we won't even get THAT far if the Community gets burned out on reviewing spam or off-topic submissions.

So, yeah... I didn't mean to marginalize the many problems inherent in the system.

Please do email [e-mail] or leave Feedback from the app.

Answer 9

Janning⭐ adding an explanation field in the quiz review would be great. Then you could say e.g. that you like a quiz, but there is still an error.

George Zach I often have the same problem. I know that there is no correct answer, but you have to pick one.
There are several questions relying on undefined behaviour. The best everybody can do is to keep reporting wrong quizzes. If you choose "other", then you could add a text.

Answer 10

Anyone can review questions i dont know if after the comunity review anyone checks if its a valid question like it should

Answer 11

Well, but that is only the pre-review.

That system helps determine which riddles seem to be interesting for the 'community', and then, when riddles bubbled up enough with the upvotes, they get approved or declined by SoloLearn.

So they do have a look at it - only not always a sufficiently scrutinizing one.

Answer 12

I want to point out one thing additionally:

Since there is a riddle reviewing system for users that predetermines which riddles will even be shown to the actual jury (that's how it's usually depicted here) one might assume riddles don't get approved only for the reason that sololearners don't review them enough.

If we look at how long it takes till lessons and assignments get approved although there is no user-side system as in the quizzes, I think it's reasonable to assume that a lack of user participation is not the only problem.

Answer 13

Janning⭐ Judging by the result of the review process, peer review is not working. I am not sure I understood the process, but do some submissions never reach the final reviewers due to peer reviews? If so, the review process should only be conducted by verified experts in the field. C requires years to master. If a question is clever, the average user/reviewer may not have enought experience to understand it and downvote it.

I would suggest any question/answer report in challenges to include a text field where anyone conducting a report can explain why something is wrong etc.
As a mentioned earlier I am engaged in academia (full time) but I could help with final reviewing, being a teaching assistant in Systems Programming using C the last 4-5 years.

Pointing fingers will not solve the problem but on the other hand not everybody can be part of the solution.