Bronze, Silver and Gold clarification

[Faalagorn]

First of all, I hope I posted in the right place - the question is related to AppDB, thought it's more a question/explanation request instead of an bug/issue/suggestion, but on the other hand it's only related to APpDB and not actual Wine project.

The ratings of Garbage and Platinum are pretty much self-explanatory, but the descriptions for Bronze, Silver and Gold leave quite for interpretation. I'm asking this, since I'm a recent maintainer of Escape from Tarkov and there's a wide disproportion in two of the current reports (mine and the other one).

I chose the "Gold" rating, mostly based on the other Unity 3D game, My Summer Car - it have the same issue with mouse, but otherwise works flawlessly, once set up. It also fits the description: "Application works flawlessly with some DLL overrides, other settings or third party software." - which suggests even with such extreme cases as using 3rd party software warrants a Gold rating, as long as there are no issues in the actual game after applying it. My rating and mainternership request was approved, so I took it as fine.

However, in another test report, the tester actually managed to install the game through Wine, when I had to resort to a Windows (virtual) machine and gave the game a bronze rating. A fellow tester had an issue with mouse lag I didn't had (maybe due to playing in forced windowed/borderless mode) and an issue with antialising I think I didn't ahd either, but may have to re-test it as I disabled AA for performance and preference reasons. The tester didn't mention a mouse bug I encountered, but I have a feeling it might been omissioned, but that's not a point.

My real question is, which one of us was more accurate in rating? We both encountered some issues, albeit a bit different. I assume we got a similarily working game in the end, but we gave a vastly different rating. Initially I was thinking of silver rating in between, but the description suggests it's not there as one of the use-cases is when "a game works fine in single-player but not in multi-player" which was clearly not a case, as all the server-side aspects of the game (multi-player, server-side trading and actual authentication/email confirmation) work fine, so initially a natural step was upping the rating for gold, even if I was leaning toward something less.

Oddly enough, the bronze description, "Application works, but it has some issues, even for normal use; a game may not redraw properly or display fonts in wrong colours, be much slower than it should etc." fits more, as the application clearly have some issues (mouse), though I can't say anything about performance, since I have no comparison (though I have a feeling that it could run better on my GPU, so it may be a case too). I didn't had any redraw issues, as graphic rendered flawlessly for me. The other tester had antialising and lag issues, so maybe that prompted to lean toward bronze.

Anyway, all in all, should I change any of the test rating, either mine or the fellow tester's, rating to something uniform? Should I let them as is? Any advice is welcome

[dimesio]

Yes, there is a lot of subjectivity in the ratings.

Looking at your test report for Escape from Tarkov, I would not have rated it gold, as you state that there is no workaround for the mouse issue, and if there are mouse issues that can't be solved, however small, then it is not working flawlessly. Whether to rate it silver or bronze is largely a matter of how much the mouse issue bothers you.

Note that the examples given in the definition of silver are merely that, examples; they are not limitations. If any feature does not work and there is no workaround, the rating should not be higher than silver.

As a maintainer, you should be checking whether the rating given is consistent with rating definitions based on the reporter's description of what works and what doesn't work. If it's not, you should change the rating to match the rating definition based on what is reported.

[Faalagorn]

Looking at your test report for Escape from Tarkov, I would not have rated it gold, as you state that there is no workaround for the mouse issue, and if there are mouse issues that can't be solved, however small, then it is not working flawlessly. Whether to rate it silver or bronze is largely a matter of how much the mouse issue bothers you.

Note that the examples given in the definition of silver are merely that, examples; they are not limitations. If any feature does not work and there is no workaround, the rating should not be higher than silver.

As a maintainer, you should be checking whether the rating given is consistent with rating definitions based on the reporter's description of what works and what doesn't work. If it's not, you should change the rating to match the rating definition based on what is reported.

Thank you for reply, you cleared up the issue a little for me, and for the time being I'll change the rating on bronze for Escape from Tarkov, as the mouse issue persists. The game might actually warrant a silver rating, though for now I matched the other tester's rating, especially since there are also big installation issues.
I still find the emphasis of multiplayer misleading in the default silver description, maybe adding an actual "e.g." or "ex." in front could help a little, but at least now I know the examples given in ratings shouldn't be taken into consideration, when instead the test results should reflect the actual application state

However, since the same bug affects My Summer Car, which for me was even more annoying, as the game relies on precise interactions screwing down small parts using mouse wheel, what about it's test ratings? I'm not a maintainer or test author of it (though I actually applied as one) and all the editors seems to rated it as gold - with two of them mentioning the mouse issue and two while not, mentioning the other issues of frame being capped to 20 and performance being heavily reduced. They most likely followed or were changed by the maintainer to uniform, based on the other test results, hence the confusing Gold rating which prompted to rate Escape from Tarkov similarily. Should the ratings be downgraded to Silver (the game have no installation issues, mouse being the only one for me)? If so then by who? Admin, maintainer or actual testers? Thank you in advance for a reply!

[dimesio]

However, since the same bug affects My Summer Car, which for me was even more annoying, as the game relies on precise interactions screwing down small parts using mouse wheel, what about it's test ratings? I'm not a maintainer or test author of it (though I actually applied as one) and all the editors seems to rated it as gold - with two of them mentioning the mouse issue and two while not, mentioning the other issues of frame being capped to 20 and performance being heavily reduced. They most likely followed or were changed by the maintainer to uniform, based on the other test results, hence the confusing Gold rating which prompted to rate Escape from Tarkov similarily. Should the ratings be downgraded to Silver (the game have no installation issues, mouse being the only one for me)? If so then by who? Admin, maintainer or actual testers? Thank you in advance for a reply!

I've changed the ratings to silver for the two test reports that mention the mouse issue, and emailed the maintainer that he should monitor the ratings better. I left the two that only mention performance at gold, because the point at which performance becomes an "issue for normal use" (bronze) is highly subjective.

[Faalagorn]

I've changed the ratings to silver for the two test reports that mention the mouse issue, and emailed the maintainer that he should monitor the ratings better. I left the two that only mention performance at gold, because the point at which performance becomes an "issue for normal use" (bronze) is highly subjective.

Thank you - it indeed seemed like a logical step, since we have no easy way to verify if the mouse issue existed for these old tests or was it a regression bug. Hopefully, it'll be fixed someday down the line (the issue is reported already, under Bug 42717 and Bug 42872), so the future test will bring the reviews back to gold or maybe even Platinum

Either way, since I now got was granted a Super Maintainer status, I've messages the other maintainer about the possible issue as well, and hopefully we're both keep the results monitored . Thank you again for shredding some light on the ratings. Hopefully the ratings will iron out over time, resulting in no ambiguous ratings, or maybe even with enough issues nailed down in Wine, any ratings other than Platinum will be less encountered, which I surely wish the Wine project . Either way, I think the issue is pretty much solved right now.