Correct method to report deceptive test results
-
- Level 1
- Posts: 5
- Joined: Mon Nov 16, 2015 1:08 am
Correct method to report deceptive test results
What is the correct method to report intentionally deceptive test results such as neglecting to disclose use of wine wrappers or wrong wine version info but still made it past whomever the app's maintainer was? The reason I ask is because I was browsing through an app's past test results and found an entry where the tester claims to have used Play on Linux to test even though third party wrappers for wine are barred from submitting test data on the test result guidelines wiki page.
Re: Correct method to report deceptive test results
Report it here. Post a link to it.
-
- Level 1
- Posts: 5
- Joined: Mon Nov 16, 2015 1:08 am
Re: Correct method to report deceptive test results
Thank you for reporting it. I've deleted that test report and warned the entry's maintainers that they should pay closer attention to the content of test reports when processing them. Not only did he mention using POL, he also said he had really used a different Wine version than the one he selected from the dropdown list. It never should have been accepted.
Re: Correct method to report deceptive test results
@dimesiodimesio wrote:Thank you for reporting it. I've deleted that test report and warned the entry's maintainers that they should pay closer attention to the content of test reports when processing them. Not only did he mention using POL, he also said he had really used a different Wine version than the one he selected from the dropdown list. It never should have been accepted.
Steve Ebey (maintainer) has actually self-posted a submission for World of Warcraft that is:
Surely this guy should know that the PBA patchset is totally unofficial?Wine 3.5 staging with PBA. Compiled with no debug symbols. Used PBA from Goldpaw repo.
...
Platinum
Should we be kicking maintainers if they do this?
Or just warn them?
It's really annoying - because WOW players tend not to be the most "technically literate" of Wine users...
Leading to unrealistic expectations of what vanilla Wine can do...
Bob
Re: Correct method to report deceptive test results
The first time, delete the test report and warn them. If they do it again, remove them as maintainers.Bob Wya wrote: Surely this guy should know that the PBA patchset is totally unofficial?
Should we be kicking maintainers if they do this?
Or just warn them?