Screwy guide on the regression testing wiki page

Open forum for end-user questions about Wine. Before asking questions, check out the Wiki as a first step.
Forum Rules
Locked
Bruni
Level 2
Level 2
Posts: 13
Joined: Fri Aug 05, 2011 3:26 pm

Screwy guide on the regression testing wiki page

Post by Bruni »

I'm confused by this:
... then you need to reset your bisect and use that "Release x.x.xx" as your first good bisect (wine-x.x.xx). This particular bisect output is just a version tag, meaning that no real code was changed (and so cannot have made your problem appear). All the "good" bisects you've set tells GIT that all that code is good, until wine-x.x.xx . GIT eliminates all that code except for the version tag, which would be the latest revision before the next version tag.
What does "meaning that no real code was changed (and so cannot have made your problem appear)" phrase mean? Same about
All the "good" bisects you've set tells GIT that all that code is good, until wine-x.x.xx
Also, I am just curious about why ccache is not specified as being used when doing regression testing at intermediate steps:
This will reposition the source code in the middle of the remaining patches. You will then recompile Wine by running

Code: Select all

./configure --verbose && make
and test for the bug again.
User avatar
André H.
Moderator
Moderator
Posts: 207
Joined: Sun Dec 07, 2008 8:33 am

Re: Screwy guide on the regression testing wiki page

Post by André H. »

Bruni wrote:I'm confused by this:
... then you need to reset your bisect and use that "Release x.x.xx" as your first good bisect (wine-x.x.xx). This particular bisect output is just a version tag, meaning that no real code was changed (and so cannot have made your problem appear). All the "good" bisects you've set tells GIT that all that code is good, until wine-x.x.xx . GIT eliminates all that code except for the version tag, which would be the latest revision before the next version tag.
What does "meaning that no real code was changed (and so cannot have made your problem appear)" phrase mean?
If you have a look at the change of Release 1.3.27 then you see that this can't introduce a regression as it only changes text, no code:
http://source.winehq.org/git/wine.git/c ... 0f0099dffe
Same about
All the "good" bisects you've set tells GIT that all that code is good, until wine-x.x.xx
Also, I am just curious about why ccache is not specified as being used when doing regression testing at intermediate steps:
This will reposition the source code in the middle of the remaining patches. You will then recompile Wine by running

Code: Select all

./configure --verbose && make
and test for the bug again.
Be free to fix this, further i also would add "-C" for configure as that caches the checks so configure runs much faster.
Bruni
Level 2
Level 2
Posts: 13
Joined: Fri Aug 05, 2011 3:26 pm

Post by Bruni »

BTW, I'm also confused by the fact that compiled Wine behaves differently depending on whether I told "./configure --disable-tests" or not. If tests are disabled, the application being tested often runs more correctly without revealing some bugs! This is why I never use "--disable-tests" to be safe.

But I wonder why this occurs.
oiaohm
Level 8
Level 8
Posts: 1020
Joined: Fri Feb 29, 2008 2:54 am

Post by oiaohm »

Bruni you just found a bug.

Disable tests should result in exactly the same binaries at end as test enabled.

Now that is going to be tricky search for someone. Build with ./configure --disable-tests and without --disable-tests in theory the only difference should be missing files of the tests not built in the --disable-tests. There is a chance that --disable-tests is changing somewhere in active code it should not be. So fixing bugs.
Bruni
Level 2
Level 2
Posts: 13
Joined: Fri Aug 05, 2011 3:26 pm

Post by Bruni »

If the bisect identifies a patch that a developer believes is harmless, you may be asked to check the result by reverting that patch.
First, reset the bisect if you haven't already:

Code: Select all

git bisect reset
Then use the revert command with the sha1 id of the bad patch:

Code: Select all

git show b821e839bb33ac8f56939cc582010ecf4d9c25d4 | patch -p1 -R
Rebuild Wine and test your program again.
When you're done testing, reset your tree to the latest Wine commit

Code: Select all

git reset --hard HEAD
I suggest to expand the above by adding the following:
Alternatively, you can also check if the bug was actually invoked by that commit without reverting suspected commit. For this, let's compare the behaviour of the program being tested right after and before this commit. To learn the program's conduct when the commit is in your Git, execute:

Code: Select all

git checkout b821e839bb33ac8f56939cc582010ecf4d9c25d4
then compile Wine and test the program. The bug must be present. If this is the case, move next.
Switch the git tree to the state just before suspected commit by running (the "^" char means that the commit preceding b821e839bb33ac8f56939cc582010ecf4d9c25d4 is now a new detached HEAD):

Code: Select all

git checkout b821e839bb33ac8f56939cc582010ecf4d9c25d4^
then rebuild Wine and see if the bug here. It must be gone away. If it's so, you have found the commit caused the bug!
The last step. Reset the HEAD to the original state:

Code: Select all

git checkout origin
In case after compilation you get non-working wine, clean up untracked files in the Git tree:

Code: Select all

git clean -dfx
Opinions?
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Post by dimesio »

I'd add a sentence explaining that the reason for doing it the alternative way is because sometimes a patch cannot be reverted cleanly.
Bruni
Level 2
Level 2
Posts: 13
Joined: Fri Aug 05, 2011 3:26 pm

Post by Bruni »

I don't know but for some cause editing the article results in broken markup.
Can anyone do it for me please?
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Post by dimesio »

Bruni wrote:I don't know but for some cause editing the article results in broken markup.
Can anyone do it for me please?
It's not you; it's a problem with the wiki that I've seen before, but seems to have gotten a lot worse. I just spent the last half hour trying to fix the page, and couldn't, so for now I've just reverted your changes.

I filed a bug for the issue with the wiki. but I wouldn't count on it getting fixed (for one thing, the wiki isn't hosted here). http://bugs.winehq.org/show_bug.cgi?id=28578
Bruni
Level 2
Level 2
Posts: 13
Joined: Fri Aug 05, 2011 3:26 pm

Post by Bruni »

dimesio wrote:I filed a bug for the issue with the wiki. but I wouldn't count on it getting fixed (for one thing, the wiki isn't hosted here). http://bugs.winehq.org/show_bug.cgi?id=28578
Can keeping Wine Wiki up-to-date be given such a low priority?
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Post by dimesio »

Bruni wrote:Can keeping Wine Wiki up-to-date be given such a low priority?
Wine is a volunteer project. Developers work on what interests them.
jjmckenzie
Moderator
Moderator
Posts: 1153
Joined: Wed Apr 27, 2011 11:01 pm

Screwy guide on the regression testing wiki page

Post by jjmckenzie »

On Wed, Oct 5, 2011 at 11:42 AM, dimesio <[email protected]> wrote:
Bruni wrote:
Can keeping Wine Wiki up-to-date be given such a low priority?
Wine is a volunteer project. Developers work on what interests them.
Also, the OP is always free to update anything on the Wiki that they
find confusing or just plain incorrect. It does not take a developer
to do this.

James
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Re: Screwy guide on the regression testing wiki page

Post by dimesio »

jjmckenzie wrote:Also, the OP is always free to update anything on the Wiki that they
find confusing or just plain incorrect. It does not take a developer
to do this.
You might try actually reading the thread you're responding to.
Locked