Performing a Regression Test

[zac]

Oh, alright... I had read that, and I knew I wasn't done yet. But it was not at all clear from the text that the next thing to do was to "compile and test one more time". Maybe that was obvious, but I didn't get it! Sorry!

That's what I'm gonna do now.

Thanks, dimesio!

[dimesio]

Oh, alright... I had read that, and I knew I wasn't done yet. But it was not at all clear from the text that the next thing to do was to "compile and test one more time". Maybe that was obvious, but I didn't get it! Sorry!

I've added that to the regression testing page.

[zac]

dimesio, that was a nice thing to do! Thanks for adding that line! It will help other beginners like me!

Well, I'm very happy! Here is the bad commit:

Code: Select all

debian:/home/zac/wine-git# git bisect bad
1a3551b630eae2418692f21790fcc7fc973f5366 is first bad commit
commit 1a3551b630eae2418692f21790fcc7fc973f5366
Author: Dylan Smith <[email protected]>
Date:   Mon Feb 2 01:32:20 2009 -0500

    richedit: Use width from EM_SETTARGETDEVICE for wrapping.

    The width for EM_SETTARGETDEVICE is used by some applications to set the
    wrapping width to a certain distance in twips.  This can be used even
    though the target device is ignored.

:040000 040000 efbf63783320529c2de51eefdbaa1bde15bb209f a237f70a5fff488280157b2876c2f44b223185e5 M      dlls

As James McKenzie pointed out, this MAY NOT be the bad commit.
So, I ran the following commands suggested by him.

Code: Select all

debian:/home/zac/wine-git# git reset --hard
HEAD is now at 1a3551b richedit: Use width from EM_SETTARGETDEVICE for wrapping.

Code: Select all

debian:/home/zac/wine-git# git show 1a3551b630eae2418692f21790fcc7fc973f5366 | patch -p1 -R
patching file dlls/riched20/context.c
patching file dlls/riched20/editor.c
patching file dlls/riched20/editstr.h
patching file dlls/riched20/tests/editor.c
patching file dlls/riched20/wrap.c
patching file dlls/riched32/tests/editor.c

and... then, './configure && make depend && make'

After that, I ran the program, and it worked!! It's better than it used to be in my older wine.

I suppose I have finally found what the problem is, then!

Would anyone like to post any comments, or add anything to this?

I think I'm over now, and the commit should be posted into bugzilla, is that it?

Thank you SO much for all your attention and help!
You've been very helpful, and very kind!

Thanks!

Oh... actually, I have some other questions. Can I keep using this wine I compiled and install the other programs on it. I have removed my old wine, because I got confused in the beginning. I'd rather use this one now, while the problem doesn't get fixed. I'd like to configure it to recognize a mounted cd .iso image that I have in my disk, to run another software. How can I do this? I suppose I have to use the terminal, but I can't get to manual pages (I think they are not here, are they?).

Well...

I'm very thankful to all of you who helped me with this!!
Thanks!!
Zac

[James McKenzie]

zac wrote:

Hello folks!
I'm back!

Regression testing is a lot of work!
I still don't know if I did everything right, though. Again, I think I need help.

I'm posting here all the results of the bisecting process. When I got the 'Bisecting: 0 revisions left to test after this' I don't seem to have received the 'bad commit' I was expecting!


Code:

debian:/home/zac/wine-git# git bisect good
Bisecting: 1211 revisions left to test after this
[853d993830509c3ecaa4113bed789bf08ec36d76] shell32: Default to the more modern location for the user profile directory.

--------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 605 revisions left to test after this
[7aa3bb3a68b1b1d5b01ace0dd9fc896c59375e99] sane.ds: Initialize the SupportedGroups field when returning a source.

-------------------------------
debian:/home/zac/wine-git# git bisect good
Bisecting: 302 revisions left to test after this
[edef19f8d60d6ce089f1562a958925668f9d4e32] d3dx8: Move some functions into core.c to match the header file layout.

--------------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 151 revisions left to test after this
[5bc54ed97ecd281a2ef4d07473d72f1e50cd9ad5] gdiplus: Implemented GdipCreateBitmapFromHICON (with tests).

----------------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 75 revisions left to test after this
[dd943665932e428c3f423ae062464d9b4d755ea5] wininet/tests: Fix tests failures on W2K and below and some early XP.

--------------------------------------
debian:/home/zac/wine-git# git bisect good
Bisecting: 37 revisions left to test after this
[f20ba244049f5473369e32df5fe108244da3e180] d3dx8: Implement D3DXFresnelTerm.

---------------------------
debian:/home/zac/wine-git# git bisect good
Bisecting: 18 revisions left to test after this
[183bb795b6c1cd1e24503a20b1a4a7b9ec12524b] mshtml: Add missing StyleSheet interfaces.

-----------------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 9 revisions left to test after this
[e7d58bdd5243d31dec0f7c735566b6b9744ddf7b] itss: Move buffer initialization a bit up (Coverity).

--------------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 4 revisions left to test after this
[edbea606f94a34872097620e2b79e9c07fa3c921] comctl32: Fix propsheet size calculation.

---------------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 1 revisions left to test after this
[8662c6fe7b60d9ff563ba790d4431ef30250249b] richedit: Implement EM_GETTEXTMODE.

------------------------------------
debian:/home/zac/wine-git# git bisect bad
Bisecting: 0 revisions left to test after this
[1a3551b630eae2418692f21790fcc7fc973f5366] richedit: Use width from EM_SETTARGETDEVICE for wrapping.

Does it work now? If not, do another git bisect bad until the problem
goes away...

If it does work see below.

Code:

Code: Select all

[/quote]
[quote]make clean
git reset --hard
git show <commit number, usually a 32 alphanumeric> | patch -p1 -R

[/quote]
[quote]

If the problem goes away, then do the first two steps above and then;

Code: Select all

git show 1a3551b630eae2418692f21790fcc7fc973f5366 | patch -p1 -R

Then do a compile to see if the program works/does not work.

If it does not work,

James McKenzie

[James McKenzie]

zac wrote:

dimesio, that was a nice thing to do! Thanks for adding that line! It will help other beginners like me!

Well, I'm very happy! Here is the bad commit:


Code:

debian:/home/zac/wine-git# git bisect bad
1a3551b630eae2418692f21790fcc7fc973f5366 is first bad commit
commit 1a3551b630eae2418692f21790fcc7fc973f5366
Author: Dylan Smith <[email protected]>
Date: Mon Feb 2 01:32:20 2009 -0500

richedit: Use width from EM_SETTARGETDEVICE for wrapping.

The width for EM_SETTARGETDEVICE is used by some applications to set the
wrapping width to a certain distance in twips. This can be used even
though the target device is ignored.

:040000 040000 efbf63783320529c2de51eefdbaa1bde15bb209f a237f70a5fff488280157b2876c2f44b223185e5 M dlls

Looks like Dylan has some work to do.

What was the program again and what is the bug number?

I'm working with him on several patches for richedit to get a function
added that may actually fix this.

James McKenzie

[James McKenzie]

James McKenzie wrote:

zac wrote:

Hello folks!
I'm back!

Never mind, you found the 'bad' commit. Now to get it fixed. Again,
program name and bug number.

BTW, good work zac. Yes you went through a long list of commits, but
just imagine if you had to do this for EACH one of those commits...(7995
commits is a long list.)

James McKenzie

[zac]

James McKenzie wrote:

BTW, good work zac. Yes you went through a long list of commits, but
just imagine if you had to do this for EACH one of those commits...(7995
commits is a long list.)

James McKenzie

Thank you, James!!
I'm very proud of myself!
You helped me a lot!

Indeed, this bisecting thing is very clever! It does save a lot of work! I enjoyed it, it's kind of fun (at least the first time you're learning to do it!)

Ok, the software can be called "Houaiss Portuguese Dictionary" for short, in English. The real name in Portuguese (if you at all need it) is "Dicionário Eletrônico Houaiss da Língua Portuguesa."

Bug 21920 http://bugs.winehq.org/show_bug.cgi?id=21920

Is that enough?
I think I'll post the bad commit into bugzilla anyway, is that alright?

Thanks!!

[mogorva]

Well, I'm very inexperienced in regression testing too, and have a question:

I filed a bugreport about a possible regression that prevents Alpha Prime from runnning correctly:
http://bugs.winehq.org/show_bug.cgi?id=21941

I tried to revert the patch that might caused the problem in the game following this guide: http://wiki.winehq.org/RegressionTesting

So I made the following in the wine-git directory:

git bisect reset
then
git show 224043d6cf8b56e9ff2537358646700211d54d1f | patch -p1 -R

but the result was this:

patching file dlls/wined3d/buffer.c
Hunk #3 FAILED at 1058.
Hunk #4 succeeded at 1098 (offset 3 lines).
Hunk #5 succeeded at 1105 (offset 3 lines).
Hunk #6 succeeded at 1148 (offset 3 lines).
Hunk #7 succeeded at 1156 (offset 3 lines).
Hunk #8 succeeded at 1251 (offset 3 lines).
1 out of 8 hunks FAILED -- saving rejects to file dlls/wined3d/buffer.c.rej

What should I do now?

[James McKenzie]

zac wrote:

James McKenzie wrote:

James McKenzie wrote:

BTW, good work zac. Yes you went through a long list of commits, but
just imagine if you had to do this for EACH one of those commits...(7995
commits is a long list.)

James McKenzie

Thank you, James!!
I'm very proud of myself!
You helped me a lot!

Indeed, this bisecting thing is very clever! It does save a lot of work! I enjoyed it, it's kind of fun (at least the first time you're learning to do it!)

Ok, the software can be called "Houaiss Portuguese Dictionary" for short, in English. The real name in Portuguese (if you at all need it) is "Dicionário Eletrônico Houaiss da Língua Portuguesa."

Bug 21920 http://bugs.winehq.org/show_bug.cgi?id=21920

Is that enough?
I think I'll post the bad commit into bugzilla anyway, is that alright?

Please do update the bugzilla entry. I'll take care of notifying the
author of that patch since we are working adding functionality to
richedit.

James McKenzie

[James McKenzie]

mogorva wrote:

Well, I'm very inexperienced in regression testing too, and have a question:

I filed a bugreport about a possible regression that prevents Alpha Prime from runnning correctly:
http://bugs.winehq.org/show_bug.cgi?id=21941

I tried to revert the patch that might caused the problem in the game following this guide: http://wiki.winehq.org/RegressionTesting

So I made the following in the wine-git directory:

git bisect reset
then
git show 224043d6cf8b56e9ff2537358646700211d54d1f | patch -p1 -R

but the result was this:

patching file dlls/wined3d/buffer.c
Hunk #3 FAILED at 1058.
Hunk #4 succeeded at 1098 (offset 3 lines).
Hunk #5 succeeded at 1105 (offset 3 lines).
Hunk #6 succeeded at 1148 (offset 3 lines).
Hunk #7 succeeded at 1156 (offset 3 lines).
Hunk #8 succeeded at 1251 (offset 3 lines).
1 out of 8 hunks FAILED -- saving rejects to file dlls/wined3d/buffer.c.rej

What should I do now?

It appears that a further patch has changed the contents of the file
from when the patch was originally published.

So, more work has to be done. You could use git blame to find out the
latest patch and then revert that patch and then revert the patch that
causes problems and then apply the follow-on patch, or you can revert
the patches, build Wine without them and see if the problem is gone.

I recommend the latter and then working through the new patches,
one-by-one to see if any of the follow-on patches cause problems as well.

James McKenzie

[abb4]

Hi, I try to do a regression test for the first time.
I got in some kind of a loop, it looks like the bad commit isn't removed
I'm stuck

Here's how I went
I found 1 bad commit

7cb044d3d8aaf1cee629c259f19177732bed37af is the first bad commit
commit 7cb044d3d8aaf1cee629c259f19177732bed37af
Author: Michael Stefaniuc <[email protected]>
Date: Wed Jul 21 23:56:02 2010 +0200

sane.ds: Remove the ignored common resource attributes.

:040000 040000 9e1890eada06b83f59052c7b58162c862cdf1681 f9326415ac4094eade2ae554e67f7f9bc88168e2 M dlls

As said, this MAY NOT be the actual bad commit.
So I did a make clean & git reset --hard,
Now I get:

HEAD is now at 7cb044d sane.ds: Remove the ignored common resource attributes.

After a ~/wine-git$ git show 7cb044d3d8aaf1cee629c259f19177732bed37af | patch -p1 -R

I get:

patching file dlls/sane.ds/sane_Da.rc
patching file dlls/sane.ds/sane_De.rc
patching file dlls/sane.ds/sane_En.rc
patching file dlls/sane.ds/sane_Es.rc
patching file dlls/sane.ds/sane_Fi.rc
patching file dlls/sane.ds/sane_Fr.rc
patching file dlls/sane.ds/sane_Hu.rc
patching file dlls/sane.ds/sane_It.rc
patching file dlls/sane.ds/sane_Ja.rc
patching file dlls/sane.ds/sane_Ko.rc
patching file dlls/sane.ds/sane_Lt.rc
patching file dlls/sane.ds/sane_Nl.rc
patching file dlls/sane.ds/sane_No.rc
patching file dlls/sane.ds/sane_Pl.rc
patching file dlls/sane.ds/sane_Pt.rc
patching file dlls/sane.ds/sane_Ro.rc
patching file dlls/sane.ds/sane_Ru.rc
patching file dlls/sane.ds/sane_Si.rc
patching file dlls/sane.ds/sane_Sv.rc
patching file dlls/sane.ds/sane_Uk.rc
patching file dlls/sane.ds/sane_Zh.rc

After ./configure && make deps && make
I test the program again, it doesn't run
so I do again a "git bisect bad"
and it returns to the same bad commit!

How do I remove this commit?
a ~/wine-git$ git show 7cb044d3d8aaf1cee629c259f19177732bed37af | patch -p1 -R did not remove it?


Any help is appreciated!

[abb4]

Any help is appreciated!

Ohh I see,
have to go back to
git bisect start
git bisect good <wine-tag where program worked>
git bisect bad <wine-tag where program stopped working>


This is quite complicated stuff

[abb4]

Any help is appreciated!

Ohh I see,
have to go back to
git bisect start
git bisect good <wine-tag where program worked>
git bisect bad <wine-tag where program stopped working>


This is quite complicated stuff

Oke, got a bit further with this
The above was mistaken.
when I recompiled it didn't work with [./configure && make deps && make ] & I lost the wine branch
But with [CC="ccache gcc" ./configure --verbose && make depend && make] it compiled well
so I completely messed things up, so to say..

Now I got stuck somewhere else, please any help is apreciated!
This is how far I got, I found the bad commit, but to put this one at test, the following happened

adri@dark:~/wine-git$ git bisect good

6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786 is the first bad commit
commit 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786
Author: Huw Davies <[email protected]>
Date: Thu Jul 22 13:37:19 2010 +0100

ole32: Implement cross-process drag and drop.

:040000 040000 ab88cc6bf0936bc4adc7bff0673415282613d919 065c727204a46


make clean

adri@dark:~/wine-git$ git reset --hard
HEAD is now at a2e1dd2 ole32/tests: Relax the reference counting tests a bit. We only care whether references are held or not.

adri@dark:~/wine-git$ git show 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786 | patch -p1 -R
patching file dlls/ole32/ole2.c
Unreversed patch detected! Ignore -R? [n] enter

Apply anyway? [n] y
Hunk #1 FAILED at 116.
Hunk #2 FAILED at 270.
Hunk #3 FAILED at 406.
Hunk #4 FAILED at 436.
Hunk #5 FAILED at 477.
Hunk #6 FAILED at 491.
Hunk #7 FAILED at 2143.
Hunk #8 FAILED at 2171.
Hunk #9 FAILED at 2183.
9 out of 9 hunks FAILED -- saving rejects to file dlls/ole32/ole2.c.rej

What should been done with a 'Unreversed patch detected!'
Or can I be sure the detected bad commit is the right one?
How to proceed?

[vitamin]

adri@dark:~/wine-git$ git reset --hard

You shouldn't of done this during bisect. Now you need to do:

Code: Select all

git bisect reset
git reset --hard 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786^

This will put your tree just before the problem commit. It [the problem patch] can't be cleanly reverted from the latest HEAD.

[abb4]

adri@dark:~/wine-git$ git reset --hard

You shouldn't of done this during bisect. Now you need to do:

Code: Select all

git bisect reset
git reset --hard 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786^

This will put your tree just before the problem commit. It [the problem patch] can't be cleanly reverted from the latest HEAD.

Oke, thanks for replying!!
I did what you said

adri@dark:~/wine-git$ git reset --hard 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786^
HEAD is now at a2e1dd2 ole32/tests: Relax the reference counting tests a bit. We only care whether references are held or not.

So now where does that leave me?
do I now compile once again & check if the program runs?
If it will I have the bad commit?
Or else? ( I'm in limbo here..)

[vitamin]

do I now compile once again & check if the program runs?

Yes.

If it will I have the bad commit?

No, your tree now on commit before bad one.

[abb4]

do I now compile once again & check if the program runs?

Yes.

If it will I have the bad commit?

No, your tree now on commit before bad one.

Thanks Vitamin
The program runs, so I guess this will bring me back to where I was.

6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786 is the first bad commit
commit 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786
Author: Huw Davies <[email protected]>
Date: Thu Jul 22 13:37:19 2010 +0100

ole32: Implement cross-process drag and drop.

:040000 040000 ab88cc6bf0936bc4adc7bff0673415282613d919 065c727204a46

I understood from this thread earlier that this still needs to be tested?
Does it not?
I need to know for sure, I don't want to sent a wrong regression test to the developers

[vitamin]

I understood from this thread earlier that this still needs to be tested?

Yeah, testing it one more time will verify that is indeed the commit that broke your program. To do that run this:

Code: Select all

git show -p 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786 | patch -p1

Compile Wine and test your program. If it doesn't work, then you've found the bad commit.

[abb4]

I understood from this thread earlier that this still needs to be tested?

Yeah, testing it one more time will verify that is indeed the commit that broke your program. To do that run this:

Code: Select all

git show -p 6d1ef3a6a64f0fabf05ce1bba5f0ec4373684786 | patch -p1

Compile Wine and test your program. If it doesn't work, then you've found the bad commit.

Great, I did the whole sequence one more time and the program doesn't run after the last Wine compilation.
I'll post this test as soon as possible
For reference sake this is about bug http://bugs.winehq.org/show_bug.cgi?id=24660
Thanks a lot Vitamin, I couldn't have done this without your help