Development process of Wine

Open forum for end-user questions about Wine. Before asking questions, check out the Wiki as a first step.
Forum Rules
Locked
Mukarrama
Newbie
Newbie
Posts: 2
Joined: Thu Nov 30, 2017 4:28 am

Development process of Wine

Post by Mukarrama »

Wine maintenance seems unusual in the sense that it appears very difficult to get working patches accepted and even many patches from longterm developers get regularly and at a high rate rejected, for arbitrary reasons. It looks a bit like a gifted developer can write a working fix in 2 weeks and spent then 5 months to split it into pieces and submit them one by one and maybe with patience get 75% of it into the source tree.

There are quite some essential bugs in the tracker that sit with working patches where the contributors have simply given up to submit them or rebase them or where the patch submission/acceptance status is completely unclear and no one of the core developers seems willing to take them up and walk you through the process. Same goes for tests. There is no real practical documentation how to write them for your code.

Given that most developers are not masochistic, are there any blog posts or developer diaries that walk you through the process of wine development? Or automated code tests?

Is there any documentation why specific pull requests were rejected? Is there any accountability?
User avatar
dimesio
Moderator
Moderator
Posts: 13202
Joined: Tue Mar 25, 2008 10:30 pm

Re: Development process of Wine

Post by dimesio »

The question is not Linux-specific, so moving to the general Help forum.

The wiki has quite a lot of documentation for developers: https://wiki.winehq.org/Category:Development.

Wine-staging was created specifically to help developers improve their patches: https://wiki.winehq.org/Wine-Staging_Development
User avatar
DarkShadow44
Level 8
Level 8
Posts: 1207
Joined: Tue Nov 22, 2016 5:39 pm

Re: Development process of Wine

Post by DarkShadow44 »

If you'r interested in an old, yet still relevant discussion of this matter: Reddit - Wine patches the black hole of code?

In short, I think it's mostly about quality. Even if a fix works, doesn't mean it doesn't break something elsewhere, or that it's a good solution to the problem at hand.
User avatar
Bob Wya
Level 12
Level 12
Posts: 3068
Joined: Sat Oct 16, 2010 7:40 pm

Re: Development process of Wine

Post by Bob Wya »

DarkShadow44 wrote:If you'r interested in an old, yet still relevant discussion of this matter: Reddit - Wine patches the black hole of code?

In short, I think it's mostly about quality. Even if a fix works, doesn't mean it doesn't break something elsewhere, or that it's a good solution to the problem at hand.
Hmmm... This!

Alexandre has a clear vision for the direction of the project.
I've had a patch rejected by him - but for all the right reasons!
It was a workaround rather than a solution to the underlying problem.
He took the time to explain why on wine-devel - which was nice.

Wine walks a fine tightrope between perfection and correct functionality.
Sure someone could have developed an amazingly performant multithreaded virtual NT kernel written in C++...
But most likely this would have ended up running nothing.

The choice to be cross-platform is perhaps quite a millstone IMHO - i.e. no SDL2.

Bob
Cybermax
Level 4
Level 4
Posts: 218
Joined: Fri Dec 01, 2017 5:26 pm

Re: Development process of Wine

Post by Cybermax »

Wine walks a fine tightrope between perfection and correct functionality.

Well, as much as i understand the "vision" and the problems developing wine to work with "everything", in my personal oppinion this cannot be done. Windows programs that have major faults following microsoft's standards as it is, is near impossible to get working with wine from a "correctness" standpoint. Sure, the idea is awesome, but at some point it should be "cleaned out". As much as I loathe the way microsoft rolled out Windows 10 - as a ad-tracking-free-if-you-let-us-monitor-everything-you-do os - even they have put the foot down on Windows XP.

What does this mean?

1. Should wine ditch hopeless old code, and NOT cater to those wanting to play that old "Pong" or whatever "really-only-for-retro-fun" programs?
2. Is it possible to move forward keeping ALL backwards compatibility things? Not even Microsoft manage to pull that off, and they make their code and have 100% access to everything. Are the Wine coders really that bright that they can reverse engineer microsoft code BETTER than even microsoft can?

Without wine, the Linux world would be a much shittier place, but if making RELEVANT patches and fixes mean you should have "pre wine-3.0" be "XP compatible" and Wine-3.0+ be Windows 7 compatible that should definately be done. Yeah, I know somethings need older wine versions, but that has more to do with bad luck breaking something than that the vision is correct.

The wine-dev bugtracker may seem perfectly understandable for a coder, but to me it looks really cumbersome. I have perhaps zero clue in how to use it, but it SEEMS counterproductive to post a bug to something that was not solved 4-5-8 years ago :)

It is impossible for the devs to test every piece of software out there, and they rely on the users of wine to do so. Then the bug-reporting system is not really that easy to use for the regular "Hey, i manage to follow some howto and install wine on my Ubuntu" user. Bugs are posted again, perhaps not really looked at, and the tracking list is flooded with unsolved cases from many years ago that may or may not be solved depending on random input of patches.

Yeah, i know its hard.. but reading the 8 year old thread that was posted, with the EXACT same issues as of this very night, how can one not loose a bit of joy :)

I would like to submit a few bugs myself, but have the distinct feeling that if i cannot produce the exact logs, the perfect description and perhaps the solution, at the right spot at the right time... it will fall down that very same black hole ppl talked about 8 years ago :)

Anyway.. nuff ranting from me.

C
User avatar
DarkShadow44
Level 8
Level 8
Posts: 1207
Joined: Tue Nov 22, 2016 5:39 pm

Re: Development process of Wine

Post by DarkShadow44 »

Cybermax wrote:Well, as much as i understand the "vision" and the problems developing wine to work with "everything", in my personal oppinion this cannot be done. Windows programs that have major faults following microsoft's standards as it is, is near impossible to get working with wine from a "correctness" standpoint. Sure, the idea is awesome, but at some point it should be "cleaned out".
Depends on what your goal is. I think Microsoft goal is more to make money, than it's for the wine community. MS might not care about some old programs, but users still may.
Cybermax wrote:As much as I loathe the way microsoft rolled out Windows 10 - as a ad-tracking-free-if-you-let-us-monitor-everything-you-do os - even they have put the foot down on Windows XP.
Yes, Microsoft did stop developing XP, but that's not really relevant here. They could have developed it further, but they have a competitor (win10) they want to sell, so it has to go. A lot of people would be happy if they continued XP for another 10 years, and I'm pretty certain they could deliver. They just don't want to...
Cybermax wrote:1. Should wine ditch hopeless old code, and NOT cater to those wanting to play that old "Pong" or whatever "really-only-for-retro-fun" programs?
Unless we don't want those programs to be lost in time, we should not ditch them. Actually that's one of the reasons I came into wine development, I just didn't want to give up my old games.
Cybermax wrote:2. Is it possible to move forward keeping ALL backwards compatibility things? Not even Microsoft manage to pull that off, and they make their code and have 100% access to everything. Are the Wine coders really that bright that they can reverse engineer microsoft code BETTER than even microsoft can?
Technically, sure, it's possible. Just adds more and more complexity... But technically, there are bugs that work reliably on a certain Windows version, it's an API quirk. We can reproduce that, if we want to, but it's a LOT of effort... But we can have tests to make sure it works how it should.
Cybermax wrote:Without wine, the Linux world would be a much shittier place, but if making RELEVANT patches and fixes mean you should have "pre wine-3.0" be "XP compatible" and Wine-3.0+ be Windows 7 compatible that should definately be done. Yeah, I know somethings need older wine versions, but that has more to do with bad luck breaking something than that the vision is correct.
And then maintain the old version of wine indefinitely? I don't see how this would work, if we want that support, we probably should have it in the latest version. After all, we can set windows versions in winecfg, that way we can make sure a certain fix is applied (or not).
Cybermax wrote:The wine-dev bugtracker may seem perfectly understandable for a coder, but to me it looks really cumbersome. I have perhaps zero clue in how to use it, but it SEEMS counterproductive to post a bug to something that was not solved 4-5-8 years ago :)
It might look a bit daunting, but the wiki helps a lot if you don't know what to do. Or the IRC. It's never counter productive to post an issue that already exists for a long time, why would it? It might only be fixed later, but keeping track of problems is important, I think.
Cybermax wrote:It is impossible for the devs to test every piece of software out there, and they rely on the users of wine to do so. Then the bug-reporting system is not really that easy to use for the regular "Hey, i manage to follow some howto and install wine on my Ubuntu" user. Bugs are posted again, perhaps not really looked at, and the tracking list is flooded with unsolved cases from many years ago that may or may not be solved depending on random input of patches.
Well, we can't test every software, but we can test the api. That's where unit tests come in. That way we can prove that behavior is correct, and stays correct. Of course, there are never enough tests, but it's still a great help. I think that it's not the problem that bugs are not looked at, it's just that the project is huge and the scope so big, and not enough manpower to fix everything.
Cybermax wrote:Yeah, i know its hard.. but reading the 8 year old thread that was posted, with the EXACT same issues as of this very night, how can one not loose a bit of joy :)
Look at the bright side, look how fare we've come the last years! You can even play some recent DX11 games.
Cybermax wrote:I would like to submit a few bugs myself, but have the distinct feeling that if i cannot produce the exact logs, the perfect description and perhaps the solution, at the right spot at the right time... it will fall down that very same black hole ppl talked about 8 years ago :)
Well, the bugtracker isn't really a black hole, it keeps everything in it, hopefully mostly categorized and sorted. Of course, bugs might get abandoned for a long time, but even very old bugs get fixed from time to time.
Cybermax
Level 4
Level 4
Posts: 218
Joined: Fri Dec 01, 2017 5:26 pm

Re: Development process of Wine

Post by Cybermax »

Thank you for the insight.

I cant help believe that making "wine work with everything that is ever made" is a very hard task. Does keeping ancient code hinder performance or make the code unnecessary complex in some way? I mean.. CAPI support? Yeah, its a module and you can choose to not compile it.. and hopefully no work is done on that part. Unless ofc im mistaken and capi is a basic function in windows that is most needed to get office to run or something. Sure, its nice to have for that 1 customer on the back of the north pole that still uses ISDN to transmit his weather data someplace.. but jeez..

What im kinda asking here is: Could wine be easier to maintain, update, implement new functions if you DON'T have to keep all those old tidbits up and running? :)

C
User avatar
DarkShadow44
Level 8
Level 8
Posts: 1207
Joined: Tue Nov 22, 2016 5:39 pm

Re: Development process of Wine

Post by DarkShadow44 »

Cybermax wrote:I cant help believe that making "wine work with everything that is ever made" is a very hard task. Does keeping ancient code hinder performance or make the code unnecessary complex in some way? I mean.. CAPI support? Yeah, its a module and you can choose to not compile it.. and hopefully no work is done on that part. Unless ofc im mistaken and capi is a basic function in windows that is most needed to get office to run or something. Sure, its nice to have for that 1 customer on the back of the north pole that still uses ISDN to transmit his weather data someplace.. but jeez..
If you're talking about capi2032.dll, this wasn't updated in about 10 years. So no, that shouldn't use any resources. It's probably not needed anymore, but it doesn't harm either. What's not used doesn't take any CPU time, and that bit of memory consumption is not worth complaining, I guess.
Cybermax wrote:What im kinda asking here is: Could wine be easier to maintain, update, implement new functions if you DON'T have to keep all those old tidbits up and running? :)
Of course it would, not supporting something old is always easier than keeping compatibility.
For example, 16bit support is still worked on, yet it's very old and even windows doesn't support that anymore. Or take other old games that don't work on windows anymore but do work on wine. Or other programs using deprecated APIs. It takes time and effort to make and keep those working, but when the official windows API doesn't support them anymore and only installing XP can help, I think it's even more important to have an open source implementation that cares about what's else lost.
Cybermax
Level 4
Level 4
Posts: 218
Joined: Fri Dec 01, 2017 5:26 pm

Re: Development process of Wine

Post by Cybermax »

DarkShadow44 wrote: Of course it would, not supporting something old is always easier than keeping compatibility.
For example, 16bit support is still worked on, yet it's very old and even windows doesn't support that anymore. Or take other old games that don't work on windows anymore but do work on wine. Or other programs using deprecated APIs. It takes time and effort to make and keep those working, but when the official windows API doesn't support them anymore and only installing XP can help, I think it's even more important to have an open source implementation that cares about what's else lost.
Sure, but in my personal opinion that is for "retro-people".. Yeah, you know those that have fun running Wolfenstein 3D on a rig with GTX1080ti :)

No doubt, in an ideal world, having "1 size fits all" would be perfect, but realistically keeping everything working does make things a lot harder than what it is.

So, am I spitting in someone's face by saying "Make two branches - 1 for performance/gaming/new stuff, and 2 for retro/compatibility stuff? Yeah, I know "staging" is worked out to be for that #1, but with a base of #2 that doesn't really make things easier either.

If I loved closed source and the big firms should make money attitude, I would go with Windows and save myself a lot of late nights... but I don't :) Still there comes a point where one have to take a small step back and review if its just as important anymore.

"Its our project, and we want this - if you dont like it, feel free to make something better" I guess would be the closing line. Nah.. keep up the great work that is the last strand keeping me from dual-booting... again... But atleast I told you my opinion :)

C
User avatar
DarkShadow44
Level 8
Level 8
Posts: 1207
Joined: Tue Nov 22, 2016 5:39 pm

Re: Development process of Wine

Post by DarkShadow44 »

Cybermax wrote:Sure, but in my personal opinion that is for "retro-people".. Yeah, you know those that have fun running Wolfenstein 3D on a rig with GTX1080ti :)
Yeah, or for legacy programs that are still used. Backwards compatibility keeps programs alive, when there's no way to run them anymore they're essentially dead.
Cybermax wrote:No doubt, in an ideal world, having "1 size fits all" would be perfect, but realistically keeping everything working does make things a lot harder than what it is.
Sure is. But on the other hand, having "1 size fits all" can be easier for the user than having to use multiple versions for different purposes, too.
Cybermax wrote:So, am I spitting in someone's face by saying "Make two branches - 1 for performance/gaming/new stuff, and 2 for retro/compatibility stuff? Yeah, I know "staging" is worked out to be for that #1, but with a base of #2 that doesn't really make things easier either.
Well, I'd say you don't always touch on legacy stuff too much when you're doing new stuff. E.g. when developing for DX11/DX12 you don't touch much DX8 code. You also need to keep in mind that staging is for patches that should end up in vanilla wine one day. So, what would having an extra "compatibility" branch help? It either needs to be rebased often or it goes out of date.
Cybermax wrote:If I loved closed source and the big firms should make money attitude, I would go with Windows and save myself a lot of late nights... but I don't :) Still there comes a point where one have to take a small step back and review if its just as important anymore.
Actually, windows gave me late nights, so I switched. xD
Well, how much backwards compatibility would you make? Cut at ISDN support, or maybe games for DX<9? Or everything from before the XP era?
Support for hardware is a bit different though, because software is essentially timeless. Most people don't have such old hardware, but you still can use programs from ages ago... If you got the right software to run it.
Cybermax wrote:"Its our project, and we want this - if you dont like it, feel free to make something better" I guess would be the closing line. Nah.. keep up the great work that is the last strand keeping me from dual-booting... again... But atleast I told you my opinion :)
Well, the great thing is that it's open. So even if the main devs would only work on modern stuff, hobby devs like me still could hack on retro support. ;)
Cybermax
Level 4
Level 4
Posts: 218
Joined: Fri Dec 01, 2017 5:26 pm

Re: Development process of Wine

Post by Cybermax »

DarkShadow44 wrote: Actually, windows gave me late nights, so I switched. xD
Well, how much backwards compatibility would you make? Cut at ISDN support, or maybe games for DX<9? Or everything from before the XP era?
Support for hardware is a bit different though, because software is essentially timeless. Most people don't have such old hardware, but you still can use programs from ages ago... If you got the right software to run it.
If keeping that code prevents implementation in a effective manner, some line could be drawn. If it does not prevent or in any way hinder development (especially performance wise) i see no harm in keeping it... but i kinda believe that making sure some old DX8 program works while working on DX11 may not be as easy as dropping DX8?

As i have said here before, i am no coder. I do however have some imagination, and we have seen time and time again that with every new os M$ as put out, they break old code. Why? Is it solely on the purpose of promoting/forcing you to buy the newest and latest? Possibly a combination.. I do hate the way every gaming company is deep into M$ when it comes to DirectX. DX9 -> XP, DX11 -> Win7, DX12 -> Win10. Ofc there is a business ploy at hand here... no doubt, but have M$ stopped making Windows 10 compatible with "all previous software platforms" just because they want to earn money on tracking your every move so they can sell you shi*? Most likely 80% yeah, and 20% "No, lets ditch this old crap to be able to squeeze some more out of this and that function". Just a guess :)

If some code makes for a multitude of loops running through old DX8 (or older) codelines "just in case" every time you do a DirectX -> OpenGL "conversion call" or whatever wine uses, it will hinder performance. With every new release of Wine, it has been larger, more code and slower performance. I know the reason for this has been "correctness" and general compatibility. Does this compatibility in any way hinder performance optimization? If so, what can be done? If dropping "old stuff" would make it a lot easier to optimize, it should be done... Speaking ofc for myself when i try to play games with wine. Maybe graphics performance is not at all prioritized in any way in that "vision"?

Should there be a "gaming" version of wine that focus on the "latest and greatest" for performance and a "application" version for compatibility that would work with the Windows 95 version of notepad.exe? If you think both things can be had in 1 version, awesome! Keep at it. If "performance is of secondary interest" for the "vision"... well...

C
Cybermax
Level 4
Level 4
Posts: 218
Joined: Fri Dec 01, 2017 5:26 pm

Re: Development process of Wine

Post by Cybermax »

It MIGHT just be that its just a handful of ppl wanting to game under Linux.. Steam does not show a huge player database, and posts on forums and whatnot from people gaming with Wine i guess is not a large % of "gamers". For gaming - install Windows.

That I personally and a handful of others WOULD like to game under Linux and use Wine for that purpose is perhaps not really a huge incentive to optimize wine towards that end? I know there are several popular programs that use some part of Wine to run under Linux, where graphics performance is of no real concern. So.. am i pleading a lost cause? :)

C
Locked