How to build the portable version?

Questions about Wine on macOS.
Post Reply
adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

How to build the portable version?

Post by adamAC » Tue Jun 23, 2020 2:52 am

Hi,

I have been given the responsibility in my company of getting a couple of our Windows apps working on macOS, so I am obviously trying to use Wine to achieve this. I am also using Wine-Bundler to create proper self-contained macOS apps.

All of that is fine but there have been some issues in the apps which mean I need to build Wine myself. If you are interested what these issues are, the first issue is that our main Windows product is very complex and CPU intensive. Running on v4.0 is fine for this but on v5.x it crashes all over the place. So, I am continuing with v4.0. The next issue is that running v4.0 on Catalina only shows some folders and files in all the file dialogs. Simply rebuilding the source for v4.0 fixes this problem for me. The last issue is with another of our Windows products which needs ScriptGetCMap() from gdi32.dll, which is not present. Someone submitted a patch for this several months ago but it has never made it in to any release (yet). So I need to apply this patch.

So, I have successfully got Wine building on macOS. And running the build from the source tree works great. But then transferring this build to the macOS app results in being unable to find "ntdll.dll.so". See below.

wine: failed to initialize: dlopen(/usr/local/lib64/wine/ntdll.dll.so, 258): image not found

When I have updated the macOS app created by Wine-Bundler I have simply dropped in the newer Wine binaries and the entire lib64 directory. Just doing this, and nothing else, produces the above error. So, all scripts in the macOS app are the same as before. With the downloaded portable v4.0 it launches fine but with the new build it fails as above.

So, I am thinking that I need to build Wine to be the portable version. But I cannot find out how to do this. I see there are portable versions available for download, so how do you make these releases?

Thanks.

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: How to build the portable version?

Post by Gcenx » Tue Jun 23, 2020 8:16 am

adamAC wrote:
Tue Jun 23, 2020 2:52 am
I have been given the responsibility in my company of getting a couple of our Windows apps working on macOS, so I am obviously trying to use Wine to achieve this. I am also using Wine-Bundler to create proper self-contained macOS apps.
Wine-Bundler doesn't handle the needed dylibs or setting needed exports so it's not really self contained.
adamAC wrote:
Tue Jun 23, 2020 2:52 am
All of that is fine but there have been some issues in the apps which mean I need to build Wine myself. If you are interested what these issues are, the first issue is that our main Windows product is very complex and CPU intensive. Running on v4.0 is fine for this but on v5.x it crashes all over the place. So, I am continuing with v4.0. The next issue is that running v4.0 on Catalina only shows some folders and files in all the file dialogs. Simply rebuilding the source for v4.0 fixes this problem for me. The last issue is with another of our Windows products which needs ScriptGetCMap() from gdi32.dll, which is not present. Someone submitted a patch for this several months ago but it has never made it in to any release (yet). So I need to apply this patch.
As you've managed to compile wine from source you really should make a bugzilla for the CPU issue with your application(s) else it won't get fixed.

The files not showing up is the new security on Catalina, wine64 should be asking for permission to access some resources
adamAC wrote:
Tue Jun 23, 2020 2:52 am
So, I have successfully got Wine building on macOS. And running the build from the source tree works great. But then transferring this build to the macOS app results in being unable to find "ntdll.dll.so". See below.

wine: failed to initialize: dlopen(/usr/local/lib64/wine/ntdll.dll.so, 258): image not found

When I have updated the macOS app created by Wine-Bundler I have simply dropped in the newer Wine binaries and the entire lib64 directory. Just doing this, and nothing else, produces the above error. So, all scripts in the macOS app are the same as before. With the downloaded portable v4.0 it launches fine but with the new build it fails as above.
That error is Linker related I believe, I have a few Winehq esk bundles uploaded https://github.com/Gcenx/macOS_Wine_builds maybe test Wine-Devel-5.11.
Grab the files from /Contents/Resources/wine , I say Wine-Devel-5.11 as I split the dylibs so /lib only contains 32Bit and /lib64 only contains 64Bit
adamAC wrote:
Tue Jun 23, 2020 2:52 am
So, I am thinking that I need to build Wine to be the portable version. But I cannot find out how to do this. I see there are portable versions available for download, so how do you make these releases?
The now deprecated Winehq macOS releases were cross-compiled using Debian stretch, using a copy of the 10.8 SDK and a patched version of Clang-3.4, some of the scripts were uploaded to the old wine-staging teams under wine-packing. But I wouldn't recommend it considering it's vastly outdated

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Tue Jun 23, 2020 9:59 am

Many thanks for your reply.

I have tried your release from GitHub but they all keel over with the CPU issue.

How are you building these releases?

Do you have a v4.0.4 release you have built as the one on WineHQ doesn't work?

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: How to build the portable version?

Post by Gcenx » Wed Jun 24, 2020 9:27 am

adamAC wrote:
Tue Jun 23, 2020 9:59 am
I have tried your release from GitHub but they all keel over with the CPU issue.
Then it’s definite a wine-5.x issue, I’m assuming you didn’t get the other issue as you never mentioned it?
adamAC wrote:
Tue Jun 23, 2020 9:59 am
How are you building these releases?
The releases on GitHub were all built on macOS Mojave using CodeWeavers LLVM/Clang8 with Xcode11.3.1 & a modified 10.14SDK that allows an i386 target.

The build environment used macports in combination with https://github.com/Gcenx/macports-wine

Some macports-ports packages are overrode ether due to upstream issues or the package being incompatible, the MoltenVK/VulkanSDK packages use the prebuilt LunarG package over attempting to build from source with its ever increasing Xcode requirement.

For macOS Mojave the attached patch allows ports to be installed as universal by using a 10.13SDK you need to provide this.

Wine(64) is compiled using macports dependencies with any needed compiler flags.
Configure options I used are listed on GitHub.

After being installed into destroot hardcoded rpaths corrected to relative paths
adamAC wrote:
Tue Jun 23, 2020 9:59 am
Do you have a v4.0.4 release you have built as the one on WineHQ doesn't work?
I haven’t built Wine-4.0.4 as it had came out after 5.0 but I could have my building created it.

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Thu Jun 25, 2020 3:02 am

Many thanks Gcenx.

I was on holiday yesterday so I will pick this up today. I was building Wine on Catalina but I do have another Mac with Mojave so can use that to try to build it. Yes, I could downgrade Catalina but I have 2 Macs specifically for testing/dev on purpose where one is Catalina and the other is pre-Catalina.

From your instructions/guidance, it looks like the crucial line is "After being installed into destroot hardcoded rpaths corrected to relative paths", which will solve my previous problem.
Gcenx wrote:
Wed Jun 24, 2020 9:27 am
Then it’s definite a wine-5.x issue, I’m assuming you didn’t get the other issue as you never mentioned it?
Are you referring to the file dialogs being blank or only certain files and folders being shown?

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Thu Jun 25, 2020 3:56 am

Gcenx wrote:
Wed Jun 24, 2020 9:27 am
After being installed into destroot hardcoded rpaths corrected to relative paths
How is this achieved? Do you have any examples of how this is done?

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: How to build the portable version?

Post by Gcenx » Thu Jun 25, 2020 8:25 am

adamAC wrote:
Thu Jun 25, 2020 3:02 am
I was building Wine on Catalina but I do have another Mac with Mojave so can use that to try to build it. Yes, I could downgrade Catalina but I have 2 Macs specifically for testing/dev on purpose where one is Catalina and the other is pre-Catalina.
If you only need wine64 then there is no need to build on Mojave over Catalina, if you also need wine then building on Mojave would be simpler.
adamAC wrote:
Thu Jun 25, 2020 3:02 am
Gcenx wrote:
Wed Jun 24, 2020 9:27 am
Then it’s definite a wine-5.x issue, I’m assuming you didn’t get the other issue as you never mentioned it?
Are you referring to the file dialogs being blank or only certain files and folders being shown?
The dialogs being blank with winehq/xquartz based builds is related to the version of libpng15 being used, I'd rebuilt it to bundle within my Wineskin fork and no longer have these issues with Winehq releases.

Certain files/folder not being shown is the change in security on macOS Catalina, multiple locations now require permissions to access the following locations;
- Desktop
- Downloads
- Documents
- Music
- Pictures
- Network Drives
adamAC wrote:
Thu Jun 25, 2020 3:56 am
Gcenx wrote:
Wed Jun 24, 2020 9:27 am
After being installed into destroot hardcoded rpaths corrected to relative paths
How is this achieved? Do you have any examples of how this is done?
You need to use install_name_tool to change the hardcoded path from /where/the/libs/are/ to rpath/libary_name.dylib, this combined with a correctly configure rpath during compilation will make a more portable build.

The alternative is configuring DYLD_FALLBACK_LIBRARY_PATH to also include wine/lib(64)

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

adamAC

Post by adamAC » Fri Jul 03, 2020 8:46 am

Hi Gcenx,

Thank you for all your help so far. You have been a star.

I have now managed to successfully build many different versions of Wine on a Mac, and produce "portable" builds. I have then used these builds to successfully create Mac apps.

I am now getting down to the nitty-gritty issues.

The first big issue is that in all file dialogs (open, save, & folder selection), some folders and most files DON'T show when running on Catalina. On Mojave, all works great. You have mentioned before that you believe this is down to a change in the security on Catalina. However, if I simply run "notepad.exe" with the same Wine binary in my Mac app I can see all folders and files. So the problem is only in our Windows app running on Catalina. We are not doing anything special in these dialogs. They work perfectly fine on Mojave but not on Catalina. As these file dialogs work fine in Notepad I don't believe it can be a security issue as you would see the same in both. Do you have any other ideas why this may be?

Thanks.

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: adamAC

Post by Gcenx » Sat Jul 04, 2020 11:47 am

adamAC wrote:
Fri Jul 03, 2020 8:46 am
I have now managed to successfully build many different versions of Wine on a Mac, and produce "portable" builds. I have then used these builds to successfully create Mac apps.
That's good to hear
adamAC wrote:
Fri Jul 03, 2020 8:46 am
I am now getting down to the nitty-gritty issues.

The first big issue is that in all file dialogs (open, save, & folder selection), some folders and most files DON'T show when running on Catalina. On Mojave, all works great. You have mentioned before that you believe this is down to a change in the security on Catalina. However, if I simply run "notepad.exe" with the same Wine binary in my Mac app I can see all folders and files. So the problem is only in our Windows app running on Catalina. We are not doing anything special in these dialogs. They work perfectly fine on Mojave but not on Catalina. As these file dialogs work fine in Notepad I don't believe it can be a security issue as you would see the same in both. Do you have any other ideas why this may be?
It's due to the new permissions within macOS Catalina, you now need permission just to view the locations I mentioned about let alone write to them. macOS Mojave you could view any location but write permissions was restricted to what your user could do without using sudo.

When launching wine(64/32on64) let's say via Terminal the resulting processes will inherit its permissions, if instead wine(64/32on64) is instead launched from within an app bundle that app bundle will require these permissions.

The other annocance that i've encountered was the tccd database can be corrupted causing the app to no request the required the required permissions, also if the code-signature is invalid permission prompts also won't happen.

An interesting effect of the inheritance system is each Port I've created using my Wineskin fork each having its own unique BundleID requires permissions per BundleID, however when launched using PortingKit (a frontend that does all configuration based on scripts that uses my Wineskin fork as the wine backend on macOS) instead the Ports inherited the permissions it has been granted.

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Mon Jul 06, 2020 5:24 am

I forgot to mention that if I build Wine on Catalina then I can see all the files and folders. But if I build it on Mojave then I can't.

I am trying to get to the point of only creating a single build that will work on both OSes (i.e. Catalina and pre-Catalina) but the Catalina build won't work correctly on Mojave. Building it on Mojave will work on Catalina except for this folder and files problem.

So, on Catalina, I have given my Mac app "Full Disk Access", and the same files and folders are still not showing.

My Mac app never requests permission to access anything. Neither does Wine when I execute that from a terminal with my Windows app.

So, I guess the tccd database could be corrupt as you say, whatever that is.

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: How to build the portable version?

Post by Gcenx » Mon Jul 06, 2020 8:42 am

adamAC wrote:
Mon Jul 06, 2020 5:24 am
I forgot to mention that if I build Wine on Catalina then I can see all the files and folders. But if I build it on Mojave then I can't.

I am trying to get to the point of only creating a single build that will work on both OSes (i.e. Catalina and pre-Catalina) but the Catalina build won't work correctly on Mojave. Building it on Mojave will work on Catalina except for this folder and files problem.

So, on Catalina, I have given my Mac app "Full Disk Access", and the same files and folders are still not showing.
This could be due to the quarantine/translocation flag(s) when moving between systems.
adamAC wrote:
Mon Jul 06, 2020 5:24 am
My Mac app never requests permission to access anything. Neither does Wine when I execute that from a terminal with my Windows app.
When wine64 is launched via Terminal it would be the one requesting permissions not wine64, when and app bundle launches wine64 the app bundle Name/BundleID would be what requests the permissions.
adamAC wrote:
Mon Jul 06, 2020 5:24 am
So, I guess the tccd database could be corrupt as you say, whatever that is.
The tccd database contain permissions like access to web camera/microphone/full disk access/etc, here is a python script I forked and removed the location reset since that requires sudo https://gist.github.com/Gcenx/668befa14 ... 589b983f2e

You could build wine64 one time on Catalina by using the following flag -mmacosx-version-min={version_here} for example I'm using -mmacosx-version-min=10.9, just a little warning if your plan is to bundle the dylibs those also need to support the same -mmacosx-version-min= configuration or not contain one.

If your plan is to code-sign and notarize this app you can't use the dylibs from XQuartz as those were not built using a new enough SDK, the lowest SDK allowed to be used currently is the 10.9 SDK

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Tue Jul 07, 2020 6:06 am

Thanks again for all your help and guidance.

I decided to re-install Catalina as I had numerous version of Wine installed and it got quite messy. Also, this will be a good test for end-user testing as this Mac had all the dev tools installed whereas after the re-install it won't.

So, after the re-install I copied my Mac app to Applications and ran it. It died almost instantly. Running it from a terminal produces this error in all the output:
Wine cannot find the FreeType font library. To enable Wine to
use TrueType fonts please install a version of FreeType greater than
or equal to 2.0.5.
In the past I have also installed XQuartx as a pre-requisite for Wine, which worked previously using WineHQ's Mac builds, but this time it didn't work. After installing XQuartz the same error is shown in the terminal. From looking at the XQuartz website it appears it should contain the FreeType font library but there is no sign of it after installation. I have searched the entire hard drive for "freetype" and there are zero results.

I tried building Wine using "--without-freetype", which built fine but then Wine failed to load my Windows app during initialisation (of Wine). It errored on something else to do with fonts, and saying there was a division by zero.

Do you have any advice on how to overcome this latest issue? Then I can get back to trying your other advice.

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: How to build the portable version?

Post by Gcenx » Tue Jul 07, 2020 12:52 pm

adamAC wrote:
Tue Jul 07, 2020 6:06 am
Thanks again for all your help and guidance.

I decided to re-install Catalina as I had numerous version of Wine installed and it got quite messy. Also, this will be a good test for end-user testing as this Mac had all the dev tools installed whereas after the re-install it won't.
I’m assuming you were using brew for dependencies that’s a good idea as brew placing symlinks into /usr/local will make it seems like everything is packaged nicely when that’s not the case.

adamAC wrote:
Tue Jul 07, 2020 6:06 am
So, after the re-install I copied my Mac app to Applications and ran it. It died almost instantly. Running it from a terminal produces this error in all the output:
Wine cannot find the FreeType font library. To enable Wine to
use TrueType fonts please install a version of FreeType greater than
or equal to 2.0.5.
Seems you didn’t provide an rpath configuration at build time so I knows you check /opt/X11/lib for libraries.
adamAC wrote:
Tue Jul 07, 2020 6:06 am
In the past I have also installed XQuartx as a pre-requisite for Wine, which worked previously using WineHQ's Mac builds, but this time it didn't work. After installing XQuartz the same error is shown in the terminal. From looking at the XQuartz website it appears it should contain the FreeType font library but there is no sign of it after installation. I have searched the entire hard drive for "freetype" and there are zero results.

I tried building Wine using "--without-freetype", which built fine but then Wine failed to load my Windows app during initialisation (of Wine). It errored on something else to do with fonts, and saying there was a division by zero.

Do you have any advice on how to overcome this latest issue? Then I can get back to trying your other advice.
Winehq compiles have rpath set for /opt/X11/lib so wine will check the standard locations like /usr/lib and /usr/local/lib etc but will also check /opt/X11/lib.

You need to search for libfreetype to find it.

Yeah don’t built wine without freetype it’s not a good idea.

Use the following exports;

Code: Select all

export C_INCLUDE_PATH="/opt/X11/include"
export LIBRARY_PATH="/opt/X11/lib”
export LFFLAGS=" -Wl,-rpath,/opt/X11/lib”
Now run configure and it will pickup the headers/dylibs included within XQuartx along with what’s found within the macOSX.sdk.

This isn’t perfect but will get you building wine using XQuartz provided files, plus being rpath set so wine64 will know where to find the needed dylibs at run time.

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Wed Jul 08, 2020 7:19 am

Many thanks again for your help.

I have added the following, as you suggested...
Gcenx wrote:
Tue Jul 07, 2020 12:52 pm
Use the following exports;

Code: Select all

export C_INCLUDE_PATH="/opt/X11/include"
export LIBRARY_PATH="/opt/X11/lib”
export LFFLAGS=" -Wl,-rpath,/opt/X11/lib”
...which runs fine on my dev Mac but now when I run Wine on Catalina it fails with the following, almost instantly:
dyld: Library not loaded: @rpath/libwine.1.dylib
Referenced from: /Applications/xxxxxxxx.app/Contents/MacOS/../Resources/wine-home/usr/bin/wine64
Reason: image not found
So, I think this is failing earlier than before.

The file it is moaning about is located in "Resources/wine-home/usr/bin/lib64" in the Mac app. Somehow in making those new exports changes it has lost where these libraries are located.

Any ideas?

Gcenx
Level 5
Level 5
Posts: 467
Joined: Mon Dec 25, 2017 12:11 pm

Re: How to build the portable version?

Post by Gcenx » Wed Jul 08, 2020 9:26 am

adamAC wrote:
Wed Jul 08, 2020 7:19 am
Many thanks again for your help.

I have added the following, as you suggested...
Gcenx wrote:
Tue Jul 07, 2020 12:52 pm
Use the following exports;

Code: Select all

export C_INCLUDE_PATH="/opt/X11/include"
export LIBRARY_PATH="/opt/X11/lib”
export LFFLAGS=" -Wl,-rpath,/opt/X11/lib”
...which runs fine on my dev Mac but now when I run Wine on Catalina it fails with the following, almost instantly:
dyld: Library not loaded: @rpath/libwine.1.dylib
Referenced from: /Applications/xxxxxxxx.app/Contents/MacOS/../Resources/wine-home/usr/bin/wine64
Reason: image not found
So, I think this is failing earlier than before.

The file it is moaning about is located in "Resources/wine-home/usr/bin/lib64" in the Mac app. Somehow in making those new exports changes it has lost where these libraries are located.

Any ideas?
The exports were to be used at configure/make time only, did you also include those within the bash wrapper?

As a workaround you could add the following to your apps bash wrapper;

Code: Select all

export dir=$(dirname "$0")
export DYLD_FALLBACK_LIBRARY_PATH=“/usr/lib:/usr/local/lib:/opt/X11/lib:${dir}/../Resources/wine-home/usr/bin/lib64”
This might be slightly off but you should get the idea, this shouldn’t be required. (SIP will wipe the contents of DYLD_FALLBACK_LIBRARY_PATH and some other exports after spawning a single process)

dir gets the current path of where it’s launched from, meaning the wrapper can be moved to another location/renames and still function that or you hardcoded it but I wouldn’t do that for an app wrapper.

adamAC
Level 1
Level 1
Posts: 9
Joined: Mon Jun 22, 2020 9:56 am

Re: How to build the portable version?

Post by adamAC » Wed Jul 08, 2020 9:37 am

Gcenx wrote:
Wed Jul 08, 2020 9:26 am
The exports were to be used at configure/make time only, did you also include those within the bash wrapper?
I only added them to the Mac I was building Wine on. I made no changes to the Mac running Catalina, except for dropping on the updated Mac app.
Gcenx wrote:
Wed Jul 08, 2020 9:26 am
As a workaround you could add the following to your apps bash wrapper;

Code: Select all

export dir=$(dirname "$0")
export DYLD_FALLBACK_LIBRARY_PATH=“/usr/lib:/usr/local/lib:/opt/X11/lib:${dir}/../Resources/wine-home/usr/bin/lib64”
This might be slightly off but you should get the idea, this shouldn’t be required. (SIP will wipe the contents of DYLD_FALLBACK_LIBRARY_PATH and some other exports after spawning a single process)

dir gets the current path of where it’s launched from, meaning the wrapper can be moved to another location/renames and still function that or you hardcoded it but I wouldn’t do that for an app wrapper.
I have done this but I get exactly the same error. I did slightly modify what you suggested as the lib64 path at the end should be "Resources/wine-home/usr/Lib64", i.e. no "bin".

Would you possibly be able to create me a build of v4.0.4 with XML support for me to test? That may give us some indication as to whether some of my problems are in the build or on Catalina. Thanks.

Post Reply