Performance degradation between wine versions when running inside a docker

[yotamoron]

Hello all.

I have a code that is using nanomsg from a Java application.
We are using a 3rd party provided DLL that runs on Windows, so we are running the entire Java application over wine.
I created a docker image from ubuntu 20.04 with both wine6 and wine8 (I compile both from source on the same docker image that will run the application) - separate images, obviously.
The code is the exact same code, all the configurations are the exact same.
As I don't need many wine modules, I compile with (same for both wine6 and wine8):
--without-alsa --without-capi --without-coreaudio --without-cups --without-dbus --without-fontconfig --without-freetype --without-gphoto --without-gssapi --without-gstreamer --without-krb5 --without-opencl --without-opengl --without-oss --without-pcap --without-pulse --without-sane --without-sdl --without-udev --without-usb --without-v4l2 --without-vulkan

Under wine8, we see a longer tcp socket write time (about 100 microseconds) then under wine6. In our context this is considered a serious degradation.

I couldn't find anything online about this, and reached a dead end in trying to debug the code or read it.

I'd appreciate any pointers to possible reasons or any ideas as to what might cause this.

The docker image build file is (the only difference between version 6 and 8 is in the git fetch/checkout):

Code: Select all

#!/bin/bash

set -e
set -x

export DEBIAN_FRONTEND="noninteractive"
WITHOUT="--without-alsa --without-capi --without-coreaudio --without-cups --without-dbus --without-fontconfig --without-freetype --without-gphoto --without-gssapi --without-gstreamer --without-krb5 --without-opencl --without-opengl --without-oss --without-pcap --without-pulse --without-sane --without-sdl --without-udev --without-usb --without-v4l2 --without-vulkan"

sed -i -E 's/^# deb-src /deb-src /g' /etc/apt/sources.list
apt-get update
apt-get install -y git
apt-get build-dep -y wine
apt-get install -y gcc-multilib

mkdir -p /wine-dirs
git clone git://source.winehq.org/git/wine.git /wine-dirs/wine-source
cd /wine-dirs/wine-source
git fetch origin refs/tags/wine-6.0.1
git checkout tags/wine-6.0.1

dpkg --add-architecture i386
apt-get update
apt-get install -y libx11-dev:i386 libfreetype6-dev:i386

mkdir -p /wine-dirs/build-32
mkdir -p /wine-dirs/build-64

cd /wine-dirs/build-64
# shellcheck disable=SC2086
../wine-source/configure --enable-win64 $WITHOUT
sed -i -E 's/#define HAVE_CLOCK_GETTIME 1//g' include/config.h
make -j "$(nproc)"
make install

cd /wine-dirs/build-32
# shellcheck disable=SC2086
../wine-source/configure --with-wine64=../build-64 $WITHOUT
sed -i -E 's/#define HAVE_CLOCK_GETTIME 1//g' include/config.h
make -j "$(nproc)"
make install

cd /
rm -rf /wine-dirs
apt-get remove -y gcc-multilib git libx11-dev:i386 libfreetype6-dev:i386 acl adwaita-icon-theme autoconf automake autopoint autotools-dev binutils binutils-common binutils-x86-64-linux-gnu bison bsdmainutils build-essential comerr-dev cpp cpp-9 dbus dbus-user-session dconf-gsettings-backend dconf-service dctrl-tools debhelper dh-autoreconf dh-strip-nondeterminism diffstat dpkg-dev dwz file flex fontconfig fontconfig-config fontforge-common fontforge-nox fonts-dejavu-core freeglut3 freeglut3-dev g++ g++-9 gcc gcc-9 gcc-9-base gettext gettext-base gir1.2-glib-2.0 gir1.2-gst-plugins-base-1.0 gir1.2-gstreamer-1.0 gir1.2-ibus-1.0 glib-networking glib-networking-common glib-networking-services groff-base gsettings-desktop-schemas gtk-update-icon-cache hicolor-icon-theme humanity-icon-theme icoutils icu-devtools imagemagick imagemagick-6-common imagemagick-6.q16 intltool-debian iso-codes khronos-api krb5-multidev libaom0 libapparmor1 libarchive-zip-perl libargon2-1 libasan5 libasound2 libasound2-data libasound2-dev libasyncns0 libatk-bridge2.0-0 libatk1.0-0 libatk1.0-data libatomic1 libatspi2.0-0 libavahi-client3 libavahi-common-data libavahi-common3 libavcodec58 libavutil56 libbinutils libblkid-dev libc-dev-bin libc6-dev libcairo-gobject2 libcairo2 libcap2 libcap2-bin libcapi20-3 libcapi20-dev libcc1-0 libcodec2-0.9 libcolord2 libcroco3 libcrypt-dev libcryptsetup12 libctf-nobfd0 libctf0 libcups2 libcups2-dev libcupsimage2 libcupsimage2-dev libdatrie1 libdbus-1-3 libdbus-1-dev libdconf1 libdebhelper-perl libdevmapper1.02.1 libdpkg-perl libdrm-amdgpu1 libdrm-common libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libdrm2 libegl-dev libegl-mesa0 libegl1 libegl1-mesa-dev libelf1 libencode-locale-perl libepoxy0 libevent-2.1-7 libexif-dev libexif12 libexpat1-dev libfaudio-dev libfaudio0 libffi-dev libfftw3-double3 libfile-listing-perl libfile-stripnondeterminism-perl libflac8 libfontconfig1 libfontconfig1-dev libfontforge3 libfreetype-dev libfreetype6 libfreetype6-dev libfribidi0 libgbm1 libgcc-9-dev libgd3 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgdraw6 libgettextpo-dev libgettextpo0 libgif7 libgirepository-1.0-1 libgl-dev libgl1 libgl1-mesa-dev libgl1-mesa-dri libglapi-mesa libgles-dev libgles1 libgles2 libgles2-mesa-dev libglib2.0-0 libglib2.0-bin libglib2.0-data libglib2.0-dev libglib2.0-dev-bin libglu1-mesa libglu1-mesa-dev libglvnd-dev libglvnd0 libglx-dev libglx-mesa0 libglx0 libgmp-dev libgmpxx4ldbl libgnutls-dane0 libgnutls-openssl27 libgnutls28-dev libgnutlsxx28 libgomp1 libgphoto2-6 libgphoto2-dev libgphoto2-port12 libgraphite2-3 libgsm1 libgsm1-dev libgssrpc4 libgstreamer-gl1.0-0 libgstreamer-plugins-base1.0-0 libgstreamer-plugins-base1.0-dev libgstreamer1.0-0 libgstreamer1.0-dev libgtk-3-0 libgtk-3-common libgudev-1.0-0 libharfbuzz0b libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-date-perl libhttp-message-perl libhttp-negotiate-perl libibus-1.0-5 libibus-1.0-dev libice-dev libice6 libicu-dev libicu66 libidn2-dev libieee1284-3 libio-html-perl libio-socket-ssl-perl libip4tc2 libisl22 libitm1 libjbig-dev libjbig0 libjpeg-dev libjpeg-turbo8 libjpeg-turbo8-dev libjpeg8 libjpeg8-dev libjson-c4 libjson-glib-1.0-0 libjson-glib-1.0-common libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkmod2 libkrb5-dev liblcms2-2 liblcms2-dev libldap2-dev libllvm12 liblqr-1-0 liblsan0 libltdl-dev libltdl7 liblwp-mediatypes-perl liblwp-protocol-https-perl liblzma-dev libmagic-mgc libmagic1 libmagickcore-6.q16-6 libmagickwand-6.q16-6 libmount-dev libmp3lame0 libmpc3 libmpdec2 libmpfr6 libmpg123-0 libmpg123-dev libmysqlclient21 libncurses-dev libncurses5-dev libnet-http-perl libnet-ssleay-perl libnuma1 libodbc1 libogg0 libopenal-data libopenal-dev libopenal1 libopengl-dev libopengl0 libopenjp2-7 libopus0 liborc-0.4-0 liborc-0.4-dev liborc-0.4-dev-bin libosmesa6 libosmesa6-dev libout123-0 libp11-kit-dev libpam-systemd libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpcap0.8 libpcap0.8-dev libpci3 libpciaccess0 libpcre16-3 libpcre2-16-0 libpcre2-32-0 libpcre2-dev libpcre2-posix2 libpcre3-dev libpcre32-3 libpcrecpp0v5 libpipeline1 libpixman-1-0 libpng-dev libpng16-16 libproxy1v5 libpthread-stubs0-dev libpulse-dev libpulse-mainloop-glib0 libpulse0 libpython3-stdlib libpython3.8 libpython3.8-minimal libpython3.8-stdlib libquadmath0 libreadline8 librest-0.7-0 librsvg2-2 librsvg2-bin librsvg2-common libsane libsane-common libsane-dev libsdl2-2.0-0 libsdl2-dev libselinux1-dev libsensors-config libsensors5 libsepol1-dev libshine3 libsigsegv2 libsm-dev libsm6 libsnappy1v5 libsndfile1 libsndio-dev libsndio7.0 libsnmp-base libsnmp35 libsoup-gnome2.4-1 libsoup2.4-1 libsoxr0 libspeex1 libspiro1 libssl-dev libstb0 libstdc++-9-dev libsub-override-perl libswresample3 libtasn1-6-dev libthai-data libthai0 libtheora0 libtiff-dev libtiff5 libtiffxx5 libtimedate-perl libtool libtry-tiny-perl libtsan0 libtwolame0 libubsan1 libuchardet0 libudev-dev libunbound8 libuninameslist1 libunwind-dev libunwind8 liburi-perl libusb-1.0-0 libv4l-0 libv4l-dev libv4l2rds0 libv4lconvert0 libva-drm2 libva-x11-2 libva2 libvdpau1 libvkd3d-dev libvkd3d-utils1 libvkd3d1 libvorbis0a libvorbisenc2 libvpx6 libvulkan-dev libvulkan1 libwavpack1 libwayland-bin libwayland-client0 libwayland-cursor0 libwayland-dev libwayland-egl1 libwayland-server0 libwebp6 libwebpmux3 libwrap0 libwww-perl libwww-robotrules-perl libx11-dev libx11-xcb-dev libx11-xcb1 libx264-155 libx265-179 libxau-dev libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-render0 libxcb-shm0 libxcb-sync1 libxcb-xfixes0 libxcb1-dev libxcomposite-dev libxcomposite1 libxcursor-dev libxcursor1 libxdamage1 libxdmcp-dev libxext-dev libxfixes-dev libxfixes3 libxi-dev libxi6 libxinerama-dev libxinerama1 libxkbcommon-dev libxkbcommon0 libxkbfile-dev libxkbfile1 libxml-libxml-perl libxml-namespacesupport-perl libxml-parser-perl libxml-sax-base-perl libxml-sax-perl libxml-simple-perl libxml2 libxml2-dev libxmu-dev libxmu-headers libxmu6 libxpm4 libxrandr-dev libxrandr2 libxrender-dev libxrender1 libxshmfence1 libxslt1-dev libxslt1.1 libxss-dev libxss1 libxt-dev libxt6 libxv-dev libxv1 libxvidcore4 libxxf86dga-dev libxxf86dga1 libxxf86vm-dev libxxf86vm1 libzvbi-common libzvbi0 linux-libc-dev m4 make man-db mime-support mysql-common nettle-dev ocl-icd-libopencl1 ocl-icd-opencl-dev odbcinst odbcinst1debian2 opencl-c-headers pci.ids perl-openssl-defaults pkg-config po-debconf python3 python3-distutils python3-lib2to3 python3-minimal python3.8 python3.8-minimal quilt readline-common shared-mime-info sharutils systemd systemd-sysv systemd-timesyncd tzdata ubuntu-mono ucf udev unicode-data unixodbc-dev uuid-dev x11-common x11proto-core-dev x11proto-dev x11proto-input-dev x11proto-randr-dev x11proto-scrnsaver-dev x11proto-xext-dev x11proto-xf86dga-dev x11proto-xf86vidmode-dev x11proto-xinerama-dev xkb-data xorg-sgml-doctools xtrans-dev xz-utils zlib1g-dev
apt-get autoremove -y

[howtoreader]

Disclaimer: I am very new here.

The wiki has a Performance page.

Now that Wine has matured to the point of running many applications correctly, people are expecting them to run as fast as on Windows. Sadly, this is not always the case. Here are a few notes related to tracking down performance issues.

The content certainly isn't network-centric, and similar to other Wine content it's full of games, but there are links to a few advanced searches of the bug tracker. Have you searched for network performance related tickets there? I'm not sure whether this issue would ultimately get more traction as a forum post or as a ticket: perhaps both?!

I presume since you have arrived at this "100 microsecond" measurement you've at least thought about "profiling" the systems involved, or perhaps you've already run various benchmarks. There are some benchmark/profiler/bottleneck-finding tools listed on the same wiki page. Otherwise people like Brendan Gregg have made decent collections of Linux performance tool info for examining different parts of the OS. Program on JVM on Wine on Container on Host is a large surface to profile. Is TCP socket write time the only notable difference in the profiles of the two systems?

I also take for granted that the container runs on (or has been isolated to) the same host, rather than one of many different container hosts, and I suppose porting this 3rd party (proprietary?) DLL to the same platform to match the native server environment has been considered.

Can you reproduce the issue using freely available code, instead of your specific stack? The simpler code you can use, and post publicly, the more likely someone can try to replicate it themselves.

Edit: I also wonder if the two versions of Wine are configured (or default to) the same Windows version? I've been tinkering with my own system and noticed the Wine config from EPEL default to "windows 7" on my machine, but after banging my head on the wall and building 9 from source it appears to default to "windows 10". Could the default Wine configuration/environment itself be a difference here, or have you made sure those are identical as well?