Can I call Linux .so libraries from a Windows program?

Questions about Wine on Linux
Post Reply
mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Can I call Linux .so libraries from a Windows program?

Post by mhanft » Sat Nov 21, 2020 3:44 am

Hi,

I know that sounds absurd, but I'm looking for a way how to call functions in a Linux .so library from my Windows software which runs with WINE.

The situation is as follows:
  • I have a set of .so libraries for Linux, from another manufacturer, this means no source code.
  • I have my own Windows program (source code available, of course) which runs with WINE.
The .so libraries export, for example, a function int DoSomething(char *string1, char* string2).

It there any change to call that function from a Windows program which runs with WINE?

If it's technically absolutely impossible, I could create an "intermediate layer" - this means, a Linux program (which calls the so. libraries just normally) which is called from my Windows program (which would change/reduce the question to "can I run Linux programs/scripts from a WINE Windows program?" - just like calling "ls -l" or so).

Thank you in advance,

-Matt

PS: Sorry, this should have gone to the Linux sub-forum, but I can't find a way to delete nor to move...

Bamm
Level 4
Level 4
Posts: 110
Joined: Thu May 22, 2008 3:18 am
Location: Manila, Philippines

Re: Can I call Linux .so libraries from a Windows program?

Post by Bamm » Sat Nov 21, 2020 7:57 am

It is possible to call a Linux executable from a Windows program running in Wine. So the easiest way would probably be to create a small command in Linux and call it from your Windows program. I even succeeded in calling a bash shell script from a Windows app.

mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Re: Can I call Linux .so libraries from a Windows program?

Post by mhanft » Sat Nov 21, 2020 11:14 am

Sounds interesting - but how? Windows system calls like "ShellExecute" etc. leave me within the Windows subsystem ("C:\blabla" which is "~/.wine/drive_c/blabla" in the Linux world). How can I call, let's say, "/usr/local/bin/bashscript.sh" from a (WINE) Windows program?

Or do I have to copy (or softlink) any Linux script into the "Windows world", and Windows execute functions recognize automagically if it's another Windows program or some Linux thing?

I guess I'll have to do some experimental physics... ;)

spoon0042
Level 5
Level 5
Posts: 456
Joined: Thu Dec 24, 2009 11:00 am

Re: Can I call Linux .so libraries from a Windows program?

Post by spoon0042 » Sun Nov 22, 2020 8:01 pm

There's an entry in the FAQ about running linux programs from windows ones, that's probably where you want to start: https://wiki.winehq.org/FAQ#How_do_I_la ... ication.3F

(That also mentions winepath which converts between windows and unix names.)

As for directly calling linux libraries... it may be possible in theory. At least I remember seeing warnings that malware could, again theoretically make linux syscalls. Or maybe it's possible (intentionally) with winelib, I'm not quite sure how that works.

... actually now that I look at it that seems to be right? Still seems like dark sorcery but here you go:
What you gain by recompiling your application with Winelib is the ability to make calls to Unix APIs, directly from your Windows source code.

mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Re: Can I call Linux .so libraries from a Windows program?

Post by mhanft » Mon Nov 23, 2020 3:13 am

Thanks a lot - that seems to be exactly what I was looking for, especially chapter 5:
https://wiki.winehq.org/Winelib_User%27 ... nelib_DLLs
For one reason or another you may find yourself with a Linux library that you want to use as if it were a Windows DLL. There are various reasons for this including the following:
  • You are porting a large application that uses several third-party libraries. One is available on Linux but you are not yet ready to link to it directly as a Linux shared library
  • ...
They even claim "The process for dealing with these situations is actually quite simple", but the description later on sounds a bit complicated to me... but I'll give it a try anyway ;)

Thanks again!

mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Re: Can I call Linux .so libraries from a Windows program?

Post by mhanft » Fri Nov 27, 2020 7:21 am

Ok, now I have dealt a lot with winemaker, winebuild, winegcc and all that... and read all the documentation ;)

But for a simple demo/test, all that is taking a sledgehammer to crack a nut (while winemaker doesn't seem to work as expected: for example, ignoring the "--wine32" parameter and more).

Now I just want to understand the most simple workflow possible - by creating a "built-in" DLL function which adds two Integers. Should be really simple - shouldn't it?

I have written a pretty short Windows program (with Delphi 7) like this:

Code: Select all

myPlusLib:=LoadLibrary('plus.dll');
myPlusFunc:=GetProcAddress(myPlusLib, 'plus');
Writeln('Result of 1+2=', myPlusFunc(1, 2));
FreeLibrary(myPlusLib)
and parallel to this, a Linux library like this

Code: Select all

#include <windef.h>
#include <stdio.h>

int WINAPI plus(int a, int b)
{
	printf("Here is plus(%d, %d)\n", a, b);
	return a+b;
}

int DllMain(void)
{
	printf("Here is DllMain\n");
	return 0;
}

int WinMain(void)
{
	printf("Here is WinMain\n");
	return 0;
}
with printf just to see what happens.

I compiled this with

winegcc -m32 -o plus.dll.so -shared plus.c

and get two warnings

Code: Select all

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /usr/lib/wine/libwinecrt0.a(dll_entry.o): warning: relocation in read-only section `.text'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: warning: creating DT_TEXTREL in a shared object
and a file plus.dll.so which I copy to /usr/lib/wine.

Now when I start wine TestPlus, I get

Code: Select all

0009:fixme:msg:ChangeWindowMessageFilterEx 0x20040 400 1 (nil)
Here is DllMain
Here is DllMain
DLL initialization failed (0x0000045A)
so my plus.dll.so seems to be found and used, but DllMain seems to be called twice, and then "initialization" fails (whatever there may be initialized).

With WINEDEBUG="+all", I found something like

Code: Select all

4096.717:0008:0009:trace:module:load_builtin_callback loaded plus.dll-yTZUi0.spec 0x18b220 0x7c2b0000
0009: load_dll( dbg_offset=0, base=7c2b0000, name=0018b248, dbg_size=0, filename=L"C:\\windows\\system32\\plus.dll-yTZUi0.spec" )
0009: load_dll() = 0
4096.717:0008:0009:trace:loaddll:load_so_dll Loaded L"C:\\windows\\system32\\plus.dll-yTZUi0.spec" at 0x7c2b0000: builtin
4096.717:0008:0009:trace:heap:RtlFreeHeap (0x110000,70000062,0x176870): returning TRUE
4096.717:0008:0009:trace:module:load_dll Loaded module L"\\??\\C:\\windows\\system32\\plus.dll" at 0x7c2b0000
4096.718:0008:0009:trace:heap:RtlFreeHeap (0x110000,70000062,0x17b660): returning TRUE
4096.718:0008:0009:trace:module:process_attach (L"plus.dll-yTZUi0.spec",(nil)) - START
4096.718:0008:0009:Call PE DLL (proc=0x7c2b5610,module=0x7c2b0000 L"plus.dll-yTZUi0.spec",reason=PROCESS_ATTACH,res=(nil))
4096.718:0008:0009:Ret  PE DLL (proc=0x7c2b5610,module=0x7c2b0000 L"plus.dll-yTZUi0.spec",reason=PROCESS_ATTACH,res=(nil)) retval=0
4096.718:0008:0009:Call PE DLL (proc=0x7c2b5610,module=0x7c2b0000 L"plus.dll-yTZUi0.spec",reason=PROCESS_DETACH,res=(nil))
4096.718:0008:0009:Ret  PE DLL (proc=0x7c2b5610,module=0x7c2b0000 L"plus.dll-yTZUi0.spec",reason=PROCESS_DETACH,res=(nil)) retval=0
4096.718:0008:0009:warn:module:process_attach Initialization of L"plus.dll-yTZUi0.spec" failed
4096.718:0008:0009:trace:module:process_attach (L"plus.dll-yTZUi0.spec",(nil)) - END
4096.718:0008:0009:trace:module:LdrUnloadDll (0x7c2b0000)
4096.718:0008:0009:trace:module:LdrUnloadDll (L"plus.dll-yTZUi0.spec") - START
4096.718:0008:0009:trace:module:MODULE_DecRefCount (L"plus.dll-yTZUi0.spec") ldr.LoadCount: 0
4096.718:0008:0009:trace:module:free_modref  unloading L"C:\\windows\\system32\\plus.dll-yTZUi0.spec"
which leaves me pretty clueless.

What am I doing wrong, what have I forgotten?

Thanks a lot,
-Matt

madewokherd
Level 3
Level 3
Posts: 84
Joined: Mon Jun 02, 2008 5:03 pm
Contact:

Re: Can I call Linux .so libraries from a Windows program?

Post by madewokherd » Fri Nov 27, 2020 11:11 pm

DllMain is called twice because of the special DLL_WINE_PREATTACH reason. If you return FALSE, Wine will use a native version of the DLL if available. This only really makes sense when replacing an existing DLL.

You probably shouldn't be calling any functions in DLL_WINE_PREATTACH. Even though printf seems to work in this case, my understanding is that linking hasn't fully completed at that point.

You have to return TRUE from DllMain when the reason is DLL_PROCESS_ATTACH for the DLL to load successfully.

BTW, the Linux "int" may not be the same as Win32 "int", in particular they are different on x86_64. You should use the all-caps version INT when you need to make sure it's compatible with your Windows code.

mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Re: Can I call Linux .so libraries from a Windows program?

Post by mhanft » Fri Dec 04, 2020 1:15 pm

Ok, meanwhile, I found out some more things... but still no success ;(

I wondered why DllMain is called twice - and in parallel, I was looking for an "unload DLL" event. So I just tried to use parameters with DllMain (didn't find any documentation for that, nowhere!), and the second parameter is actually 1 for "load" and 0 for "unload"- bingo!

So far, so good.

But: My function still cannot be found from Windows programs. What am I missing?

My plus.c now looks like this:

Code: Select all

#include <windef.h>
#include <stdio.h>

INT WINAPI plus(INT a, INT b)
{
        printf("Here is plus(%d, %d)\n", a, b);
        return a+b;
}

BOOL WINAPI DllMain(INT par1, INT par2)
{
        printf("Here is DllMain, par1=%d, par2=%d\n", par1, par2);
        return TRUE;
}
and if I run this Windows program:

Code: Select all

  myPlusDll:=LoadLibrary('plus.dll');
    Writeln('DLL is open, handle=', Format('%u', [myPlusDll]));
    myPlusFunc:=GetProcAddress(myPlusDll, 'plus');
    if Assigned(myPlusFunc) then
      Writeln('Result of 1+2=', myPlusFunc(1, 2))
    else
      Writeln('address not found');
    Writeln('FreeLibrary='+BoolToStr(FreeLibrary(myPlusDll), True));
I get

Code: Select all

0009:fixme:msg:ChangeWindowMessageFilterEx 0x20040 400 1 (nil)
Here is DllMain, par1=2084700160, par2=1
DLL is open, handle=2084700160
address not found
Here is DllMain, par1=2084700160, par2=0
FreeLibrary=True
and with an even more simple Windows program

Code: Select all

function myPlusFunc(a, b: Integer): Integer; stdcall; external 'plus.dll' name 'plus';
begin
  Writeln('Result of 1+2=', myPlusFunc(1, 2))
end.
I get

Code: Select all

Here is DllMain, par1=2119106560, par2=1
wine: Call from 0x7bc79a01 to unimplemented function plus.dll.plus, aborting
wine: Unimplemented function plus.dll.plus called at address 7BC79A01 (thread 0009), starting debugger...
002c:fixme:dbghelp:elf_search_auxv can't find symbol in module
Running nm -D plus.dll.so shows that my plus symbol is actually in the library:

Code: Select all

         w __cxa_finalize
00013118 B _end
         w __gmon_start__
         w _ITM_deregisterTMCloneTable
         w _ITM_registerTMCloneTable
0001155d T plus
         U printf
         U __wine_dll_register
I do think I'm already close to it, but one tiny detail is obviously still missing... which one is it?!

Thanks again,
-Matt

madewokherd
Level 3
Level 3
Posts: 84
Joined: Mon Jun 02, 2008 5:03 pm
Contact:

Re: Can I call Linux .so libraries from a Windows program?

Post by madewokherd » Sat Dec 05, 2020 5:10 pm

MSDN has documentation for DllMain: https://docs.microsoft.com/en-us/window ... ls/dllmain

In order to export your function from a winelib .dll, you must include it in a .spec file: https://wiki.winehq.org/Winelib_User%27 ... _Spec_file

For this particular function I would use a spec file entry of:
@ stdcall plus(long long)

mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Re: Can I call Linux .so libraries from a Windows program?

Post by mhanft » Sun Dec 06, 2020 6:14 am

Ok, now it works as expected - thanks a lot!!!

Some notes for posterity, just in case anyone finds this thread:
  • I couldn't find any DllMain documentation because I thought it was a WINE-specific function. As soon as I had recognized that it's a Windows function, everything was clear.
  • I already had found those .spec files, but didn't know what to do with them. Just existing in the working directory didn't suffice; you have to specifiy it as another input file for winegcc. So my winegcc command is now:
    winegcc -m32 -o plus.dll.so -shared -ldl plus.c plus.dll.spec
    (regarding -ldl see next item)
  • if all else fails (and for some possibly other/extended information exchange), meanwhile I have found out another way to communicate the function addresses (or any other data) to the calling process:
    • in DllMain, create a Shared Memory (by CreateFileMapping and MapViewOfFile)
    • then, open the desired .so library (could be itself, or any other arbitrary .so library) with dlopen and use dlsym to get the function addresses
    • write the function names and addresses into the Shared Memory (for example, like an INI file, such as [functions]plus=239487329)
    • in the main Windows program, after LoadLibrary('plus.dll'), open the same Shared Memory and get that list of function names and addresses.
    So the main Windows program would know that the address of the plus function is 239487329 and could call that function.
Since what I finally want to do is kind of a WINE wrapper for a binary third-party .so library, just passing the original function addresses by Shared Memory (as described above) might be easier than to write a "wrap function" for each and every function which is contained in the original third-party .so library (assumed that the parameter/result types are compatible, of course). Or maybe I could just link those third-party .so libraries to an empty DllMain and export them directly to WINE programs just by a matching .spec file? Will try this. And where to put all that stuff and how to find it (and/or do I need LD_LIBRARY_PATH) are some questions left... but I think now I'm on my way - the "proof of concept" did succeed anyway ;)

Thanks again!
-Matt

madewokherd
Level 3
Level 3
Posts: 84
Joined: Mon Jun 02, 2008 5:03 pm
Contact:

Re: Can I call Linux .so libraries from a Windows program?

Post by madewokherd » Tue Dec 08, 2020 12:38 pm

Maybe winegcc should check for -shared without a spec file and fail in this case. A dll without any exports would typically not be very useful, and one could provide an empty spec file if needed.

I've seen C# developers use something similar to that "shared memory" technique to create an interface between managed and native code. The advantage is that once you communicate that one address, the rest of the interface isn't platform-specific at all.

mhanft
Level 2
Level 2
Posts: 10
Joined: Fri Sep 15, 2017 1:46 pm

Re: Can I call Linux .so libraries from a Windows program?

Post by mhanft » Wed Dec 09, 2020 5:19 am

Meanwhile everything is working as expected - with your help; thank you again!

I didn't finally need shared memory or any other "workarounds". I just wrote a small wrapper for all functions of the third-party library, and all I had to take care of was converting Windows filenames to Linux filenames (which was a bit tricky because wine_get_unix_filename needs 16-bit chars, but common "widestring" in Linux seems to be 32-bit chars).

So the whole project is (nearly) finished now. Here's the summary how I made it (if someone should come across this thread later on):
  • The third-party manufactorer delivers a Windows version of his library (ericapi.dll) as well as a Linux version (libericapi.so).
  • Until last year's version, the Windows dll was fully compatible with WINE, so no need for extra work at all.
  • The current year's update broke with WINE compatibilty (for whatever reason - seems to claim 1 GB heap space which WINE won't grant - or something like that, see this thread)
  • Of course, the third-party manufacturer (which is, by the way, the German tax authority) says "if you use Linux, just use our Linux libs instead of Windows libs - we don't offer WINE compatibility support".
  • So i put the Linux libs into some subdirectory called "native" and wrote an own wrapper "ericapi.c" like this:
    • In DllMain(DLL_PROCESS_ATTACH), I use GetModuleFileName to find out the filename of the (my) ericapi.dll(.so), append /native to it and use dlopen to open the native libericapi.so library.
    • For example, there is a function EricInitialize(char *pluginPath, char *logPath) which I wrapped as follows:

      Code: Select all

      typedef ERICAPI_IMPORT int STDCALL (*TEricInitialize)(char *pluginPath, char *logPath);
      static TEricInitialize _EricInitialize;
      ...
      _EricInitialize=(TEricInitialize)dlsym(handle, "EricInitialize");
      ...
      INT WINAPI EricInitialize(char *pluginPath, char *LogPath)
      {
        (...convert pathnames to Linux...)
        return (*_EricInitialize)(pluginPath, logPath);
      }
    • in ericapi.dll.spec, there is
      @ stdcall EricInitialize(str str)
    • and all this is compiled by
      winegcc -m32 -o ericapi.dll -shared -ldl ericapi.c ericapi.dll.spec
      mv ericapi.dll.so ericapi.dll
    • and I put the resulting ericapi.dll to the position where the former Windows dll was, and the Windows main program doesn't even notice that it talks to a Linux .so library instead of the original Windows .dll library. Voilà!
I'd like to do the same for macOS, but I'm afraid I can't, because of several reasons:
  • I don't have a Mac.
  • The native "ericapi" lib for Mac is only available for 64-bit, and I can't link 32-bit and 64-bit together.
  • would all this run on the new Apple M1 CPU at all?
So, just two minor points still remain:
  • I know I can find out if my Windows program runs with WINE by looking for a function wine_get_version in ntdll.dll, but can I somehow find out if the "host" is Linux or Mac? I'd like to display some waning message to Mac users only ("Attention! The next update won't run any more! Switch to Windows or Linux!" or something like that).
  • From the Windows DLLs, I can get the version info by calling GetFileVersionInfo. Of course, this is not possible for my wrapper DLL(.so). But as far as I have read, I think that this version (and other) info is just kind of a resource which I could compile with wrc; I just haven't found out the correct syntax for the .rc file and how to put it all together. (Or maybe this wouldn't work at all anyway.) ;)
In any case, the (WINE's) Linux future of my software project is now assured .Thanks again!

-Matt

Post Reply