Ubuntu C ReadFileEx/WriteFileEx problems

Questions about Wine on Linux
Locked
Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

Wine version: 1.6.2
OS: Ubuntu 14.04 LTS 64-bit
There are no additional changes to winecfg or registry, and winetricks is not used. Wine prefix is set in default 64 bit.

I'm attempting to port a C program which makes extensive use of Windows libraries into Linux. The program uses the Reliable Asynchronous Transfer Protocol (RATP) that opens an Universal Asynchronous Receiver/Transmitter (UART) serial connection with an external device (hardware) to the computer.

The program itself uses ReadFileEx to read from the device into a ReadBuffer, and the CompletionRoutine of the ReadFileEx function pushes the contents of ReadBuffer into the RecieveBuffer. It's with the data in the RecieveBuffer that error checking, searching for Syn byte, data manipulation, etc is performed.

Vice-versa the WriteFileEx function writes to the device from a PendingBuffer the contents of a TransferBuffer depending on a flag, and the CompletionRoutine of the WriteFileEx function manipulates this flag. The TransferBuffer's contents is manipulated by the rest of the program to create the correct packets to transmit.

An always on loop in the main puts the program into an Alert-able wait state with SleepEx(1u, TRUE) and checks for events to process. The program also uses a Semaphore to lock/unlock the functions called by both CompletionRoutines.

The program expects specific packets to be read after transmitting into the device specific packets, and does its state calculations, ACK/SYNC, time outs, and re-transmissions in chronological order as a part of the start up sequence.

The problem I've been having is this:

On Windows, the expected behavior when compiling the C code with Microsoft Visual Studio 2010 is that ReadFileEx is passed a nNumberOfBytesToRead parameter of x. The function completes and returns success, GetLastError returns 0. The CompletionRoutine of ReadFileEx is called afterwards with a dwNumberOfBytesTransfered which is <=x that is the expected amount of data to be read from the device.

E.g.
READ START 32
READ END 32
System error: 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (contents of ReadBuffer)

event
WRITE START 7
WRITE END 7
System error: 0
<---------W1-------->

WRITE CompletionRoutine 7
Read CompletionRoutine 4
<---------R1---------> (values moved by ReadFileEx CompletionRoutine into RecieveBuffer)

READ START 32
READ END 32
System error: 0
<---------R1---------> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

event
Read CompletionRoutine 6
<---------R2---------->

READ START 32
READ END 32
System error: 0
<---------R2----------> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

event
ReadCompletionRoutine 8
<---------R3---------->

READ START 32
READ END 32
System error: 0
<---------R3----------> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
event

Done

On Ubuntu compiling with winemaker and then running the host.exe file, ReadFileEx is passed a nNumberOfBytesToRead parameter of x. The function completes and returns success, GetLastError returns 0. The CompletionRoutine of ReadFileEx is called afterwards with a dwNumberOfBytesTransfered which is always x. This throws the synchronization out of order since it appears that the ReadFileEx CompletionRoutine does not get called unless the full x amount of data is read, cutting off packets in the middle, overwriting good packets, or breaks synchronization with the respective WriteFileEx command (multiple WriteFileEx commands may have to occur before the full x Bytes can be read).

E.g (same block of code)
READ START 32
READ END 32
System error: 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (contents of ReadBuffer)
event
WRITE START 7
WRITE END 7
System error: 0
<---------W1-------->
WRITE CompletionRoutine 7
Read CompletionRoutine 32
<---------R1---------><-------------R2-----------><-----------R3(?)---------<->---------R3(?)-------->
*Received messages can share a few byte values, and the sync byte is always the same value, so it's difficult to discern where the overlap is
READ START 32
READ END 32
System error: 0
<---------R1---------><-------------R2-----------><-----------R3(?)---------<->---------R3(?)-------->
event

Done
The data itself that is read on Ubuntu-Wine correspond to the correct packets when cross referenced with windows, but it is very difficult to recover the specific packets as well as prevent the possible cutting off and overwriting of good packets.

Is there something specific about how ReadFileEx finds the "end of line" or "end of data" when reading from a device which is different from how it is handled in Wine? Or is this an issue with the Overlapped structure or alertable wait states itself? What would be a potential work around other than completely re-writing the code with out use of the Overlapped asynchronous structure?

The Overlapped Structure is declared / initialized as thus:

Code: Select all

typedef struct TAG_UARTA
{
          void* FileHandle
          BOOLEAN PortOpen

          OVERLAPPED OverlappedR;
          OVERLAPPED OverlappedT;
} UARTAFunction;

void UARTAsync_Init()
{
    memset(&itsUARTAsync.OverlappedT, 0, sizeof(itsUARTAsync.OverlappedT));
    itsUARTAsync.OverlappedT.Offset = 4096;
    itsUARTAsync.OverlappedT.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
    
    memset(&itsUARTAsync.OverlappedR, 0, sizeof(itsUARTAsync.OverlappedR));
    itsUARTAsync.OverlappedR.Offset = 4096;
    itsUARTAsync.OverlappedR.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
}
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by dimesio »

Your Wine version is over a year old; start by upgrading to the latest development release, which just came out today. If the problem is still present in 1.7.38, you might want to ask this on the developer's mailing list. Developers rarely read the forum.
Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

dimesio wrote:Your Wine version is over a year old; start by upgrading to the latest development release, which just came out today. If the problem is still present in 1.7.38, you might want to ask this on the developer's mailing list. Developers rarely read the forum.
Only reason why I chose to use 1.6.2 is because it's still listed as the stable build, but I will give it a shot.
Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

dimesio wrote:Your Wine version is over a year old; start by upgrading to the latest development release, which just came out today. If the problem is still present in 1.7.38, you might want to ask this on the developer's mailing list. Developers rarely read the forum.
Which mailing list should I submit my question to? Since wine-devel specifies that it is not for bug reports or user support questions.
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by dimesio »

I meant wine-devel; filing a bug (if one doesn't already exist) would be another alternative.
Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

Update on 1.6.2:

When COMMTIMEOUTS.ReadIntervalTimeout is set to MAXDWORD with 0 values for both ReadTotalTimeoutConstant and ReadTotalTimeoutMultiplier to force the read operation to return immediately, the CompletionRoutine does return a smaller chunk of data instead of the full x, problem being that sometimes the read data cuts off before the end of the entire packet.

E.g.
Expected:
READ START 32
READ END 32
System error: 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (contents of ReadBuffer)
……
Read CompletionRoutine 4
<---------R1--------->

Actual:
READ START 32
READ END 32
System error: 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (contents of ReadBuffer)
……
Read CompletionRoutine 4
<---------R1---

Original timeouts parameters:

Code: Select all

COMMTIMEOUTS timeouts = {0};
timeouts.ReadIntervalTimeout              = 1;
timeouts.ReadTotalTimeoutConstant          = 0;
timeouts.ReadTotalTimeoutMultiplier        = 0;
timeouts.WriteTotalTimeoutConstant         = 0;
timeouts.WriteTotalTimeoutMultiplier       = 0;
Another problem I found was that if the external device is not turned on but the USB is plugged in,

On windows WriteFileEx would return success but with an error code 87 and execute the callback, and each loop operated the same.

On Ubuntu-Wine it would return success with error code 0 once and execute the callback, then the second time it would also return success with error code 0 completion routine would not be executed and the program breaks down from that point.
Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

Update: Problem persists identically in 1.7.34.
User avatar
dimesio
Moderator
Moderator
Posts: 13367
Joined: Tue Mar 25, 2008 10:30 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by dimesio »

Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

After further analysis, it seems that a specific value (0X11U in hexadecimal or 17 in decimal) is always missing or not read by the ReadFileEx function. For instance when querying the XY coordinates set by the device, reading byte by byte (ReadFileEx with a nNumberOfBytesToRead of 1) recursively always fails when it would get to a value of 0x11U or 17.
e.g. terminal output when steadily increasing the coordinates and reading from the device

Normally each packet consists of 11 bytes:

syn sequence datalength checksum
55 4a 5 c7

data(ID (3 bytes) x y)
8b 0 0 f a

checksum high / low
15 c1
Sample output:

55 4a 5 c7 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 52 5 67 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 5a 5 e2 8b 0 0 e a ba 82 XY Input: (14, 10)
55 62 5 8 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 6a 5 8d 8b 0 0 e a ba 82 XY Input: (14, 10)
55 72 5 2d 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 7a 5 a8 8b 0 0 10 a 51 e8 XY Input: (16, 10)
55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72

The final line has the 10 byte message
55 82 5 d1 8b 0 0 b 6e 72
repeated 5 times (number of times the program will check for a message before throwing error)
This is evident elsewhere in the code as well where a fixed 0x11U or 17 value fails to be read as a part of the start up sequence of the program.

Any idea what could be causing this? The only possible connection I can find is that in the ASCII table 0x17U or 23 stands for End transmission blocks. Perhaps this is in connection with all the aforementioned problems experienced.
Tester
Level 2
Level 2
Posts: 10
Joined: Thu Mar 05, 2015 4:50 pm

Re: Ubuntu C ReadFileEx/WriteFileEx problems

Post by Tester »

Tester wrote:After further analysis, it seems that a specific value (0X11U in hexadecimal or 17 in decimal) is always missing or not read by the ReadFileEx function. For instance when querying the XY coordinates set by the device, reading byte by byte (ReadFileEx with a nNumberOfBytesToRead of 1) recursively always fails when it would get to a value of 0x11U or 17.
e.g. terminal output when steadily increasing the coordinates and reading from the device

Normally each packet consists of 11 bytes:

syn sequence datalength checksum
55 4a 5 c7

data(ID (3 bytes) x y)
8b 0 0 f a

checksum high / low
15 c1
Sample output:

55 4a 5 c7 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 52 5 67 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 5a 5 e2 8b 0 0 e a ba 82 XY Input: (14, 10)
55 62 5 8 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 6a 5 8d 8b 0 0 e a ba 82 XY Input: (14, 10)
55 72 5 2d 8b 0 0 f a 15 c1 XY Input: (15, 10)
55 7a 5 a8 8b 0 0 10 a 51 e8 XY Input: (16, 10)
55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72 55 82 5 d1 8b 0 0 b 6e 72

The final line has the 10 byte message
55 82 5 d1 8b 0 0 b 6e 72
repeated 5 times (number of times the program will check for a message before throwing error)
This is evident elsewhere in the code as well where a fixed 0x11U or 17 value fails to be read as a part of the start up sequence of the program.

Any idea what could be causing this? The only possible connection I can find is that in the ASCII table 0x17U or 23 stands for End transmission blocks. Perhaps this is in connection with all the aforementioned problems experienced.
The 0x11U issue confirmed to be a software flow control issue, fixed with stty manipulations.
Locked