WINE fails in directories with Question Marks in name

[stvs]

Is there a way to tell WINE that question marks in directory names are okay? WINE fails whenever I ask it to access a file from within a directory with a question mark in its name.

Specifically, my HTPC marks all commercials using WINE and Comskip. Whenever the schedule includes a tv show with a question mark in its title, WINE fails. Here are the logs that illustrate the failure.

Script log that calls WINE:

Code: Select all

Sun Sep  4 23:00:24 2011 - Changing directory to /Volumes/Macintosh HD2/Documents/EyeTV Archive/Curiosity_ Is There a Parallel Universe?.eyetvsched
Sun Sep  4 23:00:24 2011 - Running: /usr/bin/nice -n 14 "/opt/local/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "00000000141472b8.mpg" > /Users/name/Library/Logs/ETVComskip/336884408_comskip.log 2>&1
Sun Sep  4 23:02:11 2011 - Return code is: 768, 0x300
Sun Sep  4 23:02:11 2011 - Error code is: 3, 0x3

WINE's output log:

Code: Select all

Warning: could not find DOS drive for current working directory '/Volumes/Macintosh HD2/Documents/EyeTV Archive/Curiosity_ Is There a Parallel Universe?.eyetvsched', starting in the Windows directory.
The commandline used was:
"Z:\Library\Application Support\ETVComskip\comskip\comskip.exe" "--ini=/Library/Application Support/ETVComskip/comskip/comskip.ini" 00000000141472b8.mpg

Opening 00000000141472b8.mpg
Comskip 0.80.042, made using:
mpeg2dec-0.4.0 - by Michel Lespinasse <[email protected]> and Aaron Holtzman
No such file or directory - could not open file 00000000141472b8.mpg

[Martin Gregorie]

On Mon, 2011-09-12 at 14:32 -0500, stvs wrote:

Is there a way to tell WINE that question marks in directory names are
okay?

More specifically, you can easily create a file containing a question
mark, e.g. "doc?.txt", and you can then create a file called "doca.txt".
No problem. However, if you try to reference an existing file and there
is more than one file whose names are identical except where the
question mark occurs the open will fail because the string you gave it
matches more than one file and so the shell doesn't know which file to
select. You can escape a character to remove its special properties or
enclose the string containing it in single quotes. The common characters
that this applies to are:

? - the shell treats this as a single character wild card
* - treated as a wild card matching zero or more characters
space - the shell uses space as an argument delimiter. Actually, it
treats whitespace (space, tab and newline) as argument
delimiters.

Similar rules apply to using semicolons, single and double quotes and
ampersands in file names.

Martin

Code:
Sun Sep 4 23:00:24 2011 - Changing directory to /Volumes/Macintosh HD2/Documents/EyeTV Archive/Curiosity_ Is There a Parallel Universe?.eyetvsched
Sun Sep 4 23:00:24 2011 - Running: /usr/bin/nice -n 14 "/opt/local/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "00000000141472b8.mpg" > /Users/name/Library/Logs/ETVComskip/336884408_comskip.log 2>&1
Sun Sep 4 23:02:11 2011 - Return code is: 768, 0x300
Sun Sep 4 23:02:11 2011 - Error code is: 3, 0x3




WINE's output log:


Code:
Warning: could not find DOS drive for current working directory '/Volumes/Macintosh HD2/Documents/EyeTV Archive/Curiosity_ Is There a Parallel Universe?.eyetvsched', starting in the Windows directory.
The commandline used was:
"Z:\Library\Application Support\ETVComskip\comskip\comskip.exe" "--ini=/Library/Application Support/ETVComskip/comskip/comskip.ini" 00000000141472b8.mpg

Opening 00000000141472b8.mpg
Comskip 0.80.042, made using:
mpeg2dec-0.4.0 - by Michel Lespinasse <[email protected]> and Aaron Holtzman
No such file or directory - could not open file 00000000141472b8.mpg

[stvs]

You can escape a character to remove its special properties or
enclose the string containing it in single quotes.

Great, but how do I do that when I'm in a DIRECTORY that contains a special character? This is how this problem arises: there is a directory called 'foo?' in which there is the file bar.mpg. This fails:

$ cd 'foo?'
$ wine cmd.exe bar.mpg

WINE is unable to operate within the directory named 'foo?'. There is no way for me to escape the directory's name in the (scripted) call to wine.

[Martin Gregorie]

On Mon, 2011-09-12 at 16:58 -0500, stvs wrote:

You can escape a character to remove its special properties or
enclose the string containing it in single quotes.

Great, but how do I do that when I'm in a DIRECTORY that contains a special character? This is how this problem arises: there is a directory called 'foo?' in which there is the file bar.mpg. This fails:

$ cd 'foo?'
$ wine cmd.exe bar.mpg

WINE is unable to operate within the directory named 'foo?'. There is
no way for me to escape the directory's name in the (scripted) call to
wine.

Try this (tested, saved as xtest and made executable):

<code>
#!/bin/bash
y=$(echo "$1" | sed -e 's/\?/\\?/')
echo "y=$y"
</code>

$ xtest foo?
y=foo\?

so the following should work too:

#!/bin/bash
y=$(echo "$1" | sed -e 's/\?/\\?/')
cd "$y"
wine cmd.exe bar.mpg

Martin

[stvs]

Thanks for the suggestion, but that wil fail because it is wine that fails when executed from within a directory containing a special character. Of course you can escape the character in a shell script, but that won't help wine when it attempts to access the directory './foo?'. E.g., see the comskip logs above.

[Frédéric Delanoy]

On Mon, Sep 12, 2011 at 23:58, stvs <[email protected]> wrote:

You can escape a character to remove its special properties or
enclose the string containing it in single quotes.

Great, but how do I do that when I'm in a DIRECTORY that contains a special character? This is how this problem arises: there is a directory called 'foo?' in which there is the file bar.mpg. This fails:

$ cd 'foo?'
$ wine cmd.exe bar.mpg

WINE is unable to operate within the directory named 'foo?'. There is no way for me to escape the directory's name in the (scripted) call to wine.

'?' is an invalid character as part of file or folder names in
windows, so is it in wine.

[Martin Gregorie]

On Mon, 2011-09-12 at 18:31 -0500, stvs wrote:

Thanks for the suggestion, but that wil fail because it is wine that
fails when executed from within a directory containing a special
character.

You said 'There is no way for me to escape the directory's name in the
(scripted) call to wine', which indicates that your problem is an
inability to escale question marks in a non-Windows script. Your example
only confirms this by showing:

cd 'foo?'
wine cmd.exe bar.exe

which nowhere indicates that wine and its app has much idea what
directory its in when it fails. In fact this shows that you're not
running comskip its installation directory. Is this as accurate
illustration of what you're doing? If so, does comskip work better if
you cd to its installation directory and run it from there, e.g.

cd '/Library/Application Support/ETVComskip/comskip'
wine comskip.exe

Of course you can escape the character in a shell script, but that
won't help wine when it attempts to access the directory './foo?'.

I looked at all the logs posted this year. The ones from January are
about an entirely different problem: not a '?' in sight.

There's no indication in your logs that Wine failed to pass an absolute
pathname containing a question mark to comskip - in fact exactly the
opposite is true, since comskip reports the name correctly, though I notice
that path doesn't start with a drive letter. Should it do so?


Martin

[Frédéric Delanoy]

On Tue, Sep 13, 2011 at 02:43, Martin Gregorie <[email protected]> wrote:

On Mon, 2011-09-12 at 18:31 -0500, stvs wrote:

Thanks for the suggestion, but that wil fail because it is wine that
fails when executed from within a directory containing a special
character.

You said 'There is no way for me to escape the directory's name in the
(scripted) call to wine', which indicates that your problem is an
inability to escale question marks in a non-Windows script. Your example
only confirms this by showing:

cd 'foo?'
wine cmd.exe bar.exe

which nowhere indicates that wine and its app has much idea what
directory its in when it fails.

As already mentioned, '?' is disallowed in dir/filenames in windows,
and wine conforms to that.
See http://msdn.microsoft.com/en-us/library ... onventions
for more information.

[stvs]

Frédéric, Thanks for your answer. As a wine design question, is it reasonable for wine to have a flag that allows it to escape certain special characters and handle a greater subset of BSD names, except in the obviously clashing cases like backslash '\' and colon ':'?

[Martin Gregorie]

On Tue, 2011-09-13 at 06:17 -0500, stvs wrote:

Frédéric, Thanks for your answer. As a wine design question, is it
reasonable for wine to have a flag that allows it to escape certain
special characters and handle a greater subset of BSD names, except in
the obviously clashing cases like backslash '\' and colon ':'?

How is the data in files whose names contain the special characters
received and presented to Wine apps?

IOW, can you modify the filenames to remove/replace the special
characters before feeding them to the Wine app.


Martin

[Frédéric Delanoy]

On Tue, Sep 13, 2011 at 13:17, stvs <[email protected]> wrote:

Frédéric, Thanks for your answer. As a wine design question, is it reasonable for wine to have a flag that allows it to escape certain special characters and handle a greater subset of BSD names, except in the obviously clashing cases like backslash '\' and colon ':'?

Most likely not. AFAICT linux and BSD both accept all characters but
the '\0' in filenames. Wine OTOH must follow windows naming
conventions.
If you really need it, you could try to remove '?' from
INVALID_NT_CHARS chars list in dlls/ntdll/directory.c (line 134 or so)
and recompile wine.
No guarantee though.

Frédéric

[vitamin]

As a wine design question, is it reasonable for wine to have a flag that allows it to escape certain special characters and handle a greater subset of BSD names, except in the obviously clashing cases like backslash '\' and colon ':'?

No, it's not reasonable. Wine implements win32 api, which has specific requirements for file names. BTW Wine does allow colon ":" in file names, which on Windows represents a NTFS file stream.

[Frédéric Delanoy]

On Tue, Sep 13, 2011 at 15:35, vitamin <[email protected]> wrote:

stvs wrote:

As a wine design question, is it reasonable for wine to have a flag that allows it to escape certain special characters and handle a greater subset of BSD names, except in the obviously clashing cases like backslash '\' and colon ':'?

No, it's not reasonable. Wine implements win32 api, which has specific requirements for file names. BTW Wine does allow colon ":" in file names, which on Windows represents a NTFS file stream.

Actually it's a bit more complicated than that. It's filesystem
dependent. E.g. ':' is disallowed in directory files on NTFS, but
allowed on systems without named streams. There are also tricky rules
with filenames with ':' where applications have to scan them (e.g.
'more foo:bar' fails, but 'more < foo:bar' works...).
(put shortly, it's a mess)

[vitamin]

Actually it's a bit more complicated than that.

It always is, on Windows. On Wine, it's just that - Wine allows colon in filenames. I haven't dug deep into this in Wine code, but AJ few days ago said just that "It's kind of supported, by simply not doing anything special and creating separate files."

[Frédéric Delanoy]

On Tue, Sep 13, 2011 at 15:59, vitamin <[email protected]> wrote:

Frédéric Delanoy wrote:

Actually it's a bit more complicated than that.

It always is, on Windows. On Wine, it's just that - Wine allows colon in filenames. I haven't dug deep into this in Wine code, but AJ few days ago said just that "It's kind of supported, by simply not doing anything special and creating separate files."

Yep I tried to add some tests for them, but gave up.

[bgrupczy]

Is there a way to tell WINE that question marks in directory names are okay? WINE fails whenever I ask it to access a file from within a directory with a question mark in its name.

I had the exact same problem. The only workaround is to get that folder renamed to DOS compatible characters, let comskip run through Wine, and then change the name back again. I had to change the RecordingStarted and RecordingDone scripts to get this to happen. Only good way I could think of getting it done is to store off the Title and Episode into temporary files at the start and go read them back when EyeTV was done. I'm surprised the ETVComskip folks haven't run into this and done a workaround. I actually started doing this down in the MarkCommercials.py file but then decided that it would probably be better to do it as high in the chain of events as possible. BTW, I was also playing around with symbolic links and Wine somehow was getting the original path with the question marks.

Hope this helps some folks out there. With my luck, someone will write back with a much easier solution.

RecordingStarted snippet

Code: Select all

		set origRecordingID to recordingID
		set recordingID to recording id (recordingID as integer)
		
		set originalTitle to title of recordingID
		set originalEpisode to episode of recordingID
		set title of recordingID to unique ID of recordingID
		set episode of recordingID to ""
		
		set theFile to (path to temporary items from user domain as string) & origRecordingID & "- Title"
		open for access file theFile with write permission
		write (originalTitle) to file theFile
		close access file theFile
		
		set theFile to (path to temporary items from user domain as string) & origRecordingID & "- Episode"
		open for access file theFile with write permission
		write (originalEpisode) to file theFile
		close access file theFile
		
		set cmd to "(sleep 300 ; /usr/bin/nice -n 2 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & unique ID of recordingID & ") &> /dev/null &"
		commonScript's write_log(cmd)
		do shell script cmd

RecordingDone snippet

Code: Select all

		set origRecordingID to recordingID
		set recordingID to recording id (recordingID as integer)

		delay 15
		set theFile to (path to temporary items from user domain as string) & origRecordingID & "- Title"
		open for access file theFile
		set originalTitle to read file theFile
		close access file theFile
		set theFile to (the quoted form of POSIX path of theFile)
		set cmd to "rm -rf " & theFile & ""
		do shell script cmd
		
		set theFile to (path to temporary items from user domain as string) & origRecordingID & "- Episode"
		open for access file theFile
		set originalEpisode to ""
		try
			set originalEpisode to read file theFile
		end try
		close access file theFile
		set theFile to (the quoted form of POSIX path of theFile)
		set cmd to "rm -rf " & theFile & ""
		do shell script cmd
		
		set title of recordingID to originalTitle
		commonScript's write_log("title set")
		set episode of recordingID to originalEpisode
		commonScript's write_log("episode set")