Wine registration email - system failure

[dimesio]

At this time, we are a 'juicy' target. Put in a delay for new users
and that might scare them off.

IMO, it will still be worth their trouble as long as the spam posts are left in the mailing list archives.

In the meantime, both Dan and I are looking to see where spam is coming from and if needed blocking IPs. If the IP address could be captured and relayed to us before deleting messages from the forum, we
would appreciate it.

I thought deleting spam was your job now.

[jjmckenzie]

On Wed, Jul 20, 2011 at 9:28 AM, dimesio <[email protected]> wrote:

jjmckenzie wrote:

At this time, we are a 'juicy' target.  Put in a delay for new users
and that might scare them off.

IMO, it will still be worth their trouble as long as the spam posts are left in the mailing list archives.

I agree. However, if the moderators/admins are diligent and
record/remove the posts quickly and it takes them time to post, they
will go away and find a better target. The decision is not mine to
make but it is an option that we can look at as well as updating the
Forum Software and other things the Admins are doing.

This all adds up to a much lower signal/noise ratio. And I've was
dealing with this as a Moderator in the days of FidoNet.

James

[Martin Gregorie]

On Wed, 2011-07-20 at 07:01 -0700, James McKenzie wrote:

In the meantime, both Dan and I are looking to see where spam is
coming from and if needed blocking IPs. If the IP address could be
captured and relayed to us before deleting messages from the forum, we
would appreciate it.

A clarification: the 'From:' address in messages originating from the
users forum always of the form:

"handle" <[email protected]>

where 'handle' is the handle used to post to the forum. Quite apart from
not containing a useful mail address, the handle is very frequently
forged in spam: for instance, I see significant amounts of spam that
allegedly was sent by yourself and, even where the handle wasn't forged,
a poster doesn't have to supply a valid e-mail address to sign up - at
least its been blank in every one of the few poster profiles I've looked
at.

That said, I can certainly produce lists of the apparent senders of spam
from the user list if the details are still in my mail logs, but for the
reasons given above, I don't think they'll be particularly useful.

I can easily supply you with a list of all the domain names that have
appeared in the body of spam I've received via the Wine users mailing
list. I can produce a list of URLs and PERL regexes that match spammy
URLs almost immediately, but if you want the list reduced to just those
URLs that arrived in the Wine maillist that will take a day so to do.

Let me know which URL list you'd prefer and if you'd like a sender list.


Martin

[dimesio]

I agree. However, if the moderators/admins are diligent and
record/remove the posts quickly

Deleting posts from the forum does not delete them from the archives. Someone with the power to delete posts from the archives needs to do it.

[dimesio]

the handle is very frequently
forged in spam: for instance, I see significant amounts of spam that
allegedly was sent by yourself

Is this email you receive directly from the list, or are you accessing it via Nabble? I have noticed that Nabble mangles threads badly--posts made by other people routinely show up under my name on their site. But these same posts are correctly attributed on both the forum and in the WineHQ archives.

a poster doesn't have to supply a valid e-mail address to sign up - at
least its been blank in every one of the few poster profiles I've looked
at.

Yes, they do. If they didn't, the original poster in this thread would not be having the problem he had. The forum gives registered users the choice of whether or not to show their email address in their profile. People who don't want to give out their email address to random strangers--i.e., sensible people--check "no." If you look at my profile, you won't see my email address, either.

[Martin Gregorie]

On Wed, 2011-07-20 at 13:04 -0500, dimesio wrote:

Martin Gregorie wrote:

the handle is very frequently
forged in spam: for instance, I see significant amounts of spam that
allegedly was sent by yourself

Is this email you receive directly from the list, or are you accessing
it via Nabble? I have noticed that Nabble mangles threads badly--posts
made by other people routinely show up under my name on their site.
But these same posts are correctly attributed on both the forum and in
the WineHQ archives.

Directly from the wine-users mailing list.

I don't have a Nabble account.

Yes, they do. If they didn't, the original poster in this thread would
not be having the problem he had. The forum gives registered users
the choice of whether or not to show their email address in their
profile. People who don't want to give out their email address to
random strangers--i.e., sensible people--check "no." If you look at my
profile, you won't see my email address, either.

So I see - I just added myself to the Wine user forum today to see how
it works, and have also found out that changing the e-mail address
deactivates the account and issues a fresh challenge/response for the
new address, which is good.


Martin

[dimesio]

Directly from the wine-users mailing list.

I don't have a Nabble account.

Could you check one of the emails where you thought the user was "forged" against what's in the archives? I had always assumed it was Nabble mangling things, but perhaps it is happening at this end.

[Martin Gregorie]

On Wed, 2011-07-20 at 14:23 -0500, dimesio wrote:

Martin Gregorie wrote:

Directly from the wine-users mailing list.

I don't have a Nabble account.

Could you check one of the emails where you thought the user was
"forged" against what's in the archives? I had always assumed it was
Nabble mangling things, but perhaps it is happening at this end.

I can't do it directly because my system is currently configured to
throw spam in the bit bucket. What I can do is post the reference info
that my spam killer writes to the mail log. In the following references
the following is always true:

- MG_WHITELIST whitelists mail from addresses I've sent mail to and
MG_WINELIST is a marker for Wine user mail list messages, so
both will appear in any message received from Wine

- MG_WINESPAM is a meta rule that fires when enough subrules that
recognise phrases, URLs etc that appear in spam have fired to show
that the message is spam rather than ham. This rule has the effect of
cancelling the whitelisting applied by MG_WHITELIST.

Here are some references to messages that I think are spam with yours
and James's handles forged as the sender.

Jun 21 15:41:11
Subject: Re: [Wine] *Is In Need Of Some Help*
From: James McKenzie <[email protected]>
Rules: MG_GMAIL,MG_MONEY,MG_SALE,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

MG_GMAIL fired because James posted via Gmail
MG_MONEY says money was mentioned and MG_SALE that sales
phrases were also present

Jun 22 14:11:06
Subject: [Wine] Re: Problem with Power Point 2007 & Visio2007 on wine
From: "dimesio" <[email protected]>
Rules: MG_SPAMREF,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

This message contains a URL that I think is spam payload

Jun 30 23:51:05
Subject: SPAM: [Wine] Re: cannot activate Partsmart
From: "dimesio" <[email protected]>
Rules: MG_PRODUCT,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

This message contains a product reference

Jul 1 23:11:04
Subject: SPAM: [Wine] Re: cannot activate Partsmart
From: "dimesio" <[email protected]>
Rules: MG_PRODUCT,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

As above

Jul 7 03:21:05
Subject: SPAM: [Wine] Re: HTML Mail on Wine List
From: "jjmckenzie" <[email protected]>
Rules: MG_PRODUCT,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

As above.

Jul 11 15:31:14
Subject: SPAM: [Wine] Re: ProgramError
From: "dimesio" <[email protected]>
Rules: MG_MONEY,MG_SPAMREF,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

This contains a string that was recognised as money as well
as a URL that I think is spam payload.

If you can see what in these messages tripped the SPAMREF, PRODUCT and
SALE rules I'd be interested to know, and doubly so if any are false
positives. I'm careful to use rules that match fairly specific phrases
and tend to require combinations of hits before I mark a message as
spam. The main exception to combining rules is tests for spam-related
URLs: these have all been found in obviously spammy messages, so I take
their presence as a good spam marker. I regression test all rules
against a fairly large spam collection to make sure that individual
rules don't fire on unrelated spam but that everything in the collection
continues to be marked as spam.

I'd do the comparisons if I had copies of messages that were marked as
spam, but as I don't, if its easier for you, just send me one or two of
these complete messages and I'll do the analysis.


Martin

[dimesio]

Guessing at the ones attributed to me, based on dates/times:

Jun 22 14:11:06
Subject: [Wine] Re: Problem with Power Point 2007 & Visio2007 on wine
From: "dimesio" <[email protected]>
Rules: MG_SPAMREF,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

This message contains a URL that I think is spam payload

Probably this post: http://www.winehq.org/pipermail/wine-us ... 92670.html

Jun 30 23:51:05
Subject: SPAM: [Wine] Re: cannot activate Partsmart
From: "dimesio" <[email protected]>
Rules: MG_PRODUCT,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

This message contains a product reference

Probably this: http://www.winehq.org/pipermail/wine-us ... 93023.html

Jul 1 23:11:04
Subject: SPAM: [Wine] Re: cannot activate Partsmart
From: "dimesio" <[email protected]>
Rules: MG_PRODUCT,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

As above

Probably this: http://www.winehq.org/pipermail/wine-us ... 93076.html

Jul 11 15:31:14
Subject: SPAM: [Wine] Re: ProgramError
From: "dimesio" <[email protected]>
Rules: MG_MONEY,MG_SPAMREF,MG_WHITELIST,MG_WINELIST,MG_WINESPAM

Probably this: http://www.winehq.org/pipermail/wine-us ... 93510.html

All of the posts above are mine, and are certainly not spam. Your spam filter seems to be overly aggressive about flagging anything with a URL in it as spam, and as for the "product reference" filter, posts do often contain the name of a commercial app, because that's what the person is seeking help with.

If you really believe emails are being sent to the list under the wrong name, you need to produce an actual email that shows a different user compared to what is on the forum and in the archives.

[dimesio]

Martin: I just posted a reply to your list of emails that had been flagged as spam. You may not get it because my reply contains URLs, and it's clear your spam filter is overly aggressive about filtering those out. Check the forum if your spam filter ate it.

IMO, you've given us an excellent example of how spam filters can get things horribly wrong. Quite possibly that's what happened with his ISP.

[Martin Gregorie]

On Wed, 2011-07-20 at 18:12 -0500, dimesio wrote:

Martin: I just posted a reply to your list of emails that had been
flagged as spam. You may not get it because my reply contains URLs,
and it's clear your spam filter is overly aggressive about filtering
those out. Check the forum if your spam filter ate it.

You're right on both counts: my spam filter got your reply and I have
terms and URLs in my list that should not have been there and have been
removed:

wine-reviews.net - tagging this was clearly a bit of idiocy
the content of the message I use to regression
test catching this URL makes that quite clear

imageshack.us - a mistake: this is a pastebin equiv for images
and I missed the advertising URL I should have tagged
at the end of the test message.

I can't see why the Partsmart messages got tagged: possibly something to
do with the headers.

Anyway, thanks for the feedback and the opportunity to fix my URL hit
list. This also gives me the incentive to modify my spamkiller setup so
it puts spam in a holding tank for a few days instead of killing it
immediately.

IMO, you've give us an excellent example of how spam filters can get
things horribly wrong. Quite possibly that's what happened with his
ISP.

Yes, I agree - and of of course the fact that my filters got your
response reinforces that. Did you include messages rather than links to
them? If so that would explain it being caught because of course replies
to to messages that contain spammy references will also be tagged if
they quote the references.


Martin

[dimesio]

Did you include messages rather than links to
them?

No, just the links. FYI, my responses often include links, and sometimes my whole response is just a link. I answer a lot of questions, and t's much more efficient to post a link to the relevant part of the FAQ, AppDB, or Bugzilla than to repeat the information here. You may have been screening out quite a lot of my responses.

[Ace...]

So now, after some serious debate; what consensus conclusions can we draw?

1. Do we now think that, in fact, WineHQ is NOT a major source of spam?
or,
2. Could it still be the case (hence the isp block)?

3. Could it be that WineHQ WAS a major source of spam, and was therefore blacklisted, and remains so?
(the latter: it used to be the case that blacklisting only lasted two or three months)

OR

4. Could it be that WineHQ was never a major source of spam?

5. Could it be possible that the original log result of the refused mail; was actually, specifically correct.

Ie. Spam detected

meaning that the format, or something in the message, is triggering a spam filter?


If there is a consensus that the latter assumption could be correct, then perhaps it may be worthwhile, tidying up the registration auto email, from the perspective of "potential filter rules".

We have already identified the 'repeated word' bug, in the subject line.....
.... in fact I've just had another look at the email.

(Subject:) Welcome to WineHQ Forums Forums

(1st line ) Welcome to WineHQ Forums Forums

If the mail message is scanned subsequent to
the subject line; in 7 words, Forums is mentioned 4 times!
(+ both lines are exactly the same)

While this may have no bearing whatsoever:

a) It isn't very good
b) It is asking for trouble


Once this email has been modified, let's try it again.

[tparker]

On 07/26/2011 09:53 AM, Ace... wrote:

1. Do we now think that, in fact, WineHQ is NOT a major source of spam?

I wasn't following this and don't know if it is helpful to chime in, but
in case it is: the -only- spam I get is from the Wine email list/forum.
Fairly often the real posts seem fewer than the spam ones on any given
day. It is annoying, but I find the list useful and educational as I try
to learn how to fix things and try to help others so the spam is just
part of the price of having the list available.

[Martin Gregorie]

On Tue, 2011-07-26 at 08:53 -0500, Ace... wrote:

So now, after some serious debate; what consensus conclusions can we draw?

1. Do we now think that, in fact, WineHQ is NOT a major source of spam?

It still is a source of spam, though not a huge source. My count, for
the last week, says:
July 20 - 1
July 21 - 2
July 22 - 4
July 23 - 0
July 24 - 1
July 25 - 3
July 26 - 11 (so far at 16:00 PM GMT - my spam filter caught 7 of these)

All of this is genuine spam: I've read all of them. Today is unusual:
the spam rate for the first six days is more typical.

2. Could it still be the case (hence the isp block)?

My ISP uses greylisting. Before its implementation 80% of my mail was
spam. Post implementation its about 8%. The stuff I'm trapping is what
gets through the greylister.

3. Could it be that WineHQ WAS a major source of spam, and was therefore blacklisted, and remains so?
(the latter: it used to be the case that blacklisting only lasted two or three months)

I'd say that a day like today would be more than enough to get WineHQ
reported to at least one or more public blacklists and/or ISP's private
blacklists. Caveats:

- nobody is getting this spam unless they are subscribed to a Wine
mail list, which will probably limit those who report it to a
blacklist to casual users who think unsubscribing is more effort
that getting it blacklisted.

- Spam received from WineHQ mail lists is quite hard to trap: since the
Codewaevers MTA sends direct to subscribers' ISPs or MTAs, the usual
set of headers that trigger many Spamassassin rules are absent, so
almost all that can be used to trap this spam is the body content.
Writing general rules to catch this spam without getting false
positives on legit. Wine user messages is very difficult.

In general its a case of playing wack-a-mole by building lists of the
URLs they're advertising.

Simply doing URIBL lookups on the Wineusers output message stream
to check the URLs in the subject line and body may catch a lot of it.
IMO that would be worth a try, anyway.

4. Could it be that WineHQ was never a major source of spam?

Not the case. See above.

We have already identified the 'repeated word' bug, in the subject line.....
.... in fact I've just had another look at the email.

Unlikely, I think. I've never seen a standard Spamassassin rule that
triggers on arbitrary repeated words. If any did, they'd be looking for
words that were specific to the stuff being advertised, not something as
neutral as 'Forum'


Martin

[dimesio]

5. Could it be possible that the original log result of the refused mail; was actually, specifically correct.

Ie. Spam detected

meaning that the format, or something in the message, is triggering a spam filter?

If it is, then that spam filter is badly misconfigured.

Once this email has been modified, let's try it again.

Who said it was going to be modified?

[dimesio]

It still is a source of spam, though not a huge source. My count, for
the last week, says:
July 20 - 1
July 21 - 2
July 22 - 4
July 23 - 0
July 24 - 1
July 25 - 3
July 26 - 11 (so far at 16:00 PM GMT - my spam filter caught 7 of these)

All of this is genuine spam: I've read all of them. Today is unusual:
the spam rate for the first six days is more typical.

Spam has always tended to come in waves. Most of the spam today is from the same user, who made multiple posts within seconds of each other. It's already been removed from the forum, and I assume that user banned. It's fairly easy (although tedious) to keep the forum clean, and it generally is, but there's no way of retrieving spam once it's been forwarded to the mailing list.

[Martin Gregorie]

On Tue, 2011-07-26 at 13:04 -0500, dimesio wrote:

Spam has always tended to come in waves. Most of the spam today is
from the same user, who made multiple posts within seconds of each
other.

Indeed - often on waves from the same user.

It's already been removed from the forum, and I assume that user
banned.

Many forums are moderated, i.e. posts are passed by a human before they
appear on the forum and I assume this is true for the Wine Users forum
and that posts aren't sent to the mail list until they've passed the
moderator. Correct assumptions?

It's fairly easy (although tedious) to keep the forum clean, and it
generally is, but there's no way of retrieving spam once it's been
forwarded to the mailing list.

I'd assumed that would be the case too.


Martin

[dimesio]

Many forums are moderated, i.e. posts are passed by a human before they
appear on the forum and I assume this is true for the Wine Users forum
and that posts aren't sent to the mail list until they've passed the
moderator. Correct assumptions?

No. Forum moderators can edit and delete posts after they are made, and lock threads (which only affects the forum, not the mailing list), but there is no screening of posts beforehand.

[Ace...]

Once this email has been modified, let's try it again.

Who said it was going to be modified?

Well I suggested it should be.....
...nobody said it would be.

From a troubleshooters perspective; you ALWAYS clear out the shlite, on first principals.

It really doesn't matter whether you think that a blatent mistake is the cause, because in the time it's taken you to argue otherwise, you might just as well have sorted the prob, and moved on.

What does this give us?

The key gain, is in eliminating the question of "spam detection".

It's not about right or wrong. It is to totally eliminate "spam detection" from the equation.

ie. If there are 2 options here (if) - 1 being on the fly spam detection, 2 being pre-set blacklist..... by eliminating 1, it becomes very clear to all, where the problem lies.

This is sound engineering practice, and entirely normal in its application.
+ the email needs anyway sorting (cos it's wrong) so sorting it is not only good for the question at hand, but it's good house-keeping.

[dimesio]

If there are 2 options here (if) - 1 being on the fly spam detection, 2 being pre-set blacklist..... by eliminating 1, it becomes very clear to all, where the problem lies.

Either way, the problem is obviously with your ISP, because the confirmation email from here is not spam.

[Ace...]

Yes, I think you are right to reiterate the problem.

This thread was generated due to my ISP incorrectly flagging up the WineHQ registration email as spam.

However, IMO your statement of "either way" is confusing, because it is only repeating what I've just said ie. the problem is due to one of two potential scenarios: on the fly spam detection, or pre-configured blacklisting of the winehq email distribution address.

I had to read it 2 or 3 times to figure out what you were saying.

The point I was making was, that it would be normal to eliminate one of these options - the easiest being, the repeated word glitch, which clearly indicates that this is a computer generated message that may well be spam (even if it isn't).

We can blame the ISP, but it changes nought, cos they have umpteen million clients that they are trying to protect, and maybe there are other isp's doing a similar job, with similar defence systems.

So, we either say "it's their fault" and forget about it, or we try to find out what's wrong.

I honestly don't mind, cos I'm logged on now, using hotmail.

[dimesio]

So, we either say "it's their fault" and forget about it, or we try to find out what's wrong.

It's their fault, and their responsibility to figure out why it happened, if they want.

Most ISPs have some sort of spam filtering in place. False positives are inevitable.

[Ace...]

You may be right, but you could so easily be wrong (vis a vis it being their fault and responsibility).

We are interacting within a global system, that functions under a number of different protocols.

ideally i guess that we would want to function within as many guidelines as poss, whilst accepting that 100% coverage is evidently difficult to achieve.

However, we shouldn't lose sight of the fact that false positives might be caused by something that we can rectify - and that would be good, because in NO WAY can we rely on the isp's to create a work around, just for winehq.

It's not gonna happen (or at least, highly unlikely), particularly if the primary fault lies here, either due to 'here' being a spam source, or the source of a less than brilliant 'email message' (the primary fault not having been established as yet).

Are you 'chucking the towel in' too soon?

[dimesio]

Are you 'chucking the towel in' too soon?

It's not my decision. You are free to file a bug.