spamreports are not sent (Full Version)

All Forums >> [IceWarp Product Configuration] >> IceWarp Anti-spam Server



Message


chkla -> spamreports are not sent (9/15/2008 12:29:52 AM)

I have a strange issue here. We are sending spamreports once a day.
Last week we got complaints from customers about not receiving any spamreports anylonger.
This complaint appaears to be right because I found out that the last one sent was August 30. In the first week of september we updated Merak so checked the configuration in the admin. Looks however unchanged.
Next we started "run now with debugging". But it stopped after the second spamreport. Indeed these were the only customers who did receive a spamreport during the last 2 weeks.
We have no idea were to find the reason for this break.
Anyone an idea where we should look?

Kind regards Chris P. van der Klauw




johnt -> RE: spamreports are not sent (9/15/2008 7:06:23 AM)

We are having the same problem, and it started right after upgrading to 9.3.1. I've opened a ticket with support, however they have not been able to recommend a good fix for it. They came out with 9.3.2 and recommended that I upgrade to it to see if it fixed the problem. The upgrade to 9.3.2 does not work either. Support has told me that this has been turned over to the developers.

Have you had any problems with webmail since you upgraded your server?

Regards,

John




palidin74 -> RE: spamreports are not sent (9/16/2008 11:55:57 AM)

Sigh... This is one of the larger headaches with Merak/Icewarp. I love this product however the SPAM reports are something that if just a bit of attention was given the management of them would be so much easier. First let me say that I am by no means an expert on this product however we have been using Merak for quite a few years and I have become rather efficent in managing it. Let me start by explaining how the actual SPAM report process works as I understand it through my own troubleshooting. Please if someone out there knows something different (or better) POST since I would LOVE TO KNOW MORE about managing this process.

Here we go

Basically the SPAM report is a large script (for the lack of a better word) that generates HTML code one by one for every mailbox. There are conditions that it checks (for example if SPAM reports are not enabled, set to new or all etc) agenst the mail database. Based on those conditions the script changes its output or skips that mailbox all together. This poses a question. Do you have SPAM reports enabled on the domain level? This is one of the improvement areas. In Merak you can actually turn off the SPAM functions at the domain level however the options still show at the user level? This can be quite the issue at times when troubleshooting. If this is your case then the debug mode will show it checking the mailbox in question however it will indicate there is NO SPAM to process. If both the domain and the mailbox is flagged to use the SPAM system and there are SPAM messages within the SPAM folder (typically ~SPAM) then the script will begin processing for that mailbox. There is one more condition as well i believe. If the user has the "new" option selected where the SPAM report is only to contain messages that have arrived withing the SPAM system since the last SPAM report a data comparison is done between the messages and the spamreport.dat file contained within the users SPAM folder. If the new condition is set and there are messages with a newer date than the .dat file then the report is generated and contains only those messages. If there are no messages with a newer date than the .dat file then the mailbox is skipped. Now we get into the nitty gritty of the script. At this point all conditions are met and the processing of the messages into the report begins or a condition is not met and the script moves on to another mailbox. The script then grabs the .imap or .tmp files contained within the ~SPAM and ~quarantine folders and reads them in order to insert there information into the HTML generated report. At this point you have to realize the script is reading the messages and grabbing the appropriate fields to insert into the report for the user. Here is where things can go wrong. Unlike some applications when this script runs into a problem it simply stops processing. This is where you see a handfull of users actually getting there SPAM report and a handfull that does not. The ammount of each depends on where the script dies during its processing of the mailboxes. This is where I see the largest problem with SPAM reports. Some user recieves a mail message containing code that the script simply cannot process and it halts. The only way to fix it is to delete the spam message that is interfering with the script. So comes the big question how can you find out what message is causing this? To my knowlege there is no sure fire or simple way. You have to run the debug option within merak (the button is new in 9.3.1 and higer) however the URL method that kicks off the reports has been there since the inceptoin of the SPAM report. You can run a debug in any versoin using this method (again to my knowledge). The debug option will generate HTML as the script goes through the mailboxes and processes the SPAM report. The exact order of this i am not sure, its domain by domain i know that however i believe it uses creation date within the database table. In other works if you have mailboxes 1,2,3,4 and you created mailbox 2 then 3 then 4 then 1 each of the mailboxes were added to the SQL table as you created them, this is the same order the SPAM report script goes through and generates the reports it simply grabs the users from the table and the table happens to be 2,3,4,1 since the mailboxes were created in that order. The debug output will stop on a user, thats the user where your problem message file exists. The odd thing is this, how do you know what your last user is within the report? If the script stops is that normal or is that the problem user? How i do it is simple. I run a batch file outputing to e-mail. This file basically searches all the mail directories. When if finds a spamreport.dat file it checks the date if its more than 24hrs old then i know the report did not fully run and it prompts me to run a debug and isolate where the problem resides. There is only one problem with how I do this. When a user disables there SPAM report then the dat file remains and it shows up in my batch file. So i have to check the user each time I am prompted there might by a problem and make sure the user did not disable their SPAM report legitimately. If they did i delete the dat file so i dont get prompted by the batch file again.

I would love to see merak have a simple script that can let admins know if the script stops, if so at the very least notify the admin. Even more so I would love to see a "delete any SPAM message that prevents the SPAM report from completeing" option.

Hope this helps, and again please if anyone out there has a better way to manage this please please let us know!

Thank you




chkla -> RE: spamreports are not sent (9/17/2008 11:38:00 PM)

That's a good desciption Paladin.

This processdescription is about what my experience is.
Yesterday I did a trial and error on the spamreport. At the end I had 4 mailboxes switched off and the spamreport was running completely.
So probably your point about the content of a message is right. I 'll investigate that today by cleaning the spamboxes for those customers and see if the spamreport is still staggering on those mailboxes.

I add one thing on your description: I found out that allthough on domainlevel the spamfilter was switched off the spamreport on the individual mailboxes still keeps running.

And indeed the report is not running alphabetically but in order of older to recent time of creation of the mailbox.

Thanx for your manual. It helps a lot. Many of the processes in Merak are nog very clear. Like for instance whitelisting and all the settings for spam: there is a high level featurelist, but it says nothing about how exactly the process is running.

So we are from time to time really amazed by new discoveries about the way Merak works. See also my issue regarding whitelisting.

Kind regards, Chris




palidin74 -> RE: spamreports are not sent (9/19/2008 6:17:49 AM)

THank you Chris for the additional information. I wanted to update this a bit for any other readers that may come across this post. I kept running into SPAM reports being held up due to the script stopping at some particular mailbox. This inevitably prevents the SPAM report from being delivered to the remaining mailboxes. After narrowing down the specific message that was causing the script to hang I notified Merak Support. This way my response:

Timothy,

Sadly, this is a known issue, and what you are doing is the correct procedure. I do have a new challenge html folder I can send you(attached), but it does not fix this issue for everyone. I have been working with the developers on this, but currently don't have a solution.



So at this point we have an issue with 9.3.1 and 9.3.2 with a specific type of message that is causing the reports to hang. The devs dont have a solution as of yet. I am going to try the HTML support suggested here and I will be sure to post my results here.

Thank you




IWAdmin DanG -> RE: spamreports are not sent (9/22/2008 5:48:09 AM)

This is something that has a lot of attention being devoted to it. If you write back in, there is a new set up scripts (as of friday) that can be sent out to you, that has help a number of people. This has become such an issue that the Spam Reports are being completely re-written for version 10. Until that time, we are still working on a solution to this for everyone, that can be implemented now.




Page: [1]



2001 - 2008 © IceWarp