Score-based killfiling

Newsstar can decide not to download certain articles based on the contents of their headers (killfiling). It only reads the headers available in XOVER, to help keep the code simple. Enabling killfiling forces the xover option to be used for a server (see the section on server.cf files).

Those familiar with NNTP may be aware that XOVER only lists the contents of certain headers in a fixed order, without the header name. Newsstar reconstructs the full header as it would appear in a message by prepending it with its name, a colon and a space.

The headers available are: Newsgroups, Subject, From, Date, Message-ID, References, Size, Lines, in that order.

Newsgroups may not be the real Newsgroups header as it appears in the message, but is constructed by newsstar based on the Xref header, if present in the overview, or will simply contain the group it's examining when it finds the message. In the latter case it will not be aware of any other groups the article is crossposted to.

Size is a pseudo-header that doesn't appear in actual messages, but the data is available in the overview and may be useful for scoring.

The headers for each message are tested against each regular expression in one block, each header separated by newlines, so you may take advantage of the order and put multiple headers in one expression, but this is not recommended, in case the overview order changes.

To enable scoring for a server create a file called score.* where * is the server name, in a similar way to server config files. As usual, a sample is provided. The structure of these files is similar to the other config files, but a score value takes the place of a keyword, and an extended regular expression takes the place of a value. Regular expressions are case-insensitive by default but may be made case-sensitive by using the regex_ic option in main.cf.

Each article starts off with a score of 100, but this value may be changed in a server's config file. The headers are tested against all the lines in the score file, and each time a regular expression matches, the corresponding score is added to the article's total. If the final total is less than 0, the article is not downloaded. Therefore many of the score values you'll be using will be negative.

If the score is between 0 and the kill_score value in the server's config file a pseudo-article may be posted to the local server, containing only the headers newsstar generated by scanning the XOVER data. The default value for kill_score is 0, effectively disabling this feature.