ISC is pleased to announce that a new major release of INN is available - INN 2.5.0
Internet Systems Consortium is pleased to announce that a new major release of INN is available at:
The MD5 checksum of this release is be60e463b2c21286eca45e994df307b2. A PGP signature is available in the same directory. Due to the extensive changes between INN 2.4 and INN 2.5, no patch is available.
INN 2.5.0 is almost entirely compatible with INN 2.4, and most INN 2.4 users will be able to upgrade with a simple "make update". A few checks explained in the upgrade changes below should also be made.
All of the applicable bug fixes from the INN 2.4 STABLE series are also included in INN 2.5.
Further 2.5.x releases will be made for bug fixes. New features will go into the 2.6.x series, which is already under development. Please note that the STABLE development branch is now INN 2.5.x, and there will be no further INN 2.4.x releases after INN 2.4.6.
Many thanks to Julien ÉLIE for preparing this release.
Upgrading from 2.4 to 2.5
Perl 5.8.0 or later is recommended for INN. If you are using an earlier version, you will also need the "Encode" module for correct processing of control messages. (It is included with Perl itself in 5.8.0 and later.)
The following changes require your full attention because a manual intervention may be needed:
- In order to process control messages, controlchan now needs the "MIME::Parser" module. Packages are available from most distributions, or you can install the module directly from CPAN ("MIME-tools" in modules/by-module/MIME/, for instance on ftp.perl.org).
- Checkgroups control messages are now differently handled by controlchan: all matching lines in control.ctl will be used for a given checkgroups and a doit action will really be executed (adding, removing and changing the status of newsgroups). You should make sure that your local configuration does not rely on the previous behaviour of only mailing changes, without actually performing them.
- You should use the new control.ctl.local file shipped with INN in
*pathetc*and, at the same time, update yourcontrol.ctlandmoderatorsfiles. Also make sure that youractive.times,*distrib.patsand newsgroups files are properly encoded in UTF-8, as it is strongly recommended by RFC 3977. - The
overview.fmtfile is no longer used by INN. Two new parameters have been added toinn.conf: extraoverviewadvertised and extraoverviewhidden - Although innupgrade takes care of the change during "make update", you should make sure that your overview database is consistent with all the fields declared in
overview.fmtbecause they will all be advertised, and "Xref:full" forced as the eighth overview field. See theinn.conf(5)man page for more information about these parameters. - The innreport configuration file has slightly changed. The new innreport.conf file shipped with INN should be used and your possible changes backported to this new version.
- The $SPOOLBASE variable has been renamed to $SPOOLDIR in
innshellvarsin order to be more consistent. It impacts shell scripts only. If you importinnshellvarsand use that variable in your scripts, you will have to rename it. - gpgverify is no longer included in INN, pgpverify now has better support for GnuPG and should be used instead.
- The auth_smb authenticator program to check passwords with an SMB authentication is no longer included in INN. It was a stripped-down version of pam_smbpass, wasn't maintained, and likely had security problems. To authenticate to an SMB server such as Samba, use PAM and ckpasswd's PAM support instead.
The parameters used by nnrpd to provide TLS support are now tlscafile, tlscapath, tlscertfile and tlskeyfile in inn.conf. The sasl.conf file used for that in previous versions of INN is obsolete. innupgrade takes care of the change during "make update".
The nntpactsync parameter has been renamed to incominglogfrequency in inn.conf; innupgrade handles this renaming during the update.
In newsfeeds, innfeed should be run directly rather than through startinnfeed. innupgrade will attempt to take care of this modification during "make update".
When starting innd by hand, innd can just be run directly rather than using inndstart. If you get error messages about resetting the file descriptor limits, you may need to increase the file descriptor limits. See the sample init script in contrib for an example of how to do this.
If you are upgrading from a version prior to INN 2.4, see also "Upgrading from 2.3 to 2.4".
Major changes from 2.4 to 2.5
Ken Murchison has contributed SASL authentication support for nnrpd, implementing the AUTHINFO SASL section of RFC 4643. If the --with-sasl option is given to "configure", nnrpd will be able to authenticate clients via secure SASL mechanisms.
Julien Elie has implemented in nnrpd the new version of the NNTP protocol described in RFC 3977, RFC 4642 and RFC 4643. Consequently, nnrpd now recognizes the CAPABILITIES command, the HDR and LIST HEADERS commands, the second optional argument to specify a range of articles to LISTGROUP, the OVER command, as well as the ":bytes" and ":lines" metadata items.
Heath Kehoe has added the ability to compress overview data before it is stored in ovdb. It significantly improves the performance of this storage method and reduces the time spent by expireover. See the new --with-zlib option to "configure" and the ovdb(5) man page.
Alexander Bartolich has greatly improved innreport and especially its XHTML output (a XSL transformation is also provided, if needed, in innreport-filter.xslt, in the contrib directory).
inndstart and startinnfeed are no longer part of INN and are no longer used. Instead, a separate setuid root helper program written by Russ Allbery is used to bind to the news ports (and does only that), and is run by innd and nnrpd when necessary. This means that INN may not be able to increase file descriptor limits for itself the way that it could before. If you get error messages about resetting the file descriptor limits, you may need to increase the file descriptor limits as root before running rc.news as the news user. See the sample init script in contrib for an example of how to do this. More information on file descriptor limits can be found in INSTALL.
INN's IPv6 support was largely rewritten by Russ Allbery. IPv4 and IPv6 are now handled through the same code wherever possible, the new IPv6-aware APIs are used everywhere possible, and replacement functions are provided for systems that don't have them yet. The network code is now much more centralized, eliminating lots of duplicate code and adding better IPv6 support to some utilities.
INN now uses autoconf 2.61 or later for configuration. As a result, some "configure" options have changed slightly and more of the standard --*dir options should be supported in lieu of the old INN-specific options. See "configure --help" for the available options.
Thanks to Kirill Berezin, the buffindexed overview method now supports buffers larger than 2 GB. It is not necessary to compile INN with large file support to use such large buffers with buffindexed. Buffindexed is now also more robust with mmaped files and uses more optimized data placement.
tinyleaf, a miniature IHAVE-only leaf server written by Russ Allbery, is now included. See the tinyleaf(8) man page for more information.
controlchan recognizes the new application/news-groupinfo entity described in USEPRO and can handle character set conversions of newsgroup descriptions. The "MIME::Parser" and "Encode" modules are used. Processing control messages has been greatly improved, especially checkgroups: the active and newsgroups files are now properly updated when they are processed, and all matching lines in control.ctl for a given checkgroups are honoured (which for instance allows to use both drop and doit actions for the same checkgroups message).
A new control.ctl.local file has also been added in pathetc. Rules set in that file override rules in control.ctl, allowing administrators to specify local rules for some control messages without modifying the control.ctl configuration file that comes with INN. It also specifies encodings to use for the newsgroups file. By default, UTF-8 will be used for newsgroup descriptions, as strongly recommended by RFC 3977.
The Perl and Python filter_mode hooks are now called when innd is shutting down via either "ctlinnd shutdown" or "ctlinnd xexec" with a new mode value of "shutdown". This will allow the Perl hooks to save filter data across innd restarts without requiring that the news administrator throttle the server first. (Python already had a separate close hook that is also called.)
The legacy innshellvars.pl script has been replaced with a real INN Perl module "INN::Config" for Perl programs. The location of Perl modules can be set with the --with-libperl-dir option to "configure". All Perl scripts shipped with INN have been converted to use that module. You may want to consider using "INN::Config" in your Perl scripts, though innshellvars.pl is still provided with INN.
Support for embedded Tcl filters in innd has been removed. It hasn't worked for some time and causes innd crashes if compiled in (even if not used). If someone wants to step forward and maintain it, we recommend starting from scratch and emulating the Perl and Python filters.
If strippath is set in readers.conf, the whole user-supplied Path: header will now be stripped. Previously, the final component of the user-supplied Path: would still be retained.
news2mail can now set the envelope-from address of the mails it sends. A third optional part in news2mail.cf entries has been added by D. Stussy to achieve that.
The -g option to nnrpd is no longer supported. If you are verifying passwords against the system password database, see the ckpasswd(8) man page, and in particular the -s option. (A much better idea would be to just use PAM, which ckpasswd supports.)
Fixed a bug in "ctlinnd renumber" which was resetting the low and high water marks of empty newsgroups in the active file. This command now makes the low water mark one more than the real high water mark. The answers to LIST ACTIVE, GROUP and LISTGROUP have also been fixed to do that.
Support for bzip2-compressed batches (with bunbatch) has been added.
news.daily now processes innfeed dropped files during daily maintainance, running procbatch.
Support for runasuser and runasgroup parameters in inn.conf allows to set the news user and the news group under which the news server runs. Thanks to Ivan Shmakov for this feature.
New other options have been added to configuration files: ignore in incoming.conf, logstats, nnrpdflags and verifygroups in inn.conf, and log-time-format in innfeed.conf.
The --with-http-dir option has also been added to "configure" to set pathhttp in inn.conf.
The nntpactsync parameter has been renamed to incominglogfrequency in inn.conf.
The sasl.conf file has been removed in favour of new parameters in inn.conf to deal with TLS support: tlscafile, tlscapath, tlscertfile and tlskeyfile.
The overview.fmt file has been removed in favour of new parameters in inn.conf to deal with transition periods to accommodate overview reconfigurations. It is now possible to specify on the one hand the fields that should be advertised by nnrpd in response to LIST OVERVIEW.FMT and used for HDR, XHDR and XPAT requests (see the new extraoverviewadvertised parameter) and on the other hand the additional fields that should be silently generated (see the new extraoverviewhidden parameter).
Support for Berkeley DB versions prior to 4.3 has been dropped. You will have to use at least Berkeley DB 4.4; the recommended version is 4.7.
INN now builds entirely free of warnings from GCC with fairly aggressive warning options enabled. This involved lots of cleanup of const strings, signed versus unsigned type handling, correcting printf formats, and other changes that fixed obscure bugs and made INN's code more robust. Russ Allbery has also done considerable cleanup work on some of INN's internals, simplifying, refactoring, and removing duplicate code.
INN's test suite is now much more comprehensive and tests some high-level functions as well as more of the portability and utility function layer.
A lot of work has been done on documentation: improvements of existing documents, new documentation, and proof-reading. Sample configuration files are also more detailed.
INN is discussed on <inn-workers@lists.isc.org>. Please send any bug reports or patches to that list.
Russ Allbery
rra@isc.org
