fetchmail: IDLE - multiple daemons should be started

Bug #1021699 reported by D J Gardner
34
This bug affects 7 people
Affects Status Importance Assigned to Milestone
fetchmail (Ubuntu)
Triaged
Wishlist
Unassigned

Bug Description

Background: Fetchmail daemon can use the IMAP ILDE command, but as that never returns (hopefully!) that in turn means that only one server can be accessed per daemon. The current initscript only starts a single daemon.

I propose the attached alternative initscript which as well as the main /etc/fetchmailrc file also looks for .rc files in /etc/fetchmailrc.d. A separate daemon is launched for each config file, so that the IDLE directive can be used.
(thus /etc/fetchmailrc.d/jon_gmail.rc) can fetch from jon's gmail account, etc.)

Revision history for this message
D J Gardner (djgardner) wrote :
Revision history for this message
Matthias Andree (matthias-andree) wrote :

How do you make sure there is only one account per init script?

Revision history for this message
D J Gardner (djgardner) wrote :

A suitable grep | wc pipeline could check and issue a warning, but the whole point of my patch is that at the moment there is no way to have more than one account if one of them is using IDLE.

There is no problem with having multiple accounts per daemon (config file) if no IDLE is in use.
If IDLE is used, then that connection will only terminate on a network error, and other accounts will not be queried more than once (depending on position in the file.)

At the moment the sysadmin can 'lock up' their fetchmail daemon by adding the perfectly legal 'idle' annotation to one command.
My patch does not stop this, but provides a mechanism to use it safely.

Revision history for this message
Matthias Andree (matthias-andree) wrote : Re: [Bug 1021699] Re: fetchmail: IDLE - multiple daemons should be started

The limitation is documented, and I have always been considering killing
the IDLE option upstream. Some server NOTIFY extension that is not as
limited as IDLE is, might help, but I don't know how widespread that is.
 Else fetchmail would have to undergo internal restructuring and go on
wasting sockets. For large POP toasters, you can't use IDLE anyways, on
medium-sized, you might run out of file descriptors.

Revision history for this message
D J Gardner (djgardner) wrote :

For a home system, e.g. mine, where I have 3 or 4 accounts to check (more when the kids get older, of course), the FD limitiation is not significant at all, and of course IDLE is far nicer than polling, especially on servers which limit how often you can connect, but have no objection to IDLE use. So, personally I'd see removing IDLE as a backward step. (as far as I can see, none of the servers I connect to support NOTIFY). NOTIFY would presumably not help across multiple servers, either.

I really like fetchmail, but it struck me as annoying that the default initscript would only start the one daemon. I.e. for my use I don't see the bug as in the binary but in the initscript.

OK, a re-write of the codebase to make it capable of handling multiple concurrent connections would also solve it (sounds challenging, but I haven't looked at the source in years) and possibly use far less resources on a big system. I'm not in a position to fund such a project, sorry.

I felt my patch to the ubuntu/debian initscript a reasonable work-around until someone was.

For checking sanity of the relvant config files, how about this extra bit of shell script?

POLLING=`sed 's/#.*$//;/^\s*[Pp][Oo][Ll][Ll]\s/p;d' ${CONFIG} | wc -l `
IDLE=`sed 's/#.*$//;/^\s*[Ii][Dd][Ll][Ee]/p;d' ${CONFIG} | wc -l`
if test ${POLLING} -gt 1 -a ${IDLE} -gt 0
then
echo WARNING: ${CONFIG} uses the IDLE directive but tries to access ${POLLING} accounts. Processing will not pass the first account with an IDLE unless there is a problem with the connection.
fi

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in fetchmail (Ubuntu):
status: New → Confirmed
Bryce Harrington (bryce)
Changed in fetchmail (Ubuntu):
importance: Undecided → Wishlist
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Another suggestion would be to use systemd @service files, like the openvpn package does. You will get one daemon per vpn (in the openvpn case), and one per fetchmail config, if done properly.

Changed in fetchmail (Ubuntu):
status: Confirmed → Triaged
Revision history for this message
JanCeuleers (jan-ceuleers) wrote :

Here is a way (untested by me) to do what Andreas suggests: https://marek.dopiera.pl/2018/10/03/fetchmail-imap-idle-systemd-generators/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.