Broken /etc/fc lists on pkg update.

Check again.

IMO installpkg should be alias to upgradepkg. Or - if installpkg detects that package is already installed it will prompt what to do. It is to easy to screw your /etc/fc now.

I still think install should work exacly the same as upgrade, because there is no resonable need to do the difference other than punishing user for not using upgrade where it was needed.

But for now I just close this one.

I disagree with pjf that this ticket should be closed, therefore I reopen it.

I agree with dpc that under no circumstances should the user be punished for wrong decision on {install,upgrade}pkg in a way that leaves /etc/fc messed.

I disagree with dpc that "install should work exacly the same as upgrade", because there *is* a reasonable need to difference those modes:

• installpkg should bring the clean version of the package overriding everything concerning the package that was changed, so that if something gets broken, it can be quickly repaired. Additionally, a complete overwrite of the package (including /etc/fc) is the only way to bring new /etc/fc to the system (this is in my very concern)
• upgradepkg should be capable of being run automatically without disturbing admin and/or forcing him to update or recreate the configuration (therefore automatic updates should not include /etc/fc changes until some transition scheme is developed)

Let me summarize things:

• current status:
  • installpkg overwrites all files, including /etc/fc what makes child elements orphaned
  • upgradepkg does not overwrite /etc/fc

BOTH keep permissions for all files (r1176)

• desired status (by xmxwx):
  • installpkg removes all previous lists' child elements and then overwrites all files and their permissions
  • upgradepkg does not overwrite /etc/fc, keeps permissions of all files

I reassign the ticket to myself to notify that I've got an idea how to do it (removal prior to overwriting). Share your opinions. It would be nice if you agreed and my idea worked, because it would put an end to a neverending story of messed lists.

File permission overwriting issue has been dealt with in #97. The only thing to consider now is the idea of removing previous lists' child elements prior to overwriting /etc/fc.

...or maybe:

• upgradepkg should:
  • check whether FILELIST of the currently installed and the new package differ in /etc/fc/*
  • if they differ:
    • consider it as a fatal error since one should not upgrade to fc-incompatibile package, or
    • warn the user and let him interactively decide what to do now:
      • continue upgrade
      • perform installpkg instead of upgradepkg (with additional warning that this will remove the present configuration)
      • abort upgrade
    • warn the user, but continue without asking
  • store permissions of all files
  • remove all files except /etc/fc/* contained in the already installed pkg.tar.gz
  • extract all files except /etc/fc/*
  • restore permissions

• installpkg should:
  • remove current lists' child elements
  • remove all files (according to FILELIST in the already installed pkg), including /etc/fc/*
  • extract all files from the new package

What do you think about it?

• installpkg - I agree, it should provide a new, clean environment
• upgradepkg - we should come up with a way to let us do fc structure upgrades, imho:
  • generally, we should by default interactively warn the user and ask if he wants to upgrade the fc configuration (if not, then package upgrade is aborted)
    • if the pkg tool is run in batch mode or the user chooses some option, we should only notify him what has been changed in the configuration (so what he should probably correct)
  • I would compare the /etc/fc/* paths in FILELIST of package being upgraded with the new one, and if they differ, I would call a function in INSTALL file with first argument being the md5 sum of output of egrep 'etc/fc/' on *old* FILELIST
    • doing upgrade manually we can embed some logic and knowledge about particular configuration part being updated (consider playing with all after.sh, before.sh, deps.sh when trying to just copy the old configuration to a new structure)
    • extracted contents of *new* /etc/fc should be temporarily available somewhere for the INSTALL script (so it can just copy some new elements when it's sufficient)
    • remark: in INSTALL files I would store md5 sums in arrays/variables, not directly (consider incremental fc upgrades which will probably need to call the magic function with different, meaningful arguments)
    • interactive questions to user should have reasonable defaults in case we run in batch mode and stdin has already been closed
    • special care should be taken about list elements (well, maybe just simple loops)

Generally, we should provide an upgrade path to 2.1 and possibly to 3.0 without need to reinstall the whole system. After fixing this issue, we should maybe think how to make the system install/remove new/old packages when e.g. upgrading to 3.0.

The second idea probably for another ticket is providing a way for incremental upgrades of a *single package*, i.e. consider one did a typo in /etc/rc.d/rc.ospfd and want to fix it without need to rebuild the whole package.

Replying to pjf:

I would compare the /etc/fc/* paths in FILELIST of package being upgraded with the new one, and if they differ, I would call a function in INSTALL file with first argument being the md5 sum of output of egrep '^etc/fc/' on old FILELIST

An exit value of e.g. 2 from such function could mean that upgradepkg should just delete old fc entries and copy new ones (no support for upgrading list elements what imho would be a hassle). Such functionality would simplify cases when fc config is very simple and we just want to add a new fc entry.

Replying to pjf:

The second idea probably for another ticket is providing a way for incremental upgrades of a *single package*, i.e. consider one did a typo in /etc/rc.d/rc.ospfd and want to fix it without need to rebuild the whole package.

Hmm, how about making the pkgrel=1 packages the "starting points", and then making all pkgrel>1 packages containing only the changed files?

• we should take care about fc structure upgrades (ie. this ticket) only in case of pkgver change,
• pkgrel>1 packages should be downloaded and installed incrementally

Thus, when one want to bump the upstream software version number, change fc structure or just cause all files to be upgraded, then he will need to change pkgver and set pkgrel to 1.

A small step forward has been made in r1236.

TODO:

• tar patch to replace poor man's permission conservation
• enable /etc/fc structure upgrades (conforming to pjf's remarks)

An idea to use unionfs for /etc (#158) can appear handy in this task. During the upgrade, the new fc structure can be installed in the "underlying" filesystem not touching the current configuration.

There is a limited number of possible /etc/fc structure changes:

1. add new option

contents are to be "default" anyway

will be trivially solved by unionfs, since new files will just appear.

2. remove option

contents will be lost anyway

is not a problem since new fc parser will ignore old options. To keep everything clean, the old options can be immediately removed or (better) an ![act] can be added to let the user decide when to remove the option and its value (it could contain some important information that we cannot just remove on automated upgradepkg).

3. rename/move option

contents and meaning not changed

this can be done without user attention, however incremental upgrades and/or previous version detection is required.

4. change the storage manner

i.e. change type of field, extension of a single entry into a list of entries, list of files converted to a list of directories with the value stored somewhere inside, etc.

Just like 3 with an exception that a conversion script may turn out to be too complicated or the change could be essential so that the user should pay attention to it anyway.

Original problem fixed by Michał, next issues moved to:

• #159 - updates of Flatconf structures
• #160 - incremental package fixes

Milestone 2.1rc1 deleted