Things to do for piuparts
=========================

Please also check the BTS, especially the bugs with a severity higher than
wishlist!


for 0.6x:

- README_server.txt: rewrite style a bit more. its super easy to setup now!

- ==== piuparts.debian.org specific configuration
 ^^^^^^^ scripts should be the headline, not piu.d.o

- move more bits from README_pejacevic.txt to README_server.txt

- split out README_protocol and README_piuparts.conf? -> in piuparts.conf manpage maybe?

- support multiple architectures: #762308
  - piuparts-report should have a list of available arch and list packages
    only available on untested archs in a new state
    "(depends-)not-available-on-tested-archs"
  - master should (per default) only schedule packages which are not available
    on the master arch to slaves of different archs ->
    "schedule-evenly-to-slaves = no"

- once that works, update README_pejacevic: search for "soon shall go into operation..." :)

- piuparts-master: keep track of to whom a reservation was given

- more stats and graphs:
  - new section stats page:
    - packages processed per day and section, master writes submissions.txt
      since 0.45 for all sections.
    - generate simple diagrams: number of source + binary packages in all
      single distros: lenny, squeeze, wheezy, jessie, sid.
    - graph about piuparts stati for all sections combined? (possible ignore
      successful)

- master should create the master and backup directories, if they dont exit. If master does that remove that sentence from README_server.txt again. same with slave and tmp.

- piuparts.conf.pejacevic: maybe use mirror via nfs (faster)

- also test packages from security.d.o

- maybe compress all logfiles

- look for a solution to use the global debian mirror for debian-backports,
  too, to avoid hardcoding a specific mirror in distros.conf

- if it weren't for 'slave-bin/slave_cleanup', the slave would only need
  rights to run "sudo piuparts" but nothing else. If we can clean this up,
  the sudoers.d should recommend sudo (lsof|kill|umount) for admins.

- if there were real schroot support, piuparts could be used without sudo.
  (#708663)

- use network namespaces to disable network during the tests:
  - < weasel> says: unshare --uts --ipc --net -- sh -c 'ip addr add 127.0.0.1/8 dev lo && ip link set dev lo up && stuff'
    and points to https://anonscm.debian.org/gitweb/?p=mirror/dsa-puppet.git;a=blob;f=modules/porterbox/files/dd-schroot-cmd#l104
  - problem might be access to the mirror, either (bind mounted) nfs access will still work in the chroots or do as its done on the porterboxes:
    apt-get install -d <foo>, unshare apt-get install foo

- add a sample config with all possible keys set to some useful value
  (like /usr/share/doc/apt/examples/configure-index.gz)

- generate piuparts.1.txt automatically from piuparts.py - see this blog post
  for a nice howto:
  http://andialbrecht.wordpress.com/2009/03/17/creating-a-man-page-with-distutils-and-optparse/
  - though this seems pretty complicated... maybe rather grep for
    parser.add_option and help= in piuparts.py ?!
    - requires merging all the additional infomation in piuparts.1.txt into
      piuarts.py
    - parsing piuparts --help output may be easier than parsing piuparts.py
  - elso: examples are duplicated in piuparts.1.txt and README.txt

- check the logfiles (especially pass/) for
  - "Exception in thread"
  - java stacktraces
  - "Can't locate .* in @INC"

- we should probably have an install test with --enable-recommends and
without --warn-on-others to avoid adding artificial barriers where package
subsets are configured


for 0.7x and later:

- install from git/Makefile: remove the need for /etc/piuparts

- accept a PIUPARTS_CONF environment variable everywhere to point to a different
  piuparts.conf

- write reportbug-like wrapper for mass bug filing (start simple, make it more
  sophisticated later).

- rework known_problems:
  - split detect_well_known_errors
    - parsing the logfiles should stay there
    - generating the html should be integrated into piuparts-report
  - use a number prefix for sorting
  - add title information
  - piuparts-report: "discover" the available known_problems, dont hardcode the
    list
  - drop _issue/_error duplication, have flags inside to indicate thether to
    generate _issues.tpl (pass/) and/or _error.tpl (fail/ bugged/ affected/)
  - rework known problems to a python-friendlier format

- the templates used by update-reports.py and detect_well_known_errors should
  be taken from /etc/piuparts/templates/ and not be included in the python source

- a redirect of http://piuparts.d.o/foo to http://p.d.o/source/f/foo.html would
  be nice


for 0.8x and later:

- find_default_debian_mirrors: 
  - check whether find_default_debian_mirrors produces something useful if
    sources.list does not exist (and sources.list.d/*.list is there instead)
  - maybe just copy sources.list(.d/*) instead?

- make it possible to call aptitude (or similar) instead of apt-get and allow to
  override the commandline arguments.

- mounting /proc and perhaps others (usbfs, sysfs, /dev/pts, etc.) in the chroot
  might be a good idea because some packages might need this.

- rewrite piuparts-analyze to run over all sections and cache BTS responses

- "decorate" (strike-through) bug links generated by piuparts-analyze to
  indicate resolved state (take package version into account!)

- report:
  - write stats about the reasons for failures, as its done with shell scripts
    now (piuparts-analyze.py is an existing "fragment".)
  - RSS feeds of logs
  - do more fancy R graphs, eg. also per state
  - link (and target) to piuparts.d.o configuration is static to pejacevic. should
    refer to the actual hosts configuration if running somewhere else

- not sure if it's a sensible thing to to, but provide a way to turn off
  debugging output for piuparts.py - see
  http://docs.python.org/library/logging.html

- commandline-switches for all programms

- move shell cronjobs functionality into master, slave & report

- automated testing of piuparts using an archive of known broken packages:
  - create archive of broken packages to provide test cases for piuparts testing.
  - create emacspeak-broken-dpkg-preconfigure package for broken repo. (then later
    put more broken packages in there and use that for testing piuparts)
