Adding autopkgtests to GNU packages

[Santiago Vila]

Dear mentors:

Many GNU packages have built-in tests that test the just-built programs.

Is there a general procedure/trick/idiom to modify them so that they test
the installed version instead? (so that they can be reused with minimal
changes).

(I'd like to do this for several packages without reinventing the wheel).

Thanks.

[Santiago Ruano Rincón]

Hey tocayo,

Post by Santiago Vila
Many GNU packages have built-in tests that test the just-built programs.
Is there a general procedure/trick/idiom to modify them so that they test
the installed version instead? (so that they can be reused with minimal
changes).
(I'd like to do this for several packages without reinventing the wheel).

This is what grep does [0]:

#!/bin/sh
set -e

export DEB_BUILD_PROFILES="nocheck nodoc" DEB_BUILD_OPTIONS="nocheck nodoc"

dpkg-source --before-build `pwd`
dpkg-buildpackage -rfakeroot --target=clean
dpkg-buildpackage -rfakeroot --target=build

make -C tests check-am

[0] https://salsa.debian.org/debian/grep/-/blob/9f6e70143249f6c92fb7158a74d288332e132df4/debian/tests/upstream-test-suite

HTH,

-- Santiago

[Andreas Tille]

Hi Santiago,

Post by Santiago Ruano Rincón

Post by Santiago Vila
Many GNU packages have built-in tests that test the just-built programs.

This reminds me to my attempt to write some autopkgtest for tar. I've
created an according bug report[1] (in CC) but I did not went really far
to run the upstream test suite in an autopkgtest. I needed some local
header files from the build which is definitely not a good idea and
the final autopkgtest failed even then[2].

Currently I have no time to continue with this. However, if someone
wants to add an autopkgtest to one of our most used packages this would
be great.

Kind regards
Andreas.

Post by Santiago Ruano Rincón

Post by Santiago Vila
Is there a general procedure/trick/idiom to modify them so that they test
the installed version instead? (so that they can be reused with minimal
changes).
(I'd like to do this for several packages without reinventing the wheel).

#!/bin/sh
set -e
export DEB_BUILD_PROFILES="nocheck nodoc" DEB_BUILD_OPTIONS="nocheck nodoc"
dpkg-source --before-build `pwd`
dpkg-buildpackage -rfakeroot --target=clean
dpkg-buildpackage -rfakeroot --target=build
make -C tests check-am
[0] https://salsa.debian.org/debian/grep/-/blob/9f6e70143249f6c92fb7158a74d288332e132df4/debian/tests/upstream-test-suite

[1] https://bugs.debian.org/1068576
[2] https://salsa.debian.org/debian/tar/-/jobs/5494107

[Guillem Jover]

Hi!

Post by Santiago Ruano Rincón
#!/bin/sh
set -e
export DEB_BUILD_PROFILES="nocheck nodoc" DEB_BUILD_OPTIONS="nocheck nodoc"
dpkg-source --before-build `pwd`

Nit: I'd probably use «$(pwd)» here, or perhaps simply «.». :)

Post by Santiago Ruano Rincón
dpkg-buildpackage -rfakeroot --target=clean
dpkg-buildpackage -rfakeroot --target=build

You should not need to pass -rfakeroot, which has been the default,
if available, since dpkg 1.14.7 (from 2007-10).

Post by Santiago Ruano Rincón
make -C tests check-am

Is this really testing the installed executables? The convention is
for the «check» target to use the built binaries for the tests. If you
had only configured the package (instead of a full build), and the test
suite would not hardcode program pathnames, then I assume that might
implicitly test the installed binaries, but the above looks like might
not be doing what is expected?

Thanks,
Guillem

[Jeroen Ploemen]

On Thu, 28 Nov 2024 12:43:05 +0100

Post by Santiago Vila
Many GNU packages have built-in tests that test the just-built
programs.
Is there a general procedure/trick/idiom to modify them so that
they test the installed version instead? (so that they can be
reused with minimal changes).

What often works well with simple utils is creating a symlink
pointing to the packaged binary, in whatever location the testsuite
expects the freshly build one (typically the build root).

[Guillem Jover]

Hi!

[ Perhaps you already managed to solve this, but just in case. :) ]

Post by Santiago Vila
Many GNU packages have built-in tests that test the just-built programs.
Is there a general procedure/trick/idiom to modify them so that they test
the installed version instead? (so that they can be reused with minimal
changes).

With autotools, there is infrastructure for running the test suite but
using the installed binaries in the form of the «installcheck» target.
This might need explicit support in the build system, so you'd need to
check whether that is the case for each of these upstreams. If it is
not, then you could perhaps either ask upstream to add it or provide
a patch (it should in general not be be complicated).

See for example in dpkg, the upstream part in the files:

src/Makefile.am (installcheck-local)
build-aux/autotest.am (autotest-installcheck)

And the autopkgtest usage in for example:

debian/tests/test-not-root

Thanks,
Guillem