Chapter 9. Reporting and handling bugs

Submitters are expected to run reportbug or other tool that runs our bug script under the kernel version in question. The response to reports without this information should be a request to follow-up using reportbug. If we do not receive this information within a month of the request, the bug may be closed.

Exceptions:

Many submitters report bugs with the wrong severity. We interpret the criteria as follows and will adjust severity as appropriate:

We do not use user-tags. In order to aid bug triage we should make use of the standard tags and forwarded field defined by the BTS. In particular:

Generally we should not expect to be able to reproduce bugs without having similar hardware. We should consider:

If the bug is in upstream code and should be reported to an upstream mailing list (see Section 9.1.9, “Reporting bugs upstream”), forward the relevant message(s) there, keeping the submitter in the cc list. Mark the bug as forwarded, with a reference to the thread archive on lore.kernel.org.

Depending on the technical sophistication of the submitter and the service requirements of the system in question (e.g. whether it's a production server) we can request one or more of the following:

If the bug is in upstream code and should be reported to a bug tracker (see Section 9.1.9, “Reporting bugs upstream”), we ask the submitter to report it there in a specific category and to tell us the URL. We do not report the bug directly because follow-up questions from upstream need to go to the submitter, not to us. Given the upstream URL, we mark the bug as forwarded. bts-link then updates its status.

Many submitters search for a characteristic error message and treat this as indicating a specific bug. This can lead to many 'me too' follow-ups where, for example, the message indicates a driver bug and the second submitter is using a different driver from the original submitter.

In order to avoid the report turning into a mess of conflicting information about two or more different bugs:

Where the original report describes more than one bug ('...and other thing...'), we should clone it and deal with each separately.

Patches should normally be reviewed and accepted by the relevant upstream maintainer (aside from necessary adjustments for an older kernel version) before being applied.

We should always be polite to submitters. Not only is this implied by the Social Contract, but it is likely to lead to a faster resolution of the bug. If a submitter overrated the severity, quietly downgrade it. If a submitter has done something stupid, request that they undo that and report back. 'Sorry' and 'please' make a big difference in tone.

We will maintain general advice to submitters at https://wiki.debian.org/DebianKernelReportingBugs.

When a bug occurs in what upstream considers the current or previous stable release, or the latest release candidate, it should be reported to the upstream mailing list or bug tracker of the affected kernel component. In the relevant section of the current MAINTAINERS file, there may be a B: entry identifying where this is. Otherwise, there should be an L: entry referring to a general mailing list that can be used for bug reporting.

When the bug is in a stable/longterm branch that is still maintained, bugs should be reported to the <stable@vger.kernel.org> mailing list.

Any regressions in mainline or stable branches should additionally be reported to the <regressions@lists.linux.dev> mailing list. The kernel documentation has more information about reporting regressions.

When a bug is easy to reproduce locally but hard to get developers to reproduce (as is often true of workflow- or hardware-dependent bugs), it can be useful to compile and test a few versions to narrow down what changes introduced the regression.

You'll need to install some packages:

# apt-get install git gpg gpgv build-essential bc rsync kmod cpio bison flex libelf-dev libssl-dev

Next, fetch the Debian linux source package and the upstream source:

$ apt-get source linux  # this creates a directory linux-debian-version
$ git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux-upstream

Verify the tags for the old and new versions you want to test (usually, the last version you know works and the first version you know is broken):

$ cd linux-upstream
$ git -c gpg.program=../linux-debian-version/debian/bin/git-tag-gpg-wrapper tag -v vold-version
$ git -c gpg.program=../linux-debian-version/debian/bin/git-tag-gpg-wrapper tag -v vnew-version

Each command should show:

gpgv: Good signature from "Linus Torvalds <torvalds@linux-foundation.org>"

Check out the new version, then configure, build and test a binary package as explained in Section 4.2, “Building a custom kernel from Debian kernel source”:

$ git checkout vnew-version
$ make localmodconfig  # minimal configuration
$ scripts/config --disable DEBUG_INFO  # to keep the build reasonably small
$ scripts/config --disable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT
$ make bindeb-pkg
# dpkg -i ../linux-image-3.5.0_3.5.0-1_i386.deb  # substitute package name from the previous command
# reboot

If the bug doesn't show up, try again with the official configuration file from /boot. (If it still doesn't show up after that, declare victory and celebrate.)

Initialize the bisection process by declaring which versions worked and did not work:

$ cd linux-upstream
$ git bisect start vnew-version vold-version

Now Git checks out a version half-way in between to test. Build it, reusing the prepared configuration.

$ make silentoldconfig
$ make bindeb-pkg

Install the package, reboot, and test.

$ git bisect good  # if this version doesn't exhibit the bug
$ git bisect bad  # if it does
$ git bisect skip  # if some other bug makes it hard to test

And on to the next iteration:

$ make silentoldconfig
$ make bindeb-pkg

At the end of the process, the name of the "first bad commit" is printed, which is very useful for tracking down the bug. Narrowing down the regression range with a few rounds is useful even if you don't get that far; in that case, run git bisect log to produce a log. If you are the visual sort of person, git bisect visualize with the gitk package installed can show what is happening between steps.

See Christian Couder's article "Fighting regressions with git bisect" from kernel.org or the git-doc package for details.