Debian Bug report logs - #688303
Please support concatenated bz2 files in zlib-functions

Full log

🔗 View this message in rfc822 format

X-Loop: [email protected]
Subject: Bug#688303: fails on concatenated bzip2 compressed Packages files (regression)
Reply-To: Thijs Kinkhorst <[email protected]>, [email protected]
Resent-From: Thijs Kinkhorst <[email protected]>
Resent-To: [email protected]
Resent-CC: APT Development Team <[email protected]>
X-Loop: [email protected]
Resent-Date: Fri, 21 Sep 2012 09:30:02 +0000
Resent-Message-ID: <[email protected]>
Resent-Sender: [email protected]
X-Debian-PR-Message: report 688303
X-Debian-PR-Package: apt
X-Debian-PR-Keywords: 
X-Debian-PR-Source: apt
Received: via spool by [email protected] id=B.13482197203355
          (code B); Fri, 21 Sep 2012 09:30:02 +0000
Received: (at submit) by bugs.debian.org; 21 Sep 2012 09:28:40 +0000
X-Spam-Checker-Version: SpamAssassin 3.3.1-bugs.debian.org_2005_01_02
	(2010-03-16) on buxtehude.debian.org
X-Spam-Level: 
X-Spam-Status: No, score=-12.7 required=4.0 tests=BAYES_00,DIGITS_LETTERS,
	FOURLA,FROMDEVELOPER,HAS_PACKAGE,RCVD_IN_DNSWL_LOW,XMAILER_REPORTBUG
	autolearn=ham version=3.3.1-bugs.debian.org_2005_01_02
X-Spam-Bayes: score:0.0000 Tokens: new, 31; hammy, 151; neutral, 308; spammy,
	0. spammytokens: hammytokens:0.000-+--(unknown), 0.000-+--(unknown),
	0.000-+--(unknown), 0.000-+--(unknown), 0.000-+--(unknown)
Received: from poisson.uvt.nl ([137.56.247.187])
	by buxtehude.debian.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.72)
	(envelope-from <[email protected]>)
	id 1TEzXL-0000rh-O2
	for [email protected]; Fri, 21 Sep 2012 09:28:40 +0000
Received: from localhost (localhost [127.0.0.1])
	by poisson.uvt.nl (Postfix) with ESMTP id 20403280254;
	Fri, 21 Sep 2012 11:28:31 +0200 (CEST)
X-Virus-Scanned: Debian amavisd-new at uvt.nl
Received: from poisson.uvt.nl ([127.0.0.1])
	by localhost (poisson.uvt.nl [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id Qat25yPPWS69; Fri, 21 Sep 2012 11:28:30 +0200 (CEST)
Received: from lagrange.uvt.nl (lagrange.uvt.nl [137.56.247.171])
	by poisson.uvt.nl (Postfix) with ESMTP id 986EE2800BA;
	Fri, 21 Sep 2012 11:28:30 +0200 (CEST)
Received: from dixan.uvt.nl (dixan.uvt.nl [137.56.165.88])
	by lagrange.uvt.nl (Postfix) with ESMTP id 93322400082;
	Fri, 21 Sep 2012 11:28:30 +0200 (CEST)
Received: by dixan.uvt.nl (Postfix, from userid 523090)
	id 819EF3189BA; Fri, 21 Sep 2012 11:28:30 +0200 (CEST)
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: Thijs Kinkhorst <[email protected]>
To: Debian Bug Tracking System <[email protected]>
Message-ID: <[email protected]>
X-Mailer: reportbug 6.4.3
Date: Fri, 21 Sep 2012 11:28:30 +0200
Delivered-To: [email protected]

Package: apt
Version: 0.9.7.4
Severity: important

Hi,

Using apt from wheezy with our APT repository fails:
201 URI Done: bzip2:/var/lib/apt/lists/partial/non-free.uvt.nl_debian_dists_squeeze_uvt_binary-amd64_Packages
RecivedHash: SHA256:99efd14ebb9fe582cc986f50502540030299595badefad71f5819e78c86a0c8d
ExpectedHash: SHA256:56ea2d1eb412a7121ad50f96e389c587e870ed7e80e893de0a66b3dbd9793015
[...]
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/non-free.uvt.nl_debian_dists_squeeze_uvt_binary-amd64_Packages  Hash Sum mismatch

The problem is that our APT archive contains a Packages.bz2 which is a
concatenation of other bz2 files (of sub-archives). Two concatenated
bzip2 files again form a valid bzip2 file. The b(un)zip2 command line
utils support this and it's a documented feature. APT only decompresses
the first bzip2 file in Packages.bz2 leading indeed to a different
hash sum than expected.

This is a regression from squeeze, because apt in squeeze used the
command line utils which do this correctly. In 0.9, apt switched to
using libbz2, however, it only calls the function that decompresses
one bzip2 stream. The bzip2 command line utils handle the looping over
input themselves with the appropriate library calls (search for
bzReadGetUnused).


Cheers,
Thijs

-- Package-specific info:

-- apt-config dump --

APT "";
APT::Architecture "amd64";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "false";
APT::Install-Suggests "0";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image.*";
APT::NeverAutoRemove:: "^kfreebsd-image.*";
APT::NeverAutoRemove:: "^linux-restricted-modules.*";
APT::NeverAutoRemove:: "^linux-ubuntu-modules-.*";
APT::NeverAutoRemove:: "^gnumach$";
APT::NeverAutoRemove:: "^gnumach-image.*";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Periodic "";
APT::Periodic::AutocleanInterval "14";
APT::Cache-Limit "134217728";
APT::Architectures "";
APT::Architectures:: "amd64";
APT::Compressor "";
APT::Compressor::. "";
APT::Compressor::.::Name ".";
APT::Compressor::.::Extension "";
APT::Compressor::.::Binary "";
APT::Compressor::.::Cost "1";
APT::Compressor::gzip "";
APT::Compressor::gzip::Name "gzip";
APT::Compressor::gzip::Extension ".gz";
APT::Compressor::gzip::Binary "gzip";
APT::Compressor::gzip::Cost "2";
APT::Compressor::gzip::CompressArg "";
APT::Compressor::gzip::CompressArg:: "-9n";
APT::Compressor::gzip::UncompressArg "";
APT::Compressor::gzip::UncompressArg:: "-d";
APT::Compressor::bzip2 "";
APT::Compressor::bzip2::Name "bzip2";
APT::Compressor::bzip2::Extension ".bz2";
APT::Compressor::bzip2::Binary "bzip2";
APT::Compressor::bzip2::Cost "3";
APT::Compressor::bzip2::CompressArg "";
APT::Compressor::bzip2::CompressArg:: "-9";
APT::Compressor::bzip2::UncompressArg "";
APT::Compressor::bzip2::UncompressArg:: "-d";
APT::Compressor::xz "";
APT::Compressor::xz::Name "xz";
APT::Compressor::xz::Extension ".xz";
APT::Compressor::xz::Binary "xz";
APT::Compressor::xz::Cost "4";
APT::Compressor::xz::CompressArg "";
APT::Compressor::xz::CompressArg:: "-6";
APT::Compressor::xz::UncompressArg "";
APT::Compressor::xz::UncompressArg:: "-d";
APT::Compressor::lzma "";
APT::Compressor::lzma::Name "lzma";
APT::Compressor::lzma::Extension ".lzma";
APT::Compressor::lzma::Binary "xz";
APT::Compressor::lzma::Cost "5";
APT::Compressor::lzma::CompressArg "";
APT::Compressor::lzma::CompressArg:: "--format=lzma";
APT::Compressor::lzma::CompressArg:: "-9";
APT::Compressor::lzma::UncompressArg "";
APT::Compressor::lzma::UncompressArg:: "--format=lzma";
APT::Compressor::lzma::UncompressArg:: "-d";
APT::Compressor::::Name "";
APT::Compressor::::Extension ".";
APT::Compressor::::Binary "";
APT::Compressor::::Cost "100";
APT::Compressor::::CompressArg "";
APT::Compressor::::CompressArg:: "-9";
APT::Compressor::::UncompressArg "";
APT::Compressor::::UncompressArg:: "-d";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::solvers "";
Dir::Bin::solvers:: "/usr/lib/apt/solvers";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Bin::bzip2 "/bin/bzip2";
Dir::Bin::xz "/usr/bin/xz";
Dir::Media "";
Dir::Media::MountPath "/media/apt";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Dir::Ignore-Files-Silently:: "\.save$";
Dir::Ignore-Files-Silently:: "\.orig$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom/";
Acquire::Pdiffs "false";
Acquire::Languages "";
Acquire::Languages:: "en";
Acquire::Languages:: "none";
DPkg "";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "exec sync";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
Aptitude "";
Aptitude::Recommends-Important "false";
CommandLine "";
CommandLine::AsString "apt-config dump";

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (400, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-3-amd64 (SMP w/8 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash

Versions of packages apt depends on:
ii  debian-archive-keyring  2012.4
ii  gnupg                   1.4.12-4+b1
ii  libapt-pkg4.12          0.9.7.4
ii  libc6                   2.13-35
ii  libgcc1                 1:4.7.1-7
ii  libstdc++6              4.7.1-7

apt recommends no packages.

Versions of packages apt suggests:
pn  apt-doc     <none>
ii  aptitude    0.6.8.1-1
ii  dpkg-dev    1.16.8
ii  python-apt  0.8.4
ii  xz-utils    5.1.1alpha+20120614-1

-- no debconf information

Send a report that this bug log contains spam.