Debian Bug report logs - #942487

reprepro imposes arbitrary limits on control files that are successfully parsed by other debian tools

Reported by: Raphaël Hertzog <hertzog@debian.org> Date: Thu, 17 Oct 2019 07:12:02 UTC Severity: important Found in version reprepro/5.3.0-1 Fixed in version reprepro/5.3.0-1.1 Done: Ximin Luo <infinity0@debian.org> Bug is archived. No further changes may be made.

Toggle useless messages

Report forwarded to debian-bugs-dist@lists.debian.org, hertzog@debian.org, sylvestre@debian.org, nicoo@debian.org, wolfgang@silbermayr.at, git@rxv.cc, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 07:12:04 GMT) (full text, mbox, link).

Acknowledgement sent to Raphaël Hertzog <hertzog@debian.org> :

New Bug report received and forwarded. Copy sent to hertzog@debian.org, sylvestre@debian.org, nicoo@debian.org, wolfgang@silbermayr.at, git@rxv.cc, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 07:12:04 GMT) (full text, mbox, link).

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Raphaël Hertzog <hertzog@debian.org> To: Debian Bug Tracking System <submit@bugs.debian.org> Subject: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 08:59:18 +0200

Source: rust-web-sys Version: 0.3.28-1 Severity: critical Justification: breaks unrelated software User: devel@kali.org Usertags: origin-kali $ apt-cache show librust-web-sys-dev|grep ^Provides:|wc -c 277998 This is a serious abuse of the Provides header... for a package that provides 719 files. And it breaks unrelated software processing the Debian archive, namely reprepro: Error parsing ./lists/unstable_unstable_main_arm64_Packages line 914113: Ridiculous long (>= 256K) control chunk! For this reason, I'm going to NMU the package and disable/reduce the Provides field until you find a reasonable solution. Cheers, -- System Information: Debian Release: bullseye/sid APT prefers oldoldstable APT policy: (500, 'oldoldstable'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.2.0-3-amd64 (SMP w/4 CPU cores) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr_FR.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 07:33:09 GMT) (full text, mbox, link).

Acknowledgement sent to Ansgar <ansgar@43-1.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 07:33:09 GMT) (full text, mbox, link).

Message #10 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ansgar <ansgar@43-1.org> To: 942487@bugs.debian.org Subject: Re: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 09:25:49 +0200

Hi, in addition a 256kB Provides field seems very hard on the total size of the Packages index. Please don't do that... Ansgar

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 07:48:03 GMT) (full text, mbox, link).

Acknowledgement sent to Sylvestre Ledru <sylvestre@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 07:48:03 GMT) (full text, mbox, link).

Message #15 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Sylvestre Ledru <sylvestre@debian.org> To: Ansgar <ansgar@43-1.org>, 942487@bugs.debian.org Subject: Re: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 09:34:35 +0200

Le 17/10/2019 à 09:25, Ansgar a écrit : > Hi, > > in addition a 256kB Provides field seems very hard on the total size of > the Packages index. Please don't do that... > To be clear, this isn't done by a human. This is a tool generating this for us. I will see how to add a lintian check to block that from happening again. Sylvestre

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 07:54:10 GMT) (full text, mbox, link).

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 07:54:10 GMT) (full text, mbox, link).

Message #20 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org> To: 942487@bugs.debian.org Subject: Re: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 09:39:07 +0200

On Thu, 17 Oct 2019, Raphaël Hertzog wrote: > For this reason, I'm going to NMU the package and disable/reduce the Provides > field until you find a reasonable solution. Uploaded rust-web-sys_0.3.28-1.1_source.changes. It's still 150K but should make reprepro happy. I believe it's unreasonable to hardcode so many "interfaces" in the provides field, in particular when you represent each interface with 4 different versioned variants. Will all the package really have an auto-generated Depends line listing all those interfaces ? FWIW, IRC discussion on #debian-devel concurred that it was really not reasonable. And as a data point: 09:30 <ansgar> Longest Provides currently in unstable/amd64: 277987 librust-web-sys; 59926 librust-winapi; 7505 oca-addons-account; 3357 librust-x11+default-dev; 3280 librust-slog+default-dev 09:31 <ansgar> So at least it's only very few packages that have this problem. But from the top 5, 4 are rust packages. And this one is like 40 times bigger than the next non-rust package with a big provides line... Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 08:21:04 GMT) (full text, mbox, link).

Acknowledgement sent to Ansgar <ansgar@43-1.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 08:21:04 GMT) (full text, mbox, link).

Message #25 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ansgar <ansgar@43-1.org> To: 942487@bugs.debian.org Subject: Re: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 10:17:20 +0200

Sylvestre Ledru writes: > Le 17/10/2019 à 09:25, Ansgar a écrit : >> in addition a 256kB Provides field seems very hard on the total size of >> the Packages index. Please don't do that... >> > To be clear, this isn't done by a human. This is a tool generating > this for us. Then the tool has a problem. It really seems excessive to add four Provides for every feature: librust-web-sys+abortcontroller-dev (= 0.3.28-1) librust-web-sys-0+abortcontroller-dev (= 0.3.28-1) librust-web-sys-0.3+abortcontroller-dev (= 0.3.28-1) librust-web-sys-0.3.28+abortcontroller-dev (= 0.3.28-1) Clearly the information is already available via the version information... (I doubt even adding a Provides for every feature should be done.) Ansgar

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 10:15:23 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 10:15:23 GMT) (full text, mbox, link).

Message #30 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: Ansgar <ansgar@43-1.org>, 942487@bugs.debian.org Cc: Raphaël Hertzog <hertzog@debian.org>, Sylvestre Ledru <sylvestre@debian.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 10:06:00 +0000

Control: tags -1 + wontfix Ansgar: > Sylvestre Ledru writes: >> Le 17/10/2019 à 09:25, Ansgar a écrit : >>> in addition a 256kB Provides field seems very hard on the total size of >>> the Packages index. Please don't do that... >>> >> To be clear, this isn't done by a human. This is a tool generating >> this for us. > > Then the tool has a problem. It really seems excessive to add four > Provides for every feature: > > librust-web-sys+abortcontroller-dev (= 0.3.28-1) > librust-web-sys-0+abortcontroller-dev (= 0.3.28-1) > librust-web-sys-0.3+abortcontroller-dev (= 0.3.28-1) > librust-web-sys-0.3.28+abortcontroller-dev (= 0.3.28-1) > > Clearly the information is already available via the version > information... (I doubt even adding a Provides for every feature should > be done.) > The tool's algorithm was suggested by the maintainer of dpkg and has his blessing. It is partly due to limitations in dpkg, see #901827 for details. This bug is only really fixable once the dpkg limitation is fixed, which can't happen (apparently) for a whole stable release cycle. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Added tag(s) wontfix. Request was from Ximin Luo <infinity0@debian.org> to 942487-submit@bugs.debian.org . (Thu, 17 Oct 2019 10:15:23 GMT) (full text, mbox, link).

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 10:33:03 GMT) (full text, mbox, link).

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 10:33:03 GMT) (full text, mbox, link).

Message #37 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org> To: Ximin Luo <infinity0@debian.org> Cc: Ansgar <ansgar@43-1.org>, 942487@bugs.debian.org, Sylvestre Ledru <sylvestre@debian.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 12:30:39 +0200

On Thu, 17 Oct 2019, Ximin Luo wrote: > Control: tags -1 + wontfix This is clearly not acceptable. You can't ignore problems like this one. I saw you already broke debian-installer once with the former packages that overflowed the 16K limit of cdebootstrap. Now it's the turn of reprepro and this one is harder to fix because there are real servers running stable version of reprepro, etc. > The tool's algorithm was suggested by the maintainer of dpkg and has his > blessing. It is partly due to limitations in dpkg, see #901827 for > details. The algorithm is one thing... but the design of your tool is another thing. dpkg has dpkg-shlibdeps to build dependencies based on exported information by various package (through /var/lib/dpkg/info/*.{shlibs,symbols}). cargo should build the same infrastructure, i.e. have a /var/lib/dpkg/info/foo.cargo used by dh-cargo to build the correct dependency. Don't abuse the "Provides" field when you have such a volume of interfaces to document. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 10:33:05 GMT) (full text, mbox, link).

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 10:33:05 GMT) (full text, mbox, link).

Message #42 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org> To: Sylvestre Ledru <sylvestre@debian.org> Cc: Ansgar <ansgar@43-1.org>, 942487@bugs.debian.org Subject: Re: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 12:32:00 +0200

Hi, On Thu, 17 Oct 2019, Sylvestre Ledru wrote: > I will see how to add a lintian check to block that from happening again. FWIW, I already filed #942493 against lintian this morning. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 11:03:06 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 11:03:06 GMT) (full text, mbox, link).

Message #47 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: Raphael Hertzog <hertzog@debian.org>, 942487@bugs.debian.org Cc: Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 10:59:00 +0000

Raphael Hertzog: > On Thu, 17 Oct 2019, Ximin Luo wrote: >> Control: tags -1 + wontfix > > This is clearly not acceptable. You can't ignore problems like this one. > I saw you already broke debian-installer once with the former packages > that overflowed the 16K limit of cdebootstrap. Now it's the turn of > reprepro and this one is harder to fix because there are real servers > running stable version of reprepro, etc. > >> The tool's algorithm was suggested by the maintainer of dpkg and has his >> blessing. It is partly due to limitations in dpkg, see #901827 for >> details. > > The algorithm is one thing... but the design of your tool is another > thing. > > dpkg has dpkg-shlibdeps to build dependencies based on exported > information by various package (through > /var/lib/dpkg/info/*.{shlibs,symbols}). > > cargo should build the same infrastructure, i.e. have a > /var/lib/dpkg/info/foo.cargo used by dh-cargo to build the correct > dependency. > > Don't abuse the "Provides" field when you have such a volume of > interfaces to document. > Can you please explain why 256 KB provides field is "abuse"? Do you have some concrete suggestions on how to improve the tool to reduce this "abuse"? X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 11:09:04 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 11:09:04 GMT) (full text, mbox, link).

Message #52 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: Raphael Hertzog <hertzog@debian.org>, 942487@bugs.debian.org Cc: Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 11:05:00 +0000

Ximin Luo: > Raphael Hertzog: >> On Thu, 17 Oct 2019, Ximin Luo wrote: >>> Control: tags -1 + wontfix >> >> This is clearly not acceptable. You can't ignore problems like this one. >> I saw you already broke debian-installer once with the former packages >> that overflowed the 16K limit of cdebootstrap. Now it's the turn of >> reprepro and this one is harder to fix because there are real servers >> running stable version of reprepro, etc. >> >>> The tool's algorithm was suggested by the maintainer of dpkg and has his >>> blessing. It is partly due to limitations in dpkg, see #901827 for >>> details. >> >> The algorithm is one thing... but the design of your tool is another >> thing. >> >> dpkg has dpkg-shlibdeps to build dependencies based on exported >> information by various package (through >> /var/lib/dpkg/info/*.{shlibs,symbols}). >> >> cargo should build the same infrastructure, i.e. have a >> /var/lib/dpkg/info/foo.cargo used by dh-cargo to build the correct >> dependency. >> >> Don't abuse the "Provides" field when you have such a volume of >> interfaces to document. >> > > Can you please explain why 256 KB provides field is "abuse"? > > Do you have some concrete suggestions on how to improve the tool to reduce this "abuse"? > BTW, the tool is run not at build time but to generate the source package. So it can't use these "foo.cargo" files, because you don't need to install all of the dependencies in order to use the tool. So yes, we need a concrete suggestion on improving the tool, rather than wild hyperbole that its output is "abuse". It is 2019. If a tool can't handle 256 KB of data, I'd say the tool is at fault and not the 256 KB of data. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 11:42:06 GMT) (full text, mbox, link).

Acknowledgement sent to fin4478 fin4478 <fin4478@hotmail.com> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 11:42:06 GMT) (full text, mbox, link).

Message #57 received at 942487@bugs.debian.org (full text, mbox, reply):

From: fin4478 fin4478 <fin4478@hotmail.com> To: "942487@bugs.debian.org" <942487@bugs.debian.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 11:38:45 +0000

On Thu, 17 Oct 2019 10:06:00 +0000 Ximin Luo <infinity0@debian.org> wrote: > Control: tags -1 + wontfix > Debian should remove this kind of packages immediately. No way to make a Debian Sid installer with Simple-CDD now. The Rust language is for people who make null pointer and buffer overflow bugs and those people should not program anything. This bug proves my point: rust programmers can break even the Debian repository. A nice bomb this package is.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 14:51:03 GMT) (full text, mbox, link).

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 14:51:03 GMT) (full text, mbox, link).

Message #62 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org> To: Ximin Luo <infinity0@debian.org> Cc: 942487@bugs.debian.org, Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 16:46:25 +0200

On Thu, 17 Oct 2019, Ximin Luo wrote: > Can you please explain why 256 KB provides field is "abuse"? Because that's the amount of metadata required for 250 common packages. > Do you have some concrete suggestions on how to improve the tool to reduce this "abuse"? Yes, I gave you one. > BTW, the tool is run not at build time but to generate the source > package. So it can't use these "foo.cargo" files, because you don't need > to install all of the dependencies in order to use the tool. If you run a tool to generate the source package, you can include whatever call you want during your source package build. i.e. you control debian/rules too. And you can process the source package and/or the binary package built to create those meta-information and also to use the existing meta-information on the system. > It is 2019. If a tool can't handle 256 KB of data, I'd say the tool is > at fault and not the 256 KB of data. You are being arrogant. Replying in the same tone, I would say that the design of your tool suck. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 16:33:05 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 16:33:05 GMT) (full text, mbox, link).

Message #67 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: Raphael Hertzog <hertzog@debian.org> Cc: 942487@bugs.debian.org, Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 16:31:00 +0000

Raphael Hertzog: > On Thu, 17 Oct 2019, Ximin Luo wrote: >> Can you please explain why 256 KB provides field is "abuse"? > > Because that's the amount of metadata required for 250 common packages. > So? There are some Debian packages that have much more than 250 times the data of common packages. No tools break with that, nor are there suggestions that those big packages "suck". >> Do you have some concrete suggestions on how to improve the tool to reduce this "abuse"? > > Yes, I gave you one. > It doesn't work. >> BTW, the tool is run not at build time but to generate the source >> package. So it can't use these "foo.cargo" files, because you don't need >> to install all of the dependencies in order to use the tool. > > If you run a tool to generate the source package, you can include > whatever call you want during your source package build. i.e. you > control debian/rules too. And you can process the source package > and/or the binary package built to create those meta-information > and also to use the existing meta-information on the system. > This isn't a concrete suggestion, it's a generic vacuous statement about how package builds work, and is true for what already happens. The existing tool already "processes the source package [..] to create those meta-information", namely Provides fields corresponding to what's needed according to what's defined by upstream. >> It is 2019. If a tool can't handle 256 KB of data, I'd say the tool is >> at fault and not the 256 KB of data. > > You are being arrogant. Replying in the same tone, I would say that the > design of your tool suck. > That's cool, and it really doesn't persuade me to have any sympathy towards your issue. Note that the next time this package is automatically regenerated, your "fixes" will be undone. Please be a bit more self-critical about your own opinion. Have you considered the possibility that it is the reading tool (reprepro in this case) that "sucks" and not the writing tool? X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 17:03:02 GMT) (full text, mbox, link).

Message #70 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ansgar <ansgar@43-1.org> To: 942487@bugs.debian.org Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 19:00:15 +0200

Ximin Luo writes: > Raphael Hertzog: >> Don't abuse the "Provides" field when you have such a volume of >> interfaces to document. > > Can you please explain why 256 KB provides field is "abuse"? The Packages index is a shared resource by all packages and every Debian user has to download and process the full packages index; adding excessive amounts of data should therefore be avoided. (The 256 KB added to the Packages index are larger than the entire source (compressed) source package...) > Do you have some concrete suggestions on how to improve the tool to > reduce this "abuse"? Don't generate virtual packages (Provides) for every feature; don't generate four virtual packages for every feature. Ansgar

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 19:45:03 GMT) (full text, mbox, link).

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 19:45:03 GMT) (full text, mbox, link).

Message #75 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org> To: Ximin Luo <infinity0@debian.org> Cc: 942487@bugs.debian.org, Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 21:40:59 +0200

Hello Ximin, On Thu, 17 Oct 2019, Ximin Luo wrote: > >> Do you have some concrete suggestions on how to improve the tool to reduce this "abuse"? > > > > Yes, I gave you one. > > It doesn't work. Look, I'm not a cargo/rust expert, I won't design the tool for you but I implemented dpkg-gensymbols and the symbols support for dpkg-shlibdeps and I'm pretty confident that such a solution can work for your case too. We are not adding a provides to libc6 for each symbol that the library is exporting. And you should not add a provides for each "interface" (or whatever it's called in rust) that a package is providing. You should export the list of interfaces in a separate metadata file thas is not part of the generated "Packages" file and you should have a tool to generate the binary dependencies pointing back to the correct package that is exporting the interface. It might not be as flexible as the current approach as it might require rebuilds when the package providing the interface changes, but that's quite usual in Debian. > > You are being arrogant. Replying in the same tone, I would say that the > > design of your tool suck. > > That's cool, and it really doesn't persuade me to have any sympathy > towards your issue. Note that the next time this package is > automatically regenerated, your "fixes" will be undone. Note that if you re-introduce the issue I will ask ftpmasters to remove the package and/or ask the tech-ctte to decide about it. (I can play that game too... but it's not helping) You can't just ignore problems when you are breaking the infrastructure of derivative distributions and users... right now the problem is limited to unstable and I'm the first to have discovered it but I'm pretty confident that others will hit it as well. And as I said, those servers are not running unstable so if you really want to go down the route of fixing reprepro (while ignoring the fact that Ansgar, who is a ftpmaster, is asking you to not continue with such a Provides header), you will have to get fixes pushed to stable... > Please be a bit more self-critical about your own opinion. Have you > considered the possibility that it is the reading tool (reprepro in this > case) that "sucks" and not the writing tool? Yes, reprepro sucks in some ways. But a design that puts 270K of metadata in a single Provides line sucks too. But reprepro is in wide use and your new package is the first one to trigger the limit. You can't just ignore the reality, you have to cope with the fact that we have reprepro users and that we can't deploy a package with 270K-long Provides header currently. (And IMO we should never allow this but that's another discussion) Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 20:51:03 GMT) (full text, mbox, link).

Acknowledgement sent to kpcyrd <kpcyrd@rxv.cc> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 20:51:03 GMT) (full text, mbox, link).

Message #80 received at 942487@bugs.debian.org (full text, mbox, reply):

From: kpcyrd <kpcyrd@rxv.cc> To: Raphael Hertzog <hertzog@debian.org>, 942487@bugs.debian.org Cc: Ximin Luo <infinity0@debian.org>, Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 20:39:35 +0000

On Thu, Oct 17, 2019 at 09:40:59PM +0200, Raphael Hertzog wrote: > Look, I'm not a cargo/rust expert, I won't design the tool for you but I > implemented dpkg-gensymbols and the symbols support for dpkg-shlibdeps > and I'm pretty confident that such a solution can work for your case too. > > We are not adding a provides to libc6 for each symbol that the library is > exporting. And you should not add a provides for each "interface" (or > whatever it's called in rust) that a package is providing. > > You should export the list of interfaces in a separate metadata file thas > is not part of the generated "Packages" file and you should have a tool to > generate the binary dependencies pointing back to the correct package that > is exporting the interface. > > It might not be as flexible as the current approach as it might require > rebuilds when the package providing the interface changes, but that's > quite usual in Debian. I'm suspecting there might be some confusion here, just to make sure we're all on the same page: This isn't about symbols and we aren't generating virtual packages per interface intentionally, we're generating virtual packages based on something the rust world is referring to as "features", which are basically on/off switches that are used to drive conditional compilation and (this is important) optional dependency resolution. Not every feature gets its own package, this only happens if a feature has additional optional dependencies. For everything else we add a provides entry to keep the number of binary packages in debian low. The reason we need to depend on packages with their features specified is so we resolve the dependency tree of a rust package correctly, in a loop free way, and with support for semver similar to how cargo does it. This has worked nicely for >600 packages so far. The reason web-sys upstream has this many features and lists interfaces explicitly is likely due to LTO limitations related to wasm and the required js glue, not something we have much control over.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 22:27:06 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 22:27:06 GMT) (full text, mbox, link).

Message #85 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: Ansgar <ansgar@43-1.org>, 942487@bugs.debian.org Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 22:26:00 +0000

Ansgar: > Ximin Luo writes: >> Raphael Hertzog: >>> Don't abuse the "Provides" field when you have such a volume of >>> interfaces to document. >> >> Can you please explain why 256 KB provides field is "abuse"? > > The Packages index is a shared resource by all packages and every Debian > user has to download and process the full packages index; adding > excessive amounts of data should therefore be avoided. (The 256 KB > added to the Packages index are larger than the entire source > (compressed) source package...) > Only a few (<10) packages out of 600+ rust packages have very large Provides fields. The main Debian tools are coping with it fine. It's one derivative unofficial tool that's unable to cope. I don't see why we should introduce artificial limits (and increase workload) in order to cater to one old tool. >> Do you have some concrete suggestions on how to improve the tool to >> reduce this "abuse"? > > Don't generate virtual packages (Provides) for every feature; don't > generate four virtual packages for every feature. > Simply "not doing this" quite literally will break everything in the Debian Rust ecosystem of packages, and is quite directly analogous to asking dpkg to "ignore all Provides". Several alternatives were explored in the past and the current option was settled upon in #901827. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Thu, 17 Oct 2019 22:33:02 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Thu, 17 Oct 2019 22:33:02 GMT) (full text, mbox, link).

Message #90 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: Raphael Hertzog <hertzog@debian.org>, 942487@bugs.debian.org Cc: Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Thu, 17 Oct 2019 22:29:00 +0000

Raphael Hertzog: > [..] > > It might not be as flexible as the current approach as it might require > rebuilds when the package providing the interface changes, but that's > quite usual in Debian. > This isn't suitable for Rust, there will be too many rebuilds needed (basically half the ecosystem for every change, which happens 4-5 times a month) and I and I suspect everyone else will stop maintaining Rust packages if we are forced to do this. > [..] > > But reprepro is in wide use and your new package is the first one to > trigger the limit. You can't just ignore the reality, you have to cope > with the fact that we have reprepro users and that we can't deploy > a package with 270K-long Provides header currently. > Who is using reprepro to archive Debian Rust packages? That's the first time I've heard of this. I suspect this is a small number (its popcon [1] is less than that of rustc itself), and that they will be perfectly happy to upgrade to a fixed version of reprepro. [1] https://qa.debian.org/developer.php?login=brlink@debian.org#reprepro X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Fri, 18 Oct 2019 09:27:03 GMT) (full text, mbox, link).

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Fri, 18 Oct 2019 09:27:03 GMT) (full text, mbox, link).

Message #95 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org> To: Ximin Luo <infinity0@debian.org> Cc: 942487@bugs.debian.org, Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Fri, 18 Oct 2019 11:23:50 +0200

On Thu, 17 Oct 2019, Ximin Luo wrote: > Who is using reprepro to archive Debian Rust packages? That's the first Anybody who is mirroring Debian unstable with reprepro right now. I have no special interest in rust, but I do maintain a debian derivative that we build with reprepro merging debian testing and our own packages (and we mirror unstable as well because we cherry-pick fixes from unsatble from time to time, hence the reason why I discovered this before it has his testing). > time I've heard of this. I suspect this is a small number (its popcon > [1] is less than that of rustc itself), and that they will be perfectly > happy to upgrade to a fixed version of reprepro. Such a fixed version will not magically appear and will not be magically deployed. I have no problem upgrading to a newer reprepro once there's a fixed version but I do still believe that your use of Provides is abusive and that you should rethink the approach. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Fri, 18 Oct 2019 09:51:07 GMT) (full text, mbox, link).

Acknowledgement sent to intrigeri <intrigeri@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Fri, 18 Oct 2019 09:51:07 GMT) (full text, mbox, link).

Message #100 received at 942487@bugs.debian.org (full text, mbox, reply):

From: intrigeri <intrigeri@debian.org> To: Ximin Luo <infinity0@debian.org>, 942487@bugs.debian.org Cc: Sylvestre Ledru <sylvestre@debian.org>, Ansgar <ansgar@43-1.org>, Raphael Hertzog <hertzog@debian.org> Subject: Re: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Fri, 18 Oct 2019 11:47:05 +0200

Hi, Ximin Luo: > Who is using reprepro to archive Debian Rust packages? That's the > first time I've heard of this. I'm happy to document one specific example, in the hope that it helps this discussion adopt a user-centric approach :) Tails is taking snapshots of the Debian archive 4 times a day, so that: - We can build our images reproducibly. - We have fine control over which packages get upgraded and when, which matters a lot for our stable branches and during our code freezes. We use reprepro to do this, because back when we implemented this piece of infrastructure, it was, by far, the most suitable tool for the job; it works impressively well for us, especially considering a few experienced Debian folks had predicted it would explode when fed so much data; I don't know if there's anything better nowadays but I'm not looking forward to reimplementing something that works really nicely already. This includes Rust packages and therefore, we're affected by this bug. Cheers, -- intrigeri

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Fri, 18 Oct 2019 11:36:02 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Fri, 18 Oct 2019 11:36:02 GMT) (full text, mbox, link).

Message #105 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: intrigeri <intrigeri@debian.org>, 942487@bugs.debian.org Cc: Sylvestre Ledru <sylvestre@debian.org>, Raphael Hertzog <hertzog@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Fri, 18 Oct 2019 11:32:00 +0000

intrigeri: > Hi, > > Ximin Luo: >> Who is using reprepro to archive Debian Rust packages? That's the >> first time I've heard of this. > > I'm happy to document one specific example, in the hope that it helps > this discussion adopt a user-centric approach :) > Users do not care about "Provides" lines (as long as everything else works), but volunteer workers do care about not having to perform 20x the amount of manual labour in order to achieve the same end result. More generally I think we should stop throwing around the religious term "user-centric" to rhetorically virtue-signal during debates, it cheapens the meaning of the phrase and does not contribute much to the discussion. > Tails is taking snapshots of the Debian archive 4 times a day, [..] > This includes Rust packages and therefore, we're affected by this bug. > I'll take a look at reprepro in the next 2-3 weeks; arbitrary limits like 256K should be pretty easy to fix (have you tried simply configuring the BDB limits?). In the meantime we can also easily temporarily patch away the features for web-sys right now, since it looks like nothing uses them yet. (`aptitude search '~Dlibrust-web-sys'` shows no other packages.) A systematic change for debcargo to avoid this approach entirely is not feasible however, from a technical or worker-centric perspective. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> :

Bug#942487 ; Package src:rust-web-sys . (Mon, 23 Dec 2019 02:57:05 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net> . (Mon, 23 Dec 2019 02:57:06 GMT) (full text, mbox, link).

Message #110 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: intrigeri <intrigeri@debian.org>, 942487@bugs.debian.org Cc: Sylvestre Ledru <sylvestre@debian.org>, Raphael Hertzog <hertzog@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Mon, 23 Dec 2019 02:54:53 +0000

Control: reassign -1 reprepro 5.3.0-1 Control: retitle -1 reprepro imposes arbitrary limits on control files that are successfully parsed by other debian tools Ximin Luo: > [..] > I'll take a look at reprepro in the next 2-3 weeks; arbitrary limits like 256K should be pretty easy to fix (have you tried simply configuring the BDB limits?). The relevant code in reprepro is indexfile.c line 66: f->size = 256*1024; Change this to something like 4MB would be a short hacky fix to the current issue, I don't think even the extreme rust examples have a 4MB control field yet. A long-term fix would be to fix this: line 151-166: if (f->size - f->ofs <= 2048) { /* Adding code to enlarge the buffer in this case * is risky as hard to test properly. * * Also it is almost certainly caused by some * mis-representation of the file or perhaps * some attack. Requesting all existing memory in * those cases does not sound very useful. */ fprintf(stderr, "Error parsing %s line %d: Ridiculous long (>= 256K) control chunk!

", f->filename, f->startlinenumber); f->failed = true; return RET_ERROR; } One reasonable option would be to rip out this code and use whatever dpkg itself is using to parse the fields. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Bug reassigned from package 'src:rust-web-sys' to 'reprepro'. Request was from Ximin Luo <infinity0@debian.org> to 942487-submit@bugs.debian.org . (Mon, 23 Dec 2019 02:57:06 GMT) (full text, mbox, link).

No longer marked as found in versions rust-web-sys/0.3.28-1. Request was from Ximin Luo <infinity0@debian.org> to 942487-submit@bugs.debian.org . (Mon, 23 Dec 2019 02:57:07 GMT) (full text, mbox, link).

Marked as found in versions reprepro/5.3.0-1. Request was from Ximin Luo <infinity0@debian.org> to 942487-submit@bugs.debian.org . (Mon, 23 Dec 2019 02:57:07 GMT) (full text, mbox, link).

Changed Bug title to 'reprepro imposes arbitrary limits on control files that are successfully parsed by other debian tools' from 'rust-web-sys: Provides header is more than 256K long and it breaks reprepro...'. Request was from Ximin Luo <infinity0@debian.org> to 942487-submit@bugs.debian.org . (Mon, 23 Dec 2019 02:57:08 GMT) (full text, mbox, link).

Removed tag(s) wontfix. Request was from Ximin Luo <infinity0@debian.org> to control@bugs.debian.org . (Mon, 23 Dec 2019 03:54:04 GMT) (full text, mbox, link).

Information forwarded to debian-bugs-dist@lists.debian.org, Bernhard R. Link <brlink@debian.org> :

Bug#942487 ; Package reprepro . (Mon, 23 Dec 2019 09:48:04 GMT) (full text, mbox, link).

Acknowledgement sent to "Bernhard R. Link" <brlink@debian.org> :

Extra info received and forwarded to list. Copy sent to Bernhard R. Link <brlink@debian.org> . (Mon, 23 Dec 2019 09:48:04 GMT) (full text, mbox, link).

Message #125 received at 942487@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org> To: Ximin Luo <infinity0@debian.org>, 942487@bugs.debian.org Cc: intrigeri <intrigeri@debian.org>, Sylvestre Ledru <sylvestre@debian.org>, Raphael Hertzog <hertzog@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Mon, 23 Dec 2019 10:24:38 +0100

* Ximin Luo <infinity0@debian.org> [191223 09:16]: > A long-term fix would be to fix this: > > line 151-166: > if (f->size - f->ofs <= 2048) { > /* Adding code to enlarge the buffer in this case > * is risky as hard to test properly. > * > * Also it is almost certainly caused by some > * mis-representation of the file or perhaps > * some attack. Requesting all existing memory in > * those cases does not sound very useful. */ > > fprintf(stderr, > "Error parsing %s line %d: Ridiculous long (>= 256K) control chunk!

", > f->filename, > f->startlinenumber); > f->failed = true; > return RET_ERROR; > } > > One reasonable option would be to rip out this code and use whatever dpkg itself is using to parse the fields. As the comment describes, accepting arbitrary long control data would open all kind of security issues and require quite some hard to properly test code. Most of the attacks enabled by having longer control chunks might be able to mitigated some way, but that would require all kind of different logic that can then have some new bugs. So allowing arbitrary absurdly long control data is not something I want to support. Bernhard R. Link

Information forwarded to debian-bugs-dist@lists.debian.org, Bernhard R. Link <brlink@debian.org> :

Bug#942487 ; Package reprepro . (Mon, 23 Dec 2019 11:51:03 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Bernhard R. Link <brlink@debian.org> . (Mon, 23 Dec 2019 11:51:03 GMT) (full text, mbox, link).

Message #130 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: "Bernhard R. Link" <brlink@debian.org>, 942487@bugs.debian.org Cc: intrigeri <intrigeri@debian.org>, Sylvestre Ledru <sylvestre@debian.org>, Raphael Hertzog <hertzog@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Mon, 23 Dec 2019 11:48:24 +0000

Bernhard R. Link: > [..] > > As the comment describes, accepting arbitrary long control data would > open all kind of security issues and require quite some hard to properly > test code. Most of the attacks enabled by having longer control chunks > might be able to mitigated some way, but that would require all kind of > different logic that can then have some new bugs. > I don't see why this is the case, but nevertheless this is actually secondary to the below point: > So allowing arbitrary absurdly long control data is not something I want > to support. > dpkg and all other debian tools support it right now. It is only reprepro with this artifical constraint, which makes it not work for packages that are processable by dpkg and other debian tools. Are you suggesting that dpkg and other tools have a concrete security problem? X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Information forwarded to debian-bugs-dist@lists.debian.org, Bernhard R. Link <brlink@debian.org> :

Bug#942487 ; Package reprepro . (Tue, 24 Dec 2019 07:33:04 GMT) (full text, mbox, link).

Acknowledgement sent to "Bernhard R. Link" <brlink@debian.org> :

Extra info received and forwarded to list. Copy sent to Bernhard R. Link <brlink@debian.org> . (Tue, 24 Dec 2019 07:33:04 GMT) (full text, mbox, link).

Message #135 received at 942487@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org> To: Ximin Luo <infinity0@debian.org> Cc: 942487@bugs.debian.org, intrigeri <intrigeri@debian.org>, Sylvestre Ledru <sylvestre@debian.org>, Raphael Hertzog <hertzog@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Tue, 24 Dec 2019 08:26:49 +0100

* Ximin Luo <infinity0@debian.org> [191223 12:58]: > dpkg and all other debian tools support it right now. It is only reprepro with this artifical constraint, which makes it not work for packages that are processable by dpkg and other debian tools. If it is artifical, then it is artifically high. It is 128 times more than what almost every single package needs and more than five times what the most absurd package before needed and twice what other tools are said to have had as limit there. I will increase it in reprepro (and maybe might make it configurable to some extent), but there will always be an upper limit. > Are you suggesting that dpkg and other tools have a concrete security problem? dpkg does not check checksums of index files, so it is likely uneffected. If apt has no limit then that likely makes some attacks needlessly easy (though it might have other mitigations in that regard, and there are less things apt has to care about the way it is typically used). Accepting absurd input without confirmation is never a secure way to handle things, though. Bernhard R. Link

Severity set to 'important' from 'critical' Request was from Christoph Berg <myon@debian.org> to control@bugs.debian.org . (Mon, 06 Jan 2020 20:09:05 GMT) (full text, mbox, link).

Information forwarded to debian-bugs-dist@lists.debian.org, Bernhard R. Link <brlink@debian.org> :

Bug#942487 ; Package reprepro . (Fri, 17 Jan 2020 02:15:02 GMT) (full text, mbox, link).

Acknowledgement sent to Ximin Luo <infinity0@debian.org> :

Extra info received and forwarded to list. Copy sent to Bernhard R. Link <brlink@debian.org> . (Fri, 17 Jan 2020 02:15:02 GMT) (full text, mbox, link).

Message #142 received at 942487@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: "Bernhard R. Link" <brlink@debian.org> Cc: 942487@bugs.debian.org, intrigeri <intrigeri@debian.org>, Sylvestre Ledru <sylvestre@debian.org>, Raphael Hertzog <hertzog@debian.org>, Ansgar <ansgar@43-1.org> Subject: Re: Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro... Date: Fri, 17 Jan 2020 02:12:53 +0000

Bernhard R. Link: > * Ximin Luo <infinity0@debian.org> [191223 12:58]: >> dpkg and all other debian tools support it right now. It is only reprepro with this artifical constraint, which makes it not work for packages that are processable by dpkg and other debian tools. > > If it is artifical, then it is artifically high. It is 128 times more than > what almost every single package needs and more than five times what the most > absurd package before needed and twice what other tools are said to have > had as limit there. I will increase it in reprepro (and maybe might make it > configurable to some extent), but there will always be an upper limit. > OK, as long as it doesn't stop people from pulling in the normal Debian archive. The package in question had another RC bug filed against it (uninstallable B-D); in the process of updating it, the Provides entry is now again 277988 bytes big. Therefore I've source-only uploaded an NMU of reprepro to DELAYED/5 with the limit set to 4MB. It builds fine, so anyone who needs it urgently can apply the debdiff attached themselves. >> Are you suggesting that dpkg and other tools have a concrete security problem? > > dpkg does not check checksums of index files, so it is likely > uneffected. If apt has no limit then that likely makes some attacks > needlessly easy (though it might have other mitigations in that regard, > and there are less things apt has to care about the way it is typically > used). > Accepting absurd input without confirmation is never a secure way to handle > things, though. > I don't see any similar limits on the size of .deb files (or lists files, or whatever), so the limit on the Provides: field *inside* a .deb file seems out-of-place. Also "dput" into a reprepro repository with this size of a Provides file works totally fine, it seems the limit is only hit when pulling from another mirror. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git

Reply sent to Ximin Luo <infinity0@debian.org> :

You have taken responsibility. (Wed, 22 Jan 2020 03:03:03 GMT) (full text, mbox, link).

Notification sent to Raphaël Hertzog <hertzog@debian.org> :

Bug acknowledged by developer. (Wed, 22 Jan 2020 03:03:03 GMT) (full text, mbox, link).

Message #147 received at 942487-close@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org> To: 942487-close@bugs.debian.org Subject: Bug#942487: fixed in reprepro 5.3.0-1.1 Date: Wed, 22 Jan 2020 03:00:58 +0000

Source: reprepro Source-Version: 5.3.0-1.1 We believe that the bug you reported is fixed in the latest version of reprepro, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to 942487@bugs.debian.org, and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Ximin Luo <infinity0@debian.org> (supplier of updated reprepro package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing ftpmaster@ftp-master.debian.org) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Format: 1.8 Date: Fri, 17 Jan 2020 02:03:27 +0000 Source: reprepro Architecture: source Version: 5.3.0-1.1 Distribution: unstable Urgency: medium Maintainer: Bernhard R. Link <brlink@debian.org> Changed-By: Ximin Luo <infinity0@debian.org> Closes: 942487 Changes: reprepro (5.3.0-1.1) unstable; urgency=medium . * Non-maintainer upload. * Bump up the maxsize on a fixed-size C buffer to avoid breaking on some autogenerated rust packages. (Closes: #942487) Checksums-Sha1: ebb5a8109421fb9cd9ed87f2b78032abceac70e9 1522 reprepro_5.3.0-1.1.dsc fd0449673ad66112e07ae5706134b846d03cf93e 13592 reprepro_5.3.0-1.1.debian.tar.xz e15e7c35d91c732559b776e5857f1331998e9807 5184 reprepro_5.3.0-1.1_source.buildinfo Checksums-Sha256: 48a9fc6c1c5f2129629c645022fcdc3c28f19d52ed4d464998e7ba58059d7ba2 1522 reprepro_5.3.0-1.1.dsc f715db4207c2255f2c7c4c04ec12a95192cc6f6a533b9df42889876cf7bc9797 13592 reprepro_5.3.0-1.1.debian.tar.xz ad2716e04599c8dc2395af93a9451f476e2f050c10c0e25a76236824006e75f3 5184 reprepro_5.3.0-1.1_source.buildinfo Files: 883fc9b279e580d23bbbcccbf9e71095 1522 utils optional reprepro_5.3.0-1.1.dsc 70bea028a6c299af8ac340d32dc505fb 13592 utils optional reprepro_5.3.0-1.1.debian.tar.xz f9fb15b980014389fb55b68912e10f3f 5184 utils optional reprepro_5.3.0-1.1_source.buildinfo -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQReYinNQ9GpZ9TYcRrrH8jaRfspMAUCXiEV9QAKCRDrH8jaRfsp MBsDAQDce4BWnTzOc6UGgtgbAuSfYsvW2ThF36Jm+hOXytY4zQEArg4HQvjUO9jV nzuEeVzi6qgjzm0+jZBWLcFLuWDHoQU= =UeOc -----END PGP SIGNATURE-----

Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org . (Mon, 24 Feb 2020 07:31:36 GMT) (full text, mbox, link).

Send a report that this bug log contains spam.