RFS: ecm -- prepares CD image files so they compress better

[Loïc Martin]

Dear mentors,

I am looking for a sponsor for my package "ecm"

* Package name : ecm
Version : 1.00
Upstream Author : Neill Corlett <***@neillcorlet.com>
* URL : http://www.neillcorlett.com/ecm/
* License : GPL2+
Programming Lang: C
Description : prepares CD image files so they compress better

The ECM format allows you to reduce the size of a typical CD image
file (BIN, CDI, NRG, CCD, or any other format that uses raw sectors).
It works by eliminating the Error Correction/Detection Codes
(ECC/EDC) from each sector whenever possible. The encoder
automatically adjusts to different sector types and automatically
skips any headers it encounters.
Since the data skipped is nearly impossible to compress with
traditionnal tools, the resulting ECM file will compress far better
than the raw CD image

There's a description of how it works at
http://www.neillcorlett.com/ecm/how.html, it's really neat especially if
you like lossless compression techniques (the author has another neat
GPL compression-related program using FFTW for functionality I also
haven't found elsewhere - see http://www.neillcorlett.com/informer/ but
sadly the author has turned to .Net).

It builds this binary package: ecm

The latest entry in the Debian changelog is:

ecm (1.00-1) unstable; urgency=low

* Initial release (Closes: bug#531079).
* Repacked .zip to .tar.gz tarball, removed Windows binaries from
archive (ecm.exe, unecm.exe)

The package is lintian clean (tested with unstable lintian), no
warnings. It's been accepted in Ubuntu 9.04, and I've worked on it a bit
more for inclusion in Debian.

The package can be found on mentors.debian.net:
- URL:
http://mentors.debian.net/debian/pool/main/e/ecm
- dget line:
dget http://mentors.debian.net/debian/pool/main/e/ecm/ecm_1.00-1.dsc

[Joachim Reichel]

Hi Loïc,

Post by Loïc Martin
I am looking for a sponsor for my package "ecm"
* Package name : ecm
Version : 1.00
* URL : http://www.neillcorlett.com/ecm/
* License : GPL2+
Programming Lang: C
Description : prepares CD image files so they compress better

I had a quick look at your package and did not find a problem, though I
would have to give it a closer look before upload.

Unfortunately, ecm seems to have a problem with files larger than 2 or 4
GB (I tried the 1st Debian Lenny DVD). I ended up with a file of size
381 MB :-((. I'm aware that ecm does not claim to handle DVD images, but
it also does not refuse to handle them. I consider this a grave bug
(data loss).

I did another test with the first Debian Etch CD. ecm reduced the size
of the image by 0.03%. Compressing the original ISO and the ecm-prepared
version with bzip2 gave a 0.097% advantage for the ecm-prepared version.
Do you have any number where ecm shows a reasonable gain?

Don't most CDs/DVDs contain data (software, music, video) that is
already compressed in one way or another? Therefore I would guess that
removing some very small amount of hard-to-compress data doesn't improve
the overall situation.

For these reasons I'm currently inclined not to sponsor this package.
Also, ecm-prepared/compressed images don't seem to be widespread as far
as I can say (and the software is 7 years old).

Joachim

[Joachim Reichel]

Hi,

Post by Joachim Reichel

Post by Loïc Martin
I am looking for a sponsor for my package "ecm"
* Package name : ecm
Version : 1.00
* URL : http://www.neillcorlett.com/ecm/
* License : GPL2+
Programming Lang: C
Description : prepares CD image files so they compress better

I had a quick look at your package and did not find a problem, though I
would have to give it a closer look before upload.
Unfortunately, ecm seems to have a problem with files larger than 2 or 4
GB (I tried the 1st Debian Lenny DVD). I ended up with a file of size
381 MB :-((. I'm aware that ecm does not claim to handle DVD images, but
it also does not refuse to handle them. I consider this a grave bug
(data loss).
I did another test with the first Debian Etch CD. ecm reduced the size
of the image by 0.03%. Compressing the original ISO and the ecm-prepared
version with bzip2 gave a 0.097% advantage for the ecm-prepared version.
Do you have any number where ecm shows a reasonable gain?

Ruben Molina kindly pointed out to me that my test setup was flawed. ecm
should be applied to RAW files, not to ISO files. Another test with a
RAW image gave a similar improvement to what is reported on the ecm
homepage (~15% after compression).

Loic: I'm happy to sponsor this upload if you can get the data loss bug
fixed. There might be some small changes to the packaging needed, but
I'm confident that these can be easily sorted out.

Joachim

[Loïc Martin]

Post by Joachim Reichel
Loic: I'm happy to sponsor this upload if you can get the data loss bug
fixed. There might be some small changes to the packaging needed, but
I'm confident that these can be easily sorted out.

I haven't been able to reproduce this bug once. I tried with raw images
smaller than 2GB (1.9GB), between 2 and 4GB, and over 4GB. On raw images
smaller than 2GB, everything work fine. On images bigger than 2GB, ecm
aborts cleanly with the following error message:

Encoding foo.iso to foo.iso.ecm.
foo.iso: Value too large for defined data type

It doesn't even start creating the .ecm file. Also, in the event you had
a crash and the .ecm file didn't finish (it can take a while), the
original .iso is never modified - the .ecm file is created as a separate
file, so I also don't know where the original data was lost.

I've been using ecm on i386 and amd64 without any data loss for a few
years now. Although the program should also work on other arch, are you
using something different?

Loïc