LZX date problem handling with
the LZX plugin for Total Commander


This documentation explains mainly the problems with the archived file date in LZX archives because of the Y2K bug (year 2000 program error) and how the LZX plugin for Total Commander since v2.0 handles it. This documentation was last modified on 25.10.2006.

For a German version of this document see Handhabung des LZX-Datumsproblems mit dem LZX-Plugin für Total Commander.

This documentation is part of the archive lzx_plugin.zip available on the plugins page of the website of Total Commander.

Introduction

To unpack files from an archive created with LZX - a popular AMIGA pack format - with Total Commander, Sebastian Erbert has written a packer plugin. But the latest version 1.1 of this plugin by Sebastian does not handle the month of the archived files correct.

The last released v1.21 of LZX for the AMIGA has a really bad Y2K bug. This bug has no influence on the time and the day of the date of the archived files. But there are many problems with the month and year of the archived files because of the Y2K bug of LZX v1.21 (Aminet: lzx121r1.lha in util/arc) and the two different patches available for this bug.

Patch 1:  LZX_Y2Kfix.lha   (util/arc) written by Andrey Titov aka dr.Titus
Patch 2:  LZX121r_pch.lha  (util/arc) written by Mikolaj Calusinski

More LZX stuff and information can be found at The LZX Page by Jonathan Forbes and Tomi Poutanen. This webpage is from the authors of the LZX compression algorithm and the original LZX programs.

To solve all date problems, the plugin has been improved and v2.0 of the plugin now handles the date of the archived files correct with the help of the user. This documentation describes in detail what are the problems with the date of the archived files and what a user should do to get the correct file dates when unpacking an LZX archive.

Note:  Total Commander uses the FAT file time which cannot handle odd seconds (kills bit 0). Only NTFS precision time can save also odd seconds. So only even second values (00, 02, 04, ...) can be seen in the file list views of Total Commander and never odd second values (01, 03, 05, ...). The plugin stores the unpacked files always with precision time. Windows automatically converts it to FAT time if it is necessary.

Problem 1 - Month bug of LZX plugin v1.1 and prior

All months of archived files are displayed with the LZX plugin v1.1 with values 0-11 instead of 1-12, because LZX stores the month with a value 0-11, but Total Commander (Windows) needs 1-12. This bug is fixed in v2.0 of the LZX plugin.

See also the problem with year values which also set bit 0 of the month value (problem 4) and how the Y2K patches handle month and year of the archived files.


Problem 2 - Base year problem with file dates before 01.01.1980

An example for demonstration:

Archive:  lzx_test_unpatched.lzx
File:     LZX_Test\1978\1978-01-31.txt
Date:     31.01.1978 00:00:00

Date value inside the archive: 0xF8100000

The date value in binary format:

1111 1 000 0 001 000 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 05...00 - 6 bits - sec.  = 0x00  = 00
Bit 11...06 - 6 bits - min.  = 0x00  = 00
Bit 16...12 - 5 bits - hour  = 0x00  = 00
Bit 22...17 - 6 bits - year  = 0x08  = 08 (+1970)
Bit 26...23 - 4 bits - mon.  = 0x00  = 00 (+1)
Bit 31...27 - 5 bits - day   = 0x1F  = 31

The date values in the archive are correct, but the date is displayed as 31.00.2106 by v1.1 of the LZX plugin.
Reason: Windows uses 1980 as base year. So dates before 1980 are not possible in the Windows environment (except with NTFS precision time). In the Total Commander structure for the date/time of the archived file the year value is stored with the bits 25...31 (7 bits) in a 32 bit value.

By calculating ((1978 - 1980) << 25) >> 25) + 1980 the wrong result 2106 is produced because of -2 = 0xFFFFFFFE, shifted 25 times = 0x7E = 126, + 1980 = 2106. That is fixed by v2.0 of the LZX plugin by setting the year values 1978 and 1979 always to 1980. It can be supposed that there aren't many files in LZX archives which are really from 1978 or 1979.

Note:  Base year for the AMIGA operating system is 1978, for LZX 1970 and for Windows 1980!

Problem 3 - Y2K bug of LZX v1.21 without a Y2K patch

This version of LZX by Jonathan Forbes does not save year values after 1999 correct. The LZX compression source code is not free, so the author of this document doesn't have exact information what causes the wrong values for year values greater 1999. Following values are stored by the unpatched LZX v1.21 in the archive.

year   stored value   correct would be
2000  ........ 58 ........  30
2001  ........ 59 ........  31
2002  ........ 60 ........  32
2003  ........ 61 ........  33
2004  ........ 62 ........  34
2005  ........ 63 ........  35

That looks like this formula: (((year % 100) | 0x80) - 70) & 0x7F

The unlzx routine of the LZX plugin v1.1 adds always simply 1970 to the year value stored in the archive. That results in the wrong years 2028-2033 instead of 2000-2005.

Note:  That is not really a bug of the plugin v1.1. It is a bug of the LZX compression routine!

An example for demonstration:

Archive:  lzx_test_unpatched.lzx
File:     LZX_Test\2000\2000-01-31.txt
Date:     31.01.2000 00:00:00

Date value inside the archive: 0xF8740000

The date value in binary format:

1111 1 000 0 111 010 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 22...17 - 6 bits - year  = 0x3A  = 58 (+1970) - wrong value
Bit 26...23 - 4 bits - mon.  = 0x00  = 00 (+1)
Bit 31...27 - 5 bits - day   = 0x1F  = 31

Wrong calculated year: 2028 instead of 2000.

See the chapter handling which describes how the plugin v2.0 handles the month and year values of the archived files according to the used packer to decode the year correctly.


Problem 4 - Invalid month values for years greater 2005 of LZX v1.21

Using expected formula of problem 3 consequently for year values greater 2005 will create a return to base year 1970 with bit 0 of month always set because year value is now always greater 63. So the year would need 7 bits. This year bit in the month field results in always even month values what Andrey Titov named "six-month count system".

Example:

Archive:  lzx_test_unpatched.lzx
File:     LZX_Test\2006\2006-01-31.txt
Date:     31.01.2006 00:00:00

Date value inside the archive: 0xF8800000

The date value in binary format:

1111 1 000 1 000 000 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 22...17 - 6 bits - year  = 0x00  = 00 (+1970)  - wrong value
Bit 26...23 - 4 bits - mon.  = 0x01  = 01 (+1)    - wrong value
Bit 31...27 - 5 bits - day   = 0x1F  = 31

Because of the problem with base year 1980 and the month problem, the file date 31.01.2006 is displayed as 28.01.2098 with plugin v1.1.

It is possible to correct the year values till year 2013 by adding 2006 for LZX archives created with the unpatched packer because on an AMIGA no year value smaller 1978 is possible. For every file with year 2014 or greater stored in an archive created with unpatched LZX v1.21 the year cannot be identified correctly and will be handled wrong.

But the month value of files with a year value greater 2005 can be automatically fixed only for invalid dates, i.e. 31.02.2006 is corrected by the plugin v2.0 to 31.01.2006. But if the date is possible for the wrong month, the plugin cannot determine the correct value of the month as for example for 2012-07-31.txt which is displayed or extracted with the date 2012-08-31.

If taken also base year problem into account, LZX archives created with unpatched v1.21 can be handled by the LZX plugin 100% correct only in the range 01.01.1980 till 31.12.2005.


Y2K patch by Andrey Titov aka dr.Titus

At The LZX Page or in Aminet there are also 2 different patches available for solving the Y2K problem of LZX v1.21.

First the patch by Andrey Titov aka dr.Titus is analyzed.

In the range from 01.01.1978 to 31.12.2005 it produces exactly the same result as the unpatched LZX v1.21.

Beginning with year 2006 the year value is stored differently in comparison to the unpatched v1.21. This patch changes the base year for all year values greater 2005 to 1976. So it creates correct results till year 2033 (the values in the archive are 30-57). The range from 2034 to 2041 is saved in the archive with a year value of 0-7. Taken into account that a year before 1978 is not possible on the AMIGA, it can be detected and decoded correctly by adding 2034 to year values lower than 8.

Example 1:

Archive:  lzx_test_titov.lzx
File:     LZX_Test\2006\2006-01-31.txt
Date:     31.01.2006 00:00:00

Date value inside the archive: 0xF83C0000

The date value in binary format:

1111 1 000 0 011 110 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 22...17 - 6 bits - year  = 0x1E  = 30 (+1976)
Bit 26...23 - 4 bits - mon.  = 0x00  = 00 (+1)
Bit 31...27 - 5 bits - day   = 0x1F  = 31

Example 2:

Archive:  lzx_test_titov.lzx
File:     LZX_Test\2034\2034-11-30.txt
Date:     30.11.2034 00:00:00

Date value inside the archive: 0xF5000000

The date value in binary format:

1111 0 101 0 000 000 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 22...17 - 6 bits - year  = 0x00  = 00 (+2034)
Bit 26...23 - 4 bits - mon.  = 0x0A  = 10 (+1)
Bit 31...27 - 5 bits - day   = 0x1E  = 30

As everyone can see the patch by Titov solves also the month problem. So an LZX archive created with the patch by Titov contains always the correct file date in the full range from 01.01.1978 00:00:00 till 31.12.2041 23:59:59. But the unlzx routine must know that the LZX archive was created with LZX v1.21 patched with the patch by Titov to decode the date correctly.


Y2K patch by Mikolaj Calusinski

The second Y2K patch for LZX v1.21 at The LZX Page or in Aminet is by Mikolaj Calusinski. This patch calculates the year values inside the archive as everyone would expect it. A year from 2000 to 2033 is stored with a value of 30 to 63. So the correct result is created while unpacking by simply adding 1970. The years 2034 to 2041 are saved with values 0-7 like the other patch by Titov. So it can be also decoded correctly by adding 2034 to these values.

Example 1:

Archive:  lzx_test_calusinski.lzx
File:     LZX_Test\2000\2000-01-31.txt
Date:     31.01.2000 00:00:00

Date value inside the archive: 0xF83C0000

The date value in binary format:

1111 1 000 0 011 110 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 22...17 - 6 bits - year  = 0x1E  = 30 (+1970)
Bit 26...23 - 4 bits - mon.  = 0x00  = 00 (+1)
Bit 31...27 - 5 bits - day   = 0x1F  = 31

There is no problem with the month value for all years till 2034. But files with a year 2034 till 2041 have also the month problem 04 - bit 0 of the month is always set as Mikolaj Calusinski has also mentioned it in the readme of his patch and following example demonstrates.

Example 2:

Archive:  lzx_test_calusinski.lzx
File:     LZX_Test\2034\2034-11-30.txt
Date:     30.11.2034 00:00:00

Date value inside the archive: 0xF5800000

The date value in binary format:

1111 0 101 1 000 000 0 0000  0000 00 00 0000
  day   mon.   year   hour    min.    sec.

Bit 22...17 - 6 bits - year  = 0x00  = 00 (+2034)
Bit 26...23 - 4 bits - mon.  = 0x0B  = 11 (+1)  - wrong value
Bit 31...27 - 5 bits - day   = 0x1E  = 30

Handling of year and month values by the LZX plugin v2.0

To handle the file dates of the archived files correctly the plugin must know which version of LZX has created it. The plugin is not able to detect the packer by itself because there is no extra information about the packer in the archive.

The plugin v2.0 reads from an INI file the information which routine should be used to decode the dates of the archived files. The user can change the setting in the INI file with a text editor like Notepad. The setting is read from the INI file at every opening of an LZX archive. See INI settings of the LZX plugin for details about the INI file.

After the year is determined according to the setting in the INI file, the whole date is always checked for an invalid date. If the day of the month is not valid for the current month and the month is greater than 1 (January), the month is decreased by the value of 1 - see problem 4 - and the check is done once again. If the day is still not valid, the month from the archive is used and the day is set to the last day of this month. The leap year rule is used for February.


LZXPacker=0 ... original LZX with Y2K bug

With this setting which is also the default setting, the plugin v2.0 decodes the year and month of the archived files as follows:


With the setting LZXPacker=0 the dates of the archived files are decoded 100% correct for following date ranges:

Archive created with an original LZX:         01.01.1980 ... 31.12.2005
Archive created with Titov patched LZX:       01.01.1980 ... 31.12.2005
Archive created with Calusinski patched LZX:  01.01.1980 ... 31.12.2027

Archived files from year 1978 or 1979 are decoded with correct day and month, but the year is set to 1980.
Maybe wrong months have the archived files packed with an unpatched LZX in the year range 2006 till 2013 in the file view or after extraction.

This documentation was written in October 2006. Hopefully nobody uses an unpatched original version of LZX on an AMIGA anymore for current files.


LZXPacker=1 ... LZX v1.21 with patch by Andrey Titov aka dr.Titus

With this setting the plugin v2.0 decodes the year and month of the archived files as follows:


With the setting LZXPacker=1 the dates of the archived files are decoded 100% correct for following date ranges:

Archive created with an original LZX:         01.01.1980 ... 31.12.2005
Archive created with Titov patched LZX:       01.01.1980 ... 31.12.2041
Archive created with Calusinski patched LZX:  01.01.1980 ... 31.12.1999

Archived files from year 1978 or 1979 are decoded with correct day and month, but the year is set to 1980. Archives created with LZX with the patch by Andrey Titov and unpacked with this plugin v2.0 with the setting LZXPacker=1 will have no month problem in the full year range from 1978 till 2041.


LZXPacker=2 ... LZX v1.21 with patch by Mikolaj Calusinski

With this setting the plugin v2.0 decodes the year and month of the archived files as follows:


With the setting LZXPacker=2 the dates of the archived files are decoded 100% correct for following date ranges:

Archive created with an original LZX:         01.01.1980 ... 31.12.1999
Archive created with Titov patched LZX:       01.01.1980 ... 31.12.1999
Archive created with Calusinski patched LZX:  01.01.1980 ... 31.12.2033

Archived files from year 1978 or 1979 are decoded with correct day and month, but the year is set to 1980. Maybe wrong months have the archived files packed with LZX with the patch by Mikolaj Calusinski in the range 2033 till 2041 in the file view or after extraction. But currently (October 2006) this is not a real problem and very probably it will be never a problem.


LZX archives for testing the 3 date handling routines

The archive lzx_source.zip at the plugins page of Total Commander contains the 3 test archives mentioned in this documentation a few times. It also contains the AMIGA script file to create the LZX test archives.

Jean-Francois FABRE, author of JST, has also used LZX v1.21 (unpatched or with Titov's patch) for his JST slaves. So this plugin with the setting LZXPacker=0 or =1 must be used to correctly unpack also the date of many archived files for JST. The LZX archives for the HD installer JST can be downloaded from The JOTD Patch & HD Installers Page.

The popular hard disk installer for AMIGA demos and games WHDLoad is available as LZX package and also as LhA package which is good for testing the plugin. These archives contain also files with file comments.