DOS ain't dead

Seems I found a pretty serious bug in FreeDOS kernel - when I 7-un-ZIP a (poorly compressible) > 128 MiB file, it frequently (?) trashes the output file ... there is one block of garbage, 4 or 16 MiB in size, placed randomly (?) between 128 MiB and the end of the file. The bug doesn't occur with EDR-DOS, and, surprisingly, it doesn't occur when uncompressing a ZIP file of similar size with 7-ZIP either 7-ZIP is always happy, thus, the written data gets corrupted between 7-ZIP output and the HD ... Bug also seems not to be specific to some partition: same bug on other one, defrag has no effect, scandisk is happy ... I did some tests - eats pretty much time ... some more test still possible.

Anyone has had this or similar bug ? Anyone 7-un-ZIP's such unreasonably huge files under FreeDOS ?

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

What FreeDOS version?
What 7-Zip version?
Contents of (FD)CONFIG.SYS, AUTOEXEC.BAT?

---
Forum admin

Do you use disk cache? Did you try it disable it? And what about memory manager?

BTW I was hunting similar kind of bug in Dos Navigator + dos 6.22 + Qemm 9.0 - it happened that when I copied a lot of small files some of them was then zeroed! It happened only when DN had enough memory to cache read/writes. But it was not a DN bug - problem was in Qemm, because I replaced VGA card with different VGA ROM and Qemm used stealth ROM mapping (ST:M) technique and when ROM holes had changed it caused this different behavior (also some DJGPP/Allegro programs crashed). So I rerun Qemm optimize and then everything went fine. - Just for fun how crazy bugs can be :)

---
DOS gives me freedom to unlimited HW access.

> Seems I found a pretty serious bug in FreeDOS kernel

How do you trace the bug to the kernel?

> when I 7-un-ZIP a (poorly compressible) 128 MiB file

Two problematic parameters.

> the written data gets corrupted between 7-ZIP output and the HD

Aha!

> some more test still possible.

Necessary?

> Anyone has had this or similar bug ?

Hit anything hard enough and it will break.

> unreasonably huge files

Perfect example.

BTW - what is a MiB?

> BTW - what is a MiB?

A MebiByte -- 1024 KibiBytes

> > BTW - what is a MiB?
>
> A MebiByte -- 1024 KibiBytes

Thank you for clearing that up.

> > A MebiByte -- 1024 KibiBytes
>
> Thank you for clearing that up.

http://en.wikipedia.org/wiki/Mebibyte

> Anyone has had this or similar bug ? Anyone 7-un-ZIP's such unreasonably
> huge files under FreeDOS ?

No, never experienced this. Try again with a clean boot (no mem. managers) and with the latest kernel. But are you using your IP443 or Win32's 7ZA.EXE + HXRT or the DJGPP compile of p7zip 4.42?

---
Know your limits.h

> How do you trace the bug to the kernel?

See above.

> Hit anything hard enough and it will break.

If not, bring some TNT, if still not happy, a nuclear bomb will do the job.

> BTW - what is a MiB?

Moron's intentional Bugging

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> What FreeDOS version?

Kernel from 1.0 distro

> What 7-Zip version?

4.42 official running on HX

> FDCONFIG/FDAUTO

Nothing suspicious except HIMEMX, no cache, no DMA

> Do you use disk cache?

Of course not ... this would be the the first suspicious thing

> And what about memory manager?

HIMEMX & HDPMI32

> Try again with a clean boot

I will ... it's very slow and the bug occurs repeatedly but not always

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

if you provide all necessary files to reproduce that, I'll give it a look

FTP server address, user, password by email

Tom

> > BTW - what is a MiB?
>
> Moron's intentional Bugging

Thank you for clearing that up.

> Kernel from 1.0 distro

Latest is ke2007sep15.zip, so try that.

> > What 7-Zip version?
>
> 4.42 official running on HX

You mean the DJGPP compile or the Win32 console version?

Try both 7ZA456.ZIP (Win32 console) + HXRT and also the DJGPP compile.

> Nothing suspicious except HIMEMX, no cache, no DMA

Try XMGR or HIMEM 3.26 (no X) because HIMEMX has been known to be buggy in the past (no offense).

> Try XMGR or
> HIMEM
> 3.26 (no X) because HIMEMX has been known to be buggy in the past (no
> offense).

This is nonsense (no offense ), because FD Himem is known to be buggy in the present, which is slightly worse than having had bugs in the past.

Generally, a history of fixed bugs usually is seen as positive and quite normal. You should be more concerned and suspicious if there is NO such history available, because it might be an indication that the software isn't used at all or is premature.

---
MS-DOS forever!

> > > What 7-Zip version?
> >
> > 4.42 official running on HX
>
> You mean the DJGPP compile or the Win32 console version?
>
> Try both
> 7ZA456.ZIP
> (Win32 console) + HXRT and also the
> DJGPP
> compile.
> offense).

Michael Kostylev has a djgpp compile of p7zip 4.44.
Download: http://mik.mkw.ru/dos-stuff/p7z444b.zip

> This is nonsense (no offense ), because FD Himem is known to be buggy
> in the present, which is slightly worse than having had bugs in the past.

Maybe I should have said we need to test everything some more? More testing is surely needed for something non-trivial as this. The only way to be sure of the bug is to isolate the issue, so switching (or disabling) mem. managers is recommended.

> I'll give it a look

Thanks.

> if you provide all necessary files to reproduce that, FTP

Public FTP ? Well, I got the bug several times with at least 3 different files, thus, it seems not to be specific to one "guilty" file - so no point to send a 200 MiB file, any file providing sufficient size probably would do The action was always 7-un-ZIP'ping, the size 150 .. 350 MiB, compressibility bad (cca 1.5 ... 0.99 (!!!) times), 7-ZIP 4.42 running on HX (not only HDPMI32), HIMEMX present (indeed for no reason, and not involved except HDPMI32 cooperation), at least 2 different partitions (one of them 7 GiB FAT32, other=??? (no longer know safely)) - given that, the reproductability is unfortunately occasional, not always ... seems to depend from fragmentation / last OS access / ??? . Last time I tried to isolate out HIMEMX, but the bug didn't occur even with it ... maybe more luck next time

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> Generally, a history of fixed bugs usually is seen as positive
...
> NO such history available might be an indication that the
> software isn't used at all or is premature.

YES, I frequently wonder after downloading software xxx version zzz how in the past version yyy could work for me at all given all the horrible bugs found and fixed in zzz compared to version yyy

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Excuse me, but did you test your RAM with MEMTEST-86 or another good RAM tester? And are you sure that the electrolytic capacitors of your main board don't leak? Hardware problems can cause strange things that could be attributed to software.

> Public FTP ? Well, I got the bug several times with at least 3 different
> files, thus, it seems not to be specific to one "guilty" file - so no
> point to send a 200 MiB file, any file providing sufficient size probably
> would do The action was always 7-un-ZIP'ping, the size 150 .. 350
> MiB, compressibility bad (cca 1.5 ... 0.99 (!!!) times), 7-ZIP 4.42
> running on HX (not only HDPMI32), HIMEMX present (indeed for no
> reason, and not involved except HDPMI32 cooperation), at least 2 different
> partitions (one of them 7 GiB FAT32, other=??? (no longer know safely)) -
> given that, the reproductability is unfortunately occasional, not
> always ... seems to depend from fragmentation / last OS access /
> ??? . Last time I tried to isolate out HIMEMX, but the bug didn't occur
> even with it ... maybe more luck next time

Your "bug reports" are not really well-structured, you know? So currently you no longer have a reliable test case, which safely fails?

In this case one will have to wait until you got one again. But you can prepare at least one thing now: to make sure that you can easily switch the DOS then (FreeDOS/EDR-DOS/MS-DOS) with all other parameters unchanged (FAT partition, drivers loaded, DOS/DOSDATA loaded high, UMBPCI/EMM386, DOS Buffers).

---
MS-DOS forever!

> board don't leak? Hardware problems can cause strange things that could be
> attributed to software.

Incorrect work of RAM can cause severe, strange and irreproducible system crashes. Several years ago I have bought motherboard for AMD-K6-2 on a computer market (not new, with some chips removed...). This PC couldn't work with memory intensively because of crashes, damages of data.
MEMTEST86+ is severe and reliable test - rare test which pointed on the bug on that PC.

> Incorrect work of RAM can cause severe, strange and irreproducible system
> crashes.
> ...
> MEMTEST86+ is severe and reliable test - rare test which pointed on the
> bug on that PC.

memteste.exe 25815 8-29-06 12:53a memory tester (but not newest version)

http://rugxulo.googlepages.com/disktwo.zip

Okay, maybe you'd rather not download the whole thing just for that. BTW, IIRC it runs best with a clean boot.

http://www.geocities.com/snoopimeanie/memteste.zip (.EXE only)

P.S. Sorry for promoting my own disks so much!

> if you provide ... I'll give it a look

Late update:

1. Still have the (rarely occurring) bug as described
2. IIRC have seen cca 2 "suspicious" BUG's in FreeDOS Bugzilla that might match this problem ... regrettably the Bugzilla is dead so I can't check them anymore
3. More exact results:

- File size 633 MiB
- 5 fragments, fragment breaks at cca 50 MiB (2 breaks just one 4 KiB cluster from each other) and 220 MiB (again 2 breaks just one 4 KiB cluster from each other)
- Affected file area 80 MiB ... 96 MiB exactly 16 MiB - thus bug NOT related to fragmentation
- In affected area, 32 KiB are "lost" at the beginning, the complete (16 MiB - 32 KiB) block "moved back" by 32 KiB, final 96 MiB - 32 KiB to 96 MiB filled with garbage (??? , couldn't determine the origin)
- After 96 MiB file is OK again up to the end
- Always occurs in FreeDOS, never in EDR-DOS, so it must be a FreeDOS kernel bug (but possibly already fixed in some "CVS" - see Bugzilla)

PS: Thanks to all DOSSERS who helped debugging this, most notably Steve

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> 2. IIRC have seen cca 2 "suspicious" BUG's in FreeDOS Bugzilla that might
> match this problem ... regrettably the Bugzilla is dead so I can't check
> them anymore

E-mail Eric Auer.

> 3. More exact results:
>
> - File size 633 MiB
> - 5 fragments, fragment breaks at cca 50 MiB (2 breaks just one 4 KiB
> cluster from each other) and 220 MiB (again 2 breaks just one 4 KiB
> cluster from each other)

CWSDPMI? HDPMI32? What versions? In particular, what 7-Zip are you trying? (7z + .DLL or 7za or p7zip 4.57 or ... ???)

> - Affected file area 80 MiB ... 96 MiB exactly 16 MiB - thus bug
> NOT related to fragmentation

(J)EMM386 loaded?

> - In affected area, 32 KiB are "lost" at the beginning, the complete (16
> MiB - 32 KiB) block "moved back" by 32 KiB, final 96 MiB - 32 KiB to 96
> MiB filled with garbage (??? , couldn't determine the origin)
> - After 96 MiB file is OK again up to the end

FAT16? FAT32? FAT32+?

> - Always occurs in FreeDOS, never in EDR-DOS, so it must be a
> FreeDOS kernel bug (but possibly already fixed in some "CVS" - see
> Bugzilla)

Maybe something is incorrect in your FAT (or its backup) that only FreeDOS attempts to use??

> CWSDPMI? HDPMI32? What versions? In particular, what 7-Zip are you trying?

Always HDPMI32 -r. Win32 7-ZIP via HX. Version = ??? (cca 4 months ago)

> (J)EMM386 loaded?

NO, see above.

> FAT16? FAT32? FAT32+?

FAT32.

> Maybe something is incorrect in your FAT (or its backup) that only FreeDOS
> attempts to use??

Indeed a possibility Please not the stupid 4 "reserved" bits again !!! FAT28 sucks

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> > CWSDPMI? HDPMI32? What versions? In particular, what 7-Zip are you
> trying?
>
> Always HDPMI32 -r. Win32 7-ZIP via HX. Version = ??? (cca 4 months
> ago)

Try again with p7zip 4.57 (preferable since 4.58 seems less stable).

> > FAT16? FAT32? FAT32+?
>
> FAT32.

Okay, it may be FreeDOS' FAT32 since I don't think that got as much testing as FAT12 and FAT16.

> > Maybe something is incorrect in your FAT (or its backup) that only
> FreeDOS
> > attempts to use??
>
> Indeed a possibility Please not the stupid 4 "reserved" bits again !!!
> FAT28 sucks

I'd actually expect EDR-DOS to be buggier than FreeDOS since its FAT32 support is relatively new. But who knows (Udo??).

> 2. IIRC have seen cca 2 "suspicious" BUG's in FreeDOS Bugzilla that might
> match this problem ... regrettably the Bugzilla is dead so I can't check
> them anymore

Here's what Eric Auer says (via e-mail):

------------------------------------------------------
Bugzilla is slow but
still reacts as far as I can tell, just checked.

Unfortunately nobody tells which bug numbers this
thread on dos aint dead is talking about... Nor
which fat type is used nor whether a recent free-
dos kernel version was used... Maybe you can post
the list below into a new thread so people can say
which bugs 1. hurt most 2. they found to be fixed
in current kernels and 3. they have patches for.

Please also post that to jump directly to bug 4242
http://bugzilla.freedos.org/show_bug.cgi?id=4242
is the URL to go

Eric

------------------------------------------------------
bug_id,bug_severity,priority,bug_status,short_desc

1658,MAJOR,P2,NEW,Norton Ghost 5.1d file browser fails (kernel 2029+)
1842,MAJOR,P2,OLD,DOS=HIGH (HMA) fails on several scenarios in beta9sr2 distro
1862,MAJOR,P1,OLD,CHDIR fails on substituded drives (fixed SVN 1357?)
1908,MAJOR,P2,NEW,ren does not work in full root directory
1956,MAJOR,P2,OLD,All file opens fail under specific conditions (fixed SVN 1323?)
1959,MAJOR,P2,NEW,Opening big amount of files (+ config.sys?)

735,normal,P2,OLD,infinite retry attempts for floppy read error
1049,normal,P2,NEW,Abort, Ignore, Retry, Fail? - always retries (fixed 2032a?)
1651,normal,P2,NEW,data corruption on improper disk change during access (2029)
1688,normal,P2,OLD,Unrecognized Partitions Result In Error Message
1706,normal,P2,OLD,Indenture game: keyboard fails after a while, hardware specific
1753,normal,P2,NEW,Invalid Opcode on NetWare 16-bit Client (VLM) when logging in
1793,normal,P2,NEW,djgpp grep called from within rhide doesn't end
1814,normal,P2,NEW,Kernel crash with Opteron / RAID in initdisk
1817,normal,P3,OLD,COPY creates invalid FAT32 dir entries w/ SMARTDRV delayed write & LFN when DOSLFN not loaded
1825,normal,P3,NEW,Lost clusters after OrCAD SDT 386+ v1.21 PartList Configure
1845,normal,P2,OLD,disk access failure when booting ultimatebootcd
1854,normal,P2,OLD,turbo c++ 3.0 fails to run (fixed in SVN revision 1348)
1873,normal,P2,OLD,Kernel 2035a: stargunner display corruption (2033 okay, 2034 less broken than 2035a)
1875,normal,P2,NEW,bupdate.exe from telegard produced incorrect result
1887,normal,P3,OLD,char access to ATAPICDD, EMM386, HIMEM, CLOCK$ drivers hangs
1890,normal,P2,NEW,File Date stamp on network shares incorrect
1923,normal,P1,other,OLD,Kernel hangs when booting if IDE/SATA Compact flash card installed
1944,normal,P2,OLD,MS Himem.sys complains about VDISK app being already loaded
1953,normal,P2,OLD,21h/29h returns 0 (valid) for all drive root names up to LASTDRIVE (fixed SVN 1334?)
1957,normal,P2,OLD,GRABASC causes crash without LOADFIX

1779,minor,P2,NEW,TESTER needed: * and DJGPP RHIDE graphical corruption
1960,minor,P5,OLD,Error message at boot about illegal partition table

698,tiny,P2,REOPENED,kernel has no int 13h DMA boundary helper
1176,tiny,P2,OLD,Third floppy drive (PC-XT) not recognized
1703,tiny,P2,OLD,Freedos bugs for Chinese: No working DBCS support
1742,tiny,P2,OLD,WISH: improve config.sys menu documentation and consistency
1748,tiny,P2,OLD,DATE fails to store year > 2000 correctly on some embedded systems
1758,tiny,P2,OLD,F8 modus does not ask in rare case
1797,tiny,P2,OLD,Kernel crashes when accessing Ontracked HD without Ontrack
1828,tiny,P2,OLD,COMDRIVE remote com1 drive letter hangs for non-floppy
1861,tiny,P2,OLD,WISH: HLT in kernel idle loop, activated by config.sys option
1882,tiny,P2,OLD,WISH: Logo file supports
1903,tiny,P2,other,OLD,check whether freedos umb chain follows RBIL table #01630 style
1909,tiny,P2,MS-DOS,OLD,JO.SYS support
------------------------------------------------------

> Bugzilla is slow but
> still reacts as far as I can tell, just checked.

Can't search bugs ...

> which bug numbers this thread on dos aint dead is talking about...

Can't find it anymore

> which fat type is used

FAT32, see above.

> nor whether a recent free-dos kernel version was used

1.0 or later (multiple dates and version numbers in one kernel ...)

> http://bugzilla.freedos.org/show_bug.cgi?id=4242 is the URL to go

Doesn't exist.

http://bugzilla.freedos.org/show_bug.cgi?id=1908

Indeed I can view the bugs now

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> > nor whether a recent free-dos kernel version was used
>
> 1.0 or later (multiple dates and version numbers in one kernel ...)

2036? 2037? 2038?

> > http://bugzilla.freedos.org/show_bug.cgi?id=4242 is the URL to go
>
> Doesn't exist.

He actually claims that was just for example (i.e. made up number for demonstration purposes).

> http://bugzilla.freedos.org/show_bug.cgi?id=1908
>
> Indeed I can view the bugs now

Now get out your can of Raid ...