DOS ain't dead

BUG: should be in Developers, not Announce

Seems I found a criminal (sorry ) bug related to MZ DOS executable format.

0,1 "MZ" (or also "ZM" ???)
2,3 LastBlockSize
4,5 BlockCount

Size in bytes 2...5 covers the complete file with header (not explicitly documented, but seems to). 32 bits are used to define a size up to 512 KiB at best .. yeah, efficiency But the big confusion occurs about how the values LastBlockSize and BlockCount are calculated.

http://www.delorie.com/djgpp/doc/exe/ - that's what CWSDPMI-GO32-STUB follows, 2 KiB, BlockCount=8, LastBlockSize=0, means full block.

EOF - end of fun

Brewing a 512 bytes EXE using FASM's "format MZ" results in BlockCount=1, but LastBlockSize=$200 !!! Nobody did explicitly prohibit setting it to $200, or even more - a 32 KiB EXE file could have BlockCount=1, and LastBlockSize=$8000 !

Can it be even worse ? YES, it can

DPMIST32.BIN has 512 bytes, but BlockCount=2, and LastBlockSize=0 -> should be 1024 bytes instead ? Even more: DPMIST32.BIN did change several times in HX history, and, HX 2.5 had DPMIST32.BIN with BlockCount=1, and LastBlockSize=0 - used to be correct ? Is this change intentional ? What is the reason ?

Can it be even worse ? YES, it can

       /* get size of image */
        imageSize=headbuf.image_length_MOD_512+(headbuf.image_length_DIV_512<<9);
                // commented out because of DPMIST32.BIN
//        If (imageSize Mod 512) IsNotZero
//          Then
            imageSize-=512;
//          EndIf ;
        headerSize=(headbuf.n_header_paragraphs)<<4;
        relocSize= headbuf.n_relocation_items << 2;
        imageSize-=headerSize;

Besides of a funny programming language, Ladsoft recently implemented a horrible bug into VALX, to get around the BlockCount=2-problem !

Anyone has a valid and complete MZ EXE specification ? Or should one ask Mr. Mark_Zbikowski ?

PESTUB seems to ignore the stupid MZ header and use "GetFileSize" instead ... there is always a hack

Could this evil 2 get fixed back to 1 ?

PS: PESTUB (non-criminal) issue: accept "PX" besides "PE" on input

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> DPMIST32.BIN has 512 bytes, but BlockCount=2, and LastBlockSize=0 ->
> should be 1024 bytes instead ? Even more: DPMIST32.BIN did
> change several times in HX history, and, HX 2.5 had DPMIST32.BIN with
> BlockCount=1, and LastBlockSize=0 - used to be correct ? Is this change
> intentional ? What is the reason ?
>
> Could this evil 2 get fixed back to 1 ?

IIRC DPMIST32.BIN was formerly named DPMIST32.EXE and if it was launched accidentally it caused a crash. So it was created as an INVALID binary (there are some remnants of this issue like the "-b" option in hx's shrmzhdr.exe) which makes the DOS loader complain about it.

Since it has no .EXE extension anymore, it could possibly be created as a VALID binary without any harm now. But since it works perfectly ... never change a running system ...

---
MS-DOS forever!

Yes, it is a known issue. But it isn't discussed very often.

---
DOS-u-akbar!

> BUG: should be in Developers, not Announce

You love the word "bug", don't you? If you want a post to appear in category "Developers", you have to select that. Easy, Eh? I have moved your post now.

---
Forum admin

> You love the word "bug", don't you?



> If you want a post to appear in category "Developers", you have to select that.

I did ... or tried at least ... but didn't work for some reason

> have moved your post now.

Great ... one forum problem (and "reason" for change to php-BB3) less

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> DPMIST32.BIN was formerly named DPMIST32.EXE and if it was launched
> accidentally it caused a crash. So it was created as an INVALID binary
> (there are some remnants of this issue like the "-b" option in hx's
> shrmzhdr.exe) which makes the DOS loader complain about it.

Any attempt to prevent bugs or make a piece of software more fool-proof is good ... but having success in this is even better

1. YES, it crashes if made VALID
2. Well, it also crashes as-is INVALID - thus the "fix" doesn't work
3. There is a horrible bug in VALX becuase of this. IIRC you were the linker lover and expert - don't you care ?
4. The bug causing the crash is elsewhere - see below

> it could possibly be created as a VALID binary without any harm now.

It should definitely

> But since it works perfectly ... never change a running system ...

1. Still crashing system (I never got this crash before you gave me the hint to rename BIN into EXE and run it as-is )
2. Bugged VALX

At least, I fixed this bug and many other bugs in DPMIST32.BIN, and also did a few other improvements:

http://board.flatassembler.net/topic.php?t=8316

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

The "question" remains, why it crashes. Investigated and found the bug. BTW, I'm sure you are aware of this bug, since it becomes desperately obvious with your DEBUG version of DPMILD32

BOCHS wrote:

00000000000i[APIC?] local apic in  initializing
========================================================================
                       Bochs x86 Emulator 2.3.6
             Build from CVS snapshot, on December 24, 2007
========================================================================
00000000000i[     ]   x86-64 support: yes
00264455460e[CPU0 ] write_virtual_checks(): write beyond limit, r/w
00264455460e[CPU0 ] can_push(): esp=0, normal, wraparound? limit=ffffefff
00264455460e[CPU0 ] can_push(): esp=0, normal, wraparound? limit=ffffefff
00264455460i[CPU0 ] CPU is in protected mode (active)
00264455460i[CPU0 ] CS.d_b = 32 bit
00264455460i[CPU0 ] SS.d_b = 32 bit
00264455460i[CPU0 ] EFER   = 0x00000000
00264455460i[CPU0 ] | RAX=0000000000100000  RBX=0000000000000000
00264455460i[CPU0 ] | RCX=0000000000000002  RDX=0000000000002007
00264455460i[CPU0 ] | RSP=0000000000000000  RBP=000000000000005c
00264455460i[CPU0 ] | RSI=0000000000000020  RDI=00000000ffc01f14
00264455460i[CPU0 ] |  R8=0000000000000000   R9=0000000000000000
00264455460i[CPU0 ] | R10=0000000000000000  R11=0000000000000000
00264455460i[CPU0 ] | R12=0000000000000000  R13=0000000000000000
00264455460i[CPU0 ] | R14=0000000000000000  R15=0000000000000000
00264455460i[CPU0 ] | IOPL=3 id vip vif ac vm RF nt of df if tf sf zf af pf cf
00264455460i[CPU0 ] | SEG selector     base    limit G D
00264455460i[CPU0 ] | SEG sltr(index|ti|rpl)     base    limit G D
00264455460i[CPU0 ] |  CS:0020( 0004| 0|  0) ff801000 00006763 0 1
00264455460i[CPU0 ] |  DS:0028( 0005| 0|  0) 0002bdb0 000ffffe 1 1
00264455460i[CPU0 ] |  SS:0028( 0005| 0|  0) 0002bdb0 000ffffe 1 1
00264455460i[CPU0 ] |  ES:004b( 0009| 0|  3) 00000000 000fffff 1 1
00264455460i[CPU0 ] |  FS:0000( 0000| 0|  0) 00031040 0000ffff 0 0
00264455460i[CPU0 ] |  GS:0000( 0000| 0|  0) 0002a6c0 0000ffff 0 0
00264455460i[CPU0 ] |  MSR_FS_BASE:0000000000031040
00264455460i[CPU0 ] |  MSR_GS_BASE:000000000002a6c0
00264455460i[CPU0 ] | RIP=0000000000004be5 (0000000000004be5)
00264455460i[CPU0 ] | CR0=0x80000031 CR1=0x0 CR2=0x0000000000000000
00264455460i[CPU0 ] | CR3=0x01fff000 CR4=0x00000200
00264455460i[CPU0 ] >> push ecx : 51
00264455460e[CPU0 ] exception(): 3rd (12) exception with no resolution, shutdown status is 00h, resetting
00264455460i[SYS  ] bx_pc_system_c::Reset(SOFTWARE) called
00264455460i[CPU0 ] cpu software reset
00264455460i[APIC0] local apic in CPU 0 initializing
00264459200i[BIOS ] $Revision: 1.166 $ $Date: 2006/08/11 17:34:12 $

TripleFault inside HDPMI32 triggered by stack flooding

DPMIST32.BIN and DPMILD32.EXE run each-other many times and flood HDPMI32's stack this way ... and the only way out preventing this crazy game from running into the infinity is the TripleFault

IIRC we had the very same bug recently because of a bug in INFOPAD and "paranoid" DOS/32A

But it comes even worse: Obviously DPMILD32 indeed activates the LOOOOOOOOOONG MODE !!!

I see 2 pieces that need fixing:

- DPMILD32: (don't load DPMIST32.BIN, + other issues, see other (soon) thread)
- HDPMI32: protect from Ring0 stack overflow

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> At least, I fixed this bug and many other bugs in DPMIST32.BIN, and also
> did a few other improvements:
>
> http://board.flatassembler.net/topic.php?t=8316

What are these "many other bugs"? Please supply a list!

I'm not yet convinced that your code contains "other improvements", because the start is not really promising:

        ; Here starting from DOS: DS = ???/PSP | ES = (seg-addr) of PSP
        ; We have exactly 28 bytes left before 0,0,0,0

        mov    ax,$7202            ; 3   -> 3
        push   ax
        popf
        pushf
        pop    bx                  ; 4x1 -> 7
        cmp    ax,bx               ; 2   -> 9
        je     short llmover       ; 2   -> 11 | OK we have at least 80386

Do you know that the NT flag sometimes is "lost" when running under some v86-monitors? AFAIK NTVDM is one of those, for example. V86-monitors can fake the content of the flags register, so you should be very careful with assumptions in this regard.

After all, a test for 80386 is not really necessary, because DPMILD32 won't use 386-specific opcodes before the DPMI host has switched into protected-mode, and no host I know of will switch to protected-mode for 32bit clients on a 80286 - if you're able to find a host which runs on such a cpu at all. So it should suffice to do a very simple test to filter 8086s:

    push sp
    pop ax
    cmp ax,sp
    jnz nono

---
MS-DOS forever!

> What are these "many other bugs"? Please supply a list!

OK ... later Regrettably few of them or even none will qualify as criminal ...

> I'm not yet convinced that your code contains "other improvements",
> because the start is not really promising

> Do you know that the NT flag sometimes is "lost" when running under some

> V86-monitors can fake the content of the flags register, so you should be
> very careful with assumptions in this regard.

YES. You are just confirming that those bring nothing but trouble and supplying one reason why I don't like them "too much"

> AFAIK NTVDM is one of those, for example.

Does it refuse to run for you ? "Need 80386"

> After all, a test for 80386 is not really necessary, because DPMILD32
> won't use 386-specific opcodes before the DPMI host has
> push sp
> pop ax
> cmp ax,sp
> jnz nono

Indeed possible also ... with 80386 it just aborts faster

What about the TripleFault in HDPMI32 and the long mode ?

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> YES. You are just confirming that those bring nothing but trouble and
> supplying one reason why I don't like them "too much"

They exist and they are - still - used. That's what matters.

> Does it refuse to run for you ? "Need 80386"

If I single-step thru it with DEBUG in NTVDM - yes! The NT flag might get lost if an IRQ happens ... or just for exception 01 ... almost impossible to tell.

---
MS-DOS forever!

> The "question" remains, why it crashes. Investigated and found the bug.
> BTW, I'm sure you are aware of this bug, since it becomes
> desperately obvious with your DEBUG version of DPMILD32

I'm not aware of "this" bug, on the contrary, I hardly understand what you are talking about.

> DPMIST32.BIN and DPMILD32.EXE run each-other many times and flood
> HDPMI32's stack this way ... and the only way out preventing this crazy
> game from running into the infinity is the TripleFault

Ok, HDPMI is rebooting the machine due to a ring 0 stack underflow. Please provide a test case which I can use ... on a REAL machine, not in Box!

> But it comes even worse: Obviously DPMILD32 indeed activates the
> LOOOOOOOOOONG MODE !!!

If I'd only know what you are talking about ...

> - DPMILD32: (don't load DPMIST32.BIN, + other issues, see other (soon)
> thread)

Waiting for your test case ...

> - HDPMI32: protect from Ring0 stack overflow

Has low priority, since this only happens with some buggy programs...

---
MS-DOS forever!

Japheth wrote:

> If I single-step thru it with DEBUG in NTVDM - yes

OK ... you can make everything fail if you try hard enough

> I'm not aware of "this" bug

As you wrote in your answer 1 month back, a crash occurs if you rename DPMIST32.BIN into an EXE and run it

> on the contrary, I hardly understand what you are talking about.

Please don't be more pessimistic than necessary

> Ok, HDPMI is rebooting the machine due to a ring 0 stack underflow. Please
> provide a test case which I can use ... on a REAL machine, not in Box!

I always test on a REAL machine first ... just rename (the valid or "INVALID" version of) DPMIST32.BIN into X.EXE and run it in DOS

> > LOOOOOOOOOONG MODE !!!
> If I'd only know what you are talking about ...

BOCHS crash report, see above ...

> Waiting for your test case ...

DPMIST32.BIN renamed into X.EXE + DPMILD32.

> > - HDPMI32: protect from Ring0 stack overflow
> Has low priority, since this only happens with some buggy programs...

YES, it's just a small open problem ... and would be maybe easy to fix ?

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> > If I single-step thru it with DEBUG in NTVDM - yes
>
> OK ... you can make everything fail if you try hard enough

Running DEBUG isn't THAT hard. But feel free to ignore...

> > > LOOOOOOOOOONG MODE !!!
> > If I'd only know what you are talking about ...
>
> BOCHS crash report, see above ...

Box just displays the internal 64bit registers. That doesn't mean too much.

> > Waiting for your test case ...
>
> DPMIST32.BIN renamed into X.EXE + DPMILD32.

DPMIST32.BIN isn't supposed to be a valid binary...
Feeding DPMILD32 with a simple DOS MZ binary is no problem.

Sorry, but I'm unable to see any problem at all.

---
MS-DOS forever!

> OK ... later Regrettably few of them or even none will qualify as
> criminal ...

Doesn't matter if you call them criminal or legal. Just supply a list of at least 2 bugs in DPMIST32.ASM which you had fixed in your FASM version from 02/19/2008.

---
MS-DOS forever!

> Running DEBUG isn't THAT hard. But feel free to

No, but DOS wouldn't have too much left if one threw away anything that you can crash with help of NTVDM and DEBUG

> Box just displays the internal 64bit registers. That doesn't mean too much.

Bug of BOCHS ?

> DPMIST32.BIN isn't supposed to be a valid binary...

But it still loads and crashes, as you pointed out

> Feeding DPMILD32 with a simple DOS MZ binary is no problem.

No. As long as this binary doesn't run DPMILD32.EXE ...

> Sorry, but I'm unable to see any problem at all.

No big problem, very small problem: your fix (corrupting MZ header) doesn't work, thus I'm suggesting alternatives

> 2 bugs

ASAP

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***