DOS ain't dead - [BUG] Criminal Mark-related bug

DOS386

24.01.2008, 02:21

[BUG] Criminal Mark-related bug | DPMIST32.BIN/PESTUB/ VALX (Developers)

BUG: should be in Developers, not Announce :-(

Seems I found a criminal (sorry :clap: ) bug related to MZ DOS executable format.

0,1 "MZ" (or also "ZM" ???)
2,3 LastBlockSize
4,5 BlockCount

Size in bytes 2...5 covers the complete file with header (not explicitly documented, but seems to). 32 bits are used to define a size up to 512 KiB at best .. yeah, efficiency :clap: But the big confusion occurs about how the values LastBlockSize and BlockCount are calculated.

http://www.delorie.com/djgpp/doc/exe/ - that's what CWSDPMI-GO32-STUB follows, 2 KiB, BlockCount=8, LastBlockSize=0, means full block.

EOF - end of fun :no:

Brewing a 512 bytes EXE using FASM's "format MZ" results in BlockCount=1, but LastBlockSize=$200 !!! Nobody did explicitly prohibit setting it to $200, or even more - a 32 KiB EXE file could have BlockCount=1, and LastBlockSize=$8000 !

Can it be even worse ? YES, it can :no:

DPMIST32.BIN has 512 bytes, but BlockCount=2, and LastBlockSize=0 -> should be 1024 bytes instead ? :confused: Even more: DPMIST32.BIN did change several times in HX history, and, HX 2.5 had DPMIST32.BIN with BlockCount=1, and LastBlockSize=0 - used to be correct ? Is this change intentional ? What is the reason ?

Can it be even worse ? YES, it can :no:

/* get size of image */ imageSize=headbuf.image_length_MOD_512+(headbuf.image_length_DIV_512<<9); // commented out because of DPMIST32.BIN // If (imageSize Mod 512) IsNotZero // Then imageSize-=512; // EndIf ; headerSize=(headbuf.n_header_paragraphs)<<4; relocSize= headbuf.n_relocation_items << 2; imageSize-=headerSize;

Besides of a funny programming language, Ladsoft recently implemented a horrible bug into VALX, to get around the BlockCount=2-problem !

Anyone has a valid and complete MZ EXE specification ? Or should one ask Mr. Mark_Zbikowski ? :lol3:

PESTUB seems to ignore the stupid MZ header and use "GetFileSize" instead ... there is always a hack :clap:

Could this evil 2 get fixed back to 1 ? :hungry:

PS: PESTUB (non-criminal) issue: accept "PX" besides "PE" on input ;-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
24.01.2008, 06:05

@ DOS386

[BUG] Criminal Mark-related bug | DPMIST32.BIN/PESTUB/ VALX

Post reply

> DPMIST32.BIN has 512 bytes, but BlockCount=2, and LastBlockSize=0 ->
> should be 1024 bytes instead ? :confused: Even more: DPMIST32.BIN did
> change several times in HX history, and, HX 2.5 had DPMIST32.BIN with
> BlockCount=1, and LastBlockSize=0 - used to be correct ? Is this change
> intentional ? What is the reason ?
>
> Could this evil 2 get fixed back to 1 ? :hungry:

IIRC DPMIST32.BIN was formerly named DPMIST32.EXE and if it was launched accidentally it caused a crash. So it was created as an INVALID binary (there are some remnants of this issue like the "-b" option in hx's shrmzhdr.exe) which makes the DOS loader complain about it.

Since it has no .EXE extension anymore, it could possibly be created as a VALID binary without any harm now. But since it works perfectly :-D ... never change a running system ...

---
MS-DOS forever!

DOS386

19.02.2008, 01:41

@ Japheth

since it works perfectly .. never change a running system ?

Post reply

> DPMIST32.BIN was formerly named DPMIST32.EXE and if it was launched
> accidentally it caused a crash. So it was created as an INVALID binary
> (there are some remnants of this issue like the "-b" option in hx's
> shrmzhdr.exe) which makes the DOS loader complain about it.

Any attempt to prevent bugs or make a piece of software more fool-proof is good ... but having success in this is even better :no:

1. YES, it crashes if made VALID
2. Well, it also crashes as-is INVALID - thus the "fix" doesn't work :no:
3. There is a horrible bug in VALX becuase of this. IIRC you were the linker lover and expert :clap: - don't you care ?
4. The bug causing the crash is elsewhere - see below

> it could possibly be created as a VALID binary without any harm now.

It should definitely :hungry:

> But since it works perfectly :-D ... never change a running system ...

1. Still crashing system (I never got this crash before you gave me the hint to rename BIN into EXE and run it as-is :clap: )
2. Bugged VALX :no:

At least, I fixed this bug and many other bugs in DPMIST32.BIN, and also did a few other improvements:

http://board.flatassembler.net/topic.php?t=8316

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
19.02.2008, 19:21
(edited by Japheth, 19.02.2008, 19:33)

@ DOS386

since it works perfectly .. never change a running system ?

Post reply

> At least, I fixed this bug and many other bugs in DPMIST32.BIN, and also
> did a few other improvements:
>
> http://board.flatassembler.net/topic.php?t=8316

What are these "many other bugs"? Please supply a list!

I'm not yet convinced that your code contains "other improvements", because the start is not really promising:

; Here starting from DOS: DS = ???/PSP | ES = (seg-addr) of PSP ; We have exactly 28 bytes left before 0,0,0,0 mov ax,$7202 ; 3 -> 3 push ax popf pushf pop bx ; 4x1 -> 7 cmp ax,bx ; 2 -> 9 je short llmover ; 2 -> 11 | OK we have at least 80386

Do you know that the NT flag sometimes is "lost" when running under some v86-monitors? AFAIK NTVDM is one of those, for example. V86-monitors can fake the content of the flags register, so you should be very careful with assumptions in this regard.

After all, a test for 80386 is not really necessary, because DPMILD32 won't use 386-specific opcodes before the DPMI host has switched into protected-mode, and no host I know of will switch to protected-mode for 32bit clients on a 80286 - if you're able to find a host which runs on such a cpu at all. So it should suffice to do a very simple test to filter 8086s:

push sp pop ax cmp ax,sp jnz nono

---
MS-DOS forever!

DOS386

20.02.2008, 01:45

@ Japheth

since it works perfectly .. never change a running system ?

Post reply

> What are these "many other bugs"? Please supply a list!

OK ... later :-| Regrettably few of them or even none will qualify as criminal ...

> I'm not yet convinced that your code contains "other improvements",
> because the start is not really promising

> Do you know that the NT flag sometimes is "lost" when running under some

> V86-monitors can fake the content of the flags register, so you should be
> very careful with assumptions in this regard.

YES. You are just confirming that those bring nothing but trouble and supplying one reason why I don't like them "too much" :confused:

> AFAIK NTVDM is one of those, for example.

Does it refuse to run for you ? :clap: "Need 80386" :clap:

> After all, a test for 80386 is not really necessary, because DPMILD32
> won't use 386-specific opcodes before the DPMI host has
> push sp
> pop ax
> cmp ax,sp
> jnz nono

Indeed possible also ... with 80386 it just aborts faster :lol3:

What about the TripleFault in HDPMI32 and the long mode ?

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
20.02.2008, 08:16

@ DOS386

since it works perfectly .. never change a running system ?

Post reply

> YES. You are just confirming that those bring nothing but trouble and
> supplying one reason why I don't like them "too much" :confused:

They exist and they are - still - used. That's what matters.

> Does it refuse to run for you ? :clap: "Need 80386" :clap:

If I single-step thru it with DEBUG in NTVDM - yes! The NT flag might get lost if an IRQ happens ... or just for exception 01 ... almost impossible to tell.

---
MS-DOS forever!

Japheth

Germany (South),
21.02.2008, 10:22

@ DOS386

since it works perfectly .. never change a running system ?

Post reply

> OK ... later :-| Regrettably few of them or even none will qualify as
> criminal ...

Doesn't matter if you call them criminal or legal. Just supply a list of at least 2 bugs in DPMIST32.ASM which you had fixed in your FASM version from 02/19/2008.

---
MS-DOS forever!

DOS386

19.02.2008, 02:00

@ Japheth

SnglFault ->DblFault ->TripleFault -> BOOM - unavoidable ?

Post reply

The "question" remains, why it crashes. Investigated and found the bug. :clap: BTW, I'm sure you are aware of this bug, since it becomes desperately obvious with your DEBUG version of DPMILD32 :lol3:

BOCHS wrote:

00000000000i[APIC?] local apic in initializing ======================================================================== Bochs x86 Emulator 2.3.6 Build from CVS snapshot, on December 24, 2007 ======================================================================== 00000000000i[ ] x86-64 support: yes 00264455460e[CPU0 ] write_virtual_checks(): write beyond limit, r/w 00264455460e[CPU0 ] can_push(): esp=0, normal, wraparound? limit=ffffefff 00264455460e[CPU0 ] can_push(): esp=0, normal, wraparound? limit=ffffefff 00264455460i[CPU0 ] CPU is in protected mode (active) 00264455460i[CPU0 ] CS.d_b = 32 bit 00264455460i[CPU0 ] SS.d_b = 32 bit 00264455460i[CPU0 ] EFER = 0x00000000 00264455460i[CPU0 ] | RAX=0000000000100000 RBX=0000000000000000 00264455460i[CPU0 ] | RCX=0000000000000002 RDX=0000000000002007 00264455460i[CPU0 ] | RSP=0000000000000000 RBP=000000000000005c 00264455460i[CPU0 ] | RSI=0000000000000020 RDI=00000000ffc01f14 00264455460i[CPU0 ] | R8=0000000000000000 R9=0000000000000000 00264455460i[CPU0 ] | R10=0000000000000000 R11=0000000000000000 00264455460i[CPU0 ] | R12=0000000000000000 R13=0000000000000000 00264455460i[CPU0 ] | R14=0000000000000000 R15=0000000000000000 00264455460i[CPU0 ] | IOPL=3 id vip vif ac vm RF nt of df if tf sf zf af pf cf 00264455460i[CPU0 ] | SEG selector base limit G D 00264455460i[CPU0 ] | SEG sltr(index|ti|rpl) base limit G D 00264455460i[CPU0 ] | CS:0020( 0004| 0| 0) ff801000 00006763 0 1 00264455460i[CPU0 ] | DS:0028( 0005| 0| 0) 0002bdb0 000ffffe 1 1 00264455460i[CPU0 ] | SS:0028( 0005| 0| 0) 0002bdb0 000ffffe 1 1 00264455460i[CPU0 ] | ES:004b( 0009| 0| 3) 00000000 000fffff 1 1 00264455460i[CPU0 ] | FS:0000( 0000| 0| 0) 00031040 0000ffff 0 0 00264455460i[CPU0 ] | GS:0000( 0000| 0| 0) 0002a6c0 0000ffff 0 0 00264455460i[CPU0 ] | MSR_FS_BASE:0000000000031040 00264455460i[CPU0 ] | MSR_GS_BASE:000000000002a6c0 00264455460i[CPU0 ] | RIP=0000000000004be5 (0000000000004be5) 00264455460i[CPU0 ] | CR0=0x80000031 CR1=0x0 CR2=0x0000000000000000 00264455460i[CPU0 ] | CR3=0x01fff000 CR4=0x00000200 00264455460i[CPU0 ] >> push ecx : 51 00264455460e[CPU0 ] exception(): 3rd (12) exception with no resolution, shutdown status is 00h, resetting 00264455460i[SYS ] bx_pc_system_c::Reset(SOFTWARE) called 00264455460i[CPU0 ] cpu software reset 00264455460i[APIC0] local apic in CPU 0 initializing 00264459200i[BIOS ] $Revision: 1.166 $ $Date: 2006/08/11 17:34:12 $

TripleFault inside HDPMI32 triggered by stack flooding :lol3:

DPMIST32.BIN and DPMILD32.EXE run each-other many times and flood HDPMI32's stack this way ... and the only way out preventing this crazy game from running into the infinity is the TripleFault :lol3:

IIRC we had the very same bug recently because of a bug in INFOPAD and "paranoid" DOS/32A :no:

But it comes even worse: Obviously DPMILD32 indeed activates the LOOOOOOOOOONG MODE :lol: !!!

I see 2 pieces that need fixing:

- DPMILD32: (don't load DPMIST32.BIN, + other issues, see other (soon) thread)
- HDPMI32: protect from Ring0 stack overflow

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
20.02.2008, 10:51

@ DOS386

SnglFault ->DblFault ->TripleFault -> BOOM - unavoidable ?

Post reply

> The "question" remains, why it crashes. Investigated and found the bug.
> :clap: BTW, I'm sure you are aware of this bug, since it becomes
> desperately obvious with your DEBUG version of DPMILD32 :lol3:

I'm not aware of "this" bug, on the contrary, I hardly understand what you are talking about.

> DPMIST32.BIN and DPMILD32.EXE run each-other many times and flood
> HDPMI32's stack this way ... and the only way out preventing this crazy
> game from running into the infinity is the TripleFault :lol3:

Ok, HDPMI is rebooting the machine due to a ring 0 stack underflow. Please provide a test case which I can use ... on a REAL machine, not in Box!

> But it comes even worse: Obviously DPMILD32 indeed activates the
> LOOOOOOOOOONG MODE :lol: !!!

If I'd only know what you are talking about ...

> - DPMILD32: (don't load DPMIST32.BIN, + other issues, see other (soon)
> thread)

Waiting for your test case ...

> - HDPMI32: protect from Ring0 stack overflow

Has low priority, since this only happens with some buggy programs...

---
MS-DOS forever!

DOS386

21.02.2008, 02:33

@ Japheth

SnglFault ->DblFault ->TripleFault -> BOOM - unavoidable ?

Post reply

Japheth wrote:

> If I single-step thru it with DEBUG in NTVDM - yes

OK ... you can make everything fail if you try hard enough :lol3:

> I'm not aware of "this" bug

As you wrote in your answer 1 month back, a crash occurs if you rename DPMIST32.BIN into an EXE and run it :clap:

> on the contrary, I hardly understand what you are talking about.

Please don't be more pessimistic than necessary ;-)

> Ok, HDPMI is rebooting the machine due to a ring 0 stack underflow. Please
> provide a test case which I can use ... on a REAL machine, not in Box!

I always test on a REAL machine first :clap: ... just rename (the valid or "INVALID" version of) DPMIST32.BIN into X.EXE and run it in DOS ;-)

> > LOOOOOOOOOONG MODE :lol: !!!
> If I'd only know what you are talking about ...

BOCHS crash report, see above ...

> Waiting for your test case ...

DPMIST32.BIN renamed into X.EXE + DPMILD32.

> > - HDPMI32: protect from Ring0 stack overflow
> Has low priority, since this only happens with some buggy programs...

YES, it's just a small open problem ... and would be maybe easy to fix ? ;-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
21.02.2008, 07:05

@ DOS386

SnglFault ->DblFault ->TripleFault -> BOOM - unavoidable ?

Post reply

> > If I single-step thru it with DEBUG in NTVDM - yes
>
> OK ... you can make everything fail if you try hard enough :lol3:

Running DEBUG isn't THAT hard. But feel free to ignore...

> > > LOOOOOOOOOONG MODE :lol: !!!
> > If I'd only know what you are talking about ...
>
> BOCHS crash report, see above ...

Box just displays the internal 64bit registers. That doesn't mean too much.

> > Waiting for your test case ...
>
> DPMIST32.BIN renamed into X.EXE + DPMILD32.

DPMIST32.BIN isn't supposed to be a valid binary...
Feeding DPMILD32 with a simple DOS MZ binary is no problem.

Sorry, but I'm unable to see any problem at all.

---
MS-DOS forever!

DOS386

22.02.2008, 03:09

@ Japheth

SnglFault ->DblFault ->TripleFault -> BOOM - unavoidable ?

Post reply

> Running DEBUG isn't THAT hard. But feel free to

No, but DOS wouldn't have too much left if one threw away anything that you can crash with help of NTVDM and DEBUG ;-)

> Box just displays the internal 64bit registers. That doesn't mean too much.

Bug of BOCHS ?

> DPMIST32.BIN isn't supposed to be a valid binary...

But it still loads and crashes, as you pointed out ;-)

> Feeding DPMILD32 with a simple DOS MZ binary is no problem.

No. As long as this binary doesn't run DPMILD32.EXE ...

> Sorry, but I'm unable to see any problem at all.

No big problem, very small problem: your fix (corrupting MZ header) doesn't work, thus I'm suggesting alternatives ;-)

> 2 bugs

ASAP :hungry:

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Laaca

Czech republic,
24.01.2008, 07:33

@ DOS386

[BUG] Criminal Mark-related bug | DPMIST32.BIN/PESTUB/ VALX

Post reply

Yes, it is a known issue. But it isn't discussed very often.

---
DOS-u-akbar!

Berlin, Germany,
24.01.2008, 08:18

@ DOS386

[BUG] Criminal Mark-related bug | DPMIST32.BIN/PESTUB/ VALX

Post reply

> BUG: should be in Developers, not Announce :-(

You love the word "bug", don't you? If you want a post to appear in category "Developers", you have to select that. Easy, Eh? :-P I have moved your post now.

---
Forum admin

DOS386

19.02.2008, 01:30

@ rr

[BUG] Criminal Mark-related bug | DPMIST32.BIN/PESTUB/ VALX

Post reply

> You love the word "bug", don't you?

:lookaround:

> If you want a post to appear in category "Developers", you have to select that.

I did ... or tried at least ... but didn't work for some reason :no:

> have moved your post now.

Great ... one forum problem (and "reason" for change to php-BB3) less :clap:

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***