DOS ain't dead - [BUG] unreadable reloXection deadlocks DPMILD32

DOS386

12.06.2008, 23:58

[BUG] unreadable reloXection deadlocks DPMILD32 (DOSX)

; BUGE.ASM include "IMPORT32.INC" format PE GUI 4.0 entry llstart section '.code' code readable executable llstart: ud2 call [NotFound] call [ExitProcess] section '.idata' import data readable writeable library bd, 'BUGD.DLL', ke, 'KERNEL32.DLL' import bd, NotFound, 'NotFound' import ke, ExitProcess , 'ExitProcess'

----------------------------------------------------------------------------

; BUGD.ASM include "EXPORT.INC" format PE GUI 4.0 DLL entry llentry section '.code' code readable executable llentry: mov eax,1 ret 12 none: mov al,[none] ret section '.edata' export data readable export 'BUGD.DLL', None, 'None' section '.reloc' fixups data discardable ; readable

Seems I found a severe bug in DPMILD32: it deadlocks when a reloc section in not "readable" :-( Above very-high-quality code should trigger a "missing import" complaint, instead, it deadlocks itself at black screen with white rat arrow - reacting to mouse movements, but not to CTL-C, CTL-BEAK, APPZ/CRASH/MENU, ... -> reboot unavoidable :-( Uncommenting the "readable" or deleting complete section makes the bug vanish. Windaube, OTOH, seems to consider the section (all sections ?) unconditionally as readable.

IMHO the "readable" flag is crap - what's the point of a non-readable section ? Just ignore it and assume all readable ;-)

BTW, any plans to delete NE support from DPMILD32 ? Docs already flag it as "obsolete" , "no advantages" ... etc. - the time has come to kick it out :hungry:

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
13.06.2008, 08:40

@ DOS386

BUG? What BUG?

Post reply

> Seems I found a severe bug in DPMILD32:

Sorry, but I have severe difficulties to accept this as a bug at all.

> it deadlocks when a reloc section in not "readable" :-(

Then just don't do that!

> Windaube, OTOH, seems to consider the section (all sections ?) unconditionally as readable.

Ok, then it's a slight imcompatibility, most likely with no relevance. Please provide a real Win32 application which doesn't run in HX because of that and I will "fix" the issue immediately (yes, I've learned a lot from the DosBox developers :-D ).

> BTW, any plans to delete NE support from DPMILD32 ? Docs already flag it
> as "obsolete" , "no advantages" ... etc. - the time has come to kick it
> out :hungry:

My fantastic deb32f debugger still uses DPMILD32 and 16bit dlls. And why removing this support at all? To gain 2 kB extended-memory/disk space? And possibly introducing new bugs during the removal?

---
MS-DOS forever!

DOS386

14.06.2008, 15:02

@ Japheth

BUG? What BUG?

Post reply

> have severe difficulties to accept this as a bug

OK ... have severe doubt whether a deadlock can / should be considered as a desirable effect :confused:

> Then just don't do that!

Thanks for the hint ... trivial as hell (since I already know what the reason of the deadlock is, 3 days ago I didn't :-( ) ... but this won't damage the fact that DPMILD32 has a bug

> Please provide a real Win32 application which doesn't run in HX

IIRC real Win32 applications ( Mozilla Firefox, Adobe Photosh**, IDE of WATCOM/CC386/VCC2010/... , ... ) don't run on HX anyway because of lack of the high level GUY API :-( And since 99.99% of the "market" is occupied by M$-linker and LD, apparently setting the bit, there is not much space to find one :-|

> BUG? What BUG?

The bug I already had isolated and exhaustively described above ? :confused:

BTW, it's of course not my BUG ... I "imported" it from Tomasz's DLL example ... and since nobody has complained so far one can assume that all versions of Windaube + possibly ReactOS + possibly Wine (or even Beer ??? :lol: ) don't care about this bit. And, if you want to be very rigid about 100% PE compliance, you can raise an error message rather than a deadlock :ok:

I wonder how the answer might have differed if I just pointed about Tomasz's DLL example not working with HX, without investigating / revealing why :surprised:

> My fantastic deb32f debugger still uses DPMILD32 and 16bit dlls.

I see : This program requires Microsoft Windows :clap: ... anyway, roots of this debugger seem to be very old, parts have been converted to PE already, and IIRC cca 2 years ago you wrote something like "needs a redesign, already for 10 years" ...

> To gain 2 kB extended-memory/disk space?

Maybe a bit more ?

> And possibly introducing new bugs during the removal?

It's no way critical ;-) just the DPMILD32 source seems to be a bit ancient, PE support is "hacked" on the top of NE, and seems before a PE can be loaded, first the MZ header is read, analyzed, seeking to "next level", searching for "NE", failure, close the file, if PE is enabled at all, reopen the file, re-read MZ header, re-seek, re-read "next level" ... no big problem, since the debugger uses NE it's of course NOT a good idea for now ;-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
14.06.2008, 15:57

@ DOS386

BUG? What BUG?

Post reply

> Thanks for the hint ... trivial as hell (since I already know what the
> reason of the deadlock is, 3 days ago I didn't :-( ) ... but this won't
> damage the fact that DPMILD32 has a bug :confused:

dpmild32 doesn't care about the PE "readable" bit (IMAGE_SCN_MEM_READ) at all. And I'm not in the mood to analyze how Fasm converts its segment attributes to COFF section attributes. Since it occurs with Fasm "hand-made" PEs only, it can be ignored IMO.

> I wonder how the answer might have differed if I just pointed about
> Tomasz's DLL example not working with HX, without investigating /
> revealing why :surprised:

I doubt that this would have made any difference. It's still much work for almost nothing.

---
MS-DOS forever!

DOS386

19.06.2008, 09:04
(edited by DOS386, 19.06.2008, 09:17)

@ Japheth

1. Closed: NOT a BUG | 2. Closed: NOT a BUG | 3. Closed: NB

Post reply

> BUG? What BUG?

Actually IMHO the opposite is true: this is most likely the best bug report this forum has ever seen :lol3: (have a better one ? point me to it :hungry: anyone except me uses this forum for reporting bugs at all ? :confused: ) I provided a detailed description, reproductability of this bug is always, supplying a test case, a deep analysis - several 1'000'000's bits are involved in this bug ... and I tracked it down to just 1 single bit :clap: and detailed solution suggestions ...

> not in the mood to analyze how Fasm converts its
> segment attributes to COFF section attributes.

1:1, as usual, SSSO :clap: And it's not the problem anyway - the problem is the read permission bit only, and nothing else. Manually switching this one bit allows me to switch the bug on and off. YES !!! The reproductability of this bug is always :clap:

> dpmild32 doesn't care about the PE "readable" bit (IMAGE_SCN_MEM_READ) at all

COOL ... but what does this code do ?

; PELOAD.ASM ;*** support for PE-files *** ;*** 32-Bit DPMI clients only *** ?SKIPCOMMENTS equ 1 ;std=1, 1=skip sections not marked as r/w/e if ?SKIPCOMMENTS test byte ptr [edi].IMAGE_SECTION_HEADER.Characteristics+3, 0E0h ;is is read/write/exec? jz @F ; FAR FAR FAR below endif ;-------------------------------------------- section size into ecx call getsectionsize shr ecx, 2 push edi mov edi,[edi].IMAGE_SECTION_HEADER.VirtualAddress lea edi,[edi + esi - sizeof MZHDR] mov edx,edi xor eax, eax rep stos dword ptr [edi] pop edi

Maybe it skips my evil non-readable section ? :clap: Even worse, it doesn't skip only the loading, but also the memory clearing ? Section is not loaded, but then "fixups" are processed, thus, processing ZERO's or random garbage remaining in memory from previous activity ? The bug symptoms indeed depend from previous activity, besides aforementioned black screen, I also get various DKRNL exceptions, see shot.

[image]

IMHO the "?SKIPCOMMENTS" "feature" should get deleted, or moved after the clearing at least.

One more good idea would be in DPMILD32 to check the page block fixup sizes for:
- >=8
and
- divisibility by 4
Because a value <8 or not divisible by 4 is obviously evil :surprised: -> whine ("Can't load, fixups broken" or so) -> file is missbrewed, misslinked, corrupted, crosslinked, etc. BTW, the "PE" thingie has the very same bug: it crashes if the size is ZERO :-|

Well, implementing the fixup check + __always__ section clearing would allow to keep the "?SKIPCOMMENTS" hack enabled ;-)

This actually brings us to the next severe bug: memory clearing. DKRNL seems to correctly clear all allocated memory, also DPMILD32 clears __some__ sections (see above), however, besides sections affected by the "?SKIPCOMMENTS" "feature", it doesn't clear the stack, while Windaube seems to do this job - "almost" :-|

; ASSUME SS:(1 MiB stack at least) mov ecx,0 ; Skip bytes at the top mov esi,esp dec esi sub esi,ecx xor edx,edx ; MONVTQ EDX,0 ; Useless xor ebx,ebx ; MONVTQ EBX,0 xor eax,eax ; MONVTQ EAX,0 std ; !!! run down @@: lodsb inc ecx add ebx,eax ; Sum cmp ecx,$100000 ; 1 MiB jne short @b ; Limited loop [or dead loop] up to the crash ud2 ;----

This sums the garbage in the stack - Win provides an almost clean stack (sum is slightly random, but only cca $8000...$FFFF, located at the top), while with HX the stack is fully random, depending from previous activity :-( Actually I have some apps working poorly with HX, most likely because of this. Failing to initialize variables located in the stack is of course a severe bug of the application, nevertheless, DPMILD32 also contributes here :-( Solution:

- Always clear the stack after allocating it or before starting the application
- Add a hack - bit 14 (DPMILDR=16384) : fill stack with pseudo-random garbage instead of clearing it - this will remove the dependency of bugginess from previous activity :-) , and add a great debugging feature, allowing me to verify my suspect and give me additional ammunition for reporting such application bugs against "Won't fix - Works for me" ;-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

DOS386

19.06.2008, 09:06
(edited by DOS386, 19.06.2008, 09:21)

@ DOS386

1. Closed: NOT a BUG | 2. Closed: NOT a BUG | 3. Closed: NB

Post reply

Forum wrote:

> Text is too long. Only 5000 bytes allowed

One more DPMILD32 issue:

;*** load image in memory, dont resolve anything ;*** bx=file handle ;*** esi=IMAGE_NT_HEADER ;*** the object table is located behind the NT_HEADERS ;--- returns with C on errors in binary LoadImage proc uses edi ebp es if ?INT41SUPPORT sub esp, sizeof D386_Device_Params @loadesp ebp mov [ebp].D386_Device_Params.DD_logical_seg,0 mov dword ptr [ebp].D386_Device_Params.DD_name,0 mov word ptr [ebp].D386_Device_Params.DD_name+4,0 mov edi, [esi.MZHDR.pExeNam - sizeof MZHDR] lea edi, [edi + esi - sizeof MZHDR] call skippath mov dword ptr [ebp].D386_Device_Params.DD_sym_name,edi mov word ptr [ebp].D386_Device_Params.DD_sym_name+4,ds endif

What is this "?INT41SUPPORT" supposed to controll ? I don't see it declared anywhere ? Maybe always false, a relict ready for kick-out ?

> Since it occurs with Fasm "hand-made" PEs only, it can be ignored IMO.
> I refused to "fix" the bug because it's a lot of work for virtually nothing.
> It's still much work for almost nothing.

True, but now I've done 99% of the huge work so the 3 trivial fixes can be applied into DPMILD32 ;-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
19.06.2008, 10:29

@ DOS386

1. Closed: NOT a BUG | 2. Closed: NOT a BUG | 3. Closed: NB

Post reply

> > dpmild32 doesn't care about the PE "readable" bit (IMAGE_SCN_MEM_READ)
> at all
>
> COOL ... but what does this code do ?
>
> ; PELOAD.ASM> > ;*** support for PE-files *** > ;*** 32-Bit DPMI clients only *** > > ?SKIPCOMMENTS equ 1 ;std=1, 1=skip sections not marked as r/w/e > > if ?SKIPCOMMENTS > test byte ptr [edi].IMAGE_SECTION_HEADER.Characteristics+3, 0E0h ;is is > read/write/exec? > jz @F ; FAR FAR FAR below > endif > ;-------------------------------------------- section size into ecx > call getsectionsize > > shr ecx, 2 > push edi > mov edi,[edi].IMAGE_SECTION_HEADER.VirtualAddress > lea edi,[edi + esi - sizeof MZHDR] > mov edx,edi > xor eax, eax > rep stos dword ptr [edi] > pop edi
>
> Maybe it skips my evil non-readable section ? :clap:

Yes.

> IMHO the "?SKIPCOMMENTS" "feature" should get deleted, or moved after the
> clearing at least.

Or just set this boring "readable" bit in your app ...

> One more good idea would be in DPMILD32 to check the page block fixup
> sizes for:
> - >=8
> and
> - divisibility by 4
> Because a value <8 or not divisible by 4 is obviously evil :surprised: ->
> whine ("Can't load, fixups broken" or so) -> file is missbrewed,
> misslinked, corrupted, crosslinked, etc. BTW, the "PE" thingie has the
> very same bug: it crashes if the size is ZERO :-|

Ok, might be considered...

> This actually brings us to the next severe bug: memory clearing.
> DKRNL seems to correctly clear all allocated memory, also DPMILD32 clears
> __some__ sections (see above), however, besides sections affected by the
> "?SKIPCOMMENTS" "feature", it doesn't clear the stack, while Windaube
> seems to do this job - "almost" :-|

> - Always clear the stack after allocating it or before starting the
> application

I'm not sure if this is a good idea. If helps with buggy applications only, which fully deserve to crash and die.

---
MS-DOS forever!

DOS386

20.06.2008, 09:51

@ Japheth

1. Closed: NOT a BUG | 2. Closed: NOT a BUG | 3. Closed: NB

Post reply

> > > dpmild32 doesn't care about the PE "readable" bit
> > COOL ... but what does this code do ?
> > Maybe it skips my evil non-readable section ?
> Yes.

The truth is out :-|

> Or just set this boring "readable" bit in your app ...

Done 10 days ago (RTFS) :clap: ... I don't refuse to fix "my" bug ... but doing so won't fix the bug ("vulnerability") in DPMILD32

> > idea would be in DPMILD32 to check the page block fixup sizes
> Ok, might be considered...

:-)

> > - Always clear the stack
> I'm not sure if this is a good idea. If helps with buggy applications
> only, which fully deserve to crash and die

No :crying: please don't shoot everybody who isn't perfect. Leaving the stack random can raise random very-hard-to-debug bugginess, allows bugged apps to behave differently depending of "first run" or "non-first run" or other apps ran previously, what IMHO is very bad Clearing the stack + adding hack to fill it with mess instead makes the apps behave deterministically and helps to debug those buggy applications and brings up the evidence of bug allowing to fix or shoot them then :-D

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

DOS386

18.12.2009, 03:21

@ Japheth

Discovered a NEW BUG in DPMILD32

Post reply

if ?DPMI10 test cs:bEnvFlgs,ENVFL_DONTUSEDPMI1 jnz UseStdAlloc ;---------------------- some apps have to be loaded at their ImageBase ;---------------------- although relocs are NOT stripped!

Excellent "design" :-(

test [edx.IMAGE_NT_HEADERS.FileHeader.Characteristics],IMAGE_FILE_DLL jz @F ;---------------------- for dlls dont use the prefered load address unless ;---------------------- relocs are stripped or DPMILDR=1024 is set test cs:bEnvFlgs2, ENVFL2_USEPREFADDR jnz @F test [edx.IMAGE_NT_HEADERS.FileHeader.Characteristics],IMAGE_FILE_RELOCS_STRIPPED jz UseStdAlloc @@: mov ebx, [edx.IMAGE_NT_HEADERS.OptionalHeader.ImageBase] mov ecx, eax mov edx, 1 push eax mov ax, 0504h int 31h pop eax jc UseStdAlloc mov xmemhdl, esi mov edi, ebx jmp allocok UseStdAlloc: endif

test edx.IMAGE_NT_HEADERS.FileHeader.Characteristics],IMAGE_FILE_RELOCS_STRIPPED

is faulty, this bit is "not used" in PE. PE loaders in both ME and XP don't look at this bit, they check the directory instead.

So the correct way would be:

1. Peek both directory entries
2. If both are ZERO, assume "IMAGE_FILE_RELOCS_STRIPPED"
3. If RVA is valid and (size is >=8 and divisible by 4), assume "relox are avaiable"
4. Otherwise give up "broken relox, can't load"

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Japheth

Germany (South),
19.12.2009, 07:44

@ DOS386

Discovered a NEW BUG in DPMILD32

Post reply

>> edx.IMAGE_NT_HEADERS.FileHeader.Characteristics],IMAGE_FILE_RELOCS_STRIPPED
>
> is faulty, this bit is "not used" in PE. PE loaders in both ME and XP
> don't look at this bit, they check the directory instead.

[citation needed]

---
MS-DOS forever!

DOS386

19.12.2009, 14:04

@ Japheth

Discovered a NEW BUG in DPMILD32

Post reply

> [citation needed]

No need. A PE file having the relox directory entries valid non-ZERO and the "relox stripped" bit set will do ;-) Will you fix it if I supply one ? :hungry:

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***