DOS ain't dead - Protectmode handler called from realmode

Laaca

Czech republic,
12.08.2010, 21:12

Protectmode handler called from realmode (Developers)

Hello all. I need help with interrupt stuff where I totally got stuck.
I have a protectmode program which hooks the INT13h interrupt. Then calls some INT21h functions which internaly calls the INT13h (which is now hooked) and from this hooked 1NT13h I analyse the given arguments.
Purpose: to find out to which physical device and LBA address points the DOS drives.

I had no problem to code it in real mode in Turbo pascal.
But in Freepascal it still crashes :-(

As information resources I used these two documents:
http://www.delorie.com/djgpp/doc/dpmi/ch4.6.html
and
http://support.microsoft.com/?scid=kb%3Ben-us%3B67845&x=11&y=11

And here is my code:

uses Go32;

procedure Int13hHandler; assembler;
{I know, it does nothing but in first step would be enough not to crash}
asm
cld
lodsw;mov es:[edi+2Ah],ax {restores IP}
lodsw;mov es:[edi+2Ch],ax {restores CS}
add word ptr es:[edi+2Eh],4 {or "6"?} {sets SP}
iret
end;
procedure Int13hHandler_Dummy;begin end;

var
old13h:TsegInfo;
new13h:TsegInfo;
regs13h:trealregs; external name '___v2prt0_rmcb_regs';

begin
Get_pm_interrupt($13,old13h);
{Backup of old interrupt handler}

get_rm_callback(@Int13hHandler, regs13h, new13h);
{prepares the protectmode procedure for calling from realmode}

Set_rm_interrupt($13,new13h);
{install my interrupt handler}

asm
mov ah,32h {load info from}
mov dl,3 {disk C:}
int 21h {this call will lead to INT13h/AH=42h}
end; {I will trap it and analyze the given arguments}

writeln('Everything seems to be fine');
readln;

Set_rm_interrupt($13,new13h);
{restores normal state}

writeln('Stil fine, leaving');
end.

---
DOS-u-akbar!

ecm

Düsseldorf, Germany,
12.08.2010, 21:24

@ Laaca

Protectmode handler called from realmode

Post reply

> add word ptr es:[edi+2Eh],4 {or "6"?} {sets SP}

Did you try 6 already? Should be right. Also, you should restore the flags too.

> Set_rm_interrupt($13,new13h);
> {restores normal state}

Needs to use old13h.

With your current code, it might be a problem that other Int13 calls will get to your handler too. Instead of simulating an iret, you should chain to the old RM Int13 handler. (Simply set cs and ip appropriately, instead of from the stack.)

---
l

Laaca

Czech republic,
13.08.2010, 01:46

@ ecm

Protectmode handler called from realmode

Post reply

Fixed!
There were three problems in my code.
1) the original SP is necessary to shift by 6 (as cm suggested) not by 4 (as microsoft suggested)
2) it is necessary to save FLAGS
3) the biggest problem, I had an absolutely stupid typo in my code:
should be: Get_rm_interrupt($13,old13h);
but I had: Get_pm_interrupt($13,old13h

So the correct code is:
uses Go32;

procedure Int13hHandler; assembler;
{I know, it does nothing but in first step would be enough not to crash}
asm
cld
lodsd;mov es:[edi+2Ah],eax {restores CS:IP}
lodsd;mov es:[edi+20h],ax {flags}
add word ptr es:[edi+2Eh],6 {sets SP}
iret
end;
procedure Int13hHandler_Dummy;begin end;

var p:pointer;
r:registers;
b:byte;

old13h:TsegInfo;
new13h:TsegInfo;
regs13h:trealregs;

begin
Get_rm_interrupt($13,old13h);
{Backup of old interrupt handler}

get_rm_callback(@Int13hHandler, regs13h, new13h);
{prepares the protectmode procedure for calling from realmode}

Set_rm_interrupt($13,new13h);
{install my interrupt handler}

asm
mov ah,32h {load info from}
mov dl,3 {disk C:}
int 21h {this call will lead to INT13h/AH=42h}
end; {I will trap it and analyze the given arguments}

writeln('Everything seems to be fine');
readln;

Set_rm_interrupt($13,old13h);
free_rm_callback(new13h);
{restores normal state}

writeln('Still fine, leaving');
end.

---
DOS-u-akbar!

Japheth

Germany (South),
13.08.2010, 07:31

@ Laaca

Protectmode handler called from realmode

Post reply

> 1) the original SP is necessary to shift by 6 (as cm suggested) not by 4
> (as microsoft suggested)

It depends on how your FAR routine is supposed to exit: a RETF requires 4 bytes, an IRET requires 6 bytes.

> {I know, it does nothing but in first step would be enough not to crash}
> asm
> cld
> lodsd;mov es:[edi+2Ah],eax {restores CS:IP}
> lodsd;mov es:[edi+20h],ax {flags}
> add word ptr es:[edi+2Eh],6 {sets SP}
> iret
> end;

The second "lodsd" should be a "lodsw". There's a small risk that lodsd might cause a crash.

There is another risk that Int 13h might be called recursively. IIRC, Jack R. Ellis told me that USBDRIVE.SYS is supposed to do this occasionally.

---
MS-DOS forever!

bretjohn

Rio Rancho, NM,
13.08.2010, 18:43

@ Japheth

Protectmode handler called from realmode

Post reply

> There is another risk that Int 13h might be called recursively. IIRC, Jack
> R. Ellis told me that USBDRIVE.SYS is supposed to do this occasionally.

It's USBDRIVE.COM, not USBDRIVE.SYS, and I don't think it makes any recursive INT 13h calls (I'll verify this before the next release). Jack and I did talk about the possibility of recursive INT 13h calls as a way to design a manageable interface between UIDE and USBDRIVE, which of course never panned out.

bretjohn

Rio Rancho, NM,
13.08.2010, 19:25

@ bretjohn

EDIT: Protectmode handler called from realmode

Post reply

> I don't think it makes any recursive INT 13h calls (I'll verify this before
> the next release).

I just checked, and USBDRIVE does, in fact, make a recursive INT 13h call in one place, which I believe is justified.

I make a recursive call in INT 13h, function 8 (Get Drive Parameters). One of the values this returns is the total number of INT 13h drive currently numbers in use (in the DL register). The only way I know of to test whether a specific INT 13h drive number is valid or not is to issue a different INT 13h request, function 10h (Check Ready).

Since there is the possibility that another INT 13h TSR driver could have been installed after USBDRIVE, at the time the "Get Drive Parameters" request is made to USBDRIVE, it does not know for certain how many INT 13h drives are currently in existence. The only way to calculate this is to issue an INT 13h "Check Ready" request for each possible INT 13h disk number, and count up how many appropriate responses are received. This is what USBDRIVE does.

The other possibility is to use the value stored in the BIOS Data Area (BDA) at 0040:0075, which is supposed to be the number of INT 13h drives (according to the BIOS). However, I have been told by what I consider to be reliable sources that this number cannot always be trusted for various reasons (i.e., SYS or TSR drivers do not always update this value when they should, or drivers that get uninstalled may leave "holes" in the INT 13h numbers that can cause problems in certain cases).

It's possible that INT 13h, function 8 simply expects the number from 0040:0075 to be copied into DL for the return, but that is not what USBDRIVE does.

Any suggestions or commentary?

DOS386

26.08.2010, 09:33

@ bretjohn

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> Purpose: to find out to which physical device and LBA address points the DOS drives

Indeed badly missing:

- a call revealing what device hosts a volume
- a call taking 2 volumes as input and revealing whether to use low memory (2 different disks, at least 1 is RAMDISK or SSD or so) or high memory (both on 1 hard disk) copy strategy

Windaube AFAIK doesn't have such calls either, some people (or companies) simply never learn :clap:

> I don't think it makes any recursive INT 13h calls (I'll verify this
> I just checked, and USBDRIVE does, in fact, make a recursive INT 13h call

It has this BUG (reported cca 1 year ago) that after driver uninstall the occupied "physical disks" are not freed :-(

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

bretjohn

Rio Rancho, NM,
26.08.2010, 19:03

@ DOS386

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> > Purpose: to find out to which physical device and LBA address points the
> DOS drives

At least in theory, this is not something you're supposed to care about (though I must admit this is something that would occasionally be "nice to know"). A DOS-accessible volume doesn't have to be associated with an INT 13h physical device or CHS/LBA addressing at all (common examples include RAM drives, network drives, CD/DVD, SCSI/ASPI, USB, FireWire, etc.). That's why things are designed the way they are, and why you're supposed to only use OS calls to access data inside volumes. "Going behind the OS's back" and directly accessing data inside a volume is not a good idea, and why DOS (and Windows) don't provide a way to go back and forth between the two.

> Indeed badly missing:
>
> - a call revealing what device hosts a volume

I _might_ add something like this to a future release of USBDRIVE, though it is unlikely. If I did, however, the call would only be valid for the USB drives controlled by USBDRIVE -- it would not work for any other disks (including the MFM/RLL/ESDI/ATA/SATA/PATA/... disks that the BIOS & DOS kernel provide drivers for). In addition, there would be at least two different calls needed: one to obtain the physical information given a volume, and another to obtain the volume(s) given a physical disk.

Laaca is working on a protocol-independent way of accomplishing this: http://www.bttr-software.de/forum/forum_entry.php?id=8578

> - a call taking 2 volumes as input and revealing whether to use low memory
> (2 different disks, at least 1 is RAMDISK or SSD or so) or high memory
> (both on 1 hard disk) copy strategy

First of all, what do you mean by "high memory": UMB, HMA, EMS, XMS, or something else?

Secondly, when dealing with volumes, you should always use DOS calls, which require the memory address to be in segment:offset format, which in turn requires conventional, UMB, EMS, or in some cases, HMA. The memory must generally be compatible with all types of hardware DMA (including PCI bus mastering). It doesn't really matter whether the volumes are on the same physial disk or not, or even if they are on a physical disk at all.

> Windaube AFAIK doesn't have such calls either, some people (or companies)
> simply never learn :clap:

And they probably never will have such calls, for security and data safety reasons.

> It has this BUG (reported cca 1 year ago) that after driver uninstall the
> occupied "physical disks" are not freed :-(

That will be fixed in the next release of USBDRIVE. I've discovered that this is actually a "bug" in the way some BIOS's work, not a "bug" in USBDRIVE as such.

DOS386

27.08.2010, 03:25

@ bretjohn

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> At least in theory, this is not something you're supposed to care about

Disagree, see below about copying.

> A DOS-accessible volume doesn't have to be associated with an INT
> 13h physical device or CHS/LBA addressing at all

Right, but this doesn't invalidate the need for a such call at all (the returned info may say something else than "physical hard disk" blah blah).

> That's why things are designed the way they are, and why you're supposed to
> only use OS calls to access data inside volumes. "Going behind the OS's
> back" and directly accessing data inside a volume is not a good idea, and
> why DOS (and Windows) don't provide a way to go back and forth

Good, but see below.

> First of all, what do you mean by "high memory": UMB, HMA, EMS, XMS, or
> something else?

I meant small buffer vs big buffer. (BTW, I deprecate all memories except physical and maybe XMS, heh ...)

> It doesn't really matter whether the volumes are on the same
> physical disk or not, or even if they are on a physical disk at all.

It does matter very much, as copying to same physical disk with small buffer is very slow and exhausts the drive's durability.

> they probably never will have such calls, for security and data safety reasons

Missed the point, see above. That's why in file managers:

- There is only 1 strategy
- The user must pick the strategy
- The user must reveal what volume is on what physical disk
- Some "file managers" don't do the copying at all, they instead instruct the EXPLOITER to to the work for them :-(

> That will be fixed in the next release of USBDRIVE. I've discovered that
> this is actually a "bug" in the way some BIOS's work, not a "bug" in
> USBDRIVE as such.

Good :-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

bretjohn

Rio Rancho, NM,
27.08.2010, 16:47

@ DOS386

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> Right, but this doesn't invalidate the need for a such call at all (the
> returned info may say something else than "physical hard disk" blah blah).

Do you have a specific protocol in mind? If this is something you're wanting, you probably need to come with a design/API/protocol and at least do an initial implementation for BIOS/kernel provided drives. Like I stated earlier, if I did it for USBDRIVE I would limit my scope to USB disks, and it sounds like you want much more than that.

> Missed the point, see above. That's why in file managers:
>
> - There is only 1 strategy
> - The user must pick the strategy
> - The user must reveal what volume is on what physical disk
> - Some "file managers" don't do the copying at all, they instead instruct
> the EXPLOITER to to the work for them :-(

I'm not aware of any File Managers where the user is expected to "pick a strategy" or provide any information about the logical-to-physical relationships. Which File Manager(s) are you referring to?

Also, what EXPLOITER are you referring to? I've never seen that, either. I'm guessing that you are referring to a drive using its internal buffers to do the copy instead of actually transferring the data across the bus into RAM and back across the bus again? If that's the case, it would take much more intimate knowledge of how the disk works than the volume-to-physical relationships. You would have to know specific hardware details, like whether the disk even has an internal buffer that you can manipulate or not (e.g., USB flash drives don't, but USB-attached hard drives might).

DOS386

28.08.2010, 01:26

@ bretjohn

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> Do you have a specific protocol in mind? If this is something you're
> wanting, you probably need to come with a design/API/protocol and

under progress :hungry:

> do an initial implementation for BIOS/kernel provided drives. Like I
> stated earlier, if I did it for USBDRIVE I would limit my scope to USB
> disks, and it sounds like you want much more than that.

Knowing whether 2 volumes are on same USB HD (use big buffer) or not would be useful.

> I'm guessing that you are referring to a drive using its internal buffers
> to do the copy instead of actually transferring the data across the bus
> into RAM and back across the bus again? If that's the case, it would take
> much more intimate knowledge of how the disk works than the
> volume-to-physical relationships. You would have to know specific hardware
> details, like whether the disk even has an internal buffer that you can
> manipulate or not (e.g., USB flash drives don't, but USB-attached hard
> drives might).

Interesting, but not what I was referring to.

When copying to the same HD, you preferably use a BIG BUFFER (say 16 MiB), while in other cases (HD0->HD1, HD0->RAMDISK, HD0->USBstick0, USBstick2->USBstick2, ...) a small buffer (say 64 KiB) would be sufficient and maybe even faster.

Also storage media durability should be reported (true for HD, sticks, floppy, false for RAMDISK).

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

bretjohn

Rio Rancho, NM,
28.08.2010, 16:35

@ DOS386

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> When copying to the same HD, you preferably use a BIG BUFFER
> (say 16 MiB), while in other cases (HD0->HD1, HD0->RAMDISK,
> HD0->USBstick0, USBstick2->USBstick2, ...) a small buffer (say 64 KiB)
> would be sufficient and maybe even faster.

Nice idea, but unfortunately won't work too well without adding another INT 13h extension. The maximum number of sectors (blocks) you can transfer to/from a disk in a single INT 13h transaction is 255, or 127.5 kiB (assuming the standard 512 bytes/sector). Also, the memory address provided for the transfer is always expected to be in segment:offset format, so that automatically limits it to 64 kiB.

You could load a "large buffer" with multiple INT 13h requests and then empty the buffer the same way, which may save a little bit of head thrashing on a real hard drive (with actual platters & heads & tracks) if the sectors were in fact contiguous.

> Also storage media durability should be reported (true for HD, sticks,
> floppy, false for RAMDISK).

I would probably use a term other than "media durability" for that concept. At least to me, HD's & CD's are more durable than flash disks (which by design have a limited lifespan and should be treated as unreliable & disposable like floppies are). To me, the concept you're trying to impart is more like "data durability" (or "permanence" or "stickiness" or something like that), not "media durability".

DOS386

06.09.2010, 20:10

@ bretjohn

Volumes vs physical storage media | WAS "Protectmod handl"

Post reply

> single INT 13h transaction is 255, or 127.5 kiB

Maybe only 128 sectors (and many BIOS'es are bugged and accept even 127 sectors only ???) and DMA accepts only 64 KiB too (and buffer must be 64 KiB aligned)

> memory address provided for the transfer is always
> expected to be in segment:offset format, so that
> automatically limits it to 64 kiB.

Dumb DOS + BIOS :-(

> You could load a "large buffer" with multiple INT
> 13h requests and then empty the buffer the same way,
> which may save a little bit of head thrashing on a
> real hard drive (with actual platters & heads & tracks)

That's what I meant :-)

> more like "data durability" (or "permanence" or
> "stickiness" or something like that), not "media durability".

Heh, you are right :-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***