Back to home page

DOS ain't dead

Forum index page

Log in | Register

Back to index page

Japheth

Germany (South),
25.02.2025, 16:04

CauseWay DOS extender (Announce)

Hello,

an update to the ancient CauseWay DOS extender (originally written mostly by Michael Devore): https://github.com/Baron-von-Riedesel/CauseWay/releases

As you can see, quite a few bugs have been fixed. Also, DOS conventional memory footprint is significantly lower, especially in "DPMI mode" ( that is, when the app must run in a "DOS Box" ).

Why did I do that? Well, initially it was because of the forgotten CauseWay linker WL32 - it's such a nice 16-bit protected-mode app, that surely once needed a lot of work to make it run. But the extender itself also has a few smart features: support for a swapfile ( which hdpmi is missing ), "bimodal" support for both 16-bit and 32-bit, a nice debugger (CWD).

---
MS-DOS forever!

rr

Berlin, Germany,
25.02.2025, 20:22

@ Japheth

CauseWay DOS extender

> Why did I do that? Well, initially it was because of the forgotten CauseWay
> linker WL32 - it's such a nice 16-bit protected-mode app, that surely once
> needed a lot of work to make it run. But the extender itself also has a few
> smart features: support for a swapfile ( which hdpmi is missing ),
> "bimodal" support for both 16-bit and 32-bit, a nice debugger (CWD).

I have other priorities than DOS these days, but it's nice to see CauseWay ain't dead. :-) Thanks for your work!

---
Forum admin

tkchia

26.02.2025, 16:03

@ Japheth

CauseWay DOS extender

Hello Japheth,

Thanks for the work. I have been maintaining my own set of changes to the CauseWay extender v3.60, at https://codeberg.org/tkchia/causeway — as part of the gcc-ia16 toolchain's support for 16-bit protected mode programs. Perhaps we can compare notes. :-)

Thank you!

---
https://gitlab.com/tkchia · https://codeberg.org/tkchia · 😴 "MOV AX,0D500H+CMOS_REG_D+NMI"

Japheth

Germany (South),
27.02.2025, 15:30

@ tkchia

CauseWay DOS extender

> ... I have been maintaining my own set of changes to the
> CauseWay extender v3.60, at https://codeberg.org/tkchia/causeway — as
> part of the gcc-ia16 toolchain's support for 16-bit protected mode
> programs

Interesting. What object format is used there? COFF with 16-bit "extensions" - or OMF. And the binary format is still CauseWay's 3P?

As for CauseWay v5.0: two things remain to be done:

1. fix CauseWay's stack management. Currently host stack and real-mode stack are both located in conv. memory and "mixed" - a very bad design, IMO.

2. Not exactly a CauseWay issue, but rather an Open Watcom one: dynamic memory management of the "DOS32" branch is exceptionally slow. My "real world" test case is FTE editor: to load a 130 MB file with the Win32 variant takes about 1 sec., while the DOS32 variant needs almost 2 minutes!

---
MS-DOS forever!

Rugxulo

Usono,
28.02.2025, 05:26

@ Japheth

CauseWay DOS extender

> 2. Not exactly a CauseWay issue, but rather an Open Watcom one: dynamic
> memory management of the "DOS32" branch is exceptionally slow. My "real
> world" test case is FTE editor: to load a 130 MB file with the Win32
> variant takes about 1 sec., while the DOS32 variant needs almost 2 minutes!

(Not to pretend to be a systems developer, I'm not.)

In fairness, Windows is a multi-billion dollar company. So it's not quite a fair comparison. (The system requirements are much higher.)

Maybe it's related to 4 kb pages? Remember that CWSDPMI r7 added support for (Pentium) 4 MB pages (but can be disabled). IIRC, he said that wouldn't work under EMM386 or be swapped out.

Japheth

Germany (South),
28.02.2025, 07:14

@ Rugxulo

CauseWay DOS extender

> In fairness, Windows is a multi-billion dollar company. So it's not quite a
> fair comparison. (The system requirements are much higher.)

Hm, but the "1 second load time" is also achieved if the FTE Win32 version runs in DOS under HX - and hence without any MS code involved. So that's a strong indication that the problem is indeed the OW DOS32 runtime, isn't it?

> Maybe it's related to 4 kb pages? Remember that CWSDPMI r7 added support
> for (Pentium) 4 MB pages (but can be disabled). IIRC, he said that wouldn't
> work under EMM386 or be swapped out.

What has CWSDPMI to do with this issue? It isn't used by either OW or Win32.

---
MS-DOS forever!

Rugxulo

Usono,
28.02.2025, 10:30

@ Japheth

CauseWay DOS extender

> > In fairness, Windows is a multi-billion dollar company. So it's not quite
> a
> > fair comparison. (The system requirements are much higher.)
>
> Hm, but the "1 second load time" is also achieved if the FTE Win32 version
> runs in DOS under HX - and hence without any MS code involved. So that's a
> strong indication that the problem is indeed the OW DOS32 runtime, isn't
> it?

Does HDPMI32 support 4 MB pages?

> > Maybe it's related to 4 kb pages? Remember that CWSDPMI r7 added support
> > for (Pentium) 4 MB pages (but can be disabled). IIRC, he said that
> wouldn't
> > work under EMM386 or be swapped out.
>
> What has CWSDPMI to do with this issue? It isn't used by either OW or
> Win32.

The speed with 4 MB pages was supposed to be twice as fast on large RAM machines. 4 kb is an awfully-small amount when you have hundreds (or thousands) of MB of physical RAM. I don't think older CWSDPMI r5 would even run on such a huge machine without swapping out the page table from conventional memory.

I don't know why it's much slower here. (Obviously OpenWatcom doesn't use CWSDPMI, which is a "pure" 32-bit DPMI server without extensions.)

tkchia

28.02.2025, 16:36

@ Rugxulo

CauseWay DOS extender

Hello Rugxulo,

> Maybe it's related to 4 kb pages? Remember that CWSDPMI r7 added support
> for (Pentium) 4 MB pages (but can be disabled). IIRC, he said that wouldn't
> work under EMM386 or be swapped out.

No real need for wild guesses in my opinion. We have the source code for CauseWay. We can, like, take some time to read it...

Thank you!

---
https://gitlab.com/tkchia · https://codeberg.org/tkchia · 😴 "MOV AX,0D500H+CMOS_REG_D+NMI"

tkchia

28.02.2025, 16:51
(edited by tkchia, 28.02.2025, 18:01)

@ Japheth

CauseWay DOS extender

Hello Japheth,

> > ... I have been maintaining my own set of changes to the
> > CauseWay extender v3.60, at https://codeberg.org/tkchia/causeway — as
> > part of the gcc-ia16 toolchain's support for 16-bit protected mode
> > programs
> Interesting. What object format is used there? COFF with 16-bit
> "extensions" - or OMF. And the binary format is still CauseWay's 3P?

Yes, the executable format is still 3P. The intermediate format is ELF/x86-32 with some non-standard relocations to represent segment:offset pairs (earlier proposed by Anvin, https://git.zytor.com/users/hpa/segelf/abi/tree/segelf.txt). Segment relocations are not quite supported yet. The whole setup is rather clunky and probably needs a rethink.

Also I currently use Lindauer's xlib to rebuild the cw.lib library of runtime routines, in case that is useful.

Thank you!

---
https://gitlab.com/tkchia · https://codeberg.org/tkchia · 😴 "MOV AX,0D500H+CMOS_REG_D+NMI"

DosWorld

28.02.2025, 23:11
(edited by DosWorld, 28.02.2025, 23:32)

@ Japheth

CauseWay DOS extender

> Open Watcom one: dynamic
> memory management of the "DOS32" branch is exceptionally slow.

I am also saw this case.
It was a time when i have experiments with text editor (before dwed) and watcom c.
From my point of view, it is very bad implementation for malloc.

What i test:
I am measure time to load text file (huge work for malloc).
1. With watcom's malloc
2. With my own implementation for malloc/realloc/free. (at first, i am allocate one huge piece of memory with watcom malloc and then - use this memory for my own implementation of malloc/reallic/free).

My (very basic) implementation works fasten.
Then, i have idea about improve watcom library, ask question and see - my skills with dpmi is too low.

> I had receive big speed degradation for watcom's malloc,
> after 500k allocated strings. I think, it will be unusable after
> 1m malloc's. It is looks like watcom manage memory via linked list
> structure. So, I want refuse using watcom malloc and switch to
> my own memory manager (may be he not so fast, but he remember where
> is stored first free memory block).

win/linux has amazing memory managers - works like a cache/pool.

---
Make DOS great again!
Make Russia small again!

Japheth

Germany (South),
28.03.2025, 08:12

@ Japheth

CauseWay DOS extender v5

> As you can see, quite a few bugs have been fixed. Also, DOS conventional
> memory footprint is significantly lower, especially in "DPMI mode" ( that
> is, when the app must run in a "DOS Box" ).

So, CauseWay 5.0 is out. In short:

- better DOS/4G compatibility
- better DPMI compliance
- cleaned source code (tools: Masm v6+, JWasm, WL32)
- smaller file size, less extended memory usage
- lower DOS memory footprint, no scattered free DOS memory
- faster mode switches
- (hopefully :-D ) less bugs

---
MS-DOS forever!

RayeR

CZ,
30.03.2025, 04:03

@ Japheth

CauseWay DOS extender v5

Is it possible/what tool to re-stub existing binaries that use old CauseWay with this new one? Could be there some potential problem (e.g. compatability on very old machines)?

---
DOS gives me freedom to unlimited HW access.

Japheth

Germany (South),
30.03.2025, 05:33

@ RayeR

CauseWay DOS extender v5

> Is it possible/what tool to re-stub existing binaries that use old CauseWay
> with this new one?

AFAICS the CauseWay tools are somewhat focused on the CauseWay 3P format - they allow to translate LE/LX to 3P, replacing the stub - I suppose that's not exactly what you intend. However, the DOS/4G extender has a 4GBIND.EXE tool, that tool may allow to "restub" a LE/LX binary.

> Could be there some potential problem (e.g. compatability on very old machines)?

No idea.

---
MS-DOS forever!

tkchia

30.03.2025, 16:59

@ Japheth

CauseWay DOS extender v5

Hello Japheth,

> AFAICS the CauseWay tools are somewhat focused on the CauseWay 3P format -
> they allow to translate LE/LX to 3P, replacing the stub - I suppose that's
> not exactly what you intend. However, the DOS/4G extender has a 4GBIND.EXE
> tool, that tool may allow to "restub" a LE/LX binary.

Where (or how) may I find this 4gbind.exe tool? I tried using a search engine to look for it, but no luck.

Currently my lfanew utility can un-stubify and re-stubify PE programs, but it expressly does not support LE/LX programs yet (this is a non-trivial task).

Thank you!

---
https://gitlab.com/tkchia · https://codeberg.org/tkchia · 😴 "MOV AX,0D500H+CMOS_REG_D+NMI"

Japheth

Germany (South),
30.03.2025, 17:27

@ tkchia

CauseWay DOS extender v5

Hello tkchia,

> Where (or how) may I find this 4gbind.exe tool? I tried using
> a search engine to look for it, but no luck.

It's part of the (non-free) DOS/4G extender.
However, turned out it's not necessary. PMWBIND.EXE, part of the pmode/w extender and supplied with OW, does exactly what's needed.

---
MS-DOS forever!

tkchia

30.03.2025, 19:45

@ Japheth

CauseWay DOS extender v5

Hello Japheth,

> However, turned out it's not necessary. PMWBIND.EXE, part of the pmode/w
> extender and supplied with OW, does exactly what's needed.

Let me check that out. Thank you!

---
https://gitlab.com/tkchia · https://codeberg.org/tkchia · 😴 "MOV AX,0D500H+CMOS_REG_D+NMI"

RayeR

CZ,
01.04.2025, 16:10

@ Japheth

CauseWay DOS extender v5

How are related the 3P and LE formats? I think if Causeway use its own 3P format I need a tool for unstubify to get 3P from EXE and then I'll stub it with new Causeway. I know only this process for DJGPP programs where is tool to unstubify to COFF format and then can be restubed with CWSDPMI or PMODE/W or something else...

---
DOS gives me freedom to unlimited HW access.

Japheth

Germany (South),
02.04.2025, 14:03

@ RayeR

CauseWay DOS extender v5

> How are related the 3P and LE formats? I think if Causeway use its own 3P
> format I need a tool for unstubify to get 3P from EXE and then I'll stub it
> with new Causeway. I know only this process for DJGPP programs where is
> tool to unstubify to COFF format and then can be restubed with CWSDPMI or
> PMODE/W or something else...

CauseWay supports both 3P and LE (both formats support dlls). And there's a tool that does - among other things - convert LE to 3P, in the original CauseWay it's called CW.EXE ( I intend to change the name to something more meaningful ).

---
MS-DOS forever!

DosWorld

02.04.2025, 04:24

@ tkchia

CauseWay DOS extender v5

> LE/LX
> (this is a
> non-trivial task).

What you mean? (May be I miss something?)

---
Make DOS great again!
Make Russia small again!

tkchia

05.04.2025, 07:41

@ DosWorld

CauseWay DOS extender v5

Hello DosWorld,

> > Currently my lfanew utility can un-stubify and re-stubify
> > PE programs, but it expressly does not support LE/LX programs yet
> > (this is a non-trivial task).
> What you mean? (May be I miss something?)

The LE/LX portion contains a number of file offsets which are relative to the start of the whole file. So if I change an MZ stub to an MZ stub of different size, then I need to adjust these offsets. (I think pmwbind handles one of these offsets, but not all.)

If I simply change the MZ stub and leave the LE/LX part unchanged, the resulting program will not work.

Thank you!

---
https://gitlab.com/tkchia · https://codeberg.org/tkchia · 😴 "MOV AX,0D500H+CMOS_REG_D+NMI"

roytam

31.03.2025, 18:45

@ Japheth

CauseWay DOS extender v5

> > Is it possible/what tool to re-stub existing binaries that use old
> CauseWay
> > with this new one?
>
> AFAICS the CauseWay tools are somewhat focused on the CauseWay 3P format -
> they allow to translate LE/LX to 3P, replacing the stub - I suppose that's
> not exactly what you intend. However, the DOS/4G extender has a 4GBIND.EXE
> tool, that tool may allow to "restub" a LE/LX binary.
>
> > Could be there some potential problem (e.g. compatability on very old
> machines)?
>
> No idea.

I wonder if there is a tool to translate DOS/16M / DOS/4G "BW" format to something else?

Japheth

Germany (South),
01.04.2025, 08:56

@ roytam

CauseWay DOS extender v5

> I wonder if there is a tool to translate DOS/16M / DOS/4G "BW" format to
> something else?

And what infamous BW-apps are candidates for such translations? It's a 16-bit format, so the destination formats probably are NE or CauseWay's 3P. But what benefit does this translation offer?

---
MS-DOS forever!

roytam

01.04.2025, 14:32

@ Japheth

CauseWay DOS extender v5

> > I wonder if there is a tool to translate DOS/16M / DOS/4G "BW" format to
> > something else?
>
> And what infamous BW-apps are candidates for such translations?

maybe NDD? any those CA-Clipper programs?

> It's a 16-bit format, so the destination formats probably are NE or CauseWay's 3P.
> But what benefit does this translation offer?

to use a free/open source DOS extender so no need to use "quirky" DOS/16M or DOS/4G?

Back to index page

22395 Postings in 2076 Threads, 400 registered users, 97 users online (1 registered, 96 guests)
DOS ain't dead | Admin contact

powered by my little forum