Back to home page

DOS ain't dead

Forum index page

Log in | Register

Search:  
Back to index page
Thread view  Board view
jadoxa

Homepage E-mail

Queensland, Australia,
13.03.2022, 13:58
 

RDRVSX32: FAT32 RAM drive for HimemSX (Announce)

Post reply

I had a request (via email) to create a RAM drive greater than 4GiB. It sort of works (works on his and mine 8GiB AMD, but has issues with his 32GiB Intel) so if anyone has the time and inclination to test, that'd be great. You will need more than 4GiB RAM (in order to access super-extended memory), preferably more than 8GiB (in order to test accessing beyond 4GiB). Our testing has been simple so far: just copy large files (he was able to create a 28GiB RAM drive, copy hundreds of 64MiB files, but it fails copying a 1GiB file).
Japheth

Homepage

Germany (South),
13.03.2022, 16:50

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

Cool!

> so if anyone has the time and inclination to test, that'd be great.
I tested and got a few errors.

My test does not need a large drive, 2 GB is ok. So I can compare the results with rdisk.com.

For testing purposes, it would be great to have a rdrv32.exe version, that's a FAT32 driver restricted to standard XMS block move calls. So one can be sure that the new XMS API is NOT the source of problems.

I tried to create that version myself, but apparently the rdrvsx32.nsm source requires a "special" version of nasm - the versions that I used were not happy with it.

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
14.03.2022, 02:24

@ Japheth
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> For testing purposes, it would be great to have a rdrv32.exe version,
> that's a FAT32 driver restricted to standard XMS block move calls.
Updated the zip, adding rdrvx32.exe, which uses normal extended memory (super-extended not required).

> I tried to create that version myself, but apparently the rdrvsx32.nsm
> source requires a "special" version of nasm - the versions that I used were
> not happy with it.
I originally created it with 0.98.39, but there were bugs with v2; I eventually reported those, which I think got into a 2.14 RC release, so anything earlier than that isn't going to work; it works with the current 2.15.05 (at least the Win64 version does).
Richard

17.03.2022, 06:05

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

For my testing and comparison purposes I would like to have a FAT16 ~o.7 GB RAMdrive in XMS (to supplement FAT 16 in SXMS and FAT32 in both XMS and SXMS).

I use c:\freedos\bin\xmsdsk.exe 700000 I:

but the problem is I have to keyboard answer YES to the message

XMS driver not loaded. Want to install it (y\n)?


I have devload c:\freedos\bin\himemSX.exe in the early part of FDauto.bat.


Do I need say a "special SHSURDRV_XMS.exe" or something to compliment the SHSURDRV.EXE used to create FAT16 drives in SXMS?
jadoxa

Homepage E-mail

Queensland, Australia,
17.03.2022, 08:15

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> I use c:\freedos\bin\xmsdsk.exe 700000 I:
>
> but the problem is I have to keyboard answer YES to the message
>
> XMS driver not loaded. Want to install it (y\n)?
It's actually "XMSDSK driver", which you can autoanswer by adding /y at the end of the command.
Richard

17.03.2022, 10:27

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

>
> It's actually "XMSDSK driver", which you can autoanswer by adding
> /y at the end of the command.
Thanks - that works.

Now I have

FAT16 RAMdrive I:\ in XMS o.7 GB
FAT16 RAMdrive J:\ in SXMS 4 GB
FAT32 RAMdrive G:\ in XMS o.7 GB
FAT32 RAMdrive H:\ in SXMS 24 GB

to experiment regarding timings of large files.

So it seems all of SXMS is allocated in my 32 GB system.

Just for fun I did c:\freedos\bin\SHSURDRV 1000M,K: and it appeared to give me ~ 1 GB drive. Also did c:\freedos\bin\SHSURDRV 100M,L: and it appeared to give me ~ 100 MB drive. But I thought ALL SXMS has been allocated to J:\ + H:\ drives above?

When previously, I determined by trial and error, that a maximum of 1400M is available for RAMdrives (in XMS), a value of 1500M created all sorts of problems - I cannot understand at the moment how it is even possible to have the later drives K:\ (1 GB) and L:\(100 MB).


Question - Is there a DOS utility that gives me a memory map of the SXMS - i.e. so I can tell what the addresses in RAM for all the RAMdrives are?
jadoxa

Homepage E-mail

Queensland, Australia,
17.03.2022, 13:00

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Question - Is there a DOS utility that gives me a memory map of the SXMS -
> i.e. so I can tell what the addresses in RAM for all the RAMdrives are?
Not sure if XMSCC will work; it does for XMS using HimemSX, but I don't know if it will pick up SXMS. But since the source is available...
Zyzzle

17.03.2022, 23:06

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > Question - Is there a DOS utility that gives me a memory map of the SXMS
> -
> > i.e. so I can tell what the addresses in RAM for all the RAMdrives are?
>
> Not sure if XMSCC will work;
> it does for XMS using HimemSX, but I don't know if it will pick up SXMS.
> But since the source is available...
Yes, it works for me to pick up all SXMS, and sees all the > 4GiB XMS in my systems. Very good software.
Richard

18.03.2022, 02:35

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

Did some further experimentation with settings, now it appears for all 4 possibilities (FAT16/FAT32/XMS/SXMS) the maximum possible on my 32 GB system is


[image]
https://www.dropbox.com/s/69sbecj1xs6bac0/2022-mar-18.PNG?dl=1


Used Trace_2.01.com. In some cases cluster size of 0 bytes is reported - surely it is not possible for a bug to exist in TRACE? :) I would have thought cluster size to be 32K or 64K or something.


Is it possible to add a feature to TRACE to have displayed:-

- Total bytes (i.e. free bytes + used bytes)?

- What in RAM memory are the start and end addresses?


That way, i.e. a map of the RAMdrive for the drive being traced, I can confirm whether or not a drive is in XMS or SXMS. Because of different cluster sizes for various FAT16/32 possibilities - the addresses in HEX bytes would be probably less confusing.



> > Question - Is there a DOS utility that gives me a memory map of the SXMS
> -
> > i.e. so I can tell what the addresses in RAM for all the RAMdrives are?
>
> Not sure if XMSCC will work;
> it does for XMS using HimemSX, but I don't know if it will pick up SXMS.
> But since the source is available...

I had a look at GitHub regarding XMSCC - I am not sure that it will be suitable for me (it mentioned that it does not work with virtual drives - which I gather what a RAMdrive would be). In any case...

Question - how to create a .exe (or .com, .sys, etc) from the downloaded .zip?

Usually, in the past, everything was already built ready at the download stage and packaged in the .zip.
jadoxa

Homepage E-mail

Queensland, Australia,
18.03.2022, 06:58

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Used Trace_2.01.com. In some cases cluster size of 0 bytes is reported -
> surely it is not possible for a bug to exist in TRACE? :) I would have
> thought cluster size to be 32K or 64K or something.
Yeah, 64K, which is 65536, which is 0x10000, which is 0 in 16 bits. I'll fix it later...

> Is it possible to add a feature to TRACE to have displayed:-
>
> - Total bytes (i.e. free bytes + used bytes)?
That would make sense.

> - What in RAM memory are the start and end addresses?
That would not; it would need to be done in RDRV. Probably won't bother, though.

> I had a look at GitHub regarding XMSCC - I am not sure that it will be
> suitable for me (it mentioned that it does not work with virtual drives -
> which I gather what a RAMdrive would be). In any case...
Where do you see that?

> Question - how to create a .exe (or .com, .sys, etc) from the downloaded
> .zip?
Go to Releases, get the .exe (or the .zip, which contains the exe and readme).
Richard

19.03.2022, 03:18

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

>
> > I had a look at GitHub regarding XMSCC ...

>
> Go to Releases, get the .exe (or the .zip, which contains the exe and
> readme).
Thanks - have it now.

I had always been downloading from GitHub using the "BIG GREEN Code Button" always clearly visible at the top section even when viewing the web page is heavily cropped by me to have many windows simultaneously visible. I suppose the not so obvious/inviting "Releases" has been always there.


So to summarize:

In sequence I created

~1.4 GB FAT32 RAMdrive in XMS G:\
~ 24 GB FAT32 RAMdrive in SXMS H:\
~ 4 GB FAT16 RAMdrive in SXMS J:\
~2.5 GB FAT16 RAMdrive in XMS I:\ (or so I thought)


With the goal of trying to establish RAMdrive addresses in the 32 GB system I used XMSCC


[image]

Blocks 1, 2 and 3 match with G:\, H:\, and J:\ BUT I:\ is not shown.

Can I use the 32-bit base address (pity it was not shown in HEX) as the boundaries of the RAMdrives (at least where the actual files go)?



Also I tried XMStat, getting

[image]

and this does show my I:\ as the last entry.



Lastly, I tied XMSStat, getting

[image]

and this does show my I:\ as the last entry. However in the region section (of I:\) it goes from a larger value to a smaller value (opposite effect compared to all other entries).


So are there bugs in XMSCC and XMSSTAT regarding my I:\ RAMdrive ?


So, I now do not know if my I:\ RAMdrive (~2.5 GB) is in the XMS or not - hence why I wanted a RAM map showing where (and ranges) things such as RAMdrives are.
Japheth

Homepage

Germany (South),
19.03.2022, 05:24

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> So are there bugs in XMSCC and XMSSTAT regarding my I:\ RAMdrive ?
After creating himemsx I updated xmsstat to make it show the full 40-bit addresses of EMBs. To get it, just download the newest jemm package.

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
19.03.2022, 09:26

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> [XMSCC] Blocks 1, 2 and 3 match with G:\, H:\, and J:\ BUT I:\ is not shown.
Press Down or PgDn...
jadoxa

Homepage E-mail

Queensland, Australia,
01.04.2022, 09:33

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Used Trace_2.01.com. In some cases cluster size of 0 bytes is reported
Fixed.

> Is it possible to add a feature to TRACE to have displayed:-
>
> - Total bytes (i.e. free bytes + used bytes)?
Added (total bytes of the drive, i.e. including reserved & FAT sectors).

Trace 2.02.
Richard

24.04.2022, 07:38

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > Used Trace_2.01.com. In some cases cluster size of 0 bytes is reported
>
> Fixed.
>
> > Is it possible to add a feature to TRACE to have displayed:-
> >
> > - Total bytes (i.e. free bytes + used bytes)?
>
> Added (total bytes of the drive, i.e. including reserved & FAT sectors).
>
> Trace 2.02.



I am confused!


Using trace.com 5,679 bytes (2022-APR-02) and whichfat.com 508 bytes (2003-NOV-24) - do not agree!



[image]


Trying to create a FAT12 RAM drive.
jadoxa

Homepage E-mail

Queensland, Australia,
24.04.2022, 09:43

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Using trace.com 5,679 bytes (2022-APR-02) and whichfat.com 508 bytes
> (2003-NOV-24) - do not agree!
SHSURDRV thinks it's creating a FAT12 drive, but the DPB begs to differ.

> Trying to create a FAT12 RAM drive.
Until I fix it just make it a little smaller, so the last cluster is FF5.
Richard

24.04.2022, 13:35

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > Using trace.com 5,679 bytes (2022-APR-02) and whichfat.com 508 bytes
> > (2003-NOV-24) - do not agree!
>
> SHSURDRV thinks it's creating a FAT12 drive, but the DPB begs to differ.
>
> > Trying to create a FAT12 RAM drive.
>
> Until I fix it just make it a little smaller, so the last cluster is FF5.


Reduced size slightly of FAT12 drive.

Below are the (doslfnMS vs doslfn) timings for FAT12 USB, XMS and SMS drives for both ROOT and 1st level directories.

Note tried to make all FOLDER sets (files + directories) equal, however for the USB FAT12 drive, strange results occurred (as if 2 files are hidden from DIR - but if I recopy them to make up the "10 set of files", I get the message "do you want to overwrite...).


Hopefully all the drives below are actually FAT12.


A few minor "cosmetic" issues with the screen captures - but still usable.



[image]

[image]

[image]

[image]

[image]

[image]





[image]






Who can I suggest some minor changes to XMSSTAT.exe to?
tom

Homepage

Germany (West),
16.05.2022, 20:40

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

well. Any conclusions about your efforts?
Japheth

Homepage

Germany (South),
18.05.2022, 10:26

@ tom
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> well. Any conclusions about your efforts?
They were worth it - among other things, they brought to my attention this "screen thief" utility. It works, even if I set very uncommon text modes.

For example:
- setting VESA graphics mode 0x102, 800x600 pixels
- changing graphics to text, 37x100 chars
- changing font to 8x14, gives 43x100 chars
- changing vertical display to 602 pixel (makes the last row to be seen fully )
- changing horizontal display to 80 chars

and I get screenshots in a very nice 80x43 text mode. :-D

---
MS-DOS forever!
Richard

21.05.2022, 13:23

@ tom
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> well. Any conclusions about your efforts?


Preliminary discussion and conclusions.


The original aim of the exercise was to test out RDRVSX32 in the creation of FAT32 RAM drives - a required aspect was the creation of RAM drives larger than 4 GByte (which automatically meant to be created in SuperXtendedMemorySpace (Abbreviated to SMS in my notes).

Previously, before March, DOS users only had access to RAM drives no larger than 4 GByte due to FAT 16 limitation.


Initially, in the development process there were problems that the FAT32 RAM drive could be created on AMD machines but not my INTEL machine - which resulted in this thread being on BTTR in the first place. Eventually I overcame this INTEL/AMD difference by simply reconfiguring FDconfig.sys (I used FreeDOS RC5 at the time, now known as v1.3). Apparently non-FreeDOS systems, both INTEL and AMD, were never affected.

At present I can only do testing in FreeDOS on a bare metal setup - I have not been successful to obtain MS DOS 7.1 to install beside FreeDOS. At this stage I did not do any Virtual Machine setup testing.


From a limited number of tests, I successfully achieved RAM drives of about 28 GByte and worked with file sizes up to 4 GByte(minus 1 byte) on all trials. My hardware is limited to 32 GByte RAM and since the first 4 GByte is reserved for (XMS + 1 MByte conventional memory) - at this stage I cannot confirm/deny on whether it is possible to have RAM drives > 32 GByte.

The speed of the FAT32 RAM drive is impressive - without using timer routines to measure - to copy from say a 1 GByte file in XMS and transform, by way of binary DOS file addition, to a 4 GByte file (in SMS) took only about 1 second. On the contrary, to copy a 4 GByte file to/from RAM to/from my USB stick takes over 2 hours (using FreeDOS).

So from a very limited set of testing conditions it appeared that RDRVSX32 is highly efficient and accurate in the creation of FAT32 RAM drives (in SMS).



Early on in this thread, Japheth indicated problems when working with LFN (my tests were only with 8.3 format) - and a number of RDRVSX32 iterations were written by the author (Jadoxa).

At this point I went off in a slight "tangent" - so instead of aiming for further RDRVSX32 testing in a variety of situations, I was involved in independent testing of LFN in FAT32 RAM drive, which led me into LFN testing of FAT32 RAM drive in XMS, FAT16 (XMS/SMS) and FAT12 (XMS/SMS) and USB stick equivalents of FAT 12/16/32. No tests were done on a Hard Drive or SATA SSD and similarly no tests for my 3.5 inch USB floppy drive (which seems to be broken).

A further "tangent" occurred with using a more reliable timing method (Jadoxa's RT program) than examining time differences via DOS prompt time stamps. Also another tangent resulted from Jadoxa supplying two versons for LFN - namely doslfn and doslfnMS.

So, to try to maintain "fairness" in various testing "environments", I attempted to automate measurements via ".bat files" for all possibilities for an extremely small sample set (10 files + 4 directories). The sole intention was to determine if the two doslfn{MS} were significantly different is speed. The timing resolution I adopted was milli-seconds ( RT program can offer micro-seconds resolution). It was inconclusive with only 10 files+ - so lastly bumped up to 100+ files. Apart from the problem (mentioned below):-

Conclusion To a resolution of 0.02 seconds there appears to be no timing differences between doslfn and doslfnMS for a sample size of 100+ files - however a more realistic (for me) size
set should be a minimum of 500 files + 500 1st level sub-directories AND NOT YET TESTED at a very deep level (say 10 levels down) also. This last addition may be interesting since my experience with Windows 10 is that serious problems (in Windows) occurs when working with >6 levels down, e.g. in searches.

Problems I have observed...

SHSURdrv cannot support 100+ files in the ROOT directory of FAT12 and FAT16 - so cannot test doslfn{MS} for larger file sets.

It appears that doslfn{MS} cannot support LONG FOLDER NAMES (whereas Windows can).

When using the TIME STAMP as part of the DOS Prompt - strange timing errors occur (as if time going backwards on occassions) - the best explanation I can give for this is that my machine is INTEL (i7) with 4 cores and each core acts as 2 logical processors. Even though, as I understand, DOS in bare metal mode only is working on one core at any one time, it may not be working on only one logical processor - so, any (exe/com/sys/bat) may be simultaneously processed by two logical processors (but by the one physical core). Two important factors are involved - pipelining (which came with the advent of 486 processors) and the way instruction codes are presented (c/f 8-byte boundaries for 64 bit machines). So, with "look ahead" techniques, a code stream split over two logical processors may get "unsynchronized" slightly - so the DOS prompt executed by logical processor #1 is finished AFTER RT.exe has finished on logical processor #2. However, under otherwise identical conditions, with an AMD quad core processor - there are only as many logical processors as physical cores (hence no pipelining I think) and there is only one process execution code stream running in DOS.

Hopefully, issues with SHSURDRV ROOT tables limit and Long FOLDER Names (once verified independently to see if the problem still exists) will be fixed in the near future. However, if I am the only one to be using FAT12 and Long FOLDER Names, I can imagine it will be a very long time, if ever, before being fixed.

Another "tangent" occurred with the using of SMARTDRV (to help speed things up).

Conclusion using SMARTDRV has no observed significant speed gains when doing DIR on various media/formats.

By using SMARTDRV many things were "broken" with RAM drives - resolved by re-arranging setup sequence of all combinations of RAM drive for FAT12/16/32 + XMS/SMS.



Summary

From limited testing on a 32 Gbyte INTEL computer, it is possible to achieve a 28 GByte FAT32 RAM drive. Speed is very good - > ~1 GByte per second for file transfers (RAM to RAM). Cannot confirm/deny if >32 GByte RAM drive is possible.

doslfn and doslfnMS appear to be the same speed for up to 100+ files (restricting to 0.02 second timing resolution).

No apparent advantage in using SMARTdrv for DIR on any media/format - and consideration when simultaneously having RAM drives as FAT 12/16/32 + XMS/SMS (i.e. 6 different kinds of RAM drives).

Apparent limitations with SHSURDRV for FAT 12/16 ROOT directories (only) (literature mentions can be as low as 64 entries - USB stick (via Windows allows 100+))

doslfn{MS} does not appear to work with long FOLDER Names (need independent confirmation)

My testing limited to FreeDOS 1.3 platform - not able to test via other platform (e.g. MS DOS 7.1)

RDRVSX32 does work with both INTEL and AMD - need consideration when using FreeDOS FDconfig.sys/FDauto.bat

There appears to be timing "racing" conflicts with Time Stamps as part of DOS Prompt comparing running programs (such as RT.exe) - attributed to on only INTEL > 486 processors with more than one logical processor for each physical core (so does not exist maybe for AMD).

"Screen Thief" and "RT.exe" and "Trace.com" utilities very useful for recording of events.

Further, more rigorous testing needed, but RDRVSX32 may be good enough "as-is" for over 99.99% of applications (my guess).
tom

Homepage

Germany (West),
21.05.2022, 15:14

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> SHSURdrv cannot support 100+ files in the ROOT directory of FAT12 and FAT16
> - so cannot test doslfn{MS} for larger file sets.
FAT12 and FAT16 root entries are fixed at the time the drive is formatted or - for RAM drives - initialized.

> When using the TIME STAMP as part of the DOS Prompt - strange timing errors
> occur (as if time going backwards on occassions)
as far as I can see this happened exactly 2x. (on 09.04.2022, 4:01.26.40)



> - the best explanation I
> can give for this is that my machine is INTEL (i7) with 4 cores and each
> core acts as 2 logical processors. Even though, as I understand, DOS in
> bare metal mode only is working on one core at any one time
to my understanding BIOS only initializes the 1'st thread on the 1'st core.

> it may not be
> working on only one logical processor
 this would be VERY unusual.

> So, with "look ahead" techniques, a
> code stream split over two logical processors may get "unsynchronized"
> slightly - so the DOS prompt executed by logical processor #1 is finished
 this is nonsense, even on a multitasking system. first the time for a prompt is taken, then some command is executed, then the time for the prompt is taken again.

as the time is directly the BIOS timer tick (converted to seconds and centiseconds), I would rather suggest the kernel code to convert ticks into seconds is somehow broken (although I couldn't find a possible bug).

other than this, the BIOS timer tick count counting backward is extremely unlikely, no matter how many threads are running.
tom

Homepage

Germany (West),
21.05.2022, 17:41

@ tom
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> as the time is directly the BIOS timer tick (converted to seconds and
> centiseconds), I would rather suggest the kernel code to convert ticks into
> seconds is somehow broken (although I couldn't find a possible bug).
converting the suspicious kernel code from kernel\sysclk.c into a test routine as below 'proves' that the kernel code is correct; i.e. an increase in the BIOS tick count results in a greater converted DOS time.

I have no idea how this could happen.

#else                             // #ifdef TEST   

#undef SEEK_SET
#undef SEEK_CUR
#undef SEEK_END


#include "stdio.h"

/* compile with

        set PATH=%WATCOM%\binnt
        set INCLUDE=%WATCOM%\h
        wcl sysclk.c /DTEST /i..\hdr -q
*/     
       


main()
{     
        ticks_t ticks, bios_ticks;
    struct ClockRecord clk, oclk;
    int tmp;


        for (bios_ticks = 0; bios_ticks < 0xffff0000l; bios_ticks++)
                {   
   
        /* The scaling factor is now
           6553600/1193180 = 327680/59659 = 65536*5/59659 */

        ticks = 5 * bios_ticks;
        ticks = ((ticks / 59659u) << 16) + ((ticks % 59659u) << 16) / 59659u;

        tmp = (int)(ticks / 6000);
        clk.clkHours = tmp / 60;
        clk.clkMinutes = tmp % 60;

        tmp = (int)(ticks % 6000);
        clk.clkSeconds = tmp / 100;
        clk.clkHundredths = tmp % 100;

                if (bios_ticks < 20)
                        printf("%2d:%02d:%02d.%02d\n", clk.clkHours, clk.clkMinutes,
                                                                                clk.clkSeconds, clk.clkHundredths);
       
        if (clk.clkHours   <= oclk.clkHours &&
            clk.clkMinutes <= oclk.clkMinutes &&
            clk.clkSeconds <= oclk.clkSeconds &&
            clk.clkHundredths <= oclk.clkHundredths)
            {       
            printf("ticks %lu\n", bios_ticks);
                        printf("old %2d:%02d:%02d.%02d\n", oclk.clkHours, oclk.clkMinutes,
                                                                                oclk.clkSeconds, oclk.clkHundredths);
                        printf("now %2d:%02d:%02d.%02d\n", clk.clkHours, clk.clkMinutes,
                                                                                clk.clkSeconds, clk.clkHundredths);
            }
           
                oclk = clk; 
               
                if (clk.clkHours >= 24)
                        {
                        printf("midnight at %lu = 0x%lx\n", bios_ticks, bios_ticks);
                        break;
                        }
                }
               
        return 0;

}
#endif
jadoxa

Homepage E-mail

Queensland, Australia,
22.05.2022, 09:23

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> SHSURdrv cannot support 100+ files in the ROOT directory of FAT12 and FAT16
The default is 64, you must specify the D setting for more.

> It appears that doslfn{MS} cannot support LONG FOLDER NAMES
C:\>f:doslfn
DOSLFN 0.41f (haftmann#software & jmh 4/2022): loaded consuming 12512 bytes.
C:\>md "a long directory name"
C:\>copy nul "a long directory name\a long file name"
nul =>> a long directory name\a long file name
C:\>dir/lfn alongd~1
 Volume in drive C is FREEDOS2012
 Volume Serial Number is 124A-13F0

 Directory of C:\ALONGD~1

.                    <DIR>  22-05-2022  5:17p a long directory name
..                   <DIR>  22-05-2022  5:17p .
ALONGF~1                 0  22-05-2022  5:17p a long file name
         1 file(s)              0 bytes
         2 dir(s)     577,912,832 bytes free
Richard

23.05.2022, 13:53

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > SHSURDRV cannot support 100+ files in the ROOT directory of FAT12 and
> FAT16
>
> The default is 64, you must specify the D setting for more.
 >

Thanks for "reminding me" re "D" setting - I think it was about 2 months since I last read the readme file for SHSURDRV and so I completely forgot about the sectors etc configuration side of things. The "excitement" with working in one project with the simultaneous testing of 18 possible RAM/Physical drive/folder combinations "clouded" my programming approach to the situation.







> > It appears that doslfn{MS} cannot support LONG FOLDER NAMES
>
> C:\>f:doslfn
> DOSLFN 0.41f (haftmann#software & jmh 4/2022): loaded consuming 12512
> bytes.
> C:\>md "a long directory name"
> C:\>copy nul "a long directory name\a long file name"
> nul =>> a long directory name\a long file name
> C:\>dir/lfn alongd~1
> Volume in drive C is FREEDOS2012
> Volume Serial Number is 124A-13F0
>
> Directory of C:\ALONGD~1
>
> .                    <DIR>  22-05-2022  5:17p a long directory name
> ..                   <DIR>  22-05-2022  5:17p .
> ALONGF~1                 0  22-05-2022  5:17p a long file name
> 1 file(s)              0 bytes
> 2 dir(s)     577,912,832 bytes free
>


Have not had a chance yet to look at Long FOLDER Names (in my setup). Thanks for independently verifying that it does work. What I think might have happened to me was that with the "hundreds" of DOSLFN{MS} install-uninstall steps and coupled with SMARTDRV (which in itself caused "problems" - but workaround eventually done) I was taking - I might have copied LFN FOLDERS when the CPU was in 8.3 mode - and of course once anything is "forced" into 8.3 mode, working in LFN mode later on does not regain the "lost" name detail.

It may be about 2 weeks before I will do another comparison run, re doslfn{MS} with a very much larger file/folder set. I am going to do on another usb stick, this time on a 256 GByte one (instead of 16 GByte) - I may have issues with sizes of partitions being > 32 GByte say (soon will find out).






----------------------------------------------------------------------------

I am preparing my response to Tom's reply (which no doubt he will "critically analyze" and point errors in my ways). I do really appreciate feedback (a very powerful way to learn things, than just only spending time forever reading only). It may be a few days before I can reply to him.
Richard

26.04.2022, 05:20

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > Using trace.com 5,679 bytes (2022-APR-02) and whichfat.com 508 bytes
> > (2003-NOV-24) - do not agree!
>
> SHSURDRV thinks it's creating a FAT12 drive, but the DPB begs to differ.
>
> > Trying to create a FAT12 RAM drive.
>
> Until I fix it just make it a little smaller, so the last cluster is FF5.


I think I have a serious problem with SHSURDRV.exe ...


Reduced slightly size of FAT12 drive.

Essentially, the only changes to my testing of (doslfnMS vs doslfn) is to enlarge the tested FOLDER set - from 10 files (+4 directories) - to 100 files (+4 directories). The change involved having another folder with 90 files and just copying them to ALL FOLDER sets being tested (namely ROOT/1st_Level_sub-directory, USB/XMS/SMS memory, FAT12/FAT16/FAT32 --- ALL possible combinations).


Referring to ".bat" runs (as shown in the middle of the small overlay box at the Top Right Hand Corner of each screen capture) - c.bat and g.bat show that for SHSURDRV.exe (and only it), the copying of 90 files to the respective FOLDER set was not successful. When I BOOT up, where all the RAM drive setup occurs in FDauto.bat, I get the following error...


DOSLFN 0.41f (... 4/2022):disabled
(Another TSR grabbed Int21 and/or Int2F)
Last error:2 - couldn't expand FAT directory



Though with this error, I proceeded to run 1.bat, 2.bat, ...h.bat, i.bat and the results are below



[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]


[image]










[image]





Note the enlarged drive set for SMARTDRV is just for me to convince myself there is not point in smartdriving RAM drives - I am not even sure now that a smartdrv makes any sense for DIR. Beginning to think that the smartdrv cache is only designed for file copying, etc (not for DIR listing) - I could be wrong though.
Zyzzle

14.03.2022, 06:12

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> I had a request (via email) to create a
> RAM drive greater
> than 4GiB. It sort of works (works on his and mine 8GiB AMD, but has
> issues with his 32GiB Intel) so if anyone has the time and inclination to
> test, that'd be great.

Thanks very much for this FAT32 RAMdrive. It has been much-wanted for years. I tested the Super Extended Memory version, and didn't get errors either creating or copying files to a large RAMdrive. (I have 8G total RAM, but the program would only let me create a 5 GiB drive. Anything higher reported "Too big for system memory.) I'm using an Intel core i5 8250u laptop with 8GiB RAM, with CSM mode enabled in a UEFI BIOS which thankfully loads DOS bare metal.

I noticed some bugs:

-can only specify drive letter when used with the default option (specify ramdrive size in GiB), ie
rdrsx32 5,d creates a 5 GiB ramdrive in drive D
rdrsx32 5.5,d creates a 5 GiB ramdrive in drive D, the decimal point is ignored
rdrsx32 5500M,d creates a 5500M ramdrive in drive B, the first "drive" available in my system, NOT in drive d where I specified. Furthermore, this ramdrive in drive B gives many errors ("sector not" found) when writing files.

The ramdrive created in drive d works perfectly, in writing and copying files as large as 2 GiB. When attempting to decompress a .zip file containing a "blank" file of 4096MiB, DOS can't write this large file, giving errors. It is related to DOS not supporting unsigned filelengths, I believe. 2 GiB-1 seems to be the largest file I can create on the FAT32 RAMdrive in pure DOS (Microsoft "DOS 7.1."

Please fix the drive letter bugs, and the inability to specify floating point size of drive in GiB if possible.
jadoxa

Homepage E-mail

Queensland, Australia,
15.03.2022, 15:25

@ Zyzzle
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Please fix the drive letter bugs, and the inability to specify floating
> point size of drive in GiB if possible.
Done, but I'll hold off uploading a new version.

> Might actually be a doslfn bug that is revealed by the fat32 ramdisk.
Confirmed, still investigating.

> this should be easy to check by using FORMAT.EXE to re-format the ramdisk, and see if problems vanish.
Format is not supported, but I could put it back if desired.

> ... and the ramdisk has disappeared.
DRVON from SHSUFDRV might restore the drive.
Richard

16.03.2022, 04:30
(edited by Richard, 16.03.2022, 04:40)

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

Can someone confirm if I have correctly setup a FAT32 RAMdrive in the first 4GB of memory (XMS). The relevant FreeDOS (RC5) FDauto.bat lines are...




devload c:\freedos\bin\himemX.exe
c:\freedos\bin\rdrvX32.exe 1400M
copy d:\nulll.txt g:\G
copy d:\1 g:\1_Gbyte
dir g:\

rem himemX.exe 6,074 bytes Oct-29-2020
rem RdrvX32.exe 3,682 bytes Mar-14-2022
rem 1400M is about the maximum available for my computer




and the output produced is



Volume in drive G is RDRVX32

Dir of G:\

1_Gbyte 1,073,741,824 ...
G 0 ...

2 file(s) 1,073,741,824 bytes
0 dir(s) 392,822,784 bytes free




I booted off a USB stick (laptop in Legacy Mode) and the USB stick is the C:\ D:\ E:\ drives and drive F is a DVD drive (in the laptop).

At present I have trouble copying a file bigger than 64Mbytes from the USB stick to a FAT32 drive in the Super Extended Memory (hence why I wanted to see if I could copy to a FAT32 drive in normal Extended Memory).


Question - how can I tell if I actually have a FAT32 drive? In windows I can use File Explorer to tell me - but in DOS?
jadoxa

Homepage E-mail

Queensland, Australia,
16.03.2022, 06:44

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Can someone confirm if I have correctly setup a FAT32 RAMdrive in the first
> 4GB of memory (XMS).
> devload c:\freedos\bin\himemX.exe
> c:\freedos\bin\rdrvX32.exe 1400M
HimemX won't access beyond 4GiB, so ergo it's in the first 4GiB.

> Volume in drive G is RDRVX32
The drive is created...

> 2 file(s) 1,073,741,824 bytes
> 0 dir(s) 392,822,784 bytes free
at the size requested (~1.4GiB; the size is how much XMS to request, not how big the drive itself should be, unlike my other RAM drives).

> Question - how can I tell if I actually have a FAT32 drive?
Hm, I thought FreeDOS chkdsk would tell me that FAT32 isn't supported, but now it's saying it can't read the FAT(s). Try my Trace utility. (Or take it as given, since that's all it creates.)
rr

Homepage E-mail

Berlin, Germany,
16.03.2022, 09:01

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > Question - how can I tell if I actually have a FAT32 drive?
>
> Hm, I thought FreeDOS chkdsk would tell me that FAT32 isn't
> supported, but now it's saying it can't read the FAT(s). Try my
> Trace utility. (Or take it as
> given, since that's all it creates.)
Eric Auer's whichfat might tell you. (If it can detect the drive.)

---
Forum admin
Richard

16.03.2022, 14:14

@ rr
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

@Jason

My results so far


[image]




https://www.dropbox.com/s/s3ro7z074lkqia0/FAT32_28G.png?dl=1
Zyzzle

17.03.2022, 03:04

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> @Jason
>
> My results so far
>
>
> [image]
>
>
>
>
> https://www.dropbox.com/s/s3ro7z074lkqia0/FAT32_28G.png?dl=1
Very interesting.

How did you create / copy those 4095 MiB files on the ramdrive in DOS? The best I can do is 2047 MiB since my DOS (MS-DOS 7.1) doesn't support unsigned long integer seek / create. Which DOS are you using? How did you get it to support long unsigned file seeks? I assume you're on bare metal DOS?

It appears that you have 32 GiB total system RAM and the the new Ramdrive has problems with accessing beyond 24 (?) GiB.
Richard

17.03.2022, 05:04

@ Zyzzle
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > @Jason
> >
> > My results so far
> >
> >
> > [image]
> >
> >
> >
> >
> >
> https://www.dropbox.com/s/s3ro7z074lkqia0/FAT32_28G.png?dl=1
>
> Very interesting.
>
> How did you create / copy those 4095 MiB files on the ramdrive in DOS? The
> best I can do is 2047 MiB since my DOS (MS-DOS 7.1) doesn't support
> unsigned long integer seek / create. Which DOS are you using? How did you
> get it to support long unsigned file seeks? I assume you're on bare metal
> DOS?
>
> It appears that you have 32 GiB total system RAM and the the new Ramdrive
> has problems with accessing beyond 24 (?) GiB.


I created a USB BOOT stick using Rufus with the image file from FreeDOS RC5 (full usb image). Note RC5 has just recently been replaced with FreeDOS 1.3. As per guidelines from FreeDOS two partitions on the USB stick were created (the C:\ partition is only 512 Mbyte total (of which ~~400 MByte is used by FreeDOS).

I used the FreeDOS default FDCONFIG.SYS and FDAUTO.BAT as the framework for my configuration (and did some adjustments to suit my needs).

So with the laptop BIOS setup for "Legacy Mode", I boot up from the USB stick and the USB stick is C:\, D:\, E:\ drives of capacity 512 MB, 12 GB, 2 GB respectively (for my 16 GB USB stick). The hard drive of my laptop is not "visible" to the FreeDOS environment. Also (still having to sort things out), drive F:\ is the built-in DVD drive. So to answer your question - YES, working in bear metal mode (no virtual machine (VM) stuff at all).

The laptop has 32 GByte RAM (which is the maximum allowed for the INTEL i7 4th generation 4810MQ processor). Note that my laptop has a Q3 display (3200 x 1800) and because of this, I think that a big "chunk" of XMS (i.e. the first 4 GB of standard Extended Memory) is reserved for the display. When actually running FreeDOS the whole display area is used, so it seems a waste of display resolution for a DOS text based display (but this is what I have).

So the first 4 GB is XMS, the remaining 32-4 = 28 GB is SXMS. I set up a ~1.4 GB FAT32 G:\ RAMdrive in XMS, and ~28 GB FAT32 H:\ RAMdrive in SXMS. I think that the supplied digital photograph screenshot clearly indicates this. I used both WhichFAT.com and trace.com to verify the FAT size. I think that if you do the sums of bytes used + bytes free, for H:\, the result is 28 Gbyte. Now g:\ is only ~1.4 GB (in XMS), and this value is about the maximum, it seems, I can have for G:\. So I have "problems accessing all of the XMS", i.e. the first 4 GB of memory (keep in mind that maybe my Q3 display is a significant factor here).

So to sum up, 1.4 GB (G:\) + 28 GB (H:\) ~= 29.4 GB RAMdrives (total) as FAT32 appears to have been achieved. I don't know if I can "squeeze" out any more memory as RAMdrive.


As to the question of DOS version used - I don't know - it is whatever is the default settings of FreeDOS RC5 (or if you wish 1.3 official release).

Via a simple BASIC program (in windows 10), I created a 1 GByte file (1024 * 1024 *1024 bytes), the filename is "1", and copied it to my USB stick to the appropriate partition (D:\ when in bare metal metal). So the relevant lines in FDauto.bat are


copy d:\1 g:\1
copy g:\1 h:\1
copy h:\1 + h:\1 + h:\1 + h:\1 h:\4
copy h:\4 + h:\4 h:\8
copy h:\4 h:\4a
copy h:\4 h:\4b
copy h:\4 h:\4c
copy h:\4 h:\4d
copy h:\4 h:\4e

so the 1 GByte file, filename "1", on the USB stick D:\ partition, is copied to the FAT32 drive G:\ (in XMS), and this copied file is copied to the FAT32 H:\ (in SXMS). I used the old method of adding two files together (here they are plain ascii files) to generate a 4 GB file "4". However, as expected, if I try to create a 8 GB file "8"- only a 1 byte file is created.


This reply to you is very verbose - but I hope it answers all your questions.
jadoxa

Homepage E-mail

Queensland, Australia,
16.03.2022, 15:59

@ rr
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Eric Auer's whichfat might tell you. (If it can detect the drive.)
It detects the drive, but uses the cluster count, too.

I've updated the zip with the drive letter fix and decimal sizes, along with a fixed DOSLFN.
tom

Homepage

Germany (West),
16.03.2022, 16:20

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > Eric Auer's whichfat might tell you. (If it can detect the drive.)
>
> It detects the drive, but uses the cluster count, too.
>
> I've updated the zip with the drive letter fix and decimal sizes, along
> with a fixed DOSLFN.
so it was a bug in DOSLFN?

after all these years with DOSLFN in the wild with no (such) reported bugs I had assumed that it was a problem with RDRVSX32 as the bug seems to be easily reproduced.

what exactly was the bug (to satisfy our curiosity)

edit: just posting a 'fixed' DOSLFN.COM without posting also the sources is not nice :-( behavior in my book:-(
Japheth

Homepage

Germany (South),
16.03.2022, 17:29

@ tom
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> after all these years with DOSLFN in the wild with no (such) reported bugs
> I had assumed that it was a problem with RDRVSX32 as the bug seems to be
> easily reproduced.
I was pretty sure that the bug will be located in DOSLFN. After all, a ramdisk, once the init phase with its bureaucrazy is over, has a pretty simple job. OTOH, if you have ever looked into the DOSLFN source ... :-)

> what exactly was the bug (to satisfy our curiosity)
I'm also eager to know.

It surely isn't the last bug in doslfn...

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
17.03.2022, 04:12

@ tom
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> so it was a bug in DOSLFN?
> what exactly was the bug (to satisfy our curiosity)
Incorrect cluster count when extending the directory, so the cluster wasn't being zeroed.

> edit: just posting a 'fixed' DOSLFN.COM without posting also the sources is
> not nice :-( behavior in my book:-(
I wanted to check it really worked first; since it has I've made a new release.

> Btw, comparing FAT32 (rdrvsx32.exe) and FAT16 (modified shsurdrv that also uses super-extended memory) disk access times reveals that the FAT32 ramdisk is significantly slower.
That might be a free space issue, try with the new doslfnms.com.

> It surely isn't the last bug in doslfn...
No one else has offered to take up maintenance.
Japheth

Homepage

Germany (South),
17.03.2022, 08:01

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

deleted

(realized that the source is now available).

---
MS-DOS forever!
Japheth

Homepage

Germany (South),
16.03.2022, 17:22

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> I've updated the zip with the drive letter fix and decimal sizes, along
> with a fixed DOSLFN.
Thanks! The trashed sub-dir is gone now.

Btw, comparing FAT32 (rdrvsx32.exe) and FAT16 (modified shsurdrv that also uses super-extended memory) disk access times reveals that the FAT32 ramdisk is significantly slower. Copying a directory of about 6000 files ( virtually all are LFNs, total size is about 500 MB ) to the ramdisk needs about half a minute for FAT16, but 2 min & 15 seconds for FAT32. So the FAT32 option is probably only useful in rare cases.

---
MS-DOS forever!
tom

Homepage

Germany (West),
16.03.2022, 18:34

@ Japheth
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> > I've updated the zip with the drive letter fix and decimal sizes, along
> > with a fixed DOSLFN.
>
> Thanks! The trashed sub-dir is gone now.
>
> Btw, comparing FAT32 (rdrvsx32.exe) and FAT16 (modified shsurdrv that also
> uses super-extended memory) disk access times reveals that the FAT32
> ramdisk is significantly slower. Copying a directory of about 6000 files (
> virtually all are LFNs, total size is about 500 MB ) to the ramdisk needs
> about half a minute for FAT16, but 2 min & 15 seconds for FAT32.
to compare this in a fair way, you should have equal clustersize for both.
however a factor of 4 sounds like a bug (somewhere else).




> So the
> FAT32 option is probably only useful in rare cases.
 when you want > 4GB you need FAT32. which was the requirement in the first place.
Zyzzle

17.03.2022, 03:10

@ Japheth
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

So the
> FAT32 option is probably only useful in rare cases.
Very useful to reduce slack space (wasted space) due to large cluster sizes necessary in large FAT16 drives, when copying / storing large numbers of small files. I think a 2 GiB FAT32 drive may even use 512-byte clusters, vs 32 kb clusters for a 2 GiB FAT16 drive. The size of the FAT increases drastically when using small cluster sizes, however.

May we have the option in new rdsk32.exe to specifiy the exact cluster size of the created FAT32 ramdrive and the number of FATs and root directory entries? Does the possibility also exist to change the sector-size to 1024, 2048, or 4096 bytes?
(It doesn't hurt to ask!)
jadoxa

Homepage E-mail

Queensland, Australia,
17.03.2022, 04:22

@ Zyzzle
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> May we have the option in new rdsk32.exe to specifiy the exact cluster size
> of the created FAT32 ramdrive and the number of FATs
I've kept it simple and fixed it at a 4Ki cluster and 1 FAT. I don't intend to support FAT32 image files, so I don't see a need for anything else.

> and root directory entries?
FAT32 places the root directory in a cluster, so it will expand as necessary.

> Does the possibility also exist to change the sector-size to 1024,
> 2048, or 4096 bytes?
The possibility exists, but I won't be the one doing it. :)
Japheth

Homepage

Germany (South),
17.03.2022, 11:32

@ Japheth
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> Btw, comparing FAT32 (rdrvsx32.exe) and FAT16 (modified shsurdrv that also
> uses super-extended memory) disk access times reveals that the FAT32
> ramdisk is significantly slower. Copying a directory of about 6000 files (
> virtually all are LFNs, total size is about 500 MB ) to the ramdisk needs
> about half a minute for FAT16, but 2 min & 15 seconds for FAT32. So the
> FAT32 option is probably only useful in rare cases.
Ok, now testing with the new doslfnms.com, things look a lot better: copying to the FAT16 ramdisk needs 15 s, while the copy to FAT32 needs 17 s.

---
MS-DOS forever!
Richard

01.04.2022, 14:28

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

>
> I've updated the zip with the drive letter fix and decimal sizes, along
> with a fixed DOSLFN.

I just realized that I made a big mistake. I was assuming that when say 28 GB RAM drive is created, that it is 28 * 1024 * 1024 * 1024 bytes however I think it is only 28 * 1000 * 1000 * 1000.


So 32 GB is actually

34,359,738,368 B
33,554,432 KB
32,768 MB

in base 2 notation of the term Gigabyte.



I think everything from RdrvSX32 is as per the decimal (base 10) notation of Gigabyte.


So for the sizes, eg "1", could it be that 1024 * 1024 * 1024 bytes is allocated instead of the base 10 value?

Similarly, "1M" should be 1024 * 1024 bytes.



So, to avoid confusion (?), have


"blank", "G", "M" i.e. capitals are for the base 2 notation (no fractional part)

"g", "m" i.e. lower case are for the decimal base 10 notation (with decimal fraction resolution.



I was wondering why there was a discrepancy of about 2 GB - I thought I had a "bonus" 2 GB but in fact it was

34,359,738,368 B - 32,000,000,000 = ~ 2 GB



Most of my applications refer to the base 2 notation.


I was initially concentrating on the small details and completely missed noticing this big error of mine.
jadoxa

Homepage E-mail

Queensland, Australia,
01.04.2022, 14:58

@ Richard
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> So, to avoid confusion (?), have
I avoid confusion by saying "gibibytes" (GiB) and "mebibytes" (MiB).

If you ask for "1.5[g]" you will get 1.5 * 1024 * 1024 * 1024 bytes.
If you ask for "1.5m" you will get 1.5 * 1024 * 1024 bytes.
Japheth

Homepage

Germany (South),
14.03.2022, 15:09
(edited by Japheth, 15.03.2022, 13:06)

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

The problem I found is rather complicated:

1. in config.sys, the new himemsx.exe, doslfn 0.41d and rdrvsx32 are loaded, a 4 GB ramdisk created.

2. some LFN files are copied to the ramdisk's root directory.
The LFNs need just a bit more space than what fits in a cluster ( so the root directory then has 2 clusters ).

3. a sub-directory is created in the root directory.

4. the very same files that were copied to the root directory are now copied into the sub-directory. Just before the last file is copied, everything seems ok. After the last file has been copied, there is garbarge in the sub-directory.

I verified that the problem occurs with doslfn loaded only. And it does NOT occur if both drives (src and dest) are physical drives. Might actually be a doslfn bug that is revealed by the fat32 ramdisk.

EDIT: the problem also occurs if I use rdrvx32.exe (with a 2 GB ramdisk) and the standard HimemX. So it definitely is NOT a problem of the modifed himemsx.

---
MS-DOS forever!
tom

Homepage

Germany (West),
15.03.2022, 12:25

@ Japheth
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> I verified that the problem occurs with doslfn loaded only.

do these errors also occur without doslfn or with only short names involved?

> And it does NOT
> occur if both drives (src and dest) are physical drives. Might actually be
> a doslfn bug that is revealed by the fat32 ramdisk.
there is another difference: the ramdisk is formatted when loaded, and the formatter (integrated into RDFVSX32) is different from the external FORMAT.EXE.

if this integrated formatter is slightly off from what doslfn expects problems would also occur.

this should be easy to check by using FORMAT.EXE to re-format the ramdisk, and see if problems vanish.
Japheth

Homepage

Germany (South),
15.03.2022, 13:37

@ tom
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> do these errors also occur without doslfn or with only short names
> involved?
No. With SFNs only, there's no problem, no matter if doslfn is loaded or not.

> this should be easy to check by using FORMAT.EXE to re-format the ramdisk,
> and see if problems vanish.
It's funny. Trying to format the ramdisk K: displays:

K:\>format k:
Formatierung Laufwerk K: nicht unterstützt.
Formatierung beendet.

>Aktuelles Laufwerk nicht mehr gültig.

... and the ramdisk has disappeared.

---
MS-DOS forever!
Japheth

Homepage

Germany (South),
16.03.2022, 07:34

@ jadoxa
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

There seems to be a lower size limit for the FAT32 ramdisk: 260 MB. Trying to create a disk with 256 MB results in error message "FAT is corrupt" ( translated to English ) and the number of clusters for that drive is zero. Is this some FAT32 limitation or a bug?

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
16.03.2022, 07:51

@ Japheth
 

RDRVSX32: FAT32 RAM drive for HimemSX

Post reply

> There seems to be a lower size limit for the FAT32 ramdisk: 260 MB.
I've been testing in FreeDOS and that works fine with a 16M drive. Turns out that's what caused the chkdsk issue, since it only looks at the cluster count, so the drive needs to be over 256M to be detected as FAT32. This sounds like the same thing.
Japheth

Homepage

Germany (South),
19.03.2022, 08:09

@ jadoxa
 

another bug in doslfn

Post reply

The newest doslfn 0.41e may in a few cases report an error ( access denied, DOS error 5 ) which in fact is none.

I have a test case which should make doslfn emit this error, but I'm not quite sure if it will do so on another machine, so I examined the probable bug a bit on my own. If needed, I can provide the test case, of course.

The error occurs during a copy operation of a directory, containing more than 1000 files, mostly with LFNs.

The error is actually in doslfn, procedure "make_free_dirent_space":


        and     dx,not bit 3    ;weiter mit normalem Next_DirEnt
@@l1_:  jns     @@l1            ;(clears bit 3 of DL, tests sign of DH)
        call    Calc_Next_Cluster       ;there was no room for the longname
        jc      mfde_ret                ; so need to point to the new cluster
        call    _set_cur
        mov     bx,[Sektorp]
        jmp     @@l1_           ;Calc_Next_Cluster will clear sign


When Calc_Next_Cluster is called, it expects registers EAX and CL to be set correctly, but they aren't ( CL is supposed to contain a shift factor, but actually contains 0FFh!; and EAX does not contain the "user area" sector number for the directory that is to be enlarged ). So Calc_Next_Cluster fails, resulting in error 0005 to be returned.

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
19.03.2022, 09:29

@ Japheth
 

another bug in doslfn

Post reply

> I have a test case which should make doslfn emit this error, but I'm not
> quite sure if it will do so on another machine, so I examined the probable
> bug a bit on my own. If needed, I can provide the test case, of course.
Please provide the test case, in my testing the code was never executed. I did set CL, but I don't know why EAX would be wrong, since it comes directly from CurSector.
Japheth

Homepage

Germany (South),
19.03.2022, 10:35

@ jadoxa
 

another bug in doslfn

Post reply

> Please provide the test case, in my testing the code was never executed. I
> did set CL, but I don't know why EAX would be wrong, since it comes
> directly from CurSector.
https://github.com/Baron-von-Riedesel/Test
https://github.com/Baron-von-Riedesel/Test/releases/download/test/Test.zip

---
MS-DOS forever!
Japheth

Homepage

Germany (South),
20.03.2022, 08:10

@ jadoxa
 

another bug in doslfn

Post reply

> Please provide the test case, in my testing the code was never executed. I
> did set CL, but I don't know why EAX would be wrong, since it comes
> directly from CurSector.
Just in case the error is hard to find, there's a simple workaround:


@@l1_:  jns     @@l1            ;(clears bit 3 of DL, tests sign of DH)

;--- since the directory's size has been increased already when this code
;--- is reached, it might be sufficient to just restart the command???
        jmp make_free_dirent_space
if 0
        call    Calc_Next_Cluster       ;there was no room for the longname
        jc      mfde_retx               ; so need to point to the new cluster
        call    _set_cur
        mov     bx,[Sektorp]
        jmp     @@l1_           ;Calc_Next_Cluster will clear sign
endif


It works, AFAICS, and since this situation occurs not too often ... I'm no purist :-P

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
24.03.2022, 13:32

@ Japheth
 

another bug in doslfn

Post reply

> The error is actually in doslfn, procedure "make_free_dirent_space".
Wow, it's a wonder that ever worked. Here's an updated doslfnms (with diff from 0.41e). This time I'll hold off the release for a bit...
Japheth

Homepage

Germany (South),
25.03.2022, 08:04

@ jadoxa
 

another bug in doslfn

Post reply

> Wow, it's a wonder that ever worked. Here's an updated
> doslfnms (with
> diff from 0.41e). This time I'll hold off the release for a bit...
Yes, it's fixed, nice work!

Btw, something about the dosfsck.exe tool: if it's been run to check the FAT32 ramdisk, it wants to "cure" the content of the FS-info and BS-backup sectors. Obviously it isn't happy with the sector numbers -1 as set by the ramdisk.

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
26.03.2022, 08:49

@ Japheth
 

another bug in doslfn

Post reply

> Btw, something about the dosfsck.exe tool: if it's been run to check the
> FAT32 ramdisk, it wants to "cure" the content of the FS-info and BS-backup
> sectors. Obviously it isn't happy with the sector numbers -1 as set by the
> ramdisk.
FreeDOS: if the info & backup sectors are -1 don't use 'em.
dosfsck: if the info & backup sectors are 0 don't use 'em.

Oh, goody. New version of rdrvsx32 with valid info & backup sectors. A size < 100 is assumed gibibyte, otherwise mebibyte. You can also have a drive bigger than 32GiB (that was previously an implicit limit, 64Ki FAT sectors).
Richard

27.03.2022, 03:06

@ jadoxa
 

another bug in doslfn

Post reply

>
> Oh, goody. New version of
> rdrvsx32 with
> valid info & backup sectors. A size < 100 is assumed gibibyte, otherwise
> mebibyte. You can also have a drive bigger than 32GiB (that was previously
> an implicit limit, 64Ki FAT sectors).

It has been almost a week since I last did "DOS stuff", and then my system was in a more or less "usable state".

I just replaced relevant files (himemSX.exe rdrvX32.exe rdrvSX32.exe) with the ones from the quote link above and now my system DOES NOT WORK.


By the process of elimination (i.e. deleting relevant FDauto.bat one line at a time), BOTH the new rdrvX32.exe and rdrvSX32.exe create one (or both) of the following errors




Invalid Opcode at 16c2 06d9 0246 0000 0000 ...

(which is infinite scrolling on display repeating the same message)




Interrupt divide by zero,stack:
50c6 06d9 0246 0000 ...
Invalid Opcode at 006c 06d9 0217 3a00 9000
Invalid opcode at 01d4 0474 0246 08aa 08aa


or if POWER OFF (above via Alt Ctrl Del) and e.g.

rdrvSX32.exe 1
Interrupt divide by zero, stack:
1448 650e 4697 4646 02fd



In all error cases, I have to either (Alt Ctrl Del) or even (POWER OFF) to use my computer.


Should it be of any relevance, below is screenshot using xmSStat before attempting to use either rdrvX32.exe or rdrvSX32.exe

[image]
tom

Homepage

Germany (West),
27.03.2022, 14:41

@ Richard
 

another bug in doslfn

Post reply

> >
> > Oh, goody. New version of
> > rdrvsx32
> with
> > valid info & backup sectors. A size < 100 is assumed gibibyte,
> otherwise
> > mebibyte. You can also have a drive bigger than 32GiB (that was
> previously
> > an implicit limit, 64Ki FAT sectors).
>
>
> It has been almost a week since I last did "DOS stuff", and then my system
> was in a more or less "usable state".
>
> I just replaced relevant files (himemSX.exe rdrvX32.exe rdrvSX32.exe)
> with the ones from the quote link above and now my system DOES NOT WORK.
>
>
> By the process of elimination (i.e. deleting relevant FDauto.bat one line
> at a time), BOTH the new rdrvX32.exe and rdrvSX32.exe create one (or both)
> of the following errors
50% of bug fixing is always bug reproduction at developer's site.

as this software obviously works on other peoples' PC, you could help A LOT by
providing the minimum configuration to produce these bugs; ideally with naked device=himemsx.exe in config.sys, and rdrvx32.exe started from command line; then add additional programs until it crashes.
jadoxa

Homepage E-mail

Queensland, Australia,
28.03.2022, 05:06

@ Richard
 

another bug in doslfn

Post reply

> By the process of elimination (i.e. deleting relevant FDauto.bat one line
> at a time), BOTH the new rdrvX32.exe and rdrvSX32.exe create one (or both)
> of the following errors
Fixed (problem with not relocating high).

> Should it be of any relevance, below is screenshot using xmSStat before
> attempting to use either rdrvX32.exe or rdrvSX32.exe
How did you take the screenshot?
Richard

28.03.2022, 06:40

@ jadoxa
 

another bug in doslfn

Post reply

> > By the process of elimination (i.e. deleting relevant FDauto.bat one
> line
> > at a time), BOTH the new rdrvX32.exe and rdrvSX32.exe create one (or
> both)
> > of the following errors
>
> Fixed (problem with not relocating high).
>


Thanks - appears to be working.

After setting up himemsx I manually rdrvX32 1 and rdvXS32 30 and it appears to be working ok (refer the 6 screenshots below)

[image]

[image]

[image]

[image]

[image]

[image]




I then restored by recent FDauto.bat + FDconfig.sys to what it was a week ago, and it appears to be WORKING ok (like when a week ago). (refer the 7 screenshots below)


[image]

[image]

[image]

[image]

[image]

[image]

[image]


I have yet to do very heavy testing but so far it looks OK.
Richard

28.03.2022, 07:12

@ jadoxa
 

another bug in doslfn

Post reply

>
> > Should it be of any relevance, below is screenshot using xmSStat before
> > attempting to use either rdrvX32.exe or rdrvSX32.exe
>
> How did you take the screenshot?


I used a program from st204f.zip for the screen captures.


I do not fully understand the "legal stuff" (copyrights, licensing, free-ware, share-ware, abandon-ware etc) but if you are interested in a copy of st204f.zip (assuming you cannot source it yourself from the internet) - the .zip may just happen to come into your mail (or anyone else for that matter if interested).

:)


A screenshot, as above, was much more fun than using an old digital camera, taking a digital photo of the computer screen (needing to focus, compose, adjust camera settings, etc) removing SD card (with photo) from camera, shutting down DOS computer, installing SD card (from camera) into computer (now in windows mode), sometimes adjusting windows 10 to allow it to read the SD card, converting (in windows) the photo into format suitable for BTTR, removing SD card from computer and reinstalling same into digital camera.


A long term project of mine is to try to modify the screenshot program to require a "certain single key" to activate - rather than at present three keys to press.
jadoxa

Homepage E-mail

Queensland, Australia,
28.03.2022, 08:59

@ Richard
 

another bug in doslfn

Post reply

> I used a program from st204f.zip for the screen captures.
Ah, Screen Thief, sounds somewhat familiar.

> A screenshot, as above, was much more fun than using an old digital camera...
Yeah, it really didn't look like a photo, and it wasn't a text grab, which is what got me curious.

> A long term project of mine is to try to modify the screenshot program to
> require a "certain single key" to activate - rather than at present three
> keys to press.
Maybe try SNARF, it's Alt+S (by default, Alt is fixed, but the key is configurable).
Oso2k

04.01.2025, 00:41

@ jadoxa
 

another bug in doslfn

Post reply

@jadoxa , is http://adoxa.altervista.org/shsufdrv/rdrvsx32.zip still the latest link for this driver? I want to play with this on a couple machines. Thanks!
jadoxa

Homepage E-mail

Queensland, Australia,
04.01.2025, 06:46

@ Oso2k
 

another bug in doslfn

Post reply

Yep, haven't done anything since.
Oso2k

04.01.2025, 09:31

@ jadoxa
 

another bug in doslfn

Post reply

> Yep, haven't done anything since.
Thanks for confirming.
glennmcc

Homepage E-mail

North Jackson, Ohio (USA),
28.03.2022, 20:26

@ Richard
 

another bug in doslfn

Post reply

>
> I used a program from st204f.zip for the screen captures.
>
>
> I do not fully understand the "legal stuff" (copyrights, licensing,
> free-ware, share-ware, abandon-ware etc) but if you are interested in a
> copy of st204f.zip (assuming you cannot source it yourself from the
> internet) - the .zip may just happen to come into your mail (or anyone else
> for that matter if interested).
>
> :)
Found it here...

https://www.phatcode.net/downloads.php?id=194


Screen Thief for DOS is a freeware screen capture TSR for DOS that supports
MDA, CGA, EGA, and VGA modes. Although it was originally released as shareware,
this version of Screen Thief is freeware and can be used without restrictions.

---
--
http://glennmcc.org/
Japheth

Homepage

Germany (South),
27.03.2022, 04:48
(edited by Japheth, 27.03.2022, 08:12)

@ jadoxa
 

another bug in doslfn

Post reply

> Oh, goody. New version of
> rdrvsx32 with
> valid info & backup sectors. A size < 100 is assumed gibibyte, otherwise
> mebibyte. You can also have a drive bigger than 32GiB (that was previously
> an implicit limit, 64Ki FAT sectors).
There's a small bug in this version: the volume label entry in the root directory isn't written to the first sector of the root cluster, but to the second. This confuses MS scandisk a lot, makes it display a rather misleading error message. After clearing the volume label entry with a disk editor, the error disappears.

About doslfn: please hold off a new release! It's likely that there's still a bug in make_free_dirent_space, causing a "hang". I have no test case yet, but may be able to narrow the bug down.

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
28.03.2022, 05:08

@ Japheth
 

another bug in doslfn

Post reply

> There's a small bug in this version: the volume label entry in the root
> directory isn't written to the first sector of the root cluster
Fixed. I've also set the initial free space/next cluster, so dosfsck/scandisk on a brand new drive won't complain.

Grab the new version from the old link.
Japheth

Homepage

Germany (South),
29.03.2022, 05:45

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> About doslfn: please hold off a new release! It's likely that there's still
> a bug in make_free_dirent_space, causing a "hang". I have no test case yet,
> but may be able to narrow the bug down.
Perhaps I was able to find that bug.

With the neweset doslfn, the bug shows as a "system hang" that requires a reboot. I guess that with older versions, one gets an "access denied" error instead.

The bug happens again in make_free_dirent_space. It's a bit difficult to make a test case, because the error occurs only if the disk is pretty "full". Once the error has occured - using a debugger to intercept and so avoiding the "hang" - one can see (using WDe) that the sub-directory's size has been increased to 2 clusters, although only the first sector of the first cluster is used so far.

The suspicious code IMO seems to be here:


proc Next_Sektor pascal         ;nur FAT
;liefert n?chsten Sektor der Clusterkette bzw. des Hauptverzeichnisses
;PE: [CurSector]=momentaner Sektor
;PA: [CurSector]=n?chster Sektor, bereits gelesen
;    BX=Zeiger auf Sektor-Anfang
;    [num_cluster] inkrementiert bei Cluster-Wechsel
;VR: alle
        mov     eax,[CurSector]
        inc     eax
if 1
; this seems to fix the bug.
; DPB_LastSec contains the size of the user area of the disk ( without FAT )
; and is originally compared with CurSector, which has the FAT "included".
        mov ebx,[DPB_LastSec]
        add ebx,[DPB_UsrSec]
        cmp     ebx,eax
else
        cmp     [DPB_LastSec],eax
endif
        jc      _nde_ret        ;war User-Bereich: Ende!
        sub     eax,[DPB_UsrSec];Hauptverzeichnis-Ende?

---
MS-DOS forever!
Japheth

Homepage

Germany (South),
31.03.2022, 22:51

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

The proposed fix has a flaw: it modifies hiword(ebx). That's something that DOS ( and hence doslfn ) shouldn't do.

This is better:


;    BX=Zeiger auf Sektor-Anfang
;    [num_cluster] inkrementiert bei Cluster-Wechsel
;VR: alle
        mov     eax,[CurSector]
        inc     eax
if 0
        cmp     [DPB_LastSec],eax
        jc      _nde_ret        ;war User-Bereich: Ende!
endif
        sub     eax,[DPB_UsrSec];Hauptverzeichnis-Ende?
        cmc
        jz      _nde_ret        ;war Hauptverzeichnis: Ende! (mit CY=1)
        jnc     @@er            ;war Hauptverzeichnis: es gibt weitere
if 1
        cmp     [DPB_LastSec],eax
        jc      _nde_ret        ;war User-Bereich: Ende!
endif
        call    two2shift


So in this version the compare is unchanged, it's just a few lines later, after EAX has been adjusted.

---
MS-DOS forever!
tom

Homepage

Germany (West),
01.04.2022, 19:55

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

this is the first time I have ever seen CMC used in actual code.
probably the most underused single byte x86 instruction ;-)

besides this: thanks for your work on this. after sooo many years of (probably light) use of DOSLFN, I had assumed it to be stable software.

obviously not as it didn't take you much time to locate bugs, and eventually fix them. thanks to everyone involved:-D
jadoxa

Homepage E-mail

Queensland, Australia,
02.04.2022, 09:55

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> So in this version the compare is unchanged, it's just a few lines later,
> after EAX has been adjusted.
Done (the diff is still based on 0.41e). I also fixed the last sector itself (it should have subtracted two before adding one). I'll release this next week, if there's nothing else...


I also updated Trace again, to prevent small FAT32 drives being interpreted as FAT12.
Japheth

Homepage

Germany (South),
03.04.2022, 19:25

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> if there's nothing else...
Not really - but if you are bored and long for something useful to do:

There is a problem if you copy lots of LFN-files to the root directory. I guess this is because of that hack in doslfn that temporarily writes a file ( named 0xff,' ' ) to the root dir if a directory has to be enlarged - and if that directory is the root itself ... :-D

---
MS-DOS forever!
tom

Homepage

Germany (West),
03.04.2022, 20:25

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> > if there's nothing else...
>
> Not really - but if you are bored and long for something useful to do:
>
> There is a problem

what kind of problem?

> if you copy lots of LFN-files to the root directory.

how much is "lot's"? what is your cluster size?



> I
> guess this is because of that hack in doslfn that temporarily writes a file
> ( named 0xff,' ' ) to the root dir if a directory has to be
> enlarged - and if that directory is the root itself ... :-D
 that is entirely reasonable :-D
Japheth

Homepage

Germany (South),
04.04.2022, 07:34

@ tom
 

about to fix the "final" doslfn bug

Post reply

> what kind of problem?
wenn copying to X:, Volkoc Commander says "Cannot open file X:\filename".

> how much is "lot's"? what is your cluster size?
it's about 1100 files, 2/3 of them with LFNs, cluster size is 8 sectors.

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
06.04.2022, 15:05

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> There is a problem if you copy lots of LFN-files to the root directory.
Fixed (diff still based on 0.41e). It now simply temporarily replaces the first entry.
Japheth

Homepage

Germany (South),
06.04.2022, 18:41

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> Fixed (diff
> still based on 0.41e). It now simply temporarily replaces the first entry.
Yeah, it works! :-)
What a strange feeling to finally have a bug-free doslfn after so many years...:-D

---
MS-DOS forever!
glennmcc

Homepage E-mail

North Jackson, Ohio (USA),
06.04.2022, 20:18

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> > Fixed (diff
> > still based on 0.41e). It now simply temporarily replaces the first
> entry.
>
> Yeah, it works! :-)
> What a strange feeling to finally have a bug-free doslfn after so many
> years...:-D
Ah hell !!

You just jinxed it and now an as-yet unknown bug will crop-up. ;-)

---
--
http://glennmcc.org/
jadoxa

Homepage E-mail

Queensland, Australia,
07.04.2022, 03:02

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> What a strange feeling to finally have a bug-free doslfn after so many
> years...:-D
If I was told years ago you wouldn't have had to wait. Still, I'll wait another couple of days before making the new release.

What would you like to do with HimemSX? Are you going to incorporate the patch, or would you like a PR, or have RDRVSX32 supply its own version?
Japheth

Homepage

Germany (South),
07.04.2022, 06:58

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> What would you like to do with HimemSX? Are you going to incorporate the
> patch, or would you like a PR, or have RDRVSX32 supply its own version?
I'll add the new block move func to HimemSX, of course - and to jemmex as well.

---
MS-DOS forever!
Richard

09.04.2022, 01:11

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > ... bug-free doslfn
>
> Still, I'll wait another couple of days before making the new release.
>
>


This may NOT be a bug, but rather me not setting up Long File Names properly (never before, in DOS, used LFN's or had config/autoexec setup anything to do with LFN and related) - so here is what happened to me...

After successfully booting up and having a 30.5 Gi (32,714,765,312 bytes) FAT32 RAMdrive, creating 7 x 4Gi files + 1 x 2Gi file --> 470,548,480 bytes free (H:\)
and in XMS a 1.45 Gi FAT32 RAMdrive with a 1Gi file --> 478,707,712 bytes free (G:\) - I tried (in sequence) the following


c:\FreeDOS\BIN\doslfnms.com (4/2/2022 20,369 bytes)

which gave me an error referring to CP437uni.tbl not being present.



I copied from c:\FreeDOS\NLS\CP437uni.tbl --> C:\FreeDOS\BIN\

on running c:\> doslfnms.com - no errors reported.

C:\> copy nulll.txt 0123456789abcdef
Did a directory of c:\freedos\bin\ and it only showed me 8.3 filenames.

So I then tried
C:\> DOSLFN.com (i.e what FREEDOS RC5 supplied - and the output message was)



DOSLFN 0.41e (....) : enabled

Last error 94 - InDOS bayrao(*)i(*) vesu(*)ru(*)cu(*) kullani(*)mi(*)ni(*) SIFIRLA

(Note eg single_character(*) means the single_character I typed in is approximate, an "i" is actually a "i" with an accent (or something on top))



I then typed



C:\> CD FreeDOS ( and the error message was)




Invalid Opcode at 001E 4069 5982 D720 A200 0804 9300 ED60 A900 0204 0200 D861 AC00

Cannot terminate permanent FreeCOM instance

System halted ... eboot or poweroff now









As I said, I may not have setup LFN correctly (nothing mentioned in config/auto).
Rugxulo

Homepage

Usono,
09.04.2022, 03:38

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> As I said, I may not have setup LFN correctly (nothing mentioned in
> config/auto).
I wouldn't mix different DOSLFN versions in memory, that might confuse it.

For FreeDOS / FreeCOM, you need "set DIRCMD=/lfn" and "lfnfor on" and "lfnfor complete on", IIRC. (DR-DOS 7.03's shell also supported LFNs ... if enabled by third-party driver.)

See lfnfor for some help.

But the FreeDOS kernel doesn't support bigger than 2 GB files anyways.
Richard

09.04.2022, 06:15

@ Rugxulo
 

about to fix the "final" doslfn bug

Post reply

>
> For FreeDOS / FreeCOM, you need "set DIRCMD=/lfn" and "lfnfor on" and
> "lfnfor complete on", IIRC. (DR-DOS 7.03's shell also supported LFNs ... if
> enabled by third-party driver.)
>

Thanks for information.

in my FreeDOS RC5 FDauto.bat I inserted the following lines


c:\FreeDOS\BIN\doslfnms.com
set DIRCMD=/lfn
lfnfor on
lfnfor complete on


and it appears to work for me (i.e. both 8.3 and long file name simultaneously shown when DIR)

Question - because I am using doslfnms.com (instead of doslfn.com) should all "lfn" be replaced by "lfnms" in the last 3 lines (of the 4 lines of code shown above) to reflect the long filename program being used?






>
> But the FreeDOS kernel doesn't support bigger than 2 GB files anyways.

For my test just now, I only worked with 32K files (instead of 1 Gi file which takes about 30 minutes to load from my usb stick). For interest, it only takes about 1 second to copy FAT32 RAM to FAT32 RAM a 1 Gi file. I do not understand the problem with 4 GiB files when using FreeDOS RC5.

I copied (using RC5) the generated 4 Gi file (in FAT32 RAM) to my usb stick (it took about 2+ hours to do so) and ended up with a 4Gi file on the usb stick. Using a HEX editor (in windows) the 4 Gi appears to be OK EXCEPT for the very first byte (was changed to x1A). I did read somewhere that FAT32 allows files to be 4G-1byte (so I will keep that in mind).


Now that the LFN(ms) appears to work for me - I will try extreme testing with LFN's in FAT32.


The only reason why I tried the old lfn program was that I did not have the longer filenames displayed with doslfnms.com (which seems to be resolved by the four code lines above).

Many thanks
jadoxa

Homepage E-mail

Queensland, Australia,
09.04.2022, 12:46

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> Question - because I am using doslfnms.com (instead of doslfn.com) should
> all "lfn" be replaced by "lfnms" in the last 3 lines (of the 4 lines of
> code shown above) to reflect the long filename program being used?
No, that's not how it works. :) DOSLFNMS is built specifically for MS-DOS (at the time I believe it was the only one with the required functions).

> Now that the LFN(ms) appears to work for me - I will try extreme testing
> with LFN's in FAT32.
I was just about to release 0.41f, but since you're going to do some testing I'll hold off a bit longer. In my limited experience doslfnms.com is slower on FreeDOS, so if you notice that I'll make a doslfn.com available. (It seems the code that improves speed in MS-DOS causes a slowdown in FreeDOS.)
Richard

09.04.2022, 20:41

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

>
> I was just about to release 0.41f, but since you're going to do some
> testing I'll hold off a bit longer. In my limited experience doslfnms.com
> is slower on FreeDOS, so if you notice that I'll make a doslfn.com
> available. (It seems the code that improves speed in MS-DOS causes a
> slowdown in FreeDOS.)

I think I have a reporting protocol in place for my extreme testing of LFN with FAT32.

Below is a sample of how I wish to report back to you.

I created batch files LFN.bat and LFN2.bat for DIR C:\ and C:\FREEDOS\BIN\ respectively, which shows the start/finish times to do the DIRs, firstly for doslfnMS and then doslfn.

Are the screenshots sufficient information for you to gauge relative performance of doslfnMS with doslfn?

I was a bit lazy, so instead of writing times down, I adjusted the command prompt to show the highest resolution time.

I plan to use this format for all my extreme testing.

[image]

[image]


The batch file(s) is below

[image]




With my batch method above - I was wondering if a cache of the DIR is going to impact the second and third DIRs? i.e. 1st time for DIR would be "long", but if DIR table is now in cache, subsequent DIR of the same folder would be both shorter and the same time to do.
jadoxa

Homepage E-mail

Queensland, Australia,
10.04.2022, 04:54

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> Are the screenshots sufficient information for you to gauge relative
> performance of doslfnMS with doslfn?
If you're going to test both, then you should really be using the new version, so here it is.

> I was a bit lazy, so instead of writing times down, I adjusted the command
> prompt to show the highest resolution time.
Try rt from my Run package. Internal commands use a colon prefix: rt :dir .... Unfortunately, it writes its output to stdout, so redirecting to NUL won't work.

> With my batch method above - I was wondering if a cache of the DIR is going
> to impact the second and third DIRs? i.e. 1st time for DIR would be "long",
> but if DIR table is now in cache, subsequent DIR of the same folder would
> be both shorter and the same time to do.
Pretty much. The difference will be in disk writes. I noticed that when I extracted Japheth's test.zip each subsequent extraction was slower than the previous; the fifth was a second slower than the first. That was on the RAM drive; both versions were about the same time in MS-DOS.
Richard

10.04.2022, 11:51

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

>
>
> If you're going to test both, then you should really be using the new
> version, so here
> it is.
>
> >
>
> > With my batch method above - I was wondering if a cache of the DIR is
> going
> > to impact the second and third DIRs? i.e. 1st time for DIR would be
> "long",
> > but if DIR table is now in cache, subsequent DIR of the same folder
> would
> > be both shorter and the same time to do.
>
> Pretty much. The difference will be in disk writes. I noticed that when I
> extracted Japheth's test.zip each subsequent extraction was slower than the
> previous; the fifth was a second slower than the first. That was on the RAM
> drive; both versions were about the same time in MS-DOS.




So I think I am up to date with updates jadoxa/japheth regarding HimemSX, doslfnMS, doslfn (41f) - as of an hour ago.


I refined slightly my screenshots so that time for DIR (without lfn programs) are also included (WHITE print) to supplement doslfnMS (GREEN print) and doslfn_41f (RED print).

So the screenshots in sequence shows:-

- end part of DIR (to show how many items (files/folders))
- time to do DIR > NUL (without using lfn programs)
- (hidden is DIR of a different large folder > NUL to clear any DIR cache)
- installing doslfnMS
- time to do DIR > NUL (using doslfnMS)
- (hidden is DIR of a different large folder > NUL to clear any DIR cache)
- uninstalling doslfnMS
- installing doslfn_41f
- time to do DIR > NUL (using doslfn_41f)
- (hidden is DIR of a different large folder > NUL to clear any DIR cache)
- uninstalling doslfn_41f



and six (6) batch files created, all similar in code style, to cover the six folders

- 1.bat USB FAT16 for D:\USBFAT16\
- 2.bat USB FAT16 for D:\
- 3.bat XMS FAT32 for G:\XMS_FAT.32g\
- 4.bat XMS FAT32 for G:\
- 5.bat SXMS FAT32 for H:\SXMS_FAT.32h\
- 6.bat SXMS FAT32 for H:\





the 5.bat file is:-



cls
dir h:\sXMS_FAT.32h\
@echo OFF
dir c:\freedos\bin\ >nul
@echo ON
dir h:\sXMS_FAT.32h\ > nul
@echo OFF
dir c:\freedos\bin\ > nul
@echo ON
doslfnMS
vecho /k0/fGreen doslfnMS
dir h:\sXMS_FAT.32h\ > nul
@echo OFF
dir c:\freedos\bin\ > nul
@echo ON
vecho /a7
doslfnMS -u
doslfn
vecho /k0/fRed doslfn
dir h:\sXMS_FAT.32h\ > nul
@echo OFF
dir c:\freedos\bin\ > nul
@echo ON
vecho /a7
doslfn -u
@echo OFF
rem vecho /k0 /fGreen doslfnMS /a7
@echo ON





In the sequence 1.bat, 2.bat ...6.bat, the screenshots are:-


[image]

[image]

[image]

[image]

[image]

[image]




the respective folders for 1.bat, 2.bat ... are shown below




[image]

[image]

[image]

[image]

[image]

[image]




When doing manually a DIR with output to display, it seemed very quick (for ~ 16 or so items). However, doing output to > NUL was painfully slow for both doslfn programs (for the same ~ 16 items).


With 2.bat, an error occurred with doslfn_41f (but not doslfnMS) - I suspect it may have something to do with the 4 Gi file.see edit note at end of this reply

The (colorful) output was not consistent over all 6 batch files - I suspect a timing problem with 4.bat (XMS Root DIR of g:\).


So unless I have done something seriously wrong with the six batch files (or something silly) - it appears doslfn (all versions) have a serious timing problem with > NUL.


Any suggestions on how to do as above (i.e. with one-page screenshots) without > NUL in an automated batch file approach?


All of the above was "one particular kind of EXTREME testing of FAT32 with LFN".

I plan to do a few more.






I think I discovered the problem with 2.bat - but because this computer I am preparing this reply on is the same computer to BOOT into DOS - I will submit this reply now and edit/correct fairly soon
jadoxa

Homepage E-mail

Queensland, Australia,
10.04.2022, 14:42

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> When doing manually a DIR with output to display, it seemed very quick (for
> ~ 16 or so items). However, doing output to > NUL was painfully slow for
> both doslfn programs (for the same ~ 16 items).
This is a problem with DIR itself: it does an SFN find, then converts the short to long, so DOSLFN has to find it again. Still, about 1.5 minutes seems excessive. Here's another set of binaries with the cache enabled (it was disabled because it doesn't detect disc changes and didn't seem to impact performance).
Richard

10.04.2022, 15:43

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > When doing manually a DIR with output to display, it seemed very quick
> (for
> > ~ 16 or so items). However, doing output to > NUL was painfully slow for
> > both doslfn programs (for the same ~ 16 items).
>
> This is a problem with DIR itself: it does an SFN find, then converts the
> short to long, so DOSLFN has to find it again. Still, about 1.5 minutes
> seems excessive.
> Here's another
> set of binaries with the cache enabled (it was disabled because it doesn't
> detect disc changes and didn't seem to impact performance).


Using doslfn-c set and same batch files/folders as above, in sequence have:-


[image]


[image]


[image]


[image]


[image]


[image]
tom

Homepage

Germany (West),
10.04.2022, 22:08

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > When doing manually a DIR with output to display, it seemed very quick
> (for
> > ~ 16 or so items). However, doing output to > NUL was painfully slow for
> > both doslfn programs (for the same ~ 16 items).
>
> This is a problem with DIR itself: it does an SFN find, then converts the
> short to long, so DOSLFN has to find it again.

Nope. As I am the author of the LFN directory listing: directory listings with and without LFN were 'just fast enough', and I had never a reason to care about the difference. It might be 2x or 3x slower, but nothing relevant.

something strange is going on on his machine.
Richard

11.04.2022, 04:51

@ tom
 

about to fix the "final" doslfn bug

Post reply

>
>
> something strange is going on on his machine.
 >
>


Maybe so.


There is also the possibility that issues MAY be occurring due to the fact that I am booting off a (slow) USB stick in FREEDOS RC5. I am getting indications that FREEDOS can behave differently in certain situations.


It would be helpful if someone(s) can give precise pointers on how to have alternative (to FREEDOS). For instance, I tried to download just the files only for MSDOS 7.1 to do common things (getting confused which are internal/external commands) - but no luck with usable downloads.

What I am aiming for is for example:-

If (today only say) I was not happy with FREEDOS RC5 DIR command (when using FAT32 Ramdrives) then (for today only) I would use MSDOS71 DIR (internal) command instead. I don't know - maybe have 5x "different command.com" installed, if that makes any sense?


Question - how far can I go with "Mixing and matching" FREEDOS RC5 stuff with "anything else out there"? So on a case by case basis, I would choose which works best for me. And of course, where exactly to source the alternatives to FREEDOS (so many "dead" sites out there).


Question - can I use 4DOS stuff at all (on the same USB stick as FREEDOS)?

Question - other alternatives to FREEDOS and where to get?



My machine is a HP zBOOK 15 G2 i7 quad processor with 3200x1800 display and 32 GiByte RAM. The cpu model i7 4810MQ. It was manufactured in 2015, purchased second-hand (but was "mint" condition - as if it was never touched!) - it seems that only the original packaging and any manuals was missing. My model has been superseded by almost yearly model upgrades (up to G7 or G8 now). Windows 10 pro was already installed.
tom

Homepage

Germany (West),
10.04.2022, 17:45

@ Richard
 

about to fix the "final" doslfn bug

Post reply

something stupid is going on here:
doslfn and doslfnMS are about equally 'fast': the listing takes forever

100 seconds for a directory listing of 14 files is nonsense.
and that the times are similar for USB and xmsdisk doesn't sound right.

can you also report the time without any doslfn?
Richard

10.04.2022, 18:44

@ tom
 

about to fix the "final" doslfn bug

Post reply

> something stupid is going on here:
> doslfn and doslfnMS are about equally 'fast': the listing takes forever
>
> 100 seconds for a directory listing of 14 files is nonsense.
> and that the times are similar for USB and xmsdisk doesn't sound right.
>
> can you also report the time without any doslfn?

I re-ran the tests using doslfn-c and doslfnms-c version (the 100 seconds timings you are referring to were using doslfn-b and doslfnms-b versions). The results using the "-c" version set are in the reply immediately before your post here.

I tried to automate the timings in a set of batch files (1.bat, 2.bat ...6.bat) where all batch files are similar (only differing by file location, eg USB stick (FAT16), or XMS FAT32 or SXMS FAT32 (and either ROOT directory or a 1st level sub-directory). The batch file 5.bat is representative of the six batch files and is included with the 100-second timings reply of mine. The exact same batch files and the folders tested for timings were used in both instances (of versions -b and -c doslfn/doslfnms).

Please feel free to critically analyze my batch files (e.g. 5.bat) - I tried to prevent any possible DIR table/list caching interfering with true timings. I wanted to avoid rebooting the computer for every timing aspect (in this case a total of 18 times - 6 each of non-doslfn(ms), doslfnms, and doslfn).


Note that with my latest posting (with -c versions of doslfn and doslfnms) the timings are "color-coded":-

WHITE for plain DIR (i.e no doslfn or doslfnms used) - and is just after the end part of the displayed DIR at the top of each screen capture

GREEN for doslfnms (version -c)

RED for doslfn (version -c = version 41f latest update)
tom

Homepage

Germany (West),
10.04.2022, 19:51

@ Richard
 

about to fix the "final" doslfn bug

Post reply

your posting 11:51 :


screenshot 1: without doslfn*

DIR D:\USBFAT16 >NUL
takes 15 seconds, which is nonsense. whatever this is measuring, it's something else than you think you are measuring.

doslfn may be inefficient, but it's not probably THAT inefficient. something else is happening.



although I find it pretty stupid that the DIR's seem to be constantly changing:

your 15:43 screenshots vary between 14, 10, 16, 5, 16, 11 files respectively.

better testers are required:-(
Richard

11.04.2022, 04:05

@ tom
 

about to fix the "final" doslfn bug

Post reply

> your posting 11:51 :
>
>
> screenshot 1: without doslfn*
>
> DIR D:\USBFAT16 >NUL
>
> takes 15 seconds, which is nonsense. whatever this is measuring, it's
> something else than you think you are measuring.
>
> doslfn may be inefficient, but it's not probably THAT inefficient.
> something else is happening.
>
>
>
> although I find it pretty stupid that the DIR's seem to be constantly
> changing:
>
> your 15:43 screenshots vary between 14, 10, 16, 5, 16, 11 files
> respectively.
>
> better testers are required:-(

Thank you for your reply - it does help me when someone actually says something when something is wrong or questionable.




> your posting 11:51 :
>
>
> screenshot 1: without doslfn*
>


Relevant part of screenshot


[image]




>
> DIR D:\USBFAT16 >NUL
>
> takes 15 seconds, which is nonsense. whatever this is measuring, it's
> something else than you think you are measuring.
>


The DOS-PROMPT timestamps gives an elapsed time of

5:59:44.45
-
5:59:49.67
=
0:00:05.22 (HH:MM:SS.fraction of seconds)

So the duration is ~5 seconds (not 15 seconds - typo maybe?)

Do not know if this is nonsense, a fairly cheap "K-MART" Verbatim USB stick used which is not USB3 speed probably only USB2 or even USB1 speed).





>
> although I find it pretty stupid that the DIR's seem to be constantly
> changing:
>
> your 15:43 screenshots vary between 14, 10, 16, 5, 16, 11 files
> respectively.
 >

Yes I agree that the EXACT same files/directories set is not used in all cases. To be screen capture compatible (not missing anything) - I am now standardizing on (9 files + 5 folders) for "everything" (as far as "simple and quick" testing schemes go). The 14 and 16 variation due to "local" identifier files with the idea that they would be visible in the last lines of the DIR shown in the screen capture.



>
> better testers are required:-(

Yes, I agree - I am gaining some experience in trying to design "testing situations" for LFN - well at least I am trying to contribute in the testing of doslfnMS.


Further "reports" by me will be more complex (messy) - I am aiming for EXTREME cases - just working with 9 files with 5 directories is a fairly trivial test set.


Many thanks for your input.
jadoxa

Homepage E-mail

Queensland, Australia,
11.04.2022, 05:05

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> So the duration is ~5 seconds (not 15 seconds - typo maybe?)
That's still about 4.9 seconds longer than it should be. I tried all those trace names in a 382MiB RAM drive with DOSLFN and that was instant (under VirtualBox). This is the batch file I used (DOSLFN is the -c variant; TRACE is 2.03).


@echo off
f:rdrvx32 382
md g:\xms_fat.32g
cd g:\xms_fat.32g
f:doslfn
copy f:trace.com g:trace.asm
copy f:trace.com g:trace.com
copy f:trace.com "g:trace202 (2).co)"
copy f:trace.com "g:trace202 (3).co)"
copy f:trace.com "g:trace202 (4).co)"
copy f:trace.com "g:trace202 (5).co)"
copy f:trace.com "g:trace202 (6).co)"
copy f:trace.com "g:trace202 (7).co)"
copy f:trace.com "g:trace202 (8).co)"
copy f:trace.com "g:trace202 (9).co)"
copy f:trace.com "g:trace202 (10).co)"
copy f:trace.com "g:trace202.co)"
copy f:trace.com "g:trace203 (2).asm"
copy f:trace.com "g:trace203 (2).com"
copy nul g:z_0123456789ABCDEF
copy nul g:z_XMSFAT.32G
Richard

11.04.2022, 07:58
(edited by Richard, 11.04.2022, 08:09)

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > So the duration is ~5 seconds (not 15 seconds - typo maybe?)
>
> That's still about 4.9 seconds longer than it should be. I tried all those
> trace names in a 382MiB RAM drive with DOSLFN and that was instant (under
> VirtualBox). This is the batch file I used (DOSLFN is the -c variant; TRACE
> is 2.03).
>
>
> @echo off
> f:rdrvx32 382
> md g:\xms_fat.32g
> cd g:\xms_fat.32g
> f:doslfn
> copy f:trace.com g:trace.asm
> copy f:trace.com g:trace.com
> copy f:trace.com "g:trace202 (2).co)"
> copy f:trace.com "g:trace202 (3).co)"
> copy f:trace.com "g:trace202 (4).co)"
> copy f:trace.com "g:trace202 (5).co)"
> copy f:trace.com "g:trace202 (6).co)"
> copy f:trace.com "g:trace202 (7).co)"
> copy f:trace.com "g:trace202 (8).co)"
> copy f:trace.com "g:trace202 (9).co)"
> copy f:trace.com "g:trace202 (10).co)"
> copy f:trace.com "g:trace202.co)"
> copy f:trace.com "g:trace203 (2).asm"
> copy f:trace.com "g:trace203 (2).com"
> copy nul g:z_0123456789ABCDEF
> copy nul g:z_XMSFAT.32G
>





Help please.



I had issues with your batch file in FREEDOS RC5. So, probably for the wrong reasons, I modified it (but with the general theme in mind).


Below is the J1a.bat file I tried to run


cls
rem J1a.bat running ... (2022 APR 11)
rem .
rem FDauto.bat reduced down (no lfn etc stuff)
rem c:\freedos\bin\HimemSX.exe 6,653 bytes 4th April
rem c:\freedos\bin\doslfn.com -c variant 22,719 bytes 04April
rem c:\freedos\bin\rdvx32.exe 3,841 bytes 28 March
rem e:\trace.com 5,679 bytes (trace203.com) 2nd April
rem f:\ reserved for DVD player
rem .

@echo off
C:\freedos\bin\rdrvx32 382
md g:\xms_fat.32g
cd g:\xms_fat.32g
C:\freedos\bin\doslfn
copy e:\trace203.com g:\trace.asm
copy e:\trace203.com g:\trace203.com
copy e:\trace203.com g:\trace202 (2).co)"
copy e:\trace203.com g:\trace202 (3).co)"
copy e:\trace203.com g:\trace202 (4).co)"
copy e:\trace203.com g:\trace202 (5).co)"
copy e:\trace203.com g:\trace202 (6).co)"
copy e:\trace203.com g:\trace202 (7).co)"
copy e:\trace203.com g:\trace202 (8).co)"
copy e:\trace203.com g:\trace202 (9).co)"
copy e:\trace203.com g:\trace202 (10).co)"
copy e:\trace203.com g:\trace202.co)"
copy e:\trace203.com g:\trace203 (2).asm"
copy e:\trace203.com g:\trace203 (2).com"
copy c:\nulll.txt g:\z_0123456789ABCDEF
copy c:\nulll.txt g:\z_XMSFAT.32G
@echo ON




and the output in time sequence as follows:-

[image]

[image]

[image]






EDIT - I just realized maybe I need double quotes (") around the destination because of spaces in the destination filename
jadoxa

Homepage E-mail

Queensland, Australia,
11.04.2022, 08:47

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> rem FDauto.bat reduced down (no lfn etc stuff)
> rem c:\freedos\bin\doslfn.com -c variant 22,719 bytes 04April
> rem c:\freedos\bin\rdvx32.exe 3,841 bytes 28 March
I don't have these in FDauto.bat, which is why they're in this batch. F: is the virtual CD (C, D & E are VM drives), so I simply assume G: is the RAM disk.

> cd g:\xms_fat.32g
> copy e:\trace203.com g:\trace.asm
Do you want root (g:\) or XMS_FAT.32G (g:)?

> EDIT - I just realized maybe I need double quotes (") around the
> destination because of spaces in the destination filename

Absolutely.
Richard

11.04.2022, 11:20

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > rem FDauto.bat reduced down (no lfn etc stuff)
> > rem c:\freedos\bin\doslfn.com -c variant 22,719 bytes 04April
> > rem c:\freedos\bin\rdvx32.exe 3,841 bytes 28 March
>
> I don't have these in FDauto.bat, which is why they're in this batch. F:
> is the virtual CD (C, D & E are VM drives), so I simply assume G: is the
> RAM disk.
>
> > cd g:\xms_fat.32g
> > copy e:\trace203.com g:\trace.asm
>
> Do you want root (g:\) or XMS_FAT.32G (g:)?
>
> > EDIT - I just realized maybe I need double quotes (") around the
> > destination because of spaces in the destination filename
>
> Absolutely.


So I made a J1c.bat (shown below) and things are starting to appear to be WORKING CORRECTLY (I think). Interestingly, it appears that @echo OFF does not behave the way I thought it should - I ended up putting " > NUL" at the end of every copy line. The batch file is "messy", but I think I now have a "correct and accurate" batch file as a reference (helps to know how long things should actually take to do).


cls
rem J1c.bat running ... (2022 APR 11)
rem .
rem c:\freedos\bin\HimemSX.exe 6,653 bytes 4th April
rem c:\freedos\bin\doslfn.com -c variant 22,719 bytes 04April
rem c:\freedos\bin\rdvx32.exe 3,841 bytes 28 March
rem e:\trace.com 5,679 bytes (trace203.com) 2nd April
rem f:\ reserved for DVD player

@ECHO OFF
C:\freedos\bin\rdrvx32 -u > NUL
C:\freedos\bin\rdrvx32 382M
rem del g:\*.*
rem del g:\xms_fat.32g\*.*
C:\freedos\bin\doslfn

copy e:\trace203.com g:\trace.asm > nul
copy e:\trace203.com g:\trace203.com > nul
copy e:\trace203.com "g:\trace202 (2).co)" > nul
copy e:\trace203.com "g:\trace202 (3).co)" > nul
copy e:\trace203.com "g:\trace202 (4).co)" > nul
copy e:\trace203.com "g:\trace202 (5).co)" > nul
copy e:\trace203.com "g:\trace202 (6).co)" > nul
copy e:\trace203.com "g:\trace202 (7).co)" > nul
copy e:\trace203.com "g:\trace202 (8).co)" > nul
copy e:\trace203.com "g:\trace202 (9).co)" > nul
copy e:\trace203.com "g:\trace202 (10).co)" > nul
copy e:\trace203.com "g:\trace202.co)" > nul
copy e:\trace203.com "g:\trace203 (2).asm" > nul
copy e:\trace203.com "g:\trace203 (2).com" > nul
copy c:\nulll.txt g:\z_0123456789ABCDEF > nul
copy c:\nulll.txt g:\z_XMSFAT.32G > nul

md g:\xms_fat.32g
cd g:\xms_fat.32g
copy g:\*.* g:\XMS_FAT.32g\ > nul
@echo ON

echo .
dir g:\ > NUL
echo .
dir G:\XMS_FAT.32g\ > NUL
echo .



[image]

[image]

[image]
jadoxa

Homepage E-mail

Queensland, Australia,
11.04.2022, 12:05

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> Interestingly, it appears that @echo OFF does
> not behave the way I thought it should
It only prevents commands from being echoed, the command's output still occurs as normal.

> C:\freedos\bin\rdrvx32 -u > NUL
You could use -qq here.

> dir g:\ > NUL
> echo .
> dir G:\XMS_FAT.32g\ > NUL
That looks to be under a second. Does it get slower as the drive gets bigger?
Richard

12.04.2022, 06:03

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

>
> That looks to be under a second. Does it get slower as the drive gets
> bigger?

Will get to bigger drives very soon.

Meanwhile below is my best attempt to batch file automate the timing process to do a DIR ... > NUL for

(WHITE) without lfn installed (value pairs to determine elapsed time)
(GREEN) with doslfnMS (-c variant = latest 41f update)
(RED) with doslfn (-c variant = latest 41f update)

A (File+Directory) set of (10+4) was used for the possible combinations of:-

ROOT/1stLevelsubDirectory FOLDERS
USBFAT16/USBFAT32
XMSFAT32/SMSFAT32
XMSFAT16/SMSFAT16

Note I use the notation of SMS to designate SuperextendedMemory rather than SXMS notation.

To cover the 12 possibilities of above - 12 .bat files were created (1.bat, 2.bat,...9.bat,a.bat,b.bat,c.bat) and each .bat automatically sets up a minimal screen showing timings for with and without LFN support. Note that (hex) even .bat files are the respective ROOT FOLDERS, odd are 1st level sub-directories.

Screen CAPTURES (for timings)

1.bat D:\USB_FAT.32d\ = FAT32 USBstick
[image]



2.bat D:\ ROOT = FAT32 USBstick
[image]




3.bat G:\XMS_FAT.32g\ = XMS FAT32
[image]



4.bat G:\ ROOT = XMS FAT32
[image]




5.bat H:\SMS_FAT.32h\ = SMS FAT32
[image]




6.bat H:\ ROOT = SMS FAAT32
[image]




7.bat E:\USB_FAT.16e\ = FAT16 USBstick
[image]




8.bat E:\ ROOT = FAT16 USBstick
[image]




9.bat I:\XMS_FAT16\ = XMS FAT16
[image]




a.bat I:\ ROOT = XMS FAT16
[image]




b.bat J:\SMS_FAT.16j\ = SMS FAT16
[image]




c.bat J:\ ROOT = SMS FAT 16
[image]










the respective screenshots for the 12 folders (for confirmation purposes to establish test conditions similarity) are below - all should be (10 Files + 4 Directories)


[image]

[image]

[image]

[image]

[image]

[image]

[image]

[image]

[image]

[image]

[image]

[image]
jadoxa

Homepage E-mail

Queensland, Australia,
12.04.2022, 10:34

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> 9.bat I:\XMS_FAT16\ = XMS FAT16
>
> 12:28:45.90  C:\FREEDOS\BIN #echo .
> .
> 12:28:45.37  C:\FREEDOS\BIN #dir i:\XMS_FAT16.i\ > nul

What happened there? That makes me mistrust all the other times. I've updated RT, adding -w to write to stderr. That will let you redirect DIR to nul and still see the timing (use -2). E.g. here's a test I made to compare DIR against FOR:

C:\>lfntest.bat
C:\>lfnfor off
C:\>set dircmd=
C:\>rt -2 -w :dir fdos\bin\*.* > nul

Elapsed Time: 0.010 seconds (10 ticks).
C:\>rt -2 -w :for %j in (fdos\bin\*.*) do echo %j > nul

Elapsed Time: 0.014 seconds (14 ticks).
C:\>f:doslfn
DOSLFN 0.41f (haftmann#software & jmh 4/2022): loaded consuming 13920 bytes.
C:\>rt -2 -w :dir /lfn fdos\bin\*.* > nul

Elapsed Time: 0.083 seconds (85 ticks).
C:\>lfnfor on
C:\>rt -2 -w :for %j in (fdos\bin\*.*) do echo %j > nul

Elapsed Time: 0.016 seconds (16 ticks).
C:\>f:doslfn u
DOSLFN 0.41f (haftmann#software & jmh 4/2022): removed from memory.
Richard

12.04.2022, 11:28

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > 9.bat I:\XMS_FAT16\ = XMS FAT16
> >
> > 12:28:45.90  C:\FREEDOS\BIN #echo .
> > .
> > 12:28:45.37  C:\FREEDOS\BIN #dir i:\XMS_FAT16.i\ > nul
>
> What happened there? That makes me mistrust all the other times.

Good question.


After "sighting" thousands of DOS-PROMPT #'s in my testing and developing of the .bat files - I missed that (sorry about that) - I never thought it was even possible that the DOS-PROMPT could ever be in error! :(


I can certainly rerun everything again ( and many times) and also many times just 9.bat - which I can do very soon (I will just delay the bigger drives run, to have many more than 10 files + 4 directories).



I am just guessing here - I used ST for the screen capture - and if there was a "glitch" in the process during the actual screen capture (of the text DOS screen), of ONE BYTE for the ascii attribute (i.e. the "9" was misread say) - then ...


During the course of the test runs, the previous say 2 runs (1.bat....c.bat) there were interruptions to the test flow - and I had to re-boot mid run. Maybe the FREEDOS RC5 batch file processing (internal) program is buggy (or does not like my choice of colors).


How many times would you like me to re-run the whole set and how many times just only for 9.bat?


Thanks for pointing that anomaly out!


I will try out your RT material (never used anything like that before) - hopefully smoothly integrate into my .bat files.

I noticed that from Japheth some updates to relevant files - so will also try to change things to suit. By the way, with the JEMM side of things, am I myself suppose to make/build everything - I thought that everything would have been precompiled, etc? I suppose I now have to learn and use (for the first time) JWASM.


A quick question. I am considering implementing SMARTDRV as a permanent automatic feature. Is it a FAT16 "drive" (is there a FAT32 equivalent) - in either case how LARGE a SMARTDRV can I go. Like could I have a 16 GiB smartdrive to "cache" my 16 GB USB stick (if I wanted to go that far)?


When copying from RAMDRIVE FAT32 to USB, it takes about 2+ hours to copy one 4 GIB file with FREEDOS RC5 "copy" command. I did a general enquiry on this thread earlier ( I think to Tom) on how/where to source alternatives to FreeDOS so that I could on a case by case basis select between FREEDOS and alternatives - but no one has replied on that yet.
Japheth

Homepage

Germany (South),
12.04.2022, 12:35

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> I noticed that from Japheth some updates to relevant files - so will also
> try to change things to suit. By the way, with the JEMM side of things, am
> I myself suppose to make/build everything - I thought that everything would
> have been precompiled, etc? I suppose I now have to learn and use (for the
> first time) JWASM.
There are precompiled binaries available at github - no need to fiddle about with jwasm.

> A quick question. I am considering implementing SMARTDRV as a permanent
> automatic feature. Is it a FAT16 "drive" (is there a FAT32 equivalent) - in
> either case how LARGE a SMARTDRV can I go. Like could I have a 16 GiB
> smartdrive to "cache" my 16 GB USB stick (if I wanted to go that far)?
Smartdrv is a disk-cache and no "drive" - and its max is 32MB. I use it with 8MB. Smartdrv does speed up doslfn significantly. Be sure to load it AFTER doslfn.

> When copying from RAMDRIVE FAT32 to USB, it takes about 2+ hours to copy
> one 4 GIB file with FREEDOS RC5 "copy" command. I did a general enquiry on
> this thread earlier ( I think to Tom) on how/where to source alternatives
> to FreeDOS so that I could on a case by case basis select between FREEDOS
> and alternatives - but no one has replied on that yet.
Well, I use DOS 7, HimemSX, Doslfnms and Smartdrv and have no speed problems at all.

---
MS-DOS forever!
Richard

12.04.2022, 16:28

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

>
>
> Smartdrv is a disk-cache and no "drive" - and its max is 32MB. I use it
> with 8MB. Smartdrv does speed up doslfn significantly. Be sure to load it
> AFTER doslfn.
>
>


Question - when running in a continuous DOS session (no re-boots) and I have hundreds of install/uninstall doslfn(MS) - do I have to reload smartdrv again with each doslfn. That is, once smartdrv is installed after the first doslfn install, will it continue to work with all subsequent doslfn installs?
Japheth

Homepage

Germany (South),
12.04.2022, 17:45

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> Question - when running in a continuous DOS session (no re-boots) and I
> have hundreds of install/uninstall doslfn(MS) - do I have to reload
> smartdrv again with each doslfn. That is, once smartdrv is installed after
> the first doslfn install, will it continue to work with all subsequent
> doslfn installs?
I'm pretty sure that you can't unload doslfn after smartdrv has been loaded.

---
MS-DOS forever!
Richard

13.04.2022, 09:58

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

>
>
> Smartdrv is a disk-cache and no "drive" - and its max is 32MB. I use it
> with 8MB. Smartdrv does speed up doslfn significantly. Be sure to load it
> AFTER doslfn.
>
>

> he should start without CONFIG.SYS and AUTOEXEC.BAT (and the guarantee that no > offenders are lurking in his system)

Well, based on Tom's important suggestion that start without CONFIG.SYS/AUTOEXEC.BAT - in my case FDCONFIG.SYS/FDAUTO.BAT because I am using FreeDOS - I am building up, step by step, a workable minimal system.


To even just DEVLOAD HIMEMSX, in fact without the extension .exe I get an error message, I had to overcome the new error message

Error: free drive letter not found, increase LASTDRIVE

Apparently at the DOS PROMPT it seems, I cannot set up last drive - so I created a "new"


config.sys

!LASTDRIVE = Z
DEVLOAD himemSX.exe ' version 6,653 bytes 10 April



So I also created a "new" autoexec.bat (to save a lot of future typing) with code lines:-

RDRVSX32.exe 4 'latest version 4,016 bytes 12 April for 4 GB SXMS RAM Drive
SMARTDRV.exe A B C D E F G 32768 4096 512 '45,379 bytes 12 April (7 bytes)
doslfnMS.com ' -c variant 21,799 bytes 10 April


No matter which sequence doslfnMS/Smartdrv I have, and if doslfnMS before/after RDRVSX32 - I do not get LFN's with DIR

My smartdrv (32M) does "see" drives "A to G" and my USB stick has drives C,D and E.




Edit -Q for HIMEMSX.exe does not work
jadoxa

Homepage E-mail

Queensland, Australia,
13.04.2022, 10:33

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> DEVLOAD himemSX.exe ' version 6,653 bytes 10 April
If it's in config.sys using DEVICE= would be better.

> RDRVSX32.exe 4 'latest version 4,016 bytes 12 April for 4 GB SXMS RAM Drive
> SMARTDRV.exe A B C D E F G 32768 4096 512 '45,379 bytes 12 April (7 bytes)
No point caching a RAM drive.

> No matter which sequence doslfnMS/Smartdrv I have, and if doslfnMS
> before/after RDRVSX32 - I do not get LFN's with DIR
Did you remember to add /LFN?

> Edit -Q for HIMEMSX.exe does not work
Since the help makes no mention of it, why did you think it would?
Richard

13.04.2022, 11:07

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > DEVLOAD himemSX.exe ' version 6,653 bytes 10 April
>
> No point caching a RAM drive.
>
> > No matter which sequence doslfnMS/Smartdrv I have, and if doslfnMS
> > before/after RDRVSX32 - I do not get LFN's with DIR
>
> Did you remember to add /LFN?
>
> > Edit -Q for HIMEMSX.exe does not work
>
> Since the help makes no mention of it, why did you think it would?


I just tried putting in the lines

SET DIRCMD=/lfn
lfnFOR on
lfnFOR complete on



and now doslfnMS DOES SHOW LFN's with sequence RDRVSX32, SMARTDRV, DOSlfnMS(with SET DIRCMD=/lfn etc). I could not edit my post in time before your reply. SORRY.


Yes I did have to insert those three lines before (ONCE only) and because of maybe a thousand install/uninstall doslfn(MS) that did not have it directly (but was in the FDauto.bat) I forgot about it in my "new" autoexec.bat.




> No point caching a RAM drive.
 >

In my retesting of (doslfn vs doslfnMS) I should be able to verify your point.




Any pointers on how much to have for elementsize (how large can I go) and similarly for buffersize?

When I do a DIR of C:\ say, with a large enough buffer size, would smartdrive load up ALL directories/subdirectories tables even if I am only doing DIR C:\?
Richard

14.04.2022, 02:19

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

>
> Well, I use DOS 7, HimemSX, Doslfnms and Smartdrv and have no speed
> problems at all.


I did not see a switch to allow me to unload SMARTDRV - I do not think /R (clears the cache and restarts SMARTdrive) is the same thing.


The reason why I want to uninstall SMARTDRV is that I think I will have to try hundreds of combinations for element/buffer size (which would only be valid for my system) to attempt to get near optimal settings.

I would prefer to uninstall rather than reboot hundreds of times.


Is it possible to add in code to uninstall? I had used a HEX editor to change the required 7 bytes as per updates.


I came across the GitHub FDOS site - and there are a number of cache programs. I thought I saw one that can be uninstalled. Even maybe not as good as the MS version - but if I assume that the element/buffer size codes are similar, this might help me with that optimization. I even saw one that can cache size go larger than 32M.

I am assuming that running cache at 32M has no detrimental effects on file transfer speedswhen compared to say 8M.
Japheth

Homepage

Germany (South),
14.04.2022, 08:34

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> I would prefer to uninstall rather than reboot hundreds of times.
AFAIK smartdrv cannot be uninstalled.

> I came across the GitHub FDOS site - and there are a number of cache
> programs. I thought I saw one that can be uninstalled. Even maybe not as
> good as the MS version - but if I assume that the element/buffer size codes
> are similar, this might help me with that optimization. I even saw one that
> can cache size go larger than 32M.
Bigger is not always better. Smartdrv caches write accesses, something that the simpler comrades won't do.

---
MS-DOS forever!
tom

Homepage

Germany (West),
14.04.2022, 11:42

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> I am assuming that running cache at 32M has no detrimental effects on file
> transfer speedswhen compared to say 8M.
if you want a system with fast file transfer, you should measure file tranfer with hundreds of different settings.

currently you are mostly measuring the performance of "translate SFN to LFN".

sure, if you want the best possible performance of

C:>DIR G:\ > NUL

continue with your efforts. but otherwise, your efforts are kind of wasted time...
jadoxa

Homepage E-mail

Queensland, Australia,
12.04.2022, 16:22

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> I am just guessing here - I used ST for the screen capture - and if there
> was a "glitch" in the process during the actual screen capture (of the text
> DOS screen), of ONE BYTE for the ascii attribute (i.e. the "9" was misread
> say) - then ...
Seems pretty unlikely. But then, that time is unlikely, too.

> How many times would you like me to re-run the whole set and how many times
> just only for 9.bat?
Once (with rt) should be enough, since it's only a relative test. The real question is why it's so slow (the RAM drive in particular).

> I did a general enquiry on
> this thread earlier ( I think to Tom) on how/where to source alternatives
> to FreeDOS so that I could on a case by case basis select between FREEDOS
> and alternatives - but no one has replied on that yet.
Maybe it would help if we knew what you were actually wanting to do? 64-bit Linux might be the better choice...


I've updated RDRVSX32 (remember that? :) ) to search the parent environment for DL=? and replace the ? with the first assigned drive letter. If there's no problems with that I'll update the SHSU programs to use it.
Japheth

Homepage

Germany (South),
12.04.2022, 17:37

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> The real question is why it's so slow (the RAM drive in particular).
Perhaps it's not related to doslfn at all. It might be a BIOS bug, setting up memory beyond 4GB as "uncachable" via MTRRs. I remember that, when HimemSX was introduced, another member of this forum (Zyzzle?) had such problems.

---
MS-DOS forever!
tom

Homepage

Germany (West),
12.04.2022, 19:18

@ Japheth
 

thoughts about systematic testing

Post reply

> > The real question is why it's so slow (the RAM drive in particular).
>
> Perhaps it's not related to doslfn at all. It might be a BIOS bug, setting
> up memory beyond 4GB as "uncachable" via MTRRs. I remember that, when
> HimemSX was introduced, another member of this forum (Zyzzle?) had such
> problems.
he should start without CONFIG.SYS and AUTOEXEC.BAT (and the guarantee that no offenders are lurking in his system)

compare apples to apples.
have a batchfile to *consistently* create a directory with known size(see below) and known content. size, because a directory that once held Japhets 10000 files and at least 20000 directories will never shrink again.
huge directories are a problem for the way COMMAND generates long filenames.
when iterating over short filenames DOS knows where exactly to look for the next filename, and the sector is most often already in a buffer. to translate this short name into a long name it always starts at the beginning of the directory, and iterates on average half of the entire directory (which is clearly suboptimal).




now see what the times are for the USB drive, and if they are significantly different from WITH config/autoexec. if so, find the culprit.

then compare himemX (4 GB only) and himemsx (but also 4 GB).
are there significant differences? there shouldn't, but it's possible.

compare doslfn and doslfnMS. there shouldn't be a difference.

last, but not least: why is this interesting at all? in all cases, real people are looking longer at the directory listing than it took to print it. even on a IBM PC with 360K floppies.
Zyzzle

13.04.2022, 10:27

@ Japheth
 

about to fix the "final" doslfn bug

Post reply

> > The real question is why it's so slow (the RAM drive in particular).
>
> Perhaps it's not related to doslfn at all. It might be a BIOS bug, setting
> up memory beyond 4GB as "uncachable" via MTRRs. I remember that, when
> HimemSX was introduced, another member of this forum (Zyzzle?) had such
> problems.
Yes, that is correct. Some of the memory beyond 4GB is "uncacheable" via MTRRs and only showing a 50 mb/sec transfer speed on an i5-8250 system. That system doesn't enable MTRRs for some reason, and it slowed memory transfers down by a factor of about 500x. No system that I've tested above a 5th Gen core i5 system has successfully cached memory in MTRRs. Limitation of Intel's? BIOS limitation? I can't figure why this feature has been disabled or crippled.

When HimemSX was tested on a system which does cache memory > 4GB via MTRRs, the memory speed was something around 10 GB / sec or greater on an core i5 5600 laptop. This laptop would also properly cache MTRRs via RayeR's MTRRLFBE utility while my core i5-8250 more recent laptop doesn't (executing MTRRLFBE wc lfb freezes the system).
Richard

13.04.2022, 11:24

@ Zyzzle
 

about to fix the "final" doslfn bug

Post reply

> > > The real question is why it's so slow (the RAM drive in particular).
> >
> > Perhaps it's not related to doslfn at all. It might be a BIOS bug,
> setting
> > up memory beyond 4GB as "uncachable" via MTRRs. I remember that, when
> > HimemSX was introduced, another member of this forum (Zyzzle?) had such
> > problems.
>
> Yes, that is correct. Some of the memory beyond 4GB is "uncacheable" via
> MTRRs and only showing a 50 mb/sec transfer speed on an i5-8250 system.
> That system doesn't enable MTRRs for some reason, and it slowed memory
> transfers down by a factor of about 500x. No system that I've tested above
> a 5th Gen core i5 system has successfully cached memory in MTRRs.
> Limitation of Intel's? BIOS limitation? I can't figure why this feature has
> been disabled or crippled.
>
> When HimemSX was tested on a system which does cache memory > 4GB via
> MTRRs, the memory speed was something around 10 GB / sec or greater on an
> core i5 5600 laptop. This laptop would also properly cache MTRRs via
> RayeR's MTRRLFBE utility while my core i5-8250 more recent laptop doesn't
> (executing MTRRLFBE wc lfb freezes the system).


I am willing to try same on my HP laptop with i7-4810MQ 32GB RAM (laptop manufactured 2015 and latest BIOS updates installed).

Where to get, how to use MTRRLFBE?
Richard

13.04.2022, 16:56
(edited by Richard, 14.04.2022, 02:25)

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

>
>
> I've updated
> RDRVSX32
> (remember that? :) ) to search the parent environment for DL=?
> and replace the ? with the first assigned drive letter. If
> there's no problems with that I'll update the SHSU programs to use it.


It may be the wrong reply to respond to, but...


Although I should really be testing in a "bare-bones" CONFIG.SYS/AUTOEXEC.BAT (as per Tom's suggestion) - I was eager to apply SMARTDRV to my "real life" (???) testing and so temporarily swapped back to FDCONFIG.sys/FDAUTO.BAT.

Once the SMARTDRV cache was preloaded from the same DIR command, on the second time when displayed a folder of some ~500 items, the display lines "whizzed by" - it seemed to only take about 2 secs. When on third DIR but with > NUL, it was practically instantaneous. ALL THIS for doslfnMS (-c variant) and the LFN's shown.


HOWEVER, on running 1.bat, get error message (after successfully displaying doslfnMS timings)


Invalid opcode at 252f 32b0 0002 0462 0075 e02b 008c 53f1 36f6 0007 32b06680 0a44



On investigating further, when ever I have doslfnMS enabled and I do doslfnMS u, get the message

Another TSR grabbed Int 21 and/or Int 2F

I wonder if my screen capture programs are responsible.

The system is still responsive, but it appears the uninstall worked as no more LFN with DIR.



HOWEVER, when I now doslfn (latest -c variant) (note all these steps in sequence are in 1.bat which is for 1st level sub-directory on USB D:\ Fat 32), the output is...


DOSLFN 0.41f (....):enabled
Last error:94 - inDOS bayra ...kullanimini SIFIRLA
Invalid Opcode at 1f7e 32b0 0287 0000 01ce fff8 ff67 0002 df8d 0001 ea93 0002 eaa9
Cannot terminate permanent FreeCOM instance
System halted ... reboot or power off now


That error message (:94) is the same as per post (09.04.2022 01:11 ServerTime + 0)



Apart from the error messages/lockups - without yet doing timing comparisons - using SMARTDRV made a noticeable improvement in speed (it even looked like as if a 4x speed improvement for a 128Mbyte file - but made a 4 GiB file transfer (USB --> SXMS FAT32) much worse than before. I might have to experiment with element/buffer sizes (going for multi KB sizes may not be optimal for my system).

Still yet to try out RT.






EDIT Above results with doslfnMS then SMARTDRV - will also try order swapped

jadoxa

Homepage E-mail

Queensland, Australia,
14.04.2022, 15:23

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> On investigating further, when ever I have doslfnMS enabled and I do
> doslfnMS u, get the message
>
> Another TSR grabbed Int 21 and/or Int 2F
> ...
> HOWEVER, when I now doslfn (latest -c variant) (note all these steps in
> sequence are in 1.bat which is for 1st level sub-directory on USB D:\ Fat
> 32), the output is...
>
>
> DOSLFN 0.41f (....):enabled
> Last error:94 - inDOS bayra ...kullanimini SIFIRLA
Released DOSLFN 0.41f, which includes a basic version check (doslfn will install, not try and control doslfnms; I should probably have made it say another version was installed, but I didn't).
Richard

14.04.2022, 20:49

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

>
>
> Released DOSLFN 0.41f, which
> includes a basic version check (doslfn will install, not try
> and control doslfnms; I should probably have made it say
> another version was installed, but I didn't).
 >



Using the latest doslfn(MS) (14 April 2022), re-ran comparison of

without/with doslfn(MS)
USBFAT16/USBFAT32
XMSFAT16/XMSFAT32
SMSFAT16/SMSFAT32

for file set (10 files + 4 directories)

note SMARTDRV NOT USED in these tests and I think all relevant programs (himemSX, rdrvSX32, etc) are the latest updates.

As before (same sequence)...




1
[image]




2
[image]




3
[image]




4
[image]




5
[image]




6
[image]




7
[image]




8
[image]




9
[image]




A
[image]




B
[image]




C
[image]







I made a mistake that I did not doslfnMS -u before I started my series of .bat files (first part doslfnMS) (FDauto.bat doslsfnMS, SET COMDIR=lfn ...) and ended up with many duplicates from a sub-folder in C:\ being inserted into the C:\ ROOT. Because of all the @echo OFF (to make the screen captures look nice), if there were any error messages, I could not see them.


Will now very shortly repeat above but with much larger file set.




Is RT giving me the blank line before "Elapsed time: ...) - if so, option to not have blank line?
jadoxa

Homepage E-mail

Queensland, Australia,
15.04.2022, 04:30

@ Richard
 

about to fix the "final" doslfn bug

Post reply

Okay, those times are a bit more reasonable (although 18s is still a lot, but a cache would help with that). Of course, that's assuming RT is right and the prompt is wrong. Or if they're both right, where's the extra time coming from? Accessing the programs from the slow USB? If you're not going to use a cache, perhaps create a small RAM drive (SHSURDRV), copy the programs to that and run from there.


> Is RT giving me the blank line before "Elapsed time: ...) - if so, option
> to not have blank line?
It is, to separate its output from the program it runs. I noticed that myself after doing the release. Removing the blank automatically is probably too much trouble, but adding another option would be simple enough.
Richard

15.04.2022, 06:47
(edited by Richard, 15.04.2022, 06:59)

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> Okay, those times are a bit more reasonable (although 18s is still a lot,
> but a cache would help with that). Of course, that's assuming RT is right
> and the prompt is wrong. Or if they're both right, where's the extra time
> coming from? Accessing the programs from the slow USB? If you're not going
> to use a cache, perhaps create a small RAM drive (SHSURDRV), copy the
> programs to that and run from there.
>

I "on purpose" did not have a cache installed (SMARTDRV) or utilize RAMdrive for programs to get the "worst case " timings for comparisons.

Yes, all programs still running from USB FAT32 sub-sub-directory (C:\freedos\bin\). It is a fairly cheap 16GB verbatim (C:\,D:\ and E:\) bought from "K-mart" (10 or so years ago) and may be USB2 or even USB1 speed (on the USB2 port of the computer). I have noticed that "more expensive", "recent", "USB3" sticks from Verbatim seem to be much better, especially when multi-level sub-directories are involved (I suppose you get what you paid for).

With DOS tests, I always seem to notice that working off a sub-directory from (C:\ ROOT) seems to be noticeably slower than ROOT. C:\, my main working drive is FAT32 - and C:\ is only ~512 MB and was automatically created by (RUFUS + FreeBASIC RC5 full USB version), I had no control no control over that it had to be FAT32. Meanwhile, FreeDOS allowed me to have additional partitions (D:\ and E:\) on my USB stick (16 GB) and I allocated 12 GB to D:\ (as FAT32) and the remaining 2 GB to E:\ (as FAT16 just for fun). It is only very recently that I really did any testing etc with E:\. When I have "worn-out" my cheap USB stick, I plan to install on a brand new, higher quality (perhaps still verbatum) USB3 stick and plug into USB3 port.


The long term goal is to work "off memory" (either CACHE or most likely a RAMdrive - for all programs - where is best (assuming RAM drive) - XMSFAT16, XMSFAT32, SXMSFAT16, SXMSFAT32 - or doesn't it matter? I think I could spare some bytes off a RAMdrive for the whole DOS operating system + programs! :)

So, on reboot, probably xcopy the whole C:\ drive ( > NUL), then setup smartdrive, and make sure all paths refer only to RAMdrives.


Talking about xCOPY, on all possibilities of xCOPY C:\ --> RAMdrive, ie XMSFAT16 (~740 MB), XMSFAT32 (~740 MB), SXMSFAT16 (4 GB), SXMSFAT32 (~26 G) - the computer locks up (~~ half way through). According to Windows, C:\ drive contains 6986 items (6602 files + 385 folders) size 463 MB (size on disk 471 MB). There may be an upper limit for xCOPY which I don't know. My work around is to do it in say 5 stages (C:\ROOT then individually the four 1st sub-level-directories). Of course, over 99% on C:\ I do not really need to be there at all.







>
> > Is RT giving me the blank line before "Elapsed time: ...) - if so,
> option
> > to not have blank line?
>
> It is, to separate its output from the program it runs. I noticed that
> myself after doing the release. Removing the blank automatically is
> probably too much trouble, but adding another option would be simple
> enough.

For the next rerun of doslfn(MS) possibilities - I plan to simultaneously on screen captures, show timings w/o SMARTdrv installed, then on same capture, install SMARTdrv (won't be optimal element/buffer size values just yet), and run comparisons. Hence why I was so interested in removing the blank line before "Elapsed time:..."


EDIT each .bat run is to be with its own reboot per .bat






One thing I suppose I should really do, is to rerun exactly the same BUT change the order - eg c.bat, b.bat, a.bat, 9.bat ... 1.bat. And see if 1.bat still gives me the 18 seconds timings.
jadoxa

Homepage E-mail

Queensland, Australia,
15.04.2022, 09:34

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> The long term goal is to work "off memory" (either CACHE or most likely a
> RAMdrive - for all programs - where is best (assuming RAM drive) -
> XMSFAT16, XMSFAT32, SXMSFAT16, SXMSFAT32 - or doesn't it matter?
SX would let you keep 4Gi for normal usage. FAT32 would be better for drives over 512Mi. Otherwise I don't think it much matters.

> Talking about xCOPY...
> - the computer locks up (~~ half way through). According to Windows, C:\
> drive contains 6986 items (6602 files + 385 folders) size 463 MB (size on
> disk 471 MB).
Is that your FreeDOS install? Curious, according to dir/s/a mine is 9642 files, 1842 dirs, 439M total (1.3 installed on top of 1.2, I think it was). Worked fine under VirtualBox, to a 500Mi FAT32 drive. I'll try it in real DOS tomorrow.

> For the next rerun of doslfn(MS) possibilities - I plan to simultaneously
> on screen captures, show timings w/o SMARTdrv installed, then on same
> capture, install SMARTdrv (won't be optimal element/buffer size values just
> yet), and run comparisons. Hence why I was so interested in removing the
> blank line before "Elapsed time:..."
Released RT 1.02, which adds -c to remove the blank line. You could just redirect the entire batch: command /c 1.bat > 1.txt; vecho will still write to the screen.
Richard

17.04.2022, 09:11

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > The long term goal is to work "off memory" (either CACHE or most likely
> a
> > RAMdrive - for all programs - where is best (assuming RAM drive) -
> > XMSFAT16, XMSFAT32, SXMSFAT16, SXMSFAT32 - or doesn't it matter?
>
> SX would let you keep 4Gi for normal usage. FAT32 would be better for
> drives over 512Mi. Otherwise I don't think it much matters.
>
>
> > For the next rerun of doslfn(MS) possibilities - I plan to
> simultaneously
> > on screen captures, show timings w/o SMARTdrv installed, then on same
> > capture, install SMARTdrv (won't be optimal element/buffer size values
> just
> > yet)
>
> Released RT 1.02, which adds -c to remove the blank line. You
> could just redirect the entire batch: command /c 1.bat >
> 1.txt; vecho will still write to the screen.


Your RT -c option seems to work perfectly for me (so far). Thanks.


So I was preparing code for SMARTDRV vs nonSMARTDRV and I came across the error


There is insufficient XMS memory to load SMARTDRV


OK so I reduced my XMS RAM drives (FAT16 + FAT32) from the approximately 741M each down to ~500M each and still gives the same error message.

My FDauto.bat has the lines


xmsdsk.exe 500000 I: /y
rdrvx32.exe 500M

and 1.bat etc has

SMARTDRV C D E 8192 1024 1024


Is my SMARTDRV (attempt) as above too big? (I would have thought reducing both XMS RAMdrives down by ~ 240M each should have been plenty to cover for SMARTDRV).

Maybe I am over looking something - the only changes to my program is "tacking" the SMARTDRV setup at the end of in 1.bat etc and "repeating the code set for without SMARTDRV.
Richard

17.04.2022, 13:25

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> > The long term goal is to work "off memory" (either CACHE or most likely
> a
> > RAMdrive - for all programs - where is best (assuming RAM drive) -
> > XMSFAT16, XMSFAT32, SXMSFAT16, SXMSFAT32 - or doesn't it matter?
>
> > For the next rerun of doslfn(MS) possibilities - I plan to
> simultaneously
> > on screen captures, show timings w/o SMARTdrv installed, then on same
> > capture, install SMARTdrv (won't be optimal element/buffer size values
> just
> > yet), and run comparisons.


I had some success with SMARTDRV and XMS drive (I changed the order when the SMARTDRV is installed - NOW BEFORE any RAM drives being created).


HOWEVER, with SMARTDRV and SXMS FAT16 drive, I get the error


Invalid Drive J: (J: my chosen name)


without SMARTDRV, the exact same creation of J:\ of a SXMS FAT16 is no problem (done it a hundred times).

Below is the relevant FDAUTO.BAT code which does not seem to work for me to create a SXMS FAT16 RAMdrive 4G,


devload himemSX.exe
SMARTDRV C D E 8192 1024 1024
SHSURDRV.exe 4094M,J:

which would otherwise given me a 4G SXMS FAT16 drive and a SMARTDRV of 8M
Richard

17.04.2022, 14:40

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

>
>
> > For the next rerun of doslfn(MS) possibilities - I plan to
> simultaneously
> > on screen captures, show timings w/o SMARTdrv installed, then on same
> > capture, install SMARTdrv (won't be optimal element/buffer size values
> just
> > yet), and run comparisons.


Note, last two runs b.bat and c.bat (j:\ SXMS FAT16 drives) are in error - refer previous reply.

Some minor "cosmetic" errors on screen captures (but still usable).


So, with a "colorful Easter Egg Theme", as per other posts - the screen captures are...



[image]



[image]



[image]



[image]



[image]



[image]



[image]



[image]



[image]



[image]



[image]



[image]
Richard

19.04.2022, 06:14

@ Richard
 

about to fix the "final" doslfn bug

Post reply

> >
> >
> > > For the next rerun of doslfn(MS) possibilities - I plan to
> > simultaneously
> > > on screen captures, show timings w/o SMARTdrv installed, then on same
> > > capture, install SMARTdrv (won't be optimal element/buffer size values
> > just
> > > yet), and run comparisons.
>
>
>
> Note, last two runs b.bat and c.bat (j:\ SXMS FAT16 drives) are in error -
> refer previous reply.
>
> Some minor "cosmetic" errors on screen captures (but still usable).
>
>
> So, with a "colorful Easter Egg Theme", as per other posts - the
> screen captures are...
>
>
>
>



The set above created with "112 version" of FDauto.bat and the relevant code lines (in sequence) are...

doslfnMS
devload himemSX
smartdrv C D E 8192 1024 1024
RdrvX32 --> G:\
RdrvSX32 --> H:\
SHSURDRV --> J:\
XMSDSK --> *** invalid drive I:\ (i.e. for b.bat and c.bat test)


Changed order
113 version

doslfnMS
devload himemSX
smartdrv C D E 8192 1024 1024
SHSURDRV --> J:\
rdrvX32 --> G:\
rdrvSX32 --> *** invalid drive H:\ (i.e for 5.bat and 6.bat test)
xmsdsk --> I:\


Interestingly, somewhere along the line, my c.bat file (for version 112 test) did not exist
tom

Homepage

Germany (West),
15.04.2022, 11:32

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> Okay, those times are a bit more reasonable (although 18s is still a lot,
> but a cache would help with that). Of course, that's assuming RT is right
> and the prompt is wrong. Or if they're both right, where's the extra time
> coming from?
it might be cool to have an option -Z (time) to also display the DOS time:

Elapsed time: 04:02:09.88-04.02.10.22: 0.336 seconds

where the former is the 'official' DosGetTime time, and the latter your (more precise?) time.

in addition, RT has a couple of different ways to time things.
it would be cool to know if the times behave consistently or not

'-1', 'use the standard timer tick (55ms, 24 hours; default)', NL
'-[1]+', ' as above, but can time across days', NL
'-2', 'use the CMOS timer (1ms, 48« days)', NL
'-2+', ' as above, but restarts the timer if BIOS turns it off', NL
'-3', 'use the hi-res timer (1æs, 1 hour)', NL
'-t', 'display only the time', NL
'-r', 'display only the ticks (raw)', NL
'-w', 'write timing information to stderr', NL
'-c', 'compact, no blank line before output', NL
'-e', 'execute the program directly, rather than loading first', NL
jadoxa

Homepage E-mail

Queensland, Australia,
18.04.2022, 10:14

@ tom
 

about to fix the "final" doslfn bug

Post reply

> it might be cool to have an option -Z (time) to also display the DOS time:
Added -v (verbose) to show the command and start & finish times.

> it would be cool to know if the times behave consistently or not
Perhaps you could measure the timeout of choice? If you have it, you could use int (from RBIL) to run the BIOS wait function (int -q ah=0x86 cx=0xf dx=0x4240 for one second).
tom

Homepage

Germany (West),
18.04.2022, 12:09

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

> Perhaps you could measure the timeout of choice? If you have
> it, you could use int (from RBIL) to run the BIOS wait
> function (int -q ah=0x86 cx=0xf dx=0x4240 for one second).
the question was intended for Richard with his strange machine where sometimes
the time differences between his prompt and the times reported by rt seem to be different.
jadoxa

Homepage E-mail

Queensland, Australia,
09.04.2022, 04:29

@ Richard
 

about to fix the "final" doslfn bug

Post reply

Rugxulo beat me to it, but yeah...

> c:\FreeDOS\BIN\doslfnms.com (4/2/2022 20,369 bytes)
> ...
> C:\> DOSLFN.com (i.e what FREEDOS RC5 supplied - and the output message
> was)
...DOSLFN doesn't detect version differences, so using 0.41e to control the installed 0.41f-ms won't work. You should have unloaded the old one (doslfnms u) before loading the new one.
Laaca

Homepage

Czech republic,
11.04.2022, 11:24

@ jadoxa
 

about to fix the "final" doslfn bug

Post reply

I have an another question about DOSLFN.
Lets say that I have on my USB stick a mixture on MP3s. Some are from czech singers and some from russian singers.
The czech ones use the coding 852 and the russian ones are in cyrilic (cp-866).
As long I use this stick in windows there is no problem as everything is handled in UTF-8.
But in DOS + doslfn is a problem that it use the unicode--codepage transcoding.
If I would like to copy the USB stick to my harddisk I suppose that I have to firstly copy the czech files, then switch the DOSLFN codepage to russian (cp866uni.tbl) and run it again.
Or am I wrong and simple "copy G:\*.* C:\MUSIC" would be OK?

---
DOS-u-akbar!
jadoxa

Homepage E-mail

Queensland, Australia,
11.04.2022, 12:15

@ Laaca
 

about to fix the "final" doslfn bug

Post reply

> If I would like to copy the USB stick to my harddisk I suppose that I have
> to firstly copy the czech files, then switch the DOSLFN codepage to russian
> (cp866uni.tbl) and run it again.
> Or am I wrong and simple "copy G:\*.* C:\MUSIC" would be OK?
Nope, you're right, you'd need to change codepage. I had originally had some sort of UTF-8 support in the todo, but removed it with 0.41e. I guess it would still requiring switching: turn on UTF-8, copy, turn off UTF-8. Maybe create another topic...
Japheth

Homepage

Germany (South),
09.04.2022, 21:52

@ jadoxa
 

released new HimemSX v3.53 on GitHub

Post reply

> What would you like to do with HimemSX? Are you going to incorporate the
> patch.
Done.

Somewhat strangely, the resulting binary is 4 bytes shorter than yours - but seems to work as it should...:-)

https://github.com/Baron-von-Riedesel/HimemSX

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
10.04.2022, 04:56

@ Japheth
 

released new HimemSX v3.53 on GitHub

Post reply

> Somewhat strangely, the resulting binary is 4 bytes shorter than yours -
> but seems to work as it should...:-)
There's new code in rmcopy that's three bytes shorter, plus one byte padding.
Japheth

Homepage

Germany (South),
10.04.2022, 22:04

@ jadoxa
 

perhaps a small bug in new HimemSX?

Post reply

> There's new code in rmcopy that's three bytes shorter, plus one byte
> padding.
Ah yes, it might have been an "optimization" that I forgot.


While implementing the new block move feature in Jemm I found something that gave me a vague feeling of a bug:


        mov edx,es:[si].xms_move.dest_offset
        mov si,es:[si].xms_move.dest_handle
        call xms_get_move_addr
        mov dx,si
        pop di
        pop si
        jc @@copy_dest_is_wrong
        test di,4000h
        mov edi,eax
        mov bl,0            ; <---- isn't this missing?
        push dx
        jz @F
        mov bl,es:[si+sizeof xms_move].sxms_move.src_hi
@@:
        mov edx,es:[si].xms_move.src_offset
        mov si,es:[si].xms_move.src_handle
        call xms_get_move_addr
        pop dx
        jc @@copy_source_is_wrong


I have no test case yet, and the bug may only be revealed if you access a super-extended block with the old block-move function AH=0Bh, but in any case the thing looks suspicious. :-D

---
MS-DOS forever!
jadoxa

Homepage E-mail

Queensland, Australia,
11.04.2022, 02:34

@ Japheth
 

perhaps a small bug in new HimemSX?

Post reply

> mov bl,0 ; <---- isn't this missing?
No, it's still zero from the xor, because the mov didn't happen.
Japheth

Homepage

Germany (South),
11.04.2022, 05:38

@ jadoxa
 

perhaps a small bug in new HimemSX?

Post reply

> No, it's still zero from the xor, because the mov didn't happen.
You're right, register bx isn't modified inside xms_get_move_addr.

There's nevertheless a little bug in HimemSX, but it's there since it exists: xms_move_emb is used internally if a memory block is "reallocated"; however, it's called just once then, and is generally restricted to copy 4 GB - so reallocating a block > 4GB to a size that is is also > 4GB will fail it the block has to be moved.

---
MS-DOS forever!
Richard

11.04.2022, 05:58

@ Japheth
 

perhaps a small bug in new HimemSX?

Post reply

>
> There's nevertheless a little bug in HimemSX, but it's there since it
> exists: xms_move_emb is used internally if a memory block is "reallocated";
> however, it's called just once then, and is generally restricted to copy 4
> GB - so reallocating a block > 4GB to a size that is is also > 4GB will
> fail it the block has to be moved.


Just curious - it that any thing to do with why when I created a 4 GIB file from the copy + of four 1 GiB files why the very first byte of the 4 GiB file was changed to a 0x1A? (this was all created on a FAT32 SXMS H:\ (~ 28 GiB).

I inspected the 4 GIB file after I copied from FAT32 RAMdrive to USB stick - and inspected in windows (so I do not know "when" the 0x1A change for the first byte occurred).

I read somewhere that FAT32 files are a maximum of 4 GiB-1byte.
Japheth

Homepage

Germany (South),
11.04.2022, 10:07

@ Richard
 

perhaps a small bug in new HimemSX?

Post reply

> Just curious - it that any thing to do with why when I created a 4 GIB file
> from the copy + of four 1 GiB files why the very first byte of the 4 GiB
> file was changed to a 0x1A? (this was all created on a FAT32 SXMS H:\ (~ 28
> GiB).
That's pretty unlikely. :-D

> I read somewhere that FAT32 files are a maximum of 4 GiB-1byte.
That's true, although, IIRC, the limit, depending on what you're using, may be only just 2G-1.

---
MS-DOS forever!
Back to index page
Thread view  Board view
22583 Postings in 2101 Threads, 402 registered users, 38 users online (1 registered, 37 guests)
DOS ain't dead | Admin contact
RSS Feed
powered by my little forum