DOS ain't dead

Hello,

I can`t mount UDF filesystem to FreeDOS or Devuan.
The BD-R disc writes this
root@devuan:~# udfinfo /dev/sr0
udfinfo: Error: Virtual Allocation Table not found, maybe wrong --vatblock?
udfinfo: Warning: File Set Descriptor cannot be read
udfinfo: Warning: File Set Descriptor not found
udfinfo: Warning: Logical Volume is in inconsistent state
filename=/dev/sr0
label=12 18 2018
uuid=112c073020554446
lvid=12 18 2018
vid=UDF Volume
vsid= UDF Volume Set
fsid=
fullvsid=112C0730 UDF Volume Set
winserialnum=0x00000000
blocksize=2048
blocks=1574431
usedblocks=2
freeblocks=2297598
behindblocks=0
numfiles=0
numdirs=1
udfrev=2.01
udfwriterev=2.01
integrity=opened
accesstype=writeonce
softwriteprotect=no
hardwriteprotect=no
start=16, blocks=3, type=VRS
start=32, blocks=16, type=MVDS
start=48, blocks=16, type=RVDS
start=64, blocks=1, type=LVID
start=256, blocks=1, type=ANCHOR
start=272, blocks=2297600, type=PSPACE
root@devuan:~#

It seems to be schilly tools compiled for DOSes usefull.
cdrecord works with blue-ray burner LG :)
thanks

AFAIK there's no DOS CD/DVD driver with UDF support. There was some small attempt long time ago but was completely lost and forgotten. It seems that all optical drives will die/disappear before someone will make such driver so problem will be solved itself :)
Also beware there are more UDF versions and not all OS support all version, e.g. BluRay discs use UDF 2.x and Windows XP doesn't support it by default, you have to install some 3rd party driver (usually from BRD/DVD manufacturer).

---
DOS gives me freedom to unlimited HW access.

Just as an FYI, one of the experiments I have been performing with the USB drivers is adding support for CD, DVD, and BD to my USB drivers. I don't have much time to work on it any more, but occasionally find a little time. I'm still a LONG way from releasing anything new.

Anyway, one of the things that came up is that DVD-RAM disks are often formatted with FAT32. The issue, though, is that the sectors on optical media are 2048 bytes instead of the standard 512 bytes like floppies and hard drives. This actually isn't much of a problem for MS-DOS (MS-DOS is "compatible" with sector sizes other than 512 bytes). It is not difficult to configure MS-DOS to work with 2048 byte sectors (it is not trivial, but neither is it difficult). FreeDOS is NOT compatible AT ALL with anything other than 512 byte sectors, and many utility programs (like defraggers and caching programs) won't work with anything other than 512 byte sectors, either.

But, I think it should be possible to format all optical media as FAT32 even though it is not normally done. I haven't actually yet experimented with anything other than DVD-RAM media, but one of the things I intend to do is see if I can format other types of optical media as FAT32 and have DOS (at least MS-DOS) be able to read it. Right now, I don't see a reason why it wouldn't work but I will need to try it and see if any unforeseen issues arise. Also, one reason I would like try to do it with other optical media besides DVD-RAM is that DVD-RAM media is VERY difficult to find these days, even though it's actually superior to the other optical media in some ways.

Anyway, maybe some food for thought.

> But, I think it should be possible to format all optical media as FAT32
> even though it is not normally done. I haven't actually yet experimented
> with anything other than DVD-RAM media, but one of the things I intend to
> do is see if I can format other types of optical media as FAT32 and have
> DOS (at least MS-DOS) be able to read it. Right now, I don't see a reason
> why it wouldn't work but I will need to try it and see if any unforeseen
> issues arise. Also, one reason I would like try to do it with other
> optical media besides DVD-RAM is that DVD-RAM media is VERY difficult to
> find these days, even though it's actually superior to the other optical
> media in some ways.

But, formatting optical media (especially BD-R) as FAT32 won't get you much, as of course FAT32 will not allow files > 4GB and MS-DOS won't allow long unsigned integer files between 2 and 4 GB to be read, since only signed integer file sizes seem to be supported. It's a VERY frustrating limitation, since filesizes can't be negative, why waste the upper bit? That limitation would need to be removed first. BD-Rs are UDF exclusively, I believe, and UDF allows creation of files up to 2TB (?) I believe, certainly well beyond 4GB, at least, since discs often contain single .m2ts files that are beyond 45 GB in length.

What are some of the hurdles in coding a UDF-capable file-system driver for DOS? Does MSCDEX allow only ISO9660 format, or does it also support FAT32? I also have wondered why UDF support was never envisioned for DOS, as such a driver, like the two or three existing NTFS filesystem drivers for MS-DOS would be very, very useful. Is the old High-Sierra format compatible with 2k sectors and / or file sizes larger than 4GB? So many questions, and few answers.

Along these lines, supporting 4Kb sector-size hard disks should be possible in MS-DOS, if 2048-byte sector sizes are possible and will work. Is it possible with a simple patch to IO.SYS and COMMAND.COM?

> But, formatting optical media (especially BD-R) as FAT32 won't get you
> much, as of course FAT32 will not allow files > 4GB and MS-DOS won't allow
> long unsigned integer files between 2 and 4 GB to be read, since only
> signed integer file sizes seem to be supported. It's a VERY frustrating
> limitation, since filesizes can't be negative, why waste the upper bit?
> That limitation would need to be removed first. BD-Rs are UDF exclusively,
> I believe, and UDF allows creation of files up to 2TB (?) I believe,
> certainly well beyond 4GB, at least, since discs often contain single .m2ts
> files that are beyond 45 GB in length.

If you're wanting to read movie files from BD's, you're correct that the files sizes can be huge (far greater than FAT32 can handle). I was thinking more along the lines of using a BD (or DVD or CD) as a backup mechanism, or even a way to transfer files between computers. I don't know of a DOS program that will play BD movies (and I doubt that there ever will be one), but if you are treating a BD simply as a media storage device (and don't really care about the formatting) I think you could create something roughly equivalent to the DISKCOPY program that could copy the entire disk. Optical media has sectors much like magnetic media, though optical media is designed to be written serially rather than randomly, though I don't think it necessarily NEEDS to be that way.

Another possibility is exFAT, though that is considered proprietary by MS. The point is that you don't need to limit yourself to UDF just because everybody else does -- I think FAT32 or exFAT would work just fine on optical media (but I could be wrong). However, you cannot simply treat it like a "big floppy" since you can only write a limited number of times to the disk, and every time you write a file you also need to modify the FAT. That's one of the ways that DVD-RAM is superior to the other optical alternaitves -- you can reliably write to a DVD-RAM many more times than you can other optical media. Essentially, you can treat a DVD-RAM as a really big floppy.

> What are some of the hurdles in coding a UDF-capable file-system driver for
> DOS? Does MSCDEX allow only ISO9660 format, or does it also support FAT32?
> I also have wondered why UDF support was never envisioned for DOS, as such
> a driver, like the two or three existing NTFS filesystem drivers for MS-DOS
> would be very, very useful. Is the old High-Sierra format compatible with
> 2k sectors and / or file sizes larger than 4GB? So many questions, and few
> answers.

As alluded to earlier, the main issue with UDF is that it is a "moving target". They keep modifying it all the time so it is very difficult to spend time writing a driver that will work reliably all the time. As I was investigating and coming to the conclusion that FAT32 is probably a good option, I looked a little bit into what it would take to write a UDF driver and didn't like what I saw. The best option would probably be to try and port something from *nix.

> Along these lines, supporting 4Kb sector-size hard disks should be possible
> in MS-DOS, if 2048-byte sector sizes are possible and will work. Is it
> possible with a simple patch to IO.SYS and COMMAND.COM?

Yes it is, at least for MS-DOS. It's probably also easy for PC-DOS (since MS-DOS and PC-DOS came from the same roots), but maybe not for the other DOS variations (like PTS-DOS or DR-DOS or others). I know for sure it's NOT possible with any of the FreeDOS versions.

With MS-DOS, there are two ways you can do it. The first is to load a disk device driver in CONFIG.SYS that has the correct sector sizes. I do this with a configurable RAM Disk driver and tell it to use 2048 byte sectors. Most RAMDISKs won't allow anything other than 512 byte sectors, but there is one called TurboDisk (TDISK) that does. I set up a RAM Drive with TurboDisk and point my TEMP and TMP environment variables at it. That way I am also sure that all of my TEMP files are automatically erased when I reboot. MS RAMDRIVE will allow 1024 byte sectors and TurboDisk will allow 2048 byte sectors. I don't know of one that will allow 4096 byte sectors, but it would be possible to write one. If you load such a driver in CONFIG.SYS, the MS-DOS kernel will adjust itself to comply with the max sector size when it boots and you don't actually need to modify anything in the kernel. The MS-DOS kernel will supposedly support sector sizes of up to 8192 bytes, but I've never experimented with anything more than 2048 bytes.

The second option is to modify the MS-DOS kernel. Again, this will probably only work with MS-DOS and maybe PC-DOS but will probably not work with any other DOS. Following is an excerpt from the comments in my USBDRIVE source code that discusses how to modify the MS-DOS kernel. The only file you need to modify is MSDOS.SYS and you only need to modify one word (two bytes) in it.


;============================================================================
;Changing DOS kernel to support Sector Size larger than 512 bytes
;============================================================================
;
; Before you do this, be aware that DOS will use a LOT more memory than it
; does if you leave things at the default values. In particular, you may
; want to change the BUFFERS value in CONFIG.SYS to something smaller than
; normal. If you have 640k of RAM available to DOS (all modern computers
; do), by default DOS will create 15 buffers, each one large enough to store
; a sector's worth of data. Under normal conditions (512-byte sectors), this
; will be 7.5k of RAM dedicated to disk buffers. If you increase the sector
; size to, say, 2048 bytes (instead of 512), and leave the number of buffers
; at the default (15), there is now 30k of RAM dedicated to disk buffers
; instead of 7.5k. Unless you're actually accessing the disk with the
; larger sector size, much of the memory dedicated to disk buffers never
; gets used and is essentially wasted. In other words, you shouldn't
; change the sector size in the DOS kernel unless you have a really good
; reason -- otherwise, you're simply wasting some of your precious memory
; for no reason. Also, modern hard drives are pretty fast, so the DOS
; buffers (sort of a small-scale disk-cache) don't provide as much benefit
; like they did in the old days.
;
; DOS does have the capability to support sector sizes larger than 512
; bytes (supposedly up to 8192 bytes), but this is not enabled by default.
;
; DOS internally maintains several memory buffers, each large enough to
; store one sector's worth of data. The number of these buffers defaults
; to 15 on most modern computer systems, but can be changed using the BUFFERS
; option in your CONFIG.SYS file. The size of each buffer must be large enough
; to store a sector's-worth of data from any disk that DOS may need to access
; while running. So, the buffers must be big enough to hold the largest sector of
; ANY disk that DOS may need to access. The sector size of almost all disks
; is 512 bytes, but there are some exceptions. It is those exceptions that
; cause all of the grief.
;
;
;
; You should NOT do this unless you need to load a disk that has a sector
; size larger than 512 bytes. This will unnecessarily increase the amount
; of memory that DOS uses, and also increase the amount of memory that
; USBDRIVE uses (USBDRIVE sets up an internal buffer that is the same size
; as the DOS buffers).
;
;
;
; The size of the internal DOS sector buffers is determined at boot time using
; the default disk device drivers built into the DOS boot kernel (MSDOS.SYS
; for MS-DOS, IBMDOS.COM for PC-DOS). The kernels have a "default" minimum
; size of 128 (0080h) bytes hard-coded into the kernel. During
; the boot process, the kernel adjusts this size to match the largest
; sector size of any disks it finds (which is very rarely anything other
; than 512 bytes). The problem is, the boot kernel stops adjusting the
; size after the default disk device drivers (those built into the kernel)
; are finished processing the disks. You can continue to load disk device
; drivers after the kernel has finished, using your CONFIG.SYS file (or via
; a TSR like USBDRIVE), but the DOS kernel has already determined the maximum
; sector size it will use (and reserved the memory it needs) before it gets
; to that point, so it's too late to change it.
;
; Since it is impossible to change the maximum sector size after the DOS
; kernel has decided what it should be, you have to change the DOS kernel.
; The way you do this is to change the "default" sector size that is hard-coded
; into the kernel from 128 bytes to whatever you want it to be (presumably,
; something larger than 512 bytes). I've seen documentation that says DOS
; will support sector sizes up to 8192 bytes, though I have not personally
; tested anything larger than 2048 bytes.
;
; The easiest and fastest way to perform the patch is to use DOS's
; extremely powerful DEBUG program. DEBUG is a powerful program, and is
; therefore also extremely dangerous in the hands of someone who does not
; know what they are doing. If you're not at least a little bit familiar
; with hexadecimal arithmetic, memory segments and offsets, and similar
; concepts, you're probably better off letting someone else do this for you.
; This will take you through the steps one at a time and explain what each of
; them does, but if you're queasy about this kind of thing, let someone
; help you.
;
; First of all, you will need to locate and modify the attributes of the DOS
; boot kernel before you can edit it. The file will be located in the root
; directory of your boot drive (usually C:\, or A:\ if you boot from a
; floppy). If you're using MS-DOS (Microsoft), the file is called MSDOS.SYS.
; If you're using PC-DOS (IBM), the file is called IBMDOS.COM. If you're
; using DOS from some other manufacturer, you'll need to figure out
; which file you should be editing.
;
; The boot kernel file is a critical system file, and is "hidden" from
; normal view so that you can't move or delete or change it
; by accident. You need to change the attributes of the file so that
; you can see it and edit it. Later versions of DOS came with the ATTRIB
; utility that lets you do just that, and there are also several
; after-market utilities that will let you accomplish the same thing.
;
; Let's assume you're using MS-DOS, that you're in the root directory
; of the hard drive you booted from (C:\), and that you're using the MS-DOS
; ATTRIB utility. To make sure that the MSDOS.SYS file is there, you would
; do the following:
;
; ATTRIB MSDOS.SYS {Enter}
;
; The computer will respond with something like this:
;
; SHR C:\MSDOS.SYS
;
; Note that a "DIR MSDOS.SYS" would say the file doesn't exist, since it
; has several special attributes set (S = System, H = Hidden, and
; R = Read-only) that keep the DIR command from "seeing" it. To unset
; the S, H, and R attributes so that you can edit the file, do the following:
;
; ATTRIB -R -S -H MSDOS.SYS {Enter}
;
;
;
; Can make backup copy here just in case things get screwed up. Problem is,
; need SYS to store file (must be in specific location on disk), so a backup
; on the same drive doesn't do a lot of good. If you want a backup, you
; should have a bootable floppy or something to store the original file.
;
;
;
; Now for the fun part -- we'll get into the DOS kernel and change it using
; DEBUG. Keep in mind that everything in DEBUG uses hexadecimal numbers
; and arithmetic (not decimal):
;
; DEBUG MSDOS.SYS {Enter}
;
; The computer will simply respond with a "-", which is the DEBUG prompt
; asking you what it should do next:
;
; -
;
; The first thing you need to do is ask DEBUG how big the file it loaded
; actually is (you'll need to know this for the next step). DEBUG puts
; the file size in the CPU's CX register when it loads, so you just need
; to ask DEBUG what's stored in the CPU's CX Register:
;
; RCX {Enter}
;
; The computer will respond with something like this:
;
; CX 94FA
; :
;
; "94FA" is the hexadecimal value currently stored in the CX register
; (the number of bytes in the MSDOS.SYS file), and the ":" is DEBUG
; prompting you to enter a new value to be stored in the CX register.
; You don't want to change it, so:
;
; {Enter}
;
; DEBUG will respond with another "-" prompt:
;
; -
;
; Now, we need to Search in the kernel file to find out where the
; NUL device header is located. We will use DEBUG to Search (S), starting
; at the beginning of the file (offset 0), for the entire Length (L) of the
; file, for the string 'NUL ' (NUL followed by 5 spaces):
;
; S 0 L 94FA 'NUL ' {Enter}
;
; Note that in most cases, whether characters are upper-case and lower-case
; doesn't matter, but it DOES matter in the search string ('NUL ') --
; NUL must be in upper-case letters. The number following the 'L' should be
; the same number that was returned by DEBUG in the "RCX" step above.
;
; The computer will respond with something like this:
;
; 1924:82E2
; -
;
; The second number (the one after the colon, 82E2 in this case) is the
; offset into the file where the NUL device header is located. The location
; in the file we actually need to change is located 28 (1Ch) bytes before
; the start of the NUL device header. Since most people can't subtract
; hexadecimal numbers (like 82E2h and 1Ch) easily in their heads, we'll have
; DEBUG do it for us using the H (Hexadecimal math) function:
;
; H 82E2 1C {Enter}
;
; The computer will respond with something like this:
;
; 82FE 82C6
;
; The first number (82FE is the sum of the two numbers (82E2h + 1Ch), and the
; second number (82C6) is the difference of the two numbers (82E2h - 1Ch).
; We need the second number (the difference) for the next step, where we're
; going to make sure that we've got the correct file offset before we
; change anything. We'll use DEBUG's D (Dump memory) function to see what's
; stored in the kernel file at this location. We'll look at enough data to
; make sure we see the data we're trying to change, along with the NUL device
; header that we originally searched for:
;
; D 82C6 L 28 {Enter}
;
; The computer will respond with something like this:
;
;
; 1924:82C0 80 00-00 00 00 00 00 00 00 00 ..........
; 1924:82D0 00 00 00 00 00 00 00 00-00 00 00 00 04 80 C6 0D ................
; 1924:82E0 CC 0D 4E 55 4C 20 20 20-20 20 00 00 00 00 ..NUL ....
; -
;
; The far left column is the data address (file offset), the second
; column is the hexadecimal value of each data byte, and the last column
; is the data represented as ASCII text (with a "." in the last column
; indicating that the byte can't be represented as a printable ASCII character).
; The important thing is that the first two numbers you see are 80 00,
; and that you see a 'NUL' followed by 5 spaces (20h's) 4 bytes from the end
; of the Dump. What the other numbers are in between and after don't
; really matter.
;
; The two bytes at the beginning (80 00) are the ones we need to change.
; In the Intel architecture, data is stored in a little-endian format, which
; means that the Least Significant Byte (LSB) is stored first, followed by
; the Most Significant Byte (MSB). This actually makes a lot of sense to
; the computer, but is very confusing to humans because it is backwards
; from the way we usually write things. The two bytes, 80 00, converted to a
; human-readable form, is 0080h, which equates to 128 decimal (the
; hard-coded "default" sector size). Following are the values we may want
; to change the default sector size to:
;
; Decimal Hex DEBUG
; ------- ----- -----
; 1024 0400h 00 04
; 2048 0800h 00 08
; 4096 1000h 00 10
; 8192 2000h 00 20
;
; To change data in the file, you use DEBUG's E (Edit) command. For instance,
; to change the sector size to 2048 bytes, you would do the following:
;
; E 82C6 00 08 {Enter}
;
; The computer will simply respond with another "-" prompt:
;
; -
;
; To verify that the data was actually changed like you thought it was,
; you should do another Dump:
;
; D 82C6 L 2 {Enter}
;
; The computer will respond with something like this:
;
;
; 1924:82C0 00 08 ..
; -
;
; If you're satisfied with what you've done, you can W (Write) the file
; to make the changes permanent:
;
; W {Enter)
;
; The computer will respond with something like this:
;
; Writing 094FA bytes
; -
;
; If you're not sure you've done everything correctly, or screwed something
; up so badly that you just want to start over again, just skip the W command.
; Unless you tell DEBUG to Write the file, none of the changes you've made
; are permanent.
;
; To exit DEBUG, use the Q (Quit) command:
;
; Q {Enter}
;
; Now that the file is changed, you should set the S, H, and R attributes
; again so that you won't erase or change the file by accident:
;
; ATTRIB +R +S +H MSDOS.SYS
;
; Now, reboot and you should be all set.
;
;
; Here is a sample session to change Sector Size to 2048 for MS-DOS 6.20:
;
; C:\>attrib msdos.sys
; A SHR C:\MSDOS.SYS
;
; C:\>attrib -r -s -h msdos.sys
;
; C:\>attrib msdos.sys
; A C:\MSDOS.SYS
;
; C:\>debug msdos.sys
; -rcx
; CX 94FA
; :
; -s 0 l 94fa "NUL "
; 1E5E:82E2
; -h 82e2 1c
; 82FE 82C6
; -d 82c6 l 28
; 1E5E:82C0 80 00-00 00 00 00 00 00 00 00 ..........
; 1E5E:82D0 00 00 00 00 00 00 00 00-00 00 00 00 04 80 C6 0D ................
; 1E5E:82E0 CC 0D 4E 55 4C 20 20 20-20 20 00 00 00 00 ..NUL ....
; -e 82c6 00 08
; -d 82c6 l 28
; 1E5E:82C0 00 08-00 00 00 00 00 00 00 00 ..........
; 1E5E:82D0 00 00 00 00 00 00 00 00-00 00 00 00 04 80 C6 0D ................
; 1E5E:82E0 CC 0D 4E 55 4C 20 20 20-20 20 00 00 00 00 ..NUL ....
; -w
; Writing 094FA bytes
; -q
;
; C:\>attrib +r +s +h msdos.sys
;
; C:\>attrib msdos.sys
; A SHR C:\MSDOS.SYS
;
; C:\>
;
; Now, after a reboot, the DOS sector size is 2048 bytes.

I still have some DVD-RAM discs but use them barely now. I just take one and loaded in drive. It took quite long time to read just a directory structure. Reading every file takes another tens of seconds, sometimes got crc error. I cannot write to disc, error not enough space or media changed despite there are still some 250MB left. So I would say nothing superior over common crappy CD/DVD disc that I trashed a lot of them before...

I checked DVDForm tool (part of windows DVD-RAM driver package) an it offers formatting in UDF 1.5, 2.0, 2.5 and FAT32. I didn't try FAT32 as everywhere I read the UDF is reccomended for DVD-RAM. Maybe it's because FAT FS needs often to write to 3 different disc locations (FAT table, directory entry, data itself) that can prolong seek access time esp. on optical drives so I guess UDF is optimized for this.
I lost my interest on BluRay as it's incease of capacity is not enough to nowdays demand, drives and media are quite expansive and nobody in my neighbour have it.

BTW R. Loew already had a solution for bigger sectors/disk - TeraByte Plus Package 3.0
The TeraByte Plus Package is a set of Tools and Patches that provide support
for Hard Drives with Larger Capacities and/or Larger Sector Sizes than was
possible previously with DOS and Windows 98 and 98SE.
Support is provided for Hard Drives up to 32768TiB (16TiB for USB Drives).
Extended Support is provided for 64-Bit Addressed SCSI and USB Drives.
Not tested due to no available Drives.
Logical Sector Sizes are supported up to 32KB in DOS and 4KB in Windows.
Larger Logical Sector Sizes can be emulated on Drives with smaller Sectors.
Support is added for Clusters up to 256 Sectors in length, allowing for
extra large FAT12 and FAT16 Partitions if needed.
Support provided for Multiple Extended Partitions per Drive.

The package includes patched MSDOS 7 IO.SYS, ESDI_506.PDR and some VXDs, partitioning, formatting and bootmgr tools...

---
DOS gives me freedom to unlimited HW access.

> If you're wanting to read movie files from BD's, you're correct that the
> files sizes can be huge (far greater than FAT32 can handle). I was
> thinking more along the lines of using a BD (or DVD or CD) as a backup
> mechanism, or even a way to transfer files between computers. I don't know
> of a DOS program that will play BD movies (and I doubt that there ever will
> be one), but if you are treating a BD simply as a media storage device (and
> don't really care about the formatting) I think you could create something
> roughly equivalent to the DISKCOPY program that could copy the entire disk.
> Optical media has sectors much like magnetic media, though optical media
> is designed to be written serially rather than randomly, though I don't
> think it necessarily NEEDS to be that way.
>
> Another possibility is exFAT, though that is considered proprietary by MS.
> The point is that you don't need to limit yourself to UDF just because
> everybody else does -- I think FAT32 or exFAT would work just fine on
> optical media (but I could be wrong). However, you cannot simply treat it
> like a "big floppy" since you can only write a limited number of times to
> the disk, and every time you write a file you also need to modify the FAT.
> That's one of the ways that DVD-RAM is superior to the other optical
> alternaitves -- you can reliably write to a DVD-RAM many more times than
> you can other optical media. Essentially, you can treat a DVD-RAM as a
> really big floppy.

Khusraw's DOS port of MPLAYER can play (unencypted) Blu-ray discs, but only if formatted in ISO9660 standard, as there's no UDF driver. The problem is in MS-DOS, filesizes > 2GB aren't readable due to signed longint limitation.

There's an experimental exFAT reader for DOS as well, but the problem again is that filesizes > 2GB aren't readable, so that completely defeats the purpose of having an exFAT driver in the first place. The raw assembly code of MSDOS.SYS and / or IO.SYS needs to be modified somehow. Even if possible, I do not know if filesizes > 4GB can be directly supported.

Same issues for the R. Loew drivers (I think?), or did he somehow magically erase and / or patch the 32-bit longint limitations on filesizes as well as increase sector sizes and capacities of drives beyond the 2TB limit? I'm not sure, as I've not personally tested those drivers.

Backporting a *nix UDF driver to MS-DOS might work, if such a driver exists. I think by now, UDF standard isn't going to change very much. I think v. 2.6 is available now, but BD-Rs are written in v 2.5 anyway.

Thanks very much for pointing out how to patch MSDOS.sys file to support > 512 byte sector sizes. I figured it was probably a two-byte patch, but didn't know exactly *where* to patch, as I hadn't studied the code closely.

> Same issues for the R. Loew drivers (I think?), or did he somehow magically
> erase and / or patch the 32-bit longint limitations on filesizes as well as
> increase sector sizes and capacities of drives beyond the 2TB limit?

Yes, he also had some solution for this: Large File Emulator (for W98/XP, no sources)
This Package intercepts User Mode FileSystem API requests so that a set of
smaller Files will appear as a single large File. This allows Programs to
Create Files that are larger than 4GiB. These Files can then be Read or
Modified by other Programs. Downloading and Playing or Archiving Blu-Ray
Videos are a example of what can be done with this Package.

I neved got BRD in my hands but how the video files are stored there? In a large single file? Well, I guess yes when it use UDF. DVD used VOB files split to 2GB...

---
DOS gives me freedom to unlimited HW access.

> > Same issues for the R. Loew drivers (I think?), or did he somehow
> magically
> > erase and / or patch the 32-bit longint limitations on filesizes as well
> as
> > increase sector sizes and capacities of drives beyond the 2TB limit?
>
> Yes, he also had some solution for this: Large File Emulator (for W98/XP,
> no sources)
> This Package intercepts User Mode FileSystem API requests so that a set
> of
> smaller Files will appear as a single large File. This allows Programs to
> Create Files that are larger than 4GiB. These Files can then be Read or
> Modified by other Programs. Downloading and Playing or Archiving Blu-Ray
> Videos are a example of what can be done with this Package.
>
> I neved got BRD in my hands but how the video files are stored there? In a
> large single file? Well, I guess yes when it use UDF. DVD used VOB files
> split to 2GB...

Thank you for this information. It seems like it might solve the > 2 Gb / 4 GB file size problems.

DVDs used .VOBS which were 1 GB in size, split into several files -- up to 8.5 GB per disc. But BDR uses UDF 2.5 filesystem which supports 2 TB (?) file sizes. Most blu-ray films have a single large .m2ts file which is over 20 - 40 GiB in size. These aren't playable or viewable by dos port of MPLAYER since there's no UDF driver support or filesize support > 2 GB.

> Khusraw's DOS port of MPLAYER can play (unencypted) Blu-ray discs, but only
> if formatted in ISO9660 standard, as there's no UDF driver. The problem is
> in MS-DOS, filesizes > 2GB aren't readable due to signed longint
> limitation.
>
> There's an experimental exFAT reader for DOS as well, but the problem again
> is that filesizes > 2GB aren't readable, so that completely defeats the
> purpose of having an exFAT driver in the first place. The raw assembly code
> of MSDOS.SYS and / or IO.SYS needs to be modified somehow. Even if
> possible, I do not know if filesizes > 4GB can be directly supported.
>
> Same issues for the R. Loew drivers (I think?), or did he somehow magically
> erase and / or patch the 32-bit longint limitations on filesizes as well as
> increase sector sizes and capacities of drives beyond the 2TB limit? I'm
> not sure, as I've not personally tested those drivers.

It is possible for a file system redirector (like for ISO 9660, UDF, exFAT, NTFS, or dosemu's host FS) to support 64-bit file sizes and seeks in a manner compatible to the FAT+ support of Enhanced DR-DOS. That means expanding the SFTs with extension fields (can be stored in file handle tables of the redirector), hooking the new-style interrupt 21h service 7142h for large seeking, and service 42h to make old-style seeking somewhat aware of the large size support. I did this for dosemu2's mfs. Technologically it works, but it was rejected on other grounds.

---
l

> It is possible for a file system redirector (like for ISO 9660, UDF, exFAT,
> NTFS, or dosemu's host FS) to support 64-bit file sizes and seeks in a
> manner compatible to the FAT+ support of Enhanced DR-DOS. That means
> expanding the SFTs with extension fields (can be stored in file handle
> tables of the redirector), hooking the new-style interrupt 21h service
> 7142h for large seeking, and service 42h to make old-style seeking somewhat
> aware of the large size support. I did this
> for dosemu2's mfs.
> Technologically it works, but it was rejected on other grounds.

Indeed, with a redirector it is possible to do almost anything. That's how MSCDEX (and its clones) are able to let DOS versions with only 512-byte sector support work with CD's that have 2048 byte sectors, or allow access to network drives on servers that DOS has no idea how to access directly. That is certainly one approach, but also has limitations and issues.

The approach I'd like to use is to mount the disks at a low level (like INT 13h or via a "standard" device driver) and let DOS handle the file system itself, which it already knows how to do. I'm using a similar approach with my USB drivers and GPT disks. I have a test USB flash drive I've formatted with several partitions, some DOS-compatible and others not. I'm able to mount the DOS-compatible partitions (formatted with some version of FAT) and DOS can read and write to them with no problem. I simply don't mount the non-compatible partitions. Also note that this has nothing to do with booting from a GPT disk, only accessing data from it after DOS has already booted.

Right now the way things like MSCDEX work is not the way I think things should work. MSCDEX accesses the CD/DVD hardware directly (with hardware-specific code) and provides a redirector interface that DOS can understand. I'd like to see this as a two-step process where the ISO9660 driver is a shim rather than being hardware-specific. The side of the shim that points towards DOS could be a redirector, but the side that points towards the hardware is not hardware-specific but rather a more general purpose standard (maybe something like SCSI or INT 13h). That would allow lots of different types of hardware (like USB, SATA, IDE, etc.) to indirectly interface with DOS since DOS itself would be "extended" with the shim and could understand ISO9660 (or UDF or exFAT or ext or whatever) and not need a hardware-specific driver.

Again, maybe just some food for thought but ultimately a really bad idea.

> The approach I'd like to use is to mount the disks at a low level (like INT
> 13h or via a "standard" device driver)
unfortunately, there was never a INT 13h interface for CD/DVDs.
there IS a standard that MSCDEX/SHSUCDX use to access the hardware.


> and let DOS handle the file system
> itself, which it already knows how to do.
DOS itself doesn't. MSCDEX/SHSUCDX do.


> Right now the way things like MSCDEX work is not the way I think things
> should work. MSCDEX accesses the CD/DVD hardware directly (with
> hardware-specific code)
are you SURE about this?

MSCDEX is supposed to call some driver functions; not to do specific hardware dependent things.

hardware dependent things would be handled by a CD manufacturer-specific driver until later generic drivers like UIDE appeared.

> MSCDEX is supposed to call some driver functions; not to do specific
> hardware dependent things.

I think so, as you 1st have to load some ATAPI/AHCI/SCSI driver that is HW specific and then MSCDEX or clonees interact with such driver without knowledge what HW is below.

BTW Bret, do you plan to implement EHCI support? Unfortunatelly even EHCI became outdated and all modern MBs has XHCI only so totally incompatible with old UHCI/EHCI aware systems...

---
DOS gives me freedom to unlimited HW access.

> Indeed, with a redirector it is possible to do almost anything.

Indeed. DOS barely cares about what filesystem is on the other side.

> the way things like MSCDEX work is not the way I think things
> should work. MSCDEX accesses the CD/DVD hardware directly (with
> hardware-specific code) and provides a redirector interface...

In fact, it accesses the CD/DVD hardware using yet another interface
which is provided by hardware-specific CD/DVD drive drivers The
interface luckily is quite minimalistic, mostly linear sector based.

So basically the answer to this thread would be some sort of UDFCDEX
which talks to your standard optical drive driver on one side and
to DOS on the other side. But I fail to understand why it would be
important to access files above 2 GB with that.

BluRay contains heavy content protection and it is very unlikely
that any media player for DOS can open such files. Also, it is
quite unlikely that DOS media players have the necessary drivers
for hardware-accelerated playback of advanced high resolution
video compression formats, or are able to unleash the multi-core
power of your CPU to decompress the video fast enough without
the graphics card features for that.

So as mentioned before, FAT32 on BluRay might not be that bad
after all. And you could make a generic driver which tells DOS
that a new FAT (not CDEX) drive has appeared on one side, while
talking to a (possibly different sector size) standard CD/DVD
driver on the other side. Also nice for DVD-RAM or CD-whatever.

---
FreeDOS / DOSEMU2 / ...

Not all Blurays must use heavy DRM, you can also burn your own non-DRM content that you may like to read also under DOS. I'm not sure if other devices would understand Bluray with FAT32 when UDF is expected.
DOS Mplayer can smoothly play H264 full HD on modern PC so I belive it can do aslo H265, maybe not in full 4k but full HD is possible I belive...

---
DOS gives me freedom to unlimited HW access.

> BTW Bret, do you plan to implement EHCI support? Unfortunatelly even EHCI
> became outdated and all modern MBs has XHCI only so totally incompatible
> with old UHCI/EHCI aware systems...

It's in the pipeline, but very little time to work on it.

> It's in the pipeline, but very little time to work on it.

Yeah, I know this, I also want release my new SMBtool for ~8 yesrs but I modified a lot things that I cannot fully compile it now...

---
DOS gives me freedom to unlimited HW access.

> In fact, it accesses the CD/DVD hardware using yet another interface
> which is provided by hardware-specific CD/DVD drive drivers The
> interface luckily is quite minimalistic, mostly linear sector based.

There is a special CD device driver interface which is actually a kludgy combination of two device driver interfaces (one a character device and the other a block device). It's actually quite a mess. It really isn't designed for writing to the CD at all (it assumes all CD's will be read-only, only contain music or data files, and you will never need to write to them). The CD interface is really designed around audio/music CD's which aren't formatted as files the way almost everything else is (including modern movies). It is a type of redirector interface, though a very different type of implementation than the network redirector.

That's why the standard DOS CD thing doesn't really work all that well and really shouldn't be used for things like DVDs or even writable (or rewritable) CD's. In my USB drivers, I'm trying to include the standard CD interface where it makes sense (so "standard" CD software, like the kind that plays music, will work). But I need to go way beyond that interface to really make it work with all the modern optical media possibilities.

As mentioned elsewhere, CD's don't normally use the INT 13h interface, but I think that is really because of the legacy need to work with audio CD's. If audio CDs were formatted as files, there probably wouldn't even be a special CD interface and it might have been implemented as INT 13h (or SCSI or something similar).

> So basically the answer to this thread would be some sort of UDFCDEX
> which talks to your standard optical drive driver on one side and
> to DOS on the other side. But I fail to understand why it would be
> important to access files above 2 GB with that.

A mentioned elsewhere, it might need that to look at the huge movie files on a BD formatted with UDF. But as also mentioned, even huge movie files can be broken up into smaller files even though that is not "standard".

> BluRay contains heavy content protection and it is very unlikely
> that any media player for DOS can open such files. Also, it is
> quite unlikely that DOS media players have the necessary drivers
> for hardware-accelerated playback of advanced high resolution
> video compression formats, or are able to unleash the multi-core
> power of your CPU to decompress the video fast enough without
> the graphics card features for that.

That only applies if you're using BD's for movies. If your using BDs for file storage (like backing up a hard drive), none of the graphics stuff is needed and the speed is not really all that critical, either (it's nice if it's fast, but it's not a real requirement).

> So as mentioned before, FAT32 on BluRay might not be that bad
> after all. And you could make a generic driver which tells DOS
> that a new FAT (not CDEX) drive has appeared on one side, while
> talking to a (possibly different sector size) standard CD/DVD
> driver on the other side. Also nice for DVD-RAM or CD-whatever.

As alluded to above, I don't think the DOS side should be the standard CD interface since it's really designed around audio CDs and not files. I think the DOS side should probably be either a regular FAT-type interface or a network redirector interface.

I think that the interface to the hardware would be a FAT32 (or UDF or whatever) interface. That is, it would essentially "advertise" itself as a "FAT32 interface" (or whatever), and the hardware driver only needs to recognize that the device (or partition on the device) is formatted as FAT32 and "attach" or "bind" the hardware to the shim. The hardware driver would not need to actually "understand" how FAT32 works -- it only needs to understand how to locate specific sectors on the media. The shim is the only thing that needs to understand how FAT32 works. With this type of interface, FAT32 would not even need to be implemented in the kernel like it is now. You could "add" FAT32 support to even older versions of DOS, sort of a similar concept to the LFN drivers that currently exist for DOS. The basic idea is to separate the reading/writing of sectors (the hardware part) from the "interpretation" of the sectors and ability to tell DOS about the files (the software part).

Again, this seems like it could work, but I have no idea what kinds of issues one would run across in an actual implementation and it may turn out to be a really bad idea.

> > It's in the pipeline, but very little time to work on it.
>
> Yeah, I know this, I also want release my new SMBtool for ~8 yesrs but I
> modified a lot things that I cannot fully compile it now...

That is strange. Why do you have problems compiling your SMBtool? I assume you are using CWSDPMI extender with Watcom C compiler? Or GCC compiler, whichever works better? What new additions did you make which break the compilation? I am very interested in trying out your new version.

And, yes, I can play 2 GB demuxed h.264 files from Blu-ray movies just fine on a modern system, with the DOS port of Mplayer. These 1920x1080 .m2ts files play natively in the DOS MPLAYER port, even with sound on some PCI sound cards, but the limit is 2 GB,and they can't be played directly from BD-R discs, because of no native UDF driver in DOS. So, I think support for BD-R for both data *and* large movie files (well beyond 4GB) in DOS has a lot of potential use and utility! h.265 files, not so much, since those are not on BD-R standard, but the UHD (4k) standard, which DO NOT play in DOS, because of extreme resolution (no DOS support of 3840x2160) and extreme CPU requirements (dual / quad core, with multithreading (SMT) enabled, as a requirement to play these files).

> > It's in the pipeline, but very little time to work on it.
>
> Yeah, I know this, I also want release my new SMBtool for ~8 yesrs but I
> modified a lot things that I cannot fully compile it now...

I know this.
Do you use any kind of RCS/CVS? Maybe you could start using GitHub, GitLab, BitBucket, Mercurial, or Fossil. Isn't that hard.

Then you could make progress in smaller steps and roll back in case of problems.

---
Forum admin

BTW I checked my DVD-RAM discs and 2 of 5 ended in state when I cannot write any new files even there's still left enough space. So I decided to reformat this discs with latest ver. of Panasonic DVDForm (full format) but the formatting process ended with an error like this:
error code[20000060], SCSI Status[02033100]
I tried it to format again on ~7 various DVD drives I have (LG, Samsung, Toshiba, NEC) but it always ended with error. Only difference was the time it took, some dives gave up quite soon after few minutes, others almost finished at 95% after ~40mins but failed too. One disc then ended in write protected state and couldn't be formated anymore. Of course I tried WPtool to unprotect it but it's not possible. As a last resort I could try to format them under Linux. I bought these discs 10-14 years ago, some are Panasonic, others Verbatim, 3x speed. So I cannot longet trust DVD-RAM as reliable media for longtime backup...
I'd like to hear from other long time DVD-RAM users if they had any issues like this and what drives theys use as I don't know anybody else who still use DVD-RAM (and most of them don't use any other optical medias).

---
DOS gives me freedom to unlimited HW access.

> > Khusraw's DOS port of MPLAYER can play (unencypted) Blu-ray discs, but
> only
> > if formatted in ISO9660 standard, as there's no UDF driver. The problem
> is
> > in MS-DOS, filesizes > 2GB aren't readable due to signed longint
> > limitation.
> >
> > There's an experimental exFAT reader for DOS as well, but the problem
> again
> > is that filesizes > 2GB aren't readable, so that completely defeats the
> > purpose of having an exFAT driver in the first place. The raw assembly
> code
> > of MSDOS.SYS and / or IO.SYS needs to be modified somehow. Even if
> > possible, I do not know if filesizes > 4GB can be directly supported.
> >
> > Same issues for the R. Loew drivers (I think?), or did he somehow
> magically
> > erase and / or patch the 32-bit longint limitations on filesizes as well
> as
> > increase sector sizes and capacities of drives beyond the 2TB limit? I'm
> > not sure, as I've not personally tested those drivers.
>
> It is possible for a file system redirector (like for ISO 9660, UDF, exFAT,
> NTFS, or dosemu's host FS) to support 64-bit file sizes and seeks in a
> manner compatible to the FAT+ support of Enhanced DR-DOS. That means
> expanding the SFTs with extension fields (can be stored in file handle
> tables of the redirector), hooking the new-style interrupt 21h service
> 7142h for large seeking, and service 42h to make old-style seeking somewhat
> aware of the large size support. I did this
> for dosemu2's mfs.
> Technologically it works, but it was rejected on other grounds.

A subsequent revision of my patch got merged today.

---
l

Well, I was able to unlock my DVD-RAM disc under linux but format failed anyway. In meantime I searched for other user exp. with RAMs and found a thread on AVforums where also other users reports RAM R/W issues after 10+ years so it's quite generic, not fault on my drives. RAMs are rottig same as other optical medias, a bit dissapointed...

---
DOS gives me freedom to unlimited HW access.

> Well, I was able to unlock my DVD-RAM disc under linux but format failed
> anyway. In meantime I searched for other user exp. with RAMs and found a
> thread on AVforums where also other users reports RAM R/W issues after 10+
> years so it's quite generic, not fault on my drives. RAMs are rottig same
> as other optical medias, a bit dissapointed...

This confirms my experiences with DVD-RAM also. All of my discs are rotted. Some are 14 or 15 years old, they were never reliable to begin with. Most optical media is rubbish. The best brands, Taiyo-Yuden were discontinued years ago, and what is made now is mostly Chinese garbage. I have TY media from over 20 years ago that still plays perfectly, with no errors (c1 and low c2 error rate). For BD-R, I find Optical Quantum brand to be the most reliable made today.

I thought I might go into my idea in a little more detail since I didn't explain it all that well and it didn't generate a lot of conversation when I thought it might.

The way MSCDEX (and its clones) work now is that you first need to install a hardware-specific device driver for the CD drive. This is usually done in CONFIG.SYS, and the installed driver is a character device rather than a block device (that is, it is installed with a device name like "SCSI_CD" rather than installing as a drive letter like "E:"). When you load MSCDEX, you must tell MSCDEX which character device (name) you want it to "attach" to on the hardware side, and MSCDEX sets up what amounts to two interfaces on the DOS side -- a drive letter (like "E:") and a special interface that lets DOS access audio CDs.

In the days when MSCDEX was created, this did make sense since things like DVDs and BDs and USB and UDF and even FAT32 didn't yet exist. Even though it made sense at the time, though, it wasn't very flexible and can't handle the hardware that we have today.

I think the problem is that MSCDEX was essentially designed backwards. MSCDEX "attaches" to the character device drivers when you install it, and I think MSCDEX should simply provide two interfaces (one for audio the the other for data). The hardware-specific part is installed after MSCDEX (or at least attaches to MSCDEX after MSCDEX is installed), but there shouldn't be a "permanent" attachment or binding between MSCDEX and the hardware-specific driver like there is now. MSCDEX provides "virtual slots" that hardware can attach to and detach from on-the-fly, and which "virtual slot" gets plugged into depends on what kind of media is inserted into the hardware.

In the same modern optical driver, e.g, you can insert almost every kind of CD, DVD, or BD that has ever existed (audio CD, data CD, DVD+, DVD-, DVD-RAM, BD, read-only, writable, and rewritable, potentially formatted in all kinds of different ways). I don't think the binding between MSCDEX and the hardware should be "permanent" the way it is now.

This is conceptually what I'm trying to do with my USB drivers, where I create "virtual slots" for the USB disks that get inserted and removed and are "attached" to DOS based on how they are configured and formatted in quasi-real-time rather than making a permanent connection when the program is installed. Making permanent connections for things that are purposely designed to be removable (like optical and USB media) doesn't make a lot of sense.

Thanks for sharing experience, just confirms that DVD-RAM are not long-lasting as expected - I read ~30yrs. As normal DVD+R you can still buy Data Tresor Disc or Verbatim M-Dics that claims to use some long-lasting anorganic dye (not all burners can burn them). I read that Verbatim also eneded the production and now only the rest is going to sold out so I have to buy some soon till available. BD-R are here for shorter time and not much spreaded around users so there's much less experiences so I don't belive it woud be better. I can see there is aslo M-Disc BD 100GB (4-layer), quite expansive but still with with better ratio GB/$ than DVD, but I didn't find any PC internal burner in our shops that supports 4-layers, they can do only 2.

---
DOS gives me freedom to unlimited HW access.