DOS ain't dead

> But, formatting optical media (especially BD-R) as FAT32 won't get you
> much, as of course FAT32 will not allow files > 4GB and MS-DOS won't allow
> long unsigned integer files between 2 and 4 GB to be read, since only
> signed integer file sizes seem to be supported. It's a VERY frustrating
> limitation, since filesizes can't be negative, why waste the upper bit?
> That limitation would need to be removed first. BD-Rs are UDF exclusively,
> I believe, and UDF allows creation of files up to 2TB (?) I believe,
> certainly well beyond 4GB, at least, since discs often contain single .m2ts
> files that are beyond 45 GB in length.

If you're wanting to read movie files from BD's, you're correct that the files sizes can be huge (far greater than FAT32 can handle). I was thinking more along the lines of using a BD (or DVD or CD) as a backup mechanism, or even a way to transfer files between computers. I don't know of a DOS program that will play BD movies (and I doubt that there ever will be one), but if you are treating a BD simply as a media storage device (and don't really care about the formatting) I think you could create something roughly equivalent to the DISKCOPY program that could copy the entire disk. Optical media has sectors much like magnetic media, though optical media is designed to be written serially rather than randomly, though I don't think it necessarily NEEDS to be that way.

Another possibility is exFAT, though that is considered proprietary by MS. The point is that you don't need to limit yourself to UDF just because everybody else does -- I think FAT32 or exFAT would work just fine on optical media (but I could be wrong). However, you cannot simply treat it like a "big floppy" since you can only write a limited number of times to the disk, and every time you write a file you also need to modify the FAT. That's one of the ways that DVD-RAM is superior to the other optical alternaitves -- you can reliably write to a DVD-RAM many more times than you can other optical media. Essentially, you can treat a DVD-RAM as a really big floppy.

> What are some of the hurdles in coding a UDF-capable file-system driver for
> DOS? Does MSCDEX allow only ISO9660 format, or does it also support FAT32?
> I also have wondered why UDF support was never envisioned for DOS, as such
> a driver, like the two or three existing NTFS filesystem drivers for MS-DOS
> would be very, very useful. Is the old High-Sierra format compatible with
> 2k sectors and / or file sizes larger than 4GB? So many questions, and few
> answers.

As alluded to earlier, the main issue with UDF is that it is a "moving target". They keep modifying it all the time so it is very difficult to spend time writing a driver that will work reliably all the time. As I was investigating and coming to the conclusion that FAT32 is probably a good option, I looked a little bit into what it would take to write a UDF driver and didn't like what I saw. The best option would probably be to try and port something from *nix.

> Along these lines, supporting 4Kb sector-size hard disks should be possible
> in MS-DOS, if 2048-byte sector sizes are possible and will work. Is it
> possible with a simple patch to IO.SYS and COMMAND.COM?

Yes it is, at least for MS-DOS. It's probably also easy for PC-DOS (since MS-DOS and PC-DOS came from the same roots), but maybe not for the other DOS variations (like PTS-DOS or DR-DOS or others). I know for sure it's NOT possible with any of the FreeDOS versions.

With MS-DOS, there are two ways you can do it. The first is to load a disk device driver in CONFIG.SYS that has the correct sector sizes. I do this with a configurable RAM Disk driver and tell it to use 2048 byte sectors. Most RAMDISKs won't allow anything other than 512 byte sectors, but there is one called TurboDisk (TDISK) that does. I set up a RAM Drive with TurboDisk and point my TEMP and TMP environment variables at it. That way I am also sure that all of my TEMP files are automatically erased when I reboot. MS RAMDRIVE will allow 1024 byte sectors and TurboDisk will allow 2048 byte sectors. I don't know of one that will allow 4096 byte sectors, but it would be possible to write one. If you load such a driver in CONFIG.SYS, the MS-DOS kernel will adjust itself to comply with the max sector size when it boots and you don't actually need to modify anything in the kernel. The MS-DOS kernel will supposedly support sector sizes of up to 8192 bytes, but I've never experimented with anything more than 2048 bytes.

The second option is to modify the MS-DOS kernel. Again, this will probably only work with MS-DOS and maybe PC-DOS but will probably not work with any other DOS. Following is an excerpt from the comments in my USBDRIVE source code that discusses how to modify the MS-DOS kernel. The only file you need to modify is MSDOS.SYS and you only need to modify one word (two bytes) in it.


;============================================================================
;Changing DOS kernel to support Sector Size larger than 512 bytes
;============================================================================
;
; Before you do this, be aware that DOS will use a LOT more memory than it
; does if you leave things at the default values. In particular, you may
; want to change the BUFFERS value in CONFIG.SYS to something smaller than
; normal. If you have 640k of RAM available to DOS (all modern computers
; do), by default DOS will create 15 buffers, each one large enough to store
; a sector's worth of data. Under normal conditions (512-byte sectors), this
; will be 7.5k of RAM dedicated to disk buffers. If you increase the sector
; size to, say, 2048 bytes (instead of 512), and leave the number of buffers
; at the default (15), there is now 30k of RAM dedicated to disk buffers
; instead of 7.5k. Unless you're actually accessing the disk with the
; larger sector size, much of the memory dedicated to disk buffers never
; gets used and is essentially wasted. In other words, you shouldn't
; change the sector size in the DOS kernel unless you have a really good
; reason -- otherwise, you're simply wasting some of your precious memory
; for no reason. Also, modern hard drives are pretty fast, so the DOS
; buffers (sort of a small-scale disk-cache) don't provide as much benefit
; like they did in the old days.
;
; DOS does have the capability to support sector sizes larger than 512
; bytes (supposedly up to 8192 bytes), but this is not enabled by default.
;
; DOS internally maintains several memory buffers, each large enough to
; store one sector's worth of data. The number of these buffers defaults
; to 15 on most modern computer systems, but can be changed using the BUFFERS
; option in your CONFIG.SYS file. The size of each buffer must be large enough
; to store a sector's-worth of data from any disk that DOS may need to access
; while running. So, the buffers must be big enough to hold the largest sector of
; ANY disk that DOS may need to access. The sector size of almost all disks
; is 512 bytes, but there are some exceptions. It is those exceptions that
; cause all of the grief.
;
;
;
; You should NOT do this unless you need to load a disk that has a sector
; size larger than 512 bytes. This will unnecessarily increase the amount
; of memory that DOS uses, and also increase the amount of memory that
; USBDRIVE uses (USBDRIVE sets up an internal buffer that is the same size
; as the DOS buffers).
;
;
;
; The size of the internal DOS sector buffers is determined at boot time using
; the default disk device drivers built into the DOS boot kernel (MSDOS.SYS
; for MS-DOS, IBMDOS.COM for PC-DOS). The kernels have a "default" minimum
; size of 128 (0080h) bytes hard-coded into the kernel. During
; the boot process, the kernel adjusts this size to match the largest
; sector size of any disks it finds (which is very rarely anything other
; than 512 bytes). The problem is, the boot kernel stops adjusting the
; size after the default disk device drivers (those built into the kernel)
; are finished processing the disks. You can continue to load disk device
; drivers after the kernel has finished, using your CONFIG.SYS file (or via
; a TSR like USBDRIVE), but the DOS kernel has already determined the maximum
; sector size it will use (and reserved the memory it needs) before it gets
; to that point, so it's too late to change it.
;
; Since it is impossible to change the maximum sector size after the DOS
; kernel has decided what it should be, you have to change the DOS kernel.
; The way you do this is to change the "default" sector size that is hard-coded
; into the kernel from 128 bytes to whatever you want it to be (presumably,
; something larger than 512 bytes). I've seen documentation that says DOS
; will support sector sizes up to 8192 bytes, though I have not personally
; tested anything larger than 2048 bytes.
;
; The easiest and fastest way to perform the patch is to use DOS's
; extremely powerful DEBUG program. DEBUG is a powerful program, and is
; therefore also extremely dangerous in the hands of someone who does not
; know what they are doing. If you're not at least a little bit familiar
; with hexadecimal arithmetic, memory segments and offsets, and similar
; concepts, you're probably better off letting someone else do this for you.
; This will take you through the steps one at a time and explain what each of
; them does, but if you're queasy about this kind of thing, let someone
; help you.
;
; First of all, you will need to locate and modify the attributes of the DOS
; boot kernel before you can edit it. The file will be located in the root
; directory of your boot drive (usually C:\, or A:\ if you boot from a
; floppy). If you're using MS-DOS (Microsoft), the file is called MSDOS.SYS.
; If you're using PC-DOS (IBM), the file is called IBMDOS.COM. If you're
; using DOS from some other manufacturer, you'll need to figure out
; which file you should be editing.
;
; The boot kernel file is a critical system file, and is "hidden" from
; normal view so that you can't move or delete or change it
; by accident. You need to change the attributes of the file so that
; you can see it and edit it. Later versions of DOS came with the ATTRIB
; utility that lets you do just that, and there are also several
; after-market utilities that will let you accomplish the same thing.
;
; Let's assume you're using MS-DOS, that you're in the root directory
; of the hard drive you booted from (C:\), and that you're using the MS-DOS
; ATTRIB utility. To make sure that the MSDOS.SYS file is there, you would
; do the following:
;
; ATTRIB MSDOS.SYS {Enter}
;
; The computer will respond with something like this:
;
; SHR C:\MSDOS.SYS
;
; Note that a "DIR MSDOS.SYS" would say the file doesn't exist, since it
; has several special attributes set (S = System, H = Hidden, and
; R = Read-only) that keep the DIR command from "seeing" it. To unset
; the S, H, and R attributes so that you can edit the file, do the following:
;
; ATTRIB -R -S -H MSDOS.SYS {Enter}
;
;
;
; Can make backup copy here just in case things get screwed up. Problem is,
; need SYS to store file (must be in specific location on disk), so a backup
; on the same drive doesn't do a lot of good. If you want a backup, you
; should have a bootable floppy or something to store the original file.
;
;
;
; Now for the fun part -- we'll get into the DOS kernel and change it using
; DEBUG. Keep in mind that everything in DEBUG uses hexadecimal numbers
; and arithmetic (not decimal):
;
; DEBUG MSDOS.SYS {Enter}
;
; The computer will simply respond with a "-", which is the DEBUG prompt
; asking you what it should do next:
;
; -
;
; The first thing you need to do is ask DEBUG how big the file it loaded
; actually is (you'll need to know this for the next step). DEBUG puts
; the file size in the CPU's CX register when it loads, so you just need
; to ask DEBUG what's stored in the CPU's CX Register:
;
; RCX {Enter}
;
; The computer will respond with something like this:
;
; CX 94FA
; :
;
; "94FA" is the hexadecimal value currently stored in the CX register
; (the number of bytes in the MSDOS.SYS file), and the ":" is DEBUG
; prompting you to enter a new value to be stored in the CX register.
; You don't want to change it, so:
;
; {Enter}
;
; DEBUG will respond with another "-" prompt:
;
; -
;
; Now, we need to Search in the kernel file to find out where the
; NUL device header is located. We will use DEBUG to Search (S), starting
; at the beginning of the file (offset 0), for the entire Length (L) of the
; file, for the string 'NUL ' (NUL followed by 5 spaces):
;
; S 0 L 94FA 'NUL ' {Enter}
;
; Note that in most cases, whether characters are upper-case and lower-case
; doesn't matter, but it DOES matter in the search string ('NUL ') --
; NUL must be in upper-case letters. The number following the 'L' should be
; the same number that was returned by DEBUG in the "RCX" step above.
;
; The computer will respond with something like this:
;
; 1924:82E2
; -
;
; The second number (the one after the colon, 82E2 in this case) is the
; offset into the file where the NUL device header is located. The location
; in the file we actually need to change is located 28 (1Ch) bytes before
; the start of the NUL device header. Since most people can't subtract
; hexadecimal numbers (like 82E2h and 1Ch) easily in their heads, we'll have
; DEBUG do it for us using the H (Hexadecimal math) function:
;
; H 82E2 1C {Enter}
;
; The computer will respond with something like this:
;
; 82FE 82C6
;
; The first number (82FE is the sum of the two numbers (82E2h + 1Ch), and the
; second number (82C6) is the difference of the two numbers (82E2h - 1Ch).
; We need the second number (the difference) for the next step, where we're
; going to make sure that we've got the correct file offset before we
; change anything. We'll use DEBUG's D (Dump memory) function to see what's
; stored in the kernel file at this location. We'll look at enough data to
; make sure we see the data we're trying to change, along with the NUL device
; header that we originally searched for:
;
; D 82C6 L 28 {Enter}
;
; The computer will respond with something like this:
;
;
; 1924:82C0 80 00-00 00 00 00 00 00 00 00 ..........
; 1924:82D0 00 00 00 00 00 00 00 00-00 00 00 00 04 80 C6 0D ................
; 1924:82E0 CC 0D 4E 55 4C 20 20 20-20 20 00 00 00 00 ..NUL ....
; -
;
; The far left column is the data address (file offset), the second
; column is the hexadecimal value of each data byte, and the last column
; is the data represented as ASCII text (with a "." in the last column
; indicating that the byte can't be represented as a printable ASCII character).
; The important thing is that the first two numbers you see are 80 00,
; and that you see a 'NUL' followed by 5 spaces (20h's) 4 bytes from the end
; of the Dump. What the other numbers are in between and after don't
; really matter.
;
; The two bytes at the beginning (80 00) are the ones we need to change.
; In the Intel architecture, data is stored in a little-endian format, which
; means that the Least Significant Byte (LSB) is stored first, followed by
; the Most Significant Byte (MSB). This actually makes a lot of sense to
; the computer, but is very confusing to humans because it is backwards
; from the way we usually write things. The two bytes, 80 00, converted to a
; human-readable form, is 0080h, which equates to 128 decimal (the
; hard-coded "default" sector size). Following are the values we may want
; to change the default sector size to:
;
; Decimal Hex DEBUG
; ------- ----- -----
; 1024 0400h 00 04
; 2048 0800h 00 08
; 4096 1000h 00 10
; 8192 2000h 00 20
;
; To change data in the file, you use DEBUG's E (Edit) command. For instance,
; to change the sector size to 2048 bytes, you would do the following:
;
; E 82C6 00 08 {Enter}
;
; The computer will simply respond with another "-" prompt:
;
; -
;
; To verify that the data was actually changed like you thought it was,
; you should do another Dump:
;
; D 82C6 L 2 {Enter}
;
; The computer will respond with something like this:
;
;
; 1924:82C0 00 08 ..
; -
;
; If you're satisfied with what you've done, you can W (Write) the file
; to make the changes permanent:
;
; W {Enter)
;
; The computer will respond with something like this:
;
; Writing 094FA bytes
; -
;
; If you're not sure you've done everything correctly, or screwed something
; up so badly that you just want to start over again, just skip the W command.
; Unless you tell DEBUG to Write the file, none of the changes you've made
; are permanent.
;
; To exit DEBUG, use the Q (Quit) command:
;
; Q {Enter}
;
; Now that the file is changed, you should set the S, H, and R attributes
; again so that you won't erase or change the file by accident:
;
; ATTRIB +R +S +H MSDOS.SYS
;
; Now, reboot and you should be all set.
;
;
; Here is a sample session to change Sector Size to 2048 for MS-DOS 6.20:
;
; C:\>attrib msdos.sys
; A SHR C:\MSDOS.SYS
;
; C:\>attrib -r -s -h msdos.sys
;
; C:\>attrib msdos.sys
; A C:\MSDOS.SYS
;
; C:\>debug msdos.sys
; -rcx
; CX 94FA
; :
; -s 0 l 94fa "NUL "
; 1E5E:82E2
; -h 82e2 1c
; 82FE 82C6
; -d 82c6 l 28
; 1E5E:82C0 80 00-00 00 00 00 00 00 00 00 ..........
; 1E5E:82D0 00 00 00 00 00 00 00 00-00 00 00 00 04 80 C6 0D ................
; 1E5E:82E0 CC 0D 4E 55 4C 20 20 20-20 20 00 00 00 00 ..NUL ....
; -e 82c6 00 08
; -d 82c6 l 28
; 1E5E:82C0 00 08-00 00 00 00 00 00 00 00 ..........
; 1E5E:82D0 00 00 00 00 00 00 00 00-00 00 00 00 04 80 C6 0D ................
; 1E5E:82E0 CC 0D 4E 55 4C 20 20 20-20 20 00 00 00 00 ..NUL ....
; -w
; Writing 094FA bytes
; -q
;
; C:\>attrib +r +s +h msdos.sys
;
; C:\>attrib msdos.sys
; A SHR C:\MSDOS.SYS
;
; C:\>
;
; Now, after a reboot, the DOS sector size is 2048 bytes.