DOS ain't dead - Accessing and/or creating files

Zyzzle

30.06.2009, 23:27

Accessing and/or creating files > 2 GB in DOS? (Users)

Is there any way to access OR create a file > 2 GB in pure DOS? I think it SHOULD be possible to create a file > 2 GB but less than 4 GB in size, but I do not know how. I know it has to do with the high-order bit and its indicating positive or negative numbers. Since there are no negative file sizes, can DOS be patched to allow access from 0 to 4 GB - 1 file sizes at least?

I know FAT16 partitions with 64 kb cluters sizes (4094 MB) and FAT32 partitions would benefit greatly from 2 - 4 GB+ file size access! Has this been done?

I want to avoid using NTFS in DOS because of the large memory-overhead footprint of the drivers required to access it.

In particular, I want the DOS port of MPLAYER to be able to access video files > 2 GB in size. Is this possible or a pipe dream?

How about ExFAT (FAT64) in DOS and / or FAT32ex?

RayeR

CZ,
01.07.2009, 02:28

@ Zyzzle

Accessing and/or creating files > 2 GB in DOS?

Post reply

> Is there any way to access OR create a file > 2 GB in pure DOS? I think it
> SHOULD be possible to create a file > 2 GB but less than 4 GB in size, but

I think it's not limitation of filesystem but DOS and programs which works with files (in case of mplayer the libc functions for seeking and reading - what data types it use). There's a patch for windows 98 that allows work with 4GB files in standard way. I tried it only on FAT32 partition not on FAT16 and it works well. So I think it shouldn't be impossible under DOS too (AFAIK DJGPP use 32bit signed pointer for files so it would be needed to extend it to 64bit and also probably FreeDOS would need some patch).

---
DOS gives me freedom to unlimited HW access.

Zyzzle 01.07.2009, 04:07 @ RayeR	Accessing and/or creating files > 2 GB in DOS? Post reply
	I'd be interested in that 4GB Win98 patch for MPLAYER, if you have a link... sounds interesting.

ecm

Düsseldorf, Germany,
01.07.2009, 10:56

@ RayeR

Accessing and/or creating files > 2 GB in DOS?

Post reply

> > Is there any way to access OR create a file > 2 GB in pure DOS? I think
> it
> > SHOULD be possible to create a file > 2 GB but less than 4 GB in size,
> but
>
> I think it's not limitation of filesystem but DOS and programs which works
> with files (in case of mplayer the libc functions for seeking and reading -
> what data types it use). There's a patch for windows 98 that allows work
> with 4GB files in standard way. I tried it only on FAT32 partition not on
> FAT16 and it works well. So I think it shouldn't be impossible under DOS
> too (AFAIK DJGPP use 32bit signed pointer for files so it would be needed
> to extend it to 64bit and also probably FreeDOS would need some patch).

The low-level MS-DOS interface always uses unsigned numbers. In theory, MS-DOS can use and access files up to 4 GiB (less 1 byte). Earlier MS-DOS versions however only supported partitions up to (almost) 2 GiB. MS-DOS 7.10 supports FAT32 and supposedly (= I never tested it) files larger than 2 GiB. RBIL says the file must be opened with Open/Create (Int21.6C, or Int21.716C) with a new bit set to allow access to the part above 2 GiB. HLL libraries often still use functions 21.3D (Old open) and 21.3C (Old create/truncate) which don't allow setting that bit anyway. And even if the library uses 21.6C, it must be patched to set the bit. (And of course, ~~stupid~~ conventions of signed seek positions or negative values as error code must be dropped.)

"The FreeDOS kernel" (DOS-C) is currently being fixed to support files up to 4 GiB.

---
l

Laaca

Czech republic,
01.07.2009, 14:41

@ ecm

Accessing and/or creating files > 2 GB in DOS?

Post reply

And EDR-DOS (www.drdosprojects.de) can access files up to 256GB

But don't forget that it is support only on level of operating system. The applications may behave incorrectly with files > 2GB because as fileposition counter are usually used 32-bit signed interegers.

---
DOS-u-akbar!

RayeR

CZ,
01.07.2009, 18:40

@ Laaca

Accessing and/or creating files > 2 GB in DOS?

Post reply

> And EDR-DOS (www.drdosprojects.de) can access files up to 256GB

On what FS? It is not possible on FAT16/32.

---
DOS gives me freedom to unlimited HW access.

Khusraw

Bucharest, Romania,
01.07.2009, 19:50

@ RayeR

Accessing and/or creating files > 2 GB in DOS?

Post reply

> On what FS? It is not possible on FAT16/32.

On FAT+. :-D

---
Glory to God for all things

RayeR

CZ,
02.07.2009, 12:46

@ Khusraw

Accessing and/or creating files > 2 GB in DOS?

Post reply

> On
> FAT+. :-D

Eh, very usefull on such exotic FS...

---
DOS gives me freedom to unlimited HW access.

Rugxulo

Usono,
01.07.2009, 22:27

@ RayeR

Accessing and/or creating files > 2 GB in DOS?

Post reply

> I think it's not limitation of filesystem but DOS and programs which works
> with files (in case of mplayer the libc functions for seeking and reading -
> what data types it use). There's a patch for windows 98 that allows work
> with 4GB files in standard way. I tried it only on FAT32 partition not on
> FAT16 and it works well. So I think it shouldn't be impossible under DOS
> too (AFAIK DJGPP use 32bit signed pointer for files so it would be needed
> to extend it to 64bit and also probably FreeDOS would need some patch).

According to CWS, DJGPP 2.04 beta has improved FAT32 / 4 GB file support, but I don't know exactly what that means.

Rugxulo

Usono,
01.07.2009, 22:26

@ Zyzzle

Accessing and/or creating files > 2 GB in DOS?

Post reply

> I know FAT16 partitions with 64 kb cluters sizes (4094 MB) and FAT32
> partitions would benefit greatly from 2 - 4 GB+ file size access! Has this
> been done?

I'm not sure, but isn't 64k cluster sizes a NT thing? I don't know offhand of any pure DOS doing such.

> I want to avoid using NTFS in DOS because of the large memory-overhead
> footprint of the drivers required to access it.

Try LTOOLS instead with ext2. I'm not saying it works (I dunno), but it probably does.

> How about ExFAT (FAT64) in DOS

Don't count on this ever being implemented. Even Linux 2.6.30 removed write support for LFNs on FAT due to MS patents. And if something as old as that is patented, you can be sure that exFAT (XP, Vista, 7) is probably also.

Zyzzle 02.07.2009, 04:44 @ Rugxulo	Accessing and/or creating files > 2 GB in DOS? Post reply
	I'm able to read and write to FAT16 partitions with 64kb cluster sizes in raw MS-DOS 7.1. Haven't tested true DOS 6.2 yet... FAT+ looks interesting, but individual applications which use signed 32-bit numbers to read/write files will probably be broken with the FAT in use (?) Also, thanks all for the pointers and facts.

Khusraw

Bucharest, Romania,
02.07.2009, 07:55

@ Zyzzle

Accessing and/or creating files > 2 GB in DOS?

Post reply

> FAT+ looks interesting, but individual applications which use signed
> 32-bit numbers to read/write files will probably be broken with the FAT in
> use (?)

The applications which use signed 32-bit numbers to seek in files won't be affected, as they use another function (AH=42h). Reading/writing files is always done using unsigned 16-bit numbers.

---
Glory to God for all things

Rugxulo

Usono,
05.07.2009, 21:34

@ Rugxulo

Accessing and/or creating files > 2 GB in DOS?

Post reply

> > How about ExFAT (FAT64) in DOS
>
> Don't count on this ever being implemented. Even Linux 2.6.30 removed
> write support for LFNs on FAT due to MS patents. And if something as old
> as that is patented, you can be sure that exFAT (XP, Vista, 7) is probably
> also.

Just for reference, here's what's new in Linux land:

http://www.osnews.com/story/21766/Linux_Kernel_Patch_Works_Around_Microsoft_s_FAT_Patents

ecm

Düsseldorf, Germany,
06.07.2009, 12:15

@ Rugxulo

Accessing and/or creating files > 2 GB in DOS?

Post reply

> Just for reference, here's what's new in Linux land:
>
> http://www.osnews.com/story/21766/Linux_Kernel_Patch_Works_Around_Microsoft_s_FAT_Patents

Great, they make sure not to be sued for silly patents by writing trash into the space for the normal file name. Files with such hacked hacked directory entries won't be accessible in DOS - even if using DOSLFN.

---
l

Rugxulo

Usono,
08.07.2009, 07:15

@ ecm

Accessing and/or creating files > 2 GB in DOS?

Post reply

> Great, they make sure not to be sued for silly patents by writing trash
> into the space for the normal file name. Files with such hacked hacked
> directory entries won't be accessible in DOS - even if using DOSLFN.

Blame the patent trolls. Or those who can't live without really long filenames. Anyways, are you surprised? Linux has long since disliked DOS compatibility, so it's not like this is a new thing, more of a symptom of the real problem.

BTW, I still think StarLFN's method (basically use LONGNAME.DAT a la 4DOS' DESCRIPT.ION) is a better idea, but recently I tried compiling some stuff (latest GNU Emacs pretest) with StarLFN loaded, and it slowed down big time! So I dunno, maybe StarLFN isn't ready for primetime. :-/

ecm

Düsseldorf, Germany,
08.07.2009, 12:43

@ Rugxulo

Accessing and/or creating files > 2 GB in DOS?

Post reply

> Anyways, are you surprised? Linux has long since disliked DOS
> compatibility, so it's not like this is a new thing, more of a symptom of
> the real problem.

I think it's rather that Linux doesn't care about DOS at all. Most Linux developers probably never heard of anything but MS-DOS.

> BTW, I still think StarLFN's method (basically use LONGNAME.DAT a la 4DOS'
> DESCRIPT.ION) is a better idea, but recently I tried compiling some stuff
> (latest GNU Emacs pretest) with StarLFN loaded, and it slowed down big
> time!

Did you load some kind of disk cache? It could improve StarLFN's performance. The LONGNAME.DAT file is accessed using usual DOS functions.

> So I dunno, maybe StarLFN isn't ready for primetime. :-/

DOSLFN contains one note about what modes are implemented and what needs to be done:

; Another useful mode would be an LFNBK mode, simply using a file ; for keeping long file names - this would instantly work on all drives.

This could be used instead of the fallback mode used for unknown redirector drives. StarLFN actually does just that (in non-VFAT mode) already. (However, integrating StarLFN into DOSLFN should include changing the database layout - as mentioned in the StarLFN documentation, there are some flaws with the DESCRIPT.ION type files.)

---
l

ecm

Düsseldorf, Germany,
13.08.2009, 12:10

@ Rugxulo

Retrieving files with invalid names

Post reply

I now added some kind of "support" for files with invalid file names to my current RxDOS source code. When it's determined that the file name is invalid but the attributes show that this entry should be a file or directory (not a label/LFN/whatever), the file name is internally replaced by a representation of the directory entry's cluster and an index into the cluster. This is coded in digits 0..9 and letters A..V allowing to store 5 bits per character. Since a file name created that way is unique on the drive (as long as the directory entry doesn't move), it's unique within any given directory too. To insure that the name doesn't collide with actually existing (long or short) names in the same directory, the last character of the name is set to colon (:), which is an otherwise invalid character for both long and short names. Since the name is always zero-padded, that colon comes as the 8th character only, so the DOS name expansion code can tell it apart from an actual drive letter.

Besides the ability to see (using Find First/Find Next, aka DIR) and access (as long as the application doesn't do explicit file name checks itself) files with valid long file name only (whether DOSLFN installed/RxDOSLFN embedded or not), this method also works for any other file with corrupted file name and for files that aren't accessible because a character device with the same name is installed. (In MS-DOS, file searching with wildcards such as "DIR *.*" [or simply "DIR"] shows files and directories named as character devices but they aren't accessible.)

Since a name provided this way is actually invalid, renaming or moving the file to another place but with the same name won't work. Packing a file with such a name into an archive might be a bad idea too.

---
l

DOS386

21.10.2009, 01:20
(edited by DOS386, 21.10.2009, 01:32)

@ Zyzzle

Accessing and/or creating files > 4 GiB in DOS?

Post reply

> Is there any way to access OR create a file > 2 GB in pure DOS?

Sorry but this is an old hat

> I want to avoid using NTFS in DOS

Me too :-)

> I want the DOS port of MPLAYER to be able to access video files > 2 GB in size.

EDR-DOS has neither 2 GiB nor 4 GiB limit, nor DGJPP has (surprisingly), but you probably can't seek beyond 2 GiB or 4 GiB :-| I haven't tested > 2 GiB with MPLAYER in DOS yet ... YES, it "works" in Windaube XP on NTSC, but causes horrible fragmentation, so, NextgenerationTechnologyF******S*** indeed kept what it had promised ... BTW, FreeDOS has a 2 GiB limit (at least 1.0, retest pending for kernel 2039) ... link

> How about ExFAT (FAT64) in DOS

NO thanks :-(

> Is this possible or a pipe dream?

work work work work work work work work work work work work dream ------------------------------- ---------------------------------- ------------------------------------>>> reality work work work work work work work work work work work work

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***