[hfs-user] Catalog File Size (HFS+)

[Mark Day]

On Apr 23, 2004, at 6:36 AM, Pierre Duhem wrote:

> Is there a recommendation by Apple concerning the size of the Catalog
> File (and of the Extents Overflow File), depending on the size of the
> physical drive?

There is no official recommendation, but I'll describe the default  
behavior in Mac OS X version 10.3.  If you're curious, you can download  
the sources for newfs_hfs from Mac OS X 10.3; it is part of the  
diskdev_cmds project in Darwin.  The URL is:
<http://www.opensource.apple.com/darwinsource/tarballs/apsl/ 
diskdev_cmds-277.1.tar.gz>

For volumes less than 1GB, the default catalog node size is 4KB; for  
larger volumes, the default is 8KB.  The default extents node size is  
4KB.

The default sizes of the catalog and extents files vary with the size  
of the volume.  For volumes less than 1GB, the default size for both  
B-trees is 1/128 of the size of the volume, but at least 8 nodes.  For  
larger volumes, the following table gives the default sizes:

short clumptbl[CLUMP_ENTRIES * 2] = {
/*
  *      Volume   Catalog     Extents
  *       Size   Clump (MB)  Clump (MB)
  */
     /*   1GB */   4,         4,
     /*   2GB */   6,         4,
     /*   4GB */   8,         4,
     /*   8GB */  11,         5,
     /*  16GB */  14,         5,
     /*  32GB */  19,         6,
     /*  64GB */  25,         7,
     /* 128GB */  34,         8,
     /* 256GB */  45,         9,
     /* 512GB */  60,        11,
     /*   1TB */  80,        14,
     /*   2TB */ 107,        16,
     /*   4TB */ 144,        20,
     /*   8TB */ 192,        25,
     /*  16TB */ 256,        32
};

The "clump size" here is both the initial size of the B-tree (in  
megabytes), and is the default amount to grow the tree by when it gets  
full.  These numbers are not special; they are educated guesses based  
on general usage.

When initializing a volume, I suggest that the bitmap (allocation file)  
and extents B-tree come before (lower disk addresses) than the catalog  
B-tree.  Also, set up the nextAllocation field of the volume header to  
be well past the end of the catalog (perhaps 2-4 times the size of the  
catalog beyond the end of the catalog).  This gives the catalog lots of  
room to grow contiguously as you initially populate the volume, but  
lets you eventually use the space for data if the catalog doesn't need  
to grow that much.

> Which threshold would be reasonnable to switch to 8192-byte nodes?

Apple's default is 1GB or larger volumes.  We noticed a non-trivial  
performance gain when using 8KB nodes compared to 4KB nodes, due to the  
large number of pathname lookups in Mac OS X.  The performance  
difference was negligible for Mac OS 8 and 9 (where applications use  
the parent directory ID and leaf name much more often than full path  
names).

> Is it reasonnable to think that, for the time being, 8192 is the
> biggest node size to be used?

It's the largest default node size created by any Apple OS.  But it is  
possible for nodes to be as large as 32KB.

The HFS Plus volume format is described here:
<http://developer.apple.com/technotes/tn/tn1150.html>

That technical note was updated at the beginning of March, 2004.  A lot  
of information related to Mac OS X (including the journal, metadata  
zone, and a case-sensitive variant) was added in that update.

-Mark