Directory Monitor

With iOS 4.0, AAPL brought File Sharing to the iPhone. This feature is pretty simple to turn on (basically, you just “[a]dd the UIFileSharingEnabled key to your application’s Info.plist file and set the value of the key to YES“, and then the user can access your app’s Documents directory through iTunes) but a little tricky to handle in practice.

The wrinkle is that you’ll probably want to manipulate the files that the user adds to (or removes from) the Documents directory and, since the user can update these files at any time, that means that you’ll probably want your app to monitor that directory for any changes. Such monitoring isn’t too complicated to set up, but it does require a trip into some of the more obscure parts of the BSD underpinnings of iOS. That’s the trip we’ll be taking today, with a demo project as a roadmap.

Code and Credits

First of all, you can download the complete demo project here. Install and run the app on a device (File Sharing only works on physical devices, and not in the simulator, though the monitoring code should work both places), then add/delete files through iTunes and watch the display update. Use the “Start” and “Stop” buttons to enable or disable monitoring (it’s enabled by default).

Secondly, I should mention that the demo is based on this post by an AAPL tech. Unfortunately, the post is behind the AAPL developer firewall, so you’ll need a login to read it.

Overview

The demo is basically a “Navigation-based Application” template in which the RootViewController has been modified to (a.) display a list of the files in the app’s Documents directory, and (b.) update that list whenever the contents of the directory change.

All the interesting code has been placed in a RootViewController extension:

@interface RootViewController ()

@property (nonatomic, assign, readonly) CFFileDescriptorRef _kqRef;
@property (nonatomic, retain) NSArray* fns;

- (void)syncOrderedList:(NSArray*)a withTarget:(NSArray*)b returnInserts:(NSMutableArray*)i andDeletes:(NSMutableArray*)d;
- (void)updateFns;
- (void)kqueueFired;
- (void)startMonitor;
- (void)stopMonitor;
- (void)restartMonitor;

@end

The RootViewController class itself is declared this way:

@interface RootViewController : UITableViewController
{
	int			_dirFD;
	CFFileDescriptorRef	_kqRef;
	
	NSArray* 		fns;
}

@end

Boilerplate

The setup code in viewDidLoad initializes the title bar text and buttons, sets the initial filename list, and starts monitoring the app’s Documents directory:

- (void)viewDidLoad
{
	[super viewDidLoad];
	
	self.title = @"Documents";
	
	self.navigationItem.leftBarButtonItem = [[[UIBarButtonItem alloc] initWithTitle:@"Start" style:UIBarButtonItemStyleBordered target:self action:@selector(restartMonitor)] autorelease];
	self.navigationItem.rightBarButtonItem = [[[UIBarButtonItem alloc] initWithTitle:@"Stop" style:UIBarButtonItemStyleBordered target:self action:@selector(stopMonitor)] autorelease];

	liveshareAppDelegate* delegate = (liveshareAppDelegate*) [[UIApplication sharedApplication] delegate];
	self.fns = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:[delegate applicationDocumentsDirectory]
								       error:NULL];
	
	[self startMonitor];
}

The dealloc method stops monitoring the app’s Documents directory and releases any retained objects:

- (void)dealloc
{
	[self stopMonitor];
	[fns release];
	[super dealloc];
}

The table functions are about what you’d expect; the only exception is tableView:commitEditingStyle:forRowAtIndexPath:, which uses an NSFileManager to remove the specified file, but does not update the table or in-memory data structure — directory monitoring handles that:

// Customize the number of sections in the table view.
- (NSInteger)numberOfSectionsInTableView:(UITableView*)tableView
{
	return 1;
}


// Customize the number of rows in the table view.
- (NSInteger)tableView:(UITableView*)tableView numberOfRowsInSection:(NSInteger)section
{
	return [self.fns count];
}


// Customize the appearance of table view cells.
- (UITableViewCell*)tableView:(UITableView*)tableView cellForRowAtIndexPath:(NSIndexPath*)indexPath
{
	static NSString* CellIdentifier = @"Cell";
    
	UITableViewCell* cell = [tableView dequeueReusableCellWithIdentifier:CellIdentifier];
	if (cell == nil)
	{
		cell = [[[UITableViewCell alloc] initWithStyle:UITableViewCellStyleDefault reuseIdentifier:CellIdentifier] autorelease];
	}
    
	// Configure the cell...
	cell.textLabel.text = [self.fns objectAtIndex:indexPath.row];
	
	return cell;
}


// Override to support conditional editing of the table view.
- (BOOL)tableView:(UITableView*)tableView canEditRowAtIndexPath:(NSIndexPath*)indexPath
{
	return YES;
}


// Override to support editing the table view.
- (void)tableView:(UITableView*)tableView commitEditingStyle:(UITableViewCellEditingStyle)editingStyle forRowAtIndexPath:(NSIndexPath*)indexPath
{
	if (editingStyle == UITableViewCellEditingStyleDelete)
	{
		// Remove the file
		NSString* base = [(liveshareAppDelegate*)[[UIApplication sharedApplication] delegate] applicationDocumentsDirectory];
		NSString* path = [base stringByAppendingPathComponent:[self.fns objectAtIndex:indexPath.row]];
		[[NSFileManager defaultManager] removeItemAtPath:path error:NULL];
	}
}

Monitoring Setup

Moving on to the heart of the matter, we come to startMonitor. There’s a lot going on here, much of it best explained in-line:

- (void)startMonitor
{
	// One ping only
	if (_kqRef != NULL) return;
	
	// Fetch pathname of the directory to monitor
	liveshareAppDelegate* delegate = (liveshareAppDelegate*) [[UIApplication sharedApplication] delegate];
	NSString* docPath = [delegate applicationDocumentsDirectory];
	if (!docPath) return;

	// Open an event-only file descriptor associated with the directory
	int dirFD = open([docPath fileSystemRepresentation], O_EVTONLY);
	if (dirFD < 0) return;
	
	// Create a new kernel event queue
	int kq = kqueue();
	if (kq < 0)
	{
		close(dirFD);
		return;
	}

	// Set up a kevent to monitor
	struct kevent eventToAdd;			// Register an (ident, filter) pair with the kqueue
	eventToAdd.ident  = dirFD;			// The object to watch (the directory FD)
	eventToAdd.filter = EVFILT_VNODE;		// Watch for certain events on the VNODE spec'd by ident
	eventToAdd.flags  = EV_ADD | EV_CLEAR;		// Add a resetting kevent
	eventToAdd.fflags = NOTE_WRITE;			// The events to watch for on the VNODE spec'd by ident (writes)
	eventToAdd.data   = 0;				// No filter-specific data
	eventToAdd.udata  = NULL;			// No user data

	// Add a kevent to monitor 
	if (kevent(kq, &eventToAdd, 1, NULL, 0, NULL))
	{
		close(kq);
		close(dirFD);
		return;
	}
	
	// Wrap a CFFileDescriptor around a native FD
	CFFileDescriptorContext context = {0, self, NULL, NULL, NULL};
	_kqRef = CFFileDescriptorCreate(NULL,		// Use the default allocator
					kq,		// Wrap the kqueue
					true,		// Close the CFFileDescriptor if kq is invalidated
					KQCallback,	// Fxn to call on activity
					&context);	// Supply a context to set the callback's "info" argument
	if (_kqRef == NULL)
	{
		close(kq);
		close(dirFD);
		return;
	}
	
	// Spin out a pluggable run loop source from the CFFileDescriptorRef
	// Add it to the current run loop, then release it
	CFRunLoopSourceRef rls = CFFileDescriptorCreateRunLoopSource(NULL, _kqRef, 0);
	if (rls == NULL)
	{
		CFRelease(_kqRef); _kqRef = NULL;
		close(kq);
		close(dirFD);
		return;
	}
	CFRunLoopAddSource(CFRunLoopGetCurrent(), rls, kCFRunLoopDefaultMode);
	CFRelease(rls);

	// Store the directory FD for later closing
	_dirFD = dirFD;
	
	// Enable a one-shot (the only kind) callback
	CFFileDescriptorEnableCallBacks(_kqRef, kCFFileDescriptorReadCallBack);
}

Monitoring Setup – Comments

Note that the directory is accessed with the low-level POSIX function open, and that O_EVTONLY is supplied as the oflag. (O_EVTONLY isn’t listed as a valid option on the open manpage, but it does work.) AAPL’s “File System Performance Guidelines” give some idea of what’s going on here:

When you only want to track changes on a file or directory, be sure to open it using the O_EVTONLY flag. This flag prevents the file or directory from being marked as open or in use. This is important if you are tracking files on a removable volume and the user tries to unmount the volume. With this flag in place, the system knows it can dismiss the volume. If you had opened the files or directories without this flag, the volume would be marked as busy and would not be unmounted.

AAPL’s “File System Performance Guidelines” also explain, in general terms, how kqueues and kevents are used in this code:

The kqueue mechanism in BSD provides another way to be notified of system changes. Using this mechanism you can request notifications when specific events occur or when a specific condition becomes true. You can use this to monitor files and other system entities such as ports and processes.

The operation of kevent isn’t exactly obvious. To begin with, there’s both a struct kevent structure and a kevent() function. Furthermore, the function is used both to update the events to monitor (via its changelist and nchanges arguments) and to retrieve events from the queue (via its eventlist and nevents arguments), and the structure is used to describe both events that should be monitored and events that have been retrieved.

In this particular case, we use kevent to specify an event to monitor; we add a EVFILT_VNODE filter with a NOTE_WRITE fflags qualifier to the kqueue. This will queue an event when “[a] write [occurs] on the file referenced by the descriptor”. Since the descriptor supplied was opened on a directory, an event will be queued whenever that directory’s contents change.

Once the kqueue has been created and configured, we wrap a CFFileDescriptor around it. CFFileDescriptors can be used “to monitor file descriptors for read and write activity via CFRunLoop using callbacks” — and since a kqueue looks like a file descriptor to the rest of the system, we can use a CFFileDescriptor to deliver asynchronous notifications when an event arrives on the queue.

We create our CFFileDescriptor with a call to CFFileDescriptorCreate(). There are two things to note here: first, we specify KQCallback as the callback (we’ll see this function’s definition later) and second, we pass a CFFileDescriptorContext as the function’s context argument. The context lets us specify the info argument that will be passed to our callback when it is invoked; in this case we want to pass a pointer to the RootViewController object itself as the info argument.

The penultimate step is to create a CFRunLoopSourceRef from the CFFileDescriptor, and to add that source to the current run loop. I’d be lying if I said I had a good handle on this part; CFRunLoops have always been a bit of vague background machinery to me. (Perhaps one day I’ll have to understand them better, but not today.) The effect, at any rate, is to enable the callback to be fired when appropriate, pending the next (and final) step:

The last step is to enable callbacks on the CFFileDescriptor. Somewhat weirdly, only one-shot callbacks can be enabled; i.e., once a callback is fired, the CFFileDescriptor will reset as if callbacks had never been enabled. Still, this will do for now.

That’s a lot of moving bits to get one lousy callback to fire. To recap, the steps are:

• Open a FD on the directory you want to monitor
• Open a kqueue
• Add a monitoring kevent to the kqueue
• Wrap a CFFileDescriptor around the kqueue
• Add a CFRunLoopSourceRef from the CFFileDescriptor to the current CFRunLoop
• Enable callbacks from the CFFileDescriptor

Callback

Speaking of callbacks, here’s the actual KQCallback() function that we passed to CFFileDescriptorCreate():

static void KQCallback(CFFileDescriptorRef kqRef, CFOptionFlags callBackTypes, void *info)
{
	// Pick up the object passed in the "info" member of the CFFileDescriptorContext passed to CFFileDescriptorCreate
	RootViewController* obj = (RootViewController*) info;
	
	if ([obj isKindOfClass:[RootViewController class]]		&&	// If we can call back to the proper sort of object ...
	    (kqRef == obj._kqRef)					&&	// and the FD that issued the CB is the expected one ...
	    (callBackTypes == kCFFileDescriptorReadCallBack)		)	// and we're processing the proper sort of CB ...
	{
		[obj kqueueFired];						// Invoke the instance's CB handler
	}
}

As you can see, it doesn’t really do anything except call an object method — it only exists because you can’t pass object methods to CFFileDescriptorCreate(). Here’s the “real” callback method:

- (void)kqueueFired
{
	// Pull the native FD around which the CFFileDescriptor was wrapped
	int kq = CFFileDescriptorGetNativeDescriptor(_kqRef);
	if (kq < 0) return;
	
	// If we pull a single available event out of the queue, assume the directory was updated
	struct kevent event;
	struct timespec timeout = {0, 0};
	if (kevent(kq, NULL, 0, &event, 1, &timeout) == 1)
	{
		[self updateFns];
    	}    
	
	// (Re-)Enable a one-shot (the only kind) callback
	CFFileDescriptorEnableCallBacks(_kqRef, kCFFileDescriptorReadCallBack);
}

This method doesn't do anything too spectacular either; it makes sure that it can pull an event off the kqueue, and, assuming it can, calls the object's updateFns method and re-enables the CFFileDescriptor's one-shot callback. As for the updateFns method itself: this post is already punishingly long, and updateFns doesn't do anything particularly surprising. (It does use the synchronization algorithm I've been talking about for the past two weeks, though.)

Cleanup

On last thing: This is the code I use to tear down monitoring:

- (void)stopMonitor
{
	if (_kqRef)
	{
		close(_dirFD);
		CFFileDescriptorInvalidate(_kqRef);
		CFRelease(_kqRef);
		_kqRef = NULL;
	}
}

I think this is right, but note that the CFRunLoopSourceRef is never explicitly removed from the current CFRunLoop. I sort of assume that this happens automagically when the FD is invalidated and/or the CFFileDescriptor is released. As I said, CFRunLoops aren't my strong suit, so: fair warning.

Multitasking

You might wonder how this code integrates with multitasking. "Pretty well", it turns out. Experimentally, changes made to the Documents directory when the app is in the background result in a kevent delivered once it returns to the foreground. So that's nice.