A Guide to doing it the Right Way


Consider this document public domain or the closest equivalent in your place of residence. Even if I'm disclaiming all my rights to this text, I would still like to be attributed as the original author, out of courtesy.

The C++ source code snippets given in this document are adapted from Aegisub which is licensed under a permissive BSD license, but these adapted snippets are given to the public domain as the rest of this document is. Feel free to use them in your own software with or without attribution.

---

Introduction

Something central to the user experience in Windows' shell has always been how applications manage file type associations. File type associations have
existed since Windows 3.0, possibly even earlier (I can't confirm that right now) but it was especially with Windows 95 they became truly visible to a large portion of the users, through the more pervasive file management interface the Explorer shell introduced.

However, Windows 95 and the later operating systems in the family had a problem, and we are still fighting the bad software practices that problem caused: All users were administrators, and all users, and thus the software they ran, could change the global file associations. Furthermore, the system
used for associations wasn't very mature or flexible at the time, and importantly couldn't keep track of multiple applications being able to handle the same file type, which caused lots of contention over some types, and gave rise to a horror people are still talking about, the associations thieves.

Especially media players and photo applications have been known to ruthlessly take any and all file type associations when installed, and possibly even every time they started. Some even went as far as installing background programs that ensured they had the associations all the time, and reset it if anything changed it.

Fortunately, with Windows 2000, XP and especially Vista, Microsoft introduced new ways for applications, and more importantly, for the users, to manage
file type associations. Microsoft also introduced User Account Control, which while hated by many, gives a big advantage: You can be an unprivileged user
and an administrator user at the same time, only taking the administrator privileges when really needed. Daily managing of file type associations does
not need administrator privileges, and can be done per user, and that is what I want to describe in this document.

This guide mainly applies to applications made for Windows XP and later, if you want to run on earlier versions of Windows not everything will apply and
not all techniques will work. Some of the best techniques only work on Windows Vista and later, but I'm providing workarounds to give almost the
same features on Windows XP and Server 2003. I don't have access to a Windows 2000 system, so I can't test how much applies to that version.

I am only going to touch associating file types with programs run with regular command lines. I will not touch protocol handler registration or DDE command verbs. If you need any of those you should read the documentation Microsoft provides on MSDN. While it is a bit fragmented, it is useful enough. (But for your own sake, try to avoid the DDE stuff! It became obsolete when preemptive multitasking was introduced in Windows. Disregard that Microsoft uses it for most of the Office Suite.)

---

A note about Portable applications

Portable applications have become popular along with the portable USB flash memory sticks. Why not take an application and make sure it can keep all of its configuration on the same little portable device, instead of storing it per computer, per user? Then you can bring your favourite software and its configuration along with you.

But there is one thing a Portable application should never do: It should never touch file type associations! File type associations are still a per-computer setting, even if stored in the user profile, and a Portable application has no business touching per-computer settings.

If your application supports a Portable installation mode natively, ensure that the application knows that at runtime, and that it must never touch file type associations when running with a Portable configuration.

---

Some terminology

First we should agree on what different parts of the file type association system are called.

Registry

Registry key

Registry value

HKEY_LOCAL_MACHINE or HKLM

HKEY_CURRENT_USER or HKCU

HKEY_CLASSES_ROOT or HKCR

File extension

File type

File type association

ProgID

Verb

Perceived types

---

SPAD and Default Programs

SPAD (Set Program Access and Defaults) was introduced in Windows XP SP1 due to the antitrust cases against Microsoft. This is a control panel that lets the user configure the system defaults for web browsing, e-mail client, media playback and Java runtime environment. Unfortunately, it doesn't do anything else and configures per-system settings.

SPAD was removed again in Windows Vista, and replaced with the far better and much more useful Default Programs control panel. The Default Programs control panel contains everyhing SPAD had, but also takes over the file type associations UI that was previously part of Explorer, making managing file type associations far easier.

I'm not going to discuss SPAD either, because it is so limited in what it offers the user, but I will be discussing Default Programs plenty, and how to implement your own UI that can emulate what Default Programs offers, just inside your own application.

Windows XP does offer a bit of the functionality Default Programs has, but it's harder to find and not useful for general management of associations.

---

Registration during installation

Two thirds of managing file type associations right is doing the right thing at install time. When installing you want to do two things:

1. Register your application with the system

2. Claim any file types that aren't already globally claimed

So what do you do?

Setting up a ProgID

The first thing you do is deciding on the name of the ProgID! Let's go with Microsoft's recommendation and use this scheme:

For example:

Skip the Vendor part if you want, it might not make sense if for example you are an open source application and there is no "vendor" to speak of. (Don't pretend that you are a vendor called "OpenSource" or such.)

Okay, so create a key under HKLM\SOFTWARE\Classes with the name of your ProgID, and make the Default value for that key the human readable name for your ProgID, what will appear in Explorer as the file type name for files with extensions registered to your ProgID.


Note that since I'm using .reg file format, the backslashes need escaping with backslashes. Double quotes will also need escaping with backslashes, as seen later.

You probably also want an icon for your file type, let's pick the second icon resource in our executable:


You can also control how much of an a ProgID the user can edit from the GUI by setting the EditFlags DWORD value in your ProgID key. Read about those on MSDN.

Adding some verbs

Verbs are added below the "shell" subkey of your ProgID key. Command verbs are added to a "command" key below the verb key below the "shell" key below the ProgID key. DDE verbs would be added under a "dde" key under the verb key, but I'm not going to discuss those here.

Addition 2010-05-03: Raymond Chen has posted an example of using the IDropTarget COM interface to accept file open requests in a third, possibly more flexible, manner.

Command verbs are very simple: They simply have a Default value being the command line you want to have executed when the verb is invoked:


Two things to take note of: First, I'm enclosing the path to the executable in double quotes. Just make it explicit from the beginning instead of letting CreateProcess() guess where the executable file name ends and the parameter list begins. Second, I'm using %L and not %1. You can use %1 but then you may get a short file name instead of a full, long file name. This doesn't seem to be documented though, but I'm quite sure it works everywhere. Remember that long filenames can contain spaces and should be enclosed in quotes on the command line.

If you want a custom verb that need a user readable name, it gets a slight bit more complex, but only slight!


Now go create as many ProgID's as you think you will need. If you have a few "preferred" file types and maybe a whole lot more you, for example, can read but not write and as such they maybe aren't "preferred", consider having a ProgID for each preferred format, and one for each major logical group of other file types.

Registering ProgIDs with file extensions

File extensions are also stored in the Classes tree, under the name of the file extension. The default value of a HKLM\SOFTWARE\Classes is the system default ProgID for handling files with that extension. This is the value that was so contested during the early years, but has become less important today. It still has some importance however, and during installation you claim it if nothing else has done so already.

So, let's check if the default value exists, and write our own if it doesn't:


If there's nothing under HKCU that causes the ".jor" file extension to be associated with something else, you will now be the happy owner of that file type on the user's system! If there was already something registered for the file extension, don't despair, because we aren't done yet.

We should also register as being able to handle the type even if someone else has claimed it for theirs. We'll add our ProgID as an OpenWithProgID:


Note that here you add a value with the name of your ProgID, and the data in the value doesn't matter, just leave it as an empty string or even use a REG_NONE type value if you fancy. When you've done that, whenever the user right-clicks a file with that extension in Explorer and looks in the Open With submenu, they will see your application. Perfect!

Open With -> Select Default Program...

No wait, we aren't entirely done with the Open With thing yet. We need to register the application itself to appear in the general Open With dialogue box, so the user can also select it from there for any kind of type, or set it as the default application for a type from there.

Applications for Open With are registered in the "Applications" subkey of the Classes key. For whatever reason, you use your executable file name for key under there, and set it up with verbs as you would a ProgID.


Why two "FriendlyAppName" values? The first isn't actually used. Microsoft documents the first as being correct, but practice shows that it's the second that's correct. It also makes more sense for the second to be correct, since you're really picking a verb to perform on the file type, not an application to use for the "open" verb. (Or, the Open With dialogue box isn't restricted to the "open" verb, and it needs different "FriendlyAppName"s for each verb.)

However, this isn't entirely enough, we also need to specify what file extensions we are prepared to handle. This can be more than those you would usually register your ProgID for, just throw in anything you can possibly take. Do that in the SupportedTypes subkey of your application key:


Again, these are empty values, the data don't matter for them.


Almost done, but we still need to register for Windows Vista's Default Programs control panel to pick us up. Doing this registration is also handy if you want to have your own in-program association handling UI for earlier versions of Windows.

For this we will need a "Capabilities" key somewhere inder HKLM. If you're already storing some settings for your application under HKLM\SOFTWARE, that's a perfect location to store your Capabilities data.

Your Capabilities key should contain at least an Application Description, which apparently is intended to be a marketing blob telling not much of use, so if you have a marketing department, tell them to write a short paragraph for that now, so it's ready for when your installer goes final. The Capabilities key can also have subkeys describing what file extensions with associated ProgID's, URL types, and other things, your application can be made default for. Again, we'll focus solely on file extensions.


Since there's no single standard location to collect all those Capabilities keys, there is instead a central location they are referenced. When in doubt, add another layer of indirection.


Here, the value name is a human readable name for your application, that will appear in the Default Programs GUI (unless something else overrides it), and and value data is a string referencing a subkey of HKLM, where your Capabilities key is.

Registering Perceived Types

The last thing before we're done installing the application is handling perceived types. Perceived types are overly general groups of types, grouping them by what the user is expected to see the type as, rather than what it actually is. You can also think of it as an extremely simplified and limited kind of inheritance scheme or class mixin, but it's probably better if you don't.

There's a bunch of predefined perceived types, and it's possible to define your own if you really feel the need, I'll leave that to MSDN. The most important ones are "image", "text", "audio", "video" and "system".

We'll first register some one of our file extensions to have a perceived type, and then add ourselves to handle various preceived types.

For giving a file extension a perceived type, we'll revisit the extension's key under Classes. It's quite simple:


Now anything that's registered to handle the "text" perceived type will automagically appear in the Open With menu and the Select Default Program dialogue box for the ".horn" file extension. It's officially a text file!

Since our application also seems to handle text files and image files in general, let's claim to handle those perceived types as well so we can get the same Open With treatment as Notepad gets for text files and Windows Media Player gets for video and audio files.

Claiming to handle a perceived type happens in the "SystemFileAssociations" key under Classes. Registering for a perceived type is just like setting up a ProgID and file extension both under the same key.


Yes, no values under that key, it just has to be there. The name of the key derives from the name we set for the Open With/Select Default Program dialogue box earlier, under HKLM\SOFTWARE\Classes\Applications.

We can also add a few verbs here, that will appear unless something else overrides them:

That's it!

Yes, that's finally it, you're done doing the during-installation set-up of file type associations. You've now told Windows and your users all you can about all the fancy file types you handle, without stepping over anyone's toes or stealing associations from anyone else.

---

Uninstalling

Now that we know what to do during installation, let's quickly go over what you need to do in case the user wants to uninstall your program. The main thing to remember is to only remove things that directly relate to you, and leave anything that may have been changed alone.

Delete your ProgIDs. When your application is gone, they are worth nothing.

Delete yourself from OpenWithProgIDs in all extentions you added yourself.

Delete yourself from HKLM\SOFTWARE\Classes\Applications.

Delete your Capabilities key and remove yourself from HKLM\SOFTWARE\RegisteredApplications.

Remove anything you added under SystemFileAssociations, but be careful you don't remove something that was changed by someone else, especially do check any verbs that could be generic in some way. Don't step on someone else!

And what you should leave alone: Leave whatever the ProgID for any extensions you may or may not have taken during installation. It doesn't hurt with stale ProgIDs being referenced there and you just risk stepping on someone else. Leave any PerceivedTypes you set for any extensions, they are still valid.

---

After installation

Let's start with a simple use case you may want to implement in your application, and how to handle it on modern versions of Windows, i.e. Vista and later. We'll leave in a stub for earlier versions too.

The user wants to customise what file types your application takes, and you provide a button in your Options GUI to take the user to somewhere to
configure that.

Here's an example in C++ of what that button could do:

That's easy, isn't it? The user clicks the button, the user gets the standard Windows UI for managing file type associations for an application. If just this had been introduced ealier... later I'll present a C++ class that does the most important things on older systems.


Now, what if you want to check on start-up whether your application actually is the default for all file types and offer the user to take over, it's also quite easy on Windows Vista and later systems. It takes some more code than just showing the application association registration UI, mostly because you first need to ask the user!


Remember to provide a way for the user to disable the check for being default application, and remember that regardless of whether the user clicks Yes or No to the "take defaults?" question, you should still store whether the user wants to be asked again. The IApplicationAssociationRegistration interface has many more functions, be sure to look up the documentation on it.

Before proceeding to handling the case of old Windows version with our own own code, let's take a quick look at where Explorer (or rather the conglomerate of COM classes that makes up the Windows Shell) resolves file type associations.

---

Reverse engineering default programs for a file type

Let's start by taking a look at MSDN. Somewhat strangely, it seems to be the page discussing perceived types that mentions the priority order of the association locations. See near the bottom of the linked article.

Here's my own short discussion of each of these:

1. "User customised"
2. "ProgID"
3. "SystemFileAssociations"
4. "PerceivedType"
5. "Base Class"
6. "AllFilesystemObjects"

Regular applications are supposed to have no business with any of these during regular operation, though you may want to touch 1. and 2. at times. Locations 3., 5., and 6. are mostly intended for shell extensions and things that look and feel like shell extensions, while the others are for more regular file creation and viewing software.

As to why it's safe to touch the location in 1., despite it being undocumented, the argument will be: Windows Vista and newer provides a well-documented programmatic interface to manipulate the user defaults. Use that interface when it's available. When it's not available, you're on an older version of Windows, and you know that all newer versions will support the well-documented interface. The undocumented interface won't magically disappear from old and already-published versions of Windows, and it won't be redefined in those either. Therefore it's safe to use it, as long as the new programmatic interface is not available.

The undocumented "user defaults" location on Windows NT 5.x is HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts.

This key stores mostly a whole Classes mini-subtree containing all kinds of user preferences and MRU lists. Stuff that changes during daily use. Do note that the format of this key did change slightly from Windows XP to Windows Vista! Just to prove that it should indeed be considered undocumented territory.

Under the Explorer\FileExts key you will find a bunch of keys, each for one file extension. A FileExt subkey can have a value specifying an override ProgID or Application. The primary thing to take note of is that if a FileExt subkey has a value named "Progid" or "Application", then the ProgID seen in HKCR for the extension should be ignored. There should never be both a "Progid" and an "Application" value present, but if both are, the "Progid" takes precedence.

Here's an example of something you would not see:


Currently, double-clicking a .txt file would invoke jor.exe on it, per the Flubware.Hornjor.Text.1 ProgID in HKLM Classes. If the "Progid" value was removed from the Explorer\FileExts\.txt key, then .txt files would be opening with WordPad instead. If you also remove the "Application" value, they would be back to opening with Notepad, or whatever is reflected in HKCR.

Let's take advantage of how HKCU Classes override HKLM Classes and do as little mocking around in Explorer's undocumented keys as we can.

We'll use the Explorer\FileExts subkeys to help check whether we have file types, but write our own defaults to HKCU Classes file extension keys, and then just delete any user preferences set from Explorer.

If you look inside the Explorer\FileExts subkeys on Windows XP, you will also note some OpenWithList and OpenWithProgids subkeys of those, they are user-customised MRU lists of what applications the user has been opening that file type with. We don't need to touch those.

---

Aside: Shell New

One of those little niceties if you have an application that creates documents is having a Shell New registration. Providing one via template files is easy and you can be sure users will be thankful. It might not be something you ever really hear of, but do it anyway!

Create a document file in your application that would constiture a "blank" document. Include that file in your installer package and install it to the CSIDL_COMMON_TEMPLATES directory (ask SHGetSpecialFolderPath().) The file should be named something relevant to your application, and have the proper file extension!

Now ensure that HKCR\.ext\ProgID\ShellNew has a value named "FileName" with data as the path to your installed template file:


You can also try to do simpler, and not install any template, and instead put a value named "NullFile" with no data, and you create zero byte files. Just make sure your application code supports that case.

On uninstall, remember to delete the template file and remove the registry entries for it.

---

Managing default applications in Windows 5 with C++

Let the rest of this file be commented C++ source code.