Debugging ActiveX Controls

Up to now I have talked about cracking using some pretty basic tools. Most of my cracks are done with nothing more than FileMon, RegMon, Peek, and Notepad. I just don't write a lot about using disassemlers or debuggers.

The fact is that with ActiveX controls, most of the time it really doesn't take more than the basic tools. We rarely need to dig into code just to get our license strings. They are usually in plain sight. But there are those times when we do actually need to break out SoftIce and W32DASM.

This essay assumes that you are either competent with SoftIce or you know how to search the internet to find some SoftIce tutorials to become competent. I will not explain a lot of the details of what to do, but rather give a general overview of what to expect when debugging ActiveX controls.  I don't want to teach how to use SoftIce, just how to use it specifically for ActiveX controls.

The cool thing about debugging ActiveX controls is that they are almost always the same. ActiveX controls use the COM interface. And the purpose of COM (or any interface for that matter) is to standardize how things work. To be an ActiveX control, it has to use the COM interface. And unless they come up with some proprietary licensing methods, they are all cracked the same way.  And even if they do come up with their own method, it still comes down to either looking in the registry or looking at some file contents.

But before we start debugging the controls, I need to explain some things about ActiveX technology. Basically, an ActiveX component is something that adds functionality to your application. The concept is that if you need a spreadsheet, you buy a spreadsheet control and just drop it into your application and set a few properties.

ActiveX technology goes by several names. Sometimes it is called OLE Automation, COM, or whatever sounds best at the time. An ActiveX component doesn't have to be a control. It can also be a set of functions, an ActiveX document, an ASP component, an IIS component, or a whole application. But whatever form it takes, it will still communicate using the COM interface.

One decision you must make when creating a component is whether you want it to run in-process or out-of-process. The difference between is basically that out-of-process controls are more scalable and in-process components have better performance. Being in-process means that the control is run as a part of the parent application's process. In other words, if you debug an ActiveX control in VB6, it will look like you are debugging VB6 itself. If you debug a control in a compiled executable, it will look like you are debugging that executable (or sometimes the VB or VC runtimes). Most ActiveX controls with an .OCX extension are in-process, .DLL files can be either, and EXE controls are usually run out-of-process.

With that in mind, the next step is to find a breakpoint that will trigger SoftIce at just the right place in code. That place should be somewhere where we can find the license string. I prefer targeting the time when a control is first placed on a form because that is where most ActiveX containers will request a license string and also the place where you will usually see a nag screen.

The actual breakpoint I use at this point is based on the information I have gathered about the control. The decision is based on which tools were used to create the controls and whether it looks at the registry or a file for a license string.

Below are some guidellines to help you decide which breakpoint will work best for a particular control. Of course, these are general guidelines. There will always be one control that does things its own way.

Controls Created With Visual Basic
Standard Registry Licensing
The standard licensing is used when the developer checks the Require License Key box in the Project Properties dialog. The license key is stored in HKEY_CLASSES_ROOT\Licenses key in the registry. The ActiveX control will look at the value in your registry and compares it to what it should be. To catch this process in SoftIce, you need to load the exports for Advapi32.dll. First, set a breakpoint on regqueryvalue and then place a control on the form. When SoftIce pops up, remove the first breakpoint and set a new one on lstrcmp. Press CTRL+D to continue and the next time SoftIce pops up, you will be in lstrcmp and you can step through that (using F8) to find the real value and what it is being compared to.

Other Registry Licensing
If a control uses the GetSetting function in VB to use its own licensing method, set a breakpoint on rtcGetSetting. To do this, you will have to load the exports of the appropriate VB runtimes (msvbm60.dll or msvbm50.dll).

License Files
I have never seen this, but a control created in Visual Basic could also use a .lic file to store the license string. If you do find this, you could set a breakpoint on __vbaFileOpen in the appropriate VB runtimes.


Controls Created With Visual C++ Using MFC
Standard Registry Licensing
I am starting to see more and more MFC controls use the standard registry licensing methods. If you run into these, use the same steps explained above with Visual Basic controls.

Other Registry Licensing
If a MFC control uses its own licensing method using the registry, regqueryvalue is a good place to set a breakpoint.

License Files
Usually, you will not ever need to resort to SoftIce with this type of licensing. Most often, you can open up the .OCX file in a hex editor and just browse for the license string. I have explained this in my other essay Using LIC Files.


Controls Created With ATL
Standard Registry Licensing
ATL controls usually don't use this method, but if they do, use the same techniques as with Visual Basic controls.

Other Registry Licensing
Setting a breakpoint on regqueryvalue will usually get you to the right place.

License Files
If the developer uses the Control Wizard when developing an ATL control, the following code is created:

static const TCHAR BASED_CODE _szLicFileName[] = _T("License.lic");
static const WCHAR BASED_CODE _szLicString[] = L"Copyright (c) 1995 ";

/////////////////////////////////////////////////////////////////////////////
// CLicenseCtrl::CLicenseCtrlFactory::VerifyUserLicense
// Checks for existence of a user license

BOOL CLicenseCtrl::CLicenseCtrlFactory::VerifyUserLicense()
{
return AfxVerifyLicFile(AfxGetInstanceHandle(),
_szLicFileName, _szLicString);
}

/////////////////////////////////////////////////////////////////////////////
// CLicenseCtrl::CLicenseCtrlFactory::GetLicenseKey -
// Returns a runtime licensing key

BOOL CLicenseCtrl::CLicenseCtrlFactory::GetLicenseKey(DWORD dwReserved,
BSTR FAR* pbstrKey)
{
if (pbstrKey == NULL)
return FALSE;

*pbstrKey = SysAllocString(_szLicString);
return (*pbstrKey != NULL);
}

As you can see, the license string (_szLicString) is stored as a constant in plaintext. As with MFC controls, you can open up the .OCX file in a hex editor and view this string. You can then place this string into a .lic file and you are registered. There is no need to use a debugger for this type of licensing.

However, if the developer decides to use his own code instead of the generic code above, you could set a breakpoint on Kernel32!ReadFile and go from there.  And if none of these work, you could always set a breakpoint on CoGetClassObject in ole32.dll. This is where control licensing is handled. 

So now that you know these functions, you will find that you rarely use anything else as breakpoints.  There really isn't much more to it.  They key is knowing your target.