It turns out to be quite simple.

Add the expander to your WPF control surface:

<telerik:RadExpander Height="24" Name="radExpander2" Width="170" />

Specifically set the Header property in the XAML like so:

<telerik:RadExpander Height="24" Name="radExpander1" Width="170">
    <telerik:RadExpander.Header>
        <Image Height="20" Source="Gfx/Alerts.ico"/>
    </telerik:RadExpander.Header>
</telerik:RadExpander>

This code assumes that you have a graphics file in a folder named ‘Gfx’ in your solution.