I'm archiving this here for my own benefit. I didn't come up with this originally. Credit due to @ajkgordon on superuser.com (original link) for the original.
To delete the Outlook profile, go to Finder > Applications > right-click / CTRL-click on Outlook > Show Package Contents > Contents > SharedSupport > Outlook Profile Manager > Select profile > Click the minus sign to remove. Then empty trash.
After this, the user can launch Outlook and setup their profile as if it was first being setup.
In case you're looking for PC instructions, here the are from memory.
Open Control Panel (the legacy control panel, not the new Windows 10 settings)
Open the Mail control panel
Select Show Profiles
Remove the profile
Restart Outlook and setup the profile again.