+1 Looks good to me

If something has gone wrong, should we raise an Exception here?

No, because the fact that we get here isn't a problem in itself. That just means that we weren't configured on installation, in which case we shouldn't do anything for uninstallation.
That's not an issue, just expected.

In that case, can you change the comment to state that instead of saying something went wrong? Maybe a "The installation didn't record a domain, so there is nothing to do here."?

Maybe s/, we I/. I/

LGTM otherwise!

I suggest fixing that typo, but I don't need to re-review if you do. LGTM!