Yes, it will work for any webpage you preview from within the Web CMS.  The webpage/site does not have to be published for the Siteimprove plugin to check it for any accessibility errors.

Make sure you have selected the following filters in the plugin:

• Choose conformance level: A AA AA conformance (not AAA conformance)
• Choose severity: Error (not Warning or Review)
• Choose responsibility: Editor (not Webmaster or Developer)