Until today, the only thing you couldn't delete from a web page was the homepage, but now you can delete it too. To delete an entire site, you can select all the pages and remove them. Unfortunately, you'll have to delete the uploaded files one at a time or hide the site in the settings. But even if you delete everything from the web site, there's no way to delete the site itself.
Another improvement is that any page can become a homepage, so you can have multiple potential homepages. To change the homepage, go to the settings and select one of the pages you've created from the Homepage option.
It would be nice to see more features in Google Page Creator that could make it a part of Google Docs: collaborative editing, site search, folders, custom templates.