I was installing the RC0 version of SQL Server 2008 this weekend and kept running into the following error:
This access control list is not in canonical form and therefore cannot be modified.
There were a lot of these errors in the Feb CTP related to the registry which seemed to have been resolved. The errors from the RC0 install were related to the directory structure and the permissions being out of order.
I was able to resolve the issue by navigating to the %Program Files%\Microsoft SQL Server\100\Setup Bootstrap\Log\ directory and opening the details.txt file to find out the directory that was having the issue. Using that information I went to the directory, right clicked and went to properties. By clicking the 'Security' tab the OS warns you of the issue and then fixes it. Hitting retry at this point allows the installation to continue.
In total there were 5 directories that caused this issue, all of them located in the %Program Files%\Microsoft SQL Server directory structure. While I didn't capture each of them it is an easy fix that produces a clean install of SQL Server. Hopefully this will save you some time if you are running into the same problem. I was installing on Windows Server 2003 EE R2 SP2.