OK, we had a lot of fun this summer at Cool Blue Interactive overhauling the layered maps for the Georgia Conservancy's Coastal Georgia Land Conservation Initiative. The "before" was a Flash-based map that had non-intuitive layer navigation, and no way for the Georgia Conservancy to add/update/remove layers from the map display, nor a way to integrate third-party layers like Google. Internally they use ArcGIS, but due to budget constraints the ArcGIS server options were off the table. As a solution set, we chose GeoServer and OpenLayers to do the heavy lifting to provide our client with a standards-compliant and robust web mapping solution.
GeoServer is a Java-based web app that provides standards-compliant WMS, WFS and WCS support for web-based GIS solutions. We fired it up on an Ubuntu server running Tomcat, and after some tuning have a very serviceable server to point OpenLayers to.
OpenLayers, for those not familiar, is a Javascript-based mapping solution for the web that resulted from a group of developers reverse engineering Google's tile-based mapping system. It provides support for defining a single base map and "overlays" that can be stacked on top of the base map to give a layered effect - hence the name. OpenLayers is incredibly robust and easy to use in its simplest implementations, but has an extensive library that allows you to customize the map to the Nth degree.
The third and final part was getting the layers from ArcGIS into open standard map formats (aka "projections") that OpenLayers and GeoServer could manage. We used the Quantum GIS open source workstation system to import, manipulate, and export the final layers into GeoServer.
Once it was completed, there were 200+ layers in ~70GB of cached map tiles being managed by GeoServer to cover all 11 coastal counties, with anywhere from 10-22 layers per county. Performance is very browser dependent as OpenLayers is Javascript-based and very graphically intensive. Google's Chrome is almost real-time with FF and IE coming in with slower performance, but still very usable considering the amount of data being managed (upwards of 300 map tile images at a time).
We hand-built and tuned our mapping solution, but if you are looking for a packaged option check out OpenGeo. Their stack is free to download, and they offer paid support services (and I believe hosted solutions as well). If a Flash-based solution that needs to work with native ArcGIS layers works for you, also look into the freely available ArcGIS Viewer for Flex.

