I’d rather not see that content deleted entirely. We do offer the packages right back to 1.0, and if a user needs to do an upgrade from an ancient version, they’ll want to see the upgrade notes / known issues for each version as they follow the upgrade path.
That said, I do agree the search index issue is annoying. Could we move older manuals to a PDF download? Then the people who really need them can still get them, but it’s de facto not indexable 
+1 on the rest of this, especially redirects in Apache (awkward, but what can you do?). I just don’t want to make it harder for users of old versions to upgrade - otherwise they won’t upgrade 