I would first only move attacker spawns closer (except 32, leave that far, so defenders have enough time to man the walls at round start) and remove the doors. I wouldn't do anything else, unless this still would be too easy to defend, which I don't think will be the case.