“It is a fair summary of history to say that the safeguards of liberty have frequently been forged in cases involving not very nice people”