@timcliff All those upvotes (assuming the voters had positive reputation which is usually the case) increased the reputation. The later downvote reduce the reputation, reversing the effect of the upvotes. As long as the downvotes aren't extreme and overwhelming (which usually only happens in the case of serious abuse), reputation will usually be only slightly affected or even increase a little if the payout is merely reduced and not driven all the way to zero and beyond (which would be a waste of vote power by the downvoters).