Those with great wealth inevitably find ways to use it to secure more political favors for themselves at the expense of the rest of us. The system inevitably becomes increasingly rigged in their favor. To restrict their ability to use their wealth to influence elections is undeniably a violation of personal freedom, but so are the numerous subsidies, liability limits, licensing requirements, IP monopolies, military contracts, and competition stifling regulations that elites tend to support. As long as there is a state capable of imposing its will upon the rest of us, there will always be self-serving elites using it for their own gain. While electoral democracy may be more accountable than single party rule or monarchy, it still puts coercive power in the hands of politicians who are far removed from the people they supposedly represent.
Campaign finance reform is at best a band-aid measure with its own problems. Political Candidates are a part of the state, and limiting the transactions they can make in their official capacity as candidates is arguably little different than limiting the ability cops or judges to take bribes. That however does not prevent those with great wealth from finding unofficial means of influencing elections such as by influencing major media outlets. Preventing this would require major restrictions of freedom.
Bitcoin tips welcome: