Also just adding on to this concentration of power note:
1. AI Integration in Government - In other words, as government agencies adopt AI into administrative decision-making, how do we ensure that oversight remains effective? For many, AI functions as a black box, and I fear that government officials may lack the necessary technical expertise to scrutinize AI-integrated systems effectively. Are Congress / inspectors general / independent oversight bodies equipped with the technical resources and legal authority necessary for effective oversight? We should also prevent AI from concentrating additional discretionary power within executive agencies. Relatedly, AI should not diminish the ability of government oversight institutions to challenge decisions made by government agencies.
2. Government Dependence on Frontier Labs- As government entities become more dependent on AI , we should prevent the handful of private frontier labs from having disproportionate influence over gov operations. Ideally, government contracts should establish mechanisms for replacing AI vendors with little disruption to government operations, thereby mitigating the risk of regulatory capture (and reducing vendor lock in).
3. Concentration of Compute - while open-weight models and AI dev tools proliferate, I'd say that computational infrastructure still remains comparatively expensive. Currently, compute is concentrated among the few who have the requisite capital, granting such companies significant control over the direction of AI research. Consequently, I'd imagine competition policy and public investment in computing infrastructure may help to preserve independent research, thereby preventing excessive dependence on the dominant industry players.
I won't argue that AI lowered the barriers to entry for aspiring entrepreneurs; personally, I welcome this new wave of software development, especially insofar as it empowers aspiring founders who otherwise don't have the technical faculties to pursue their ambitions. At the same time, I fear that instances like the one below are harbingers of software security deterioration en masse.
AI assists in coding acceleration, sure, but it seems like it's still incapable of completely replacing the judgement of necessary cybersecurity teams.
https://t.co/RpnRwnQxH1
@_NathanCalvin I think it's also worth mentioning that with the increasing usage of models in their safety research, "misaligned behavior...could undermine that work." Pretty concerned about evaluation integrity itself
A frontier lab will claim that its incumbent on them as a "responsible" entity to develop the "safest" AGI first as the rationale behind increasingly rapid development timelines, but this sentiment, quite frankly, seems disingenuous now.
OpenAI will fire three of their safety researchers for sharing information with METR and Anthropic will draft a concerningly light-touch AI safety bill that effectively has less teeth than the FRONTIER Act while simultaneously advocating for regulatory oversight. As an aside, it's not entirely clear to me that the Hugging Face incident itself would even warrant penalties (unless it separately violates some reporting / safety framework requirements) under this bill draft.
Reading the writing on the wall right now, the professed commitment to AI safety as the motivating factor for development seems like a guise for underlying commercial incentives, especially with the prospect of impending IPO's. I just find it funny how these labs reconcile their conviction in the possibility of human extinction with the fact that shareholder value still matters in such a future.