Liberating AI: The End of Surveillance – 🦛 💌 Hippogram #24

Reflecting on the past two years of open-source AI advancements in large language models, it’s evident that open-source communities are leading innovation faster, more affordably, and inclusively than proprietary efforts from companies like OpenAI, Anthropic, and Google. This progress has even extended to edge AI, where models operate independently of Big Tech's cloud infrastructure. Contrary to the belief that open-source strengthens Big Tech or demands excessive energy, the community has focused on creating efficient, smaller models.

Open-source innovation frees us from the surveillance capitalist model, enabling full control over our data while making devices intelligent without internet connectivity. The availability of local and open-source LLMs, similar to Linux, allows us to embed intelligence in every device without being surveilled.

Open Source vs. Proprietary Models

A regulatory consultant opposing open-source AI recently claimed it would remain a niche product for enthusiasts. This perspective is as misguided as calling Linux niche. Linux powers everything from web servers to the world’s top supercomputers. It's the backbone of modern technology, running routers, smart TVs, and IoT devices, and it’s the foundation of Android, the most popular OS on the planet.

Open-source LLMs offer a diverse array of sizes, architectures, and computational needs, driving hardware vendors to develop specialized chips. This diversity fuels progress and innovation.

New Hardware Opportunities

Take Groq, for instance. Founded by Jonathan Ross, the engineer behind Google's TPUs, Groq developed specialized chips to accelerate LLM inference. If LLMs were monopolized, companies like Groq wouldn't exist. A few months ago I shared a simple benchmark test comparing OpenAI’s hosted GPT-4 with LLama + Groq, and guess what? Groq was 54 times cheaper and faster. Meanwhile, Etched, a US startup, announced their new Sohu AI chip runs models 20x cheaper than Nvidia's H100 GPUs. Open Source creates market opportunities and distributes power, monopolies stifle innovation and inflate costs, leading to market failures. Who would have thought?

AI Policies and the Effective Altruist Lobby

EU policies often favor potential monopolies, with many EU and US policymakers failing to distance themselves from NGOs like the Future of Life Institute. This organization, funded by billionaires like Elon Musk, Jaan Tallinn, and Dustin Moskovitz, promotes an anti-open-source stance and policy papers based on weak evidence to advocate for banning such models. All of them are major investors in leading AI companies and view open source as a threat to their business model.

There is no solid evidence proving that open-source large language models significantly contribute to bioterrorism risks. The regulatory thresholds they publicly advocate create an oligopoly of 10 companies (see FLI statement).

I was recently criticized, and even threatened, as I mentioned that some of our EU policy makers opinion might have been corrupted by this organization. But hey, I work in healthcare and this feel very much as inviting the tobacco industry to the public health policy table. Yes, our public health policies were once swayed by Big Tobacco's influence on academic research, but through awareness and stringent regulations, we successfully mitigated this, a lesson now relevant as we confront similar issues with policy research on AI.

The Threat of Surveillance

The billionaire, Jaan Tallinn, a major influence on AI policies such as the EU AI Act, recently called for making high-level graphics cards "illegal" and for "more pervasive surveillance of software." This, of course, is without mentioning that he is the first major investor in Anthropic AI, a closed-source competitor to OpenAI backed by Amazon and Google and worth nearly $20 billion.

As my friend Daniel Jeffries mentioned in a recent tweet, "outlawing chip development and creating a software surveillance state has no place in any country that values freedom and democracy. Nobody with these kind of belief structures can make a sane, sound policy recommendations. People with these beliefs cannot make sound policy recommendations. These are totalitarian values, and they should be as repugnant as a bug on your sandwich."

Such proposals are incompatible with any country that values freedom and democracy. People with such totalitarian beliefs cannot make sensible policy recommendations.

Important detail as well, Jaan Tallin is the initial lead investor behind Anthropic AI, a startup backed by Google and Amazon. It's current value US$ 20 Billion.

The Power of Small, On-Edge Models

The Raspberry Pi AI Kit, designed for the Raspberry Pi 5, includes a Hailo AI module with a Neural Processing Unit (NPU). The Hailo-8L chip can perform 13 tera-operations per second (TOPS). If you're not familiar with Raspberry Pi, it's a series of small single-board computers developed in the UK by the Raspberry Pi Foundation. Cheap, efficient, and versatile. What's not to love? MediaTek's Genio chipsets are another game-changer, designed for edge AI devices, combining CPU, GPU, and AI Processing Unit capabilities. Then there's Sipeed, a company from Shenzhen, producing IoT boards based on open-source RISC-V architecture. You can run a small large language model, Phi-3 3.8B, on the Sipeed M4N AI-Box at 4.4 tokens per second. And the cost? A mere EUR 50 on Aliexpress. Talk about making AI accessible!

Decentralized open sourced AI: the hero we need against surveillance infrastructures.

I couldn't find any figures on how many IoT devices are not connected to the internet. All my household appliances at homer are IoT devices that can be connected to the internet but are not. Frankly, using an app to control my washing machine sounds pointless.  Some say 40% of people avoid IoT due to privacy and security concerns. The real value of IoT devices? For vendors to collect data and understand user behavior. But decentralized edge AI appliances? Now we're talking. Finally, a way to tell my washing machine to "shut up" without those annoying beeps. No, seriously, I eagerly anticipate the day I can command my German Miele washing machine to wash 10 white shirts at 11 PM and follow its instructions.

Privacy First Healthcare

Now, imagine a healthcare revolution with privacy-first, decentralized architecture. Yet, we remain stuck in the mud of centralized data collection, blinding entrepreneurs to devices that truly serve the user. Picture it: a weight scale you can talk to right out of the box, guiding you and syncing data like Apple AirDrop, while not having to connect it to the internet. The dream: a device that actually puts you first.

Those who know me well probably remember my infamous tales from 2010 about connecting my first "intelligent" weight scale to my Twitter account to leverage peer pressure for weight loss. And the hilarious twist? My cleaning lady, unbeknownst to me, was using my weight scale while I was away, broadcasting her weight to my Twitter followers. My tweeps were baffled, wondering how I'd managed to lose 10kg in one week.

I've experimented with and used technology to optimize my healthcare needs for years, and if there's one lesson I've learned, it's this: the information asymmetries created by mass surveillance are leading to new power asymmetries, and sometimes are being weaponized against you. Enter the European Health Data Space (EHDS) with policies granting the state access to our IoT health data during a pandemic. Chilling, right? It’s like a flashback to COVID-19 when public health became a convenient excuse for state surveillance expansion.

Juli Zeh's novel "The Method" paints a grim picture of mid-21st century Germany, a totalitarian surveillance state fixated on health and disease prevention. The "Method" rules with an iron fist, enforcing mandatory health testing and strict regulation of diet, exercise, and sleep. Non-compliance? Severe penalties await. Use a banned substance, get fined. Mass surveillance and data collection through IoT devices are justified as protecting individuals and ensuring fair judgment, while personal freedoms are eradicated under the guise of public and private good alignment. In her Novel, Zeh questions individual liberty versus state control, the ethics of preventive health measures, and the dangers of a surveillance state.

Conslusion

The open human collaboration of open source AI in healthcare offers significant benefits for privacy, security and the democratization of AI.

It allows us to give everyone the opportunity to develop and create medical innovations independently of large corporations. A researcher in Africa may not be able to pay the publication fees of the scientific publishing industry and become part of the global scientific community, but thanks to open source developments, they can develop AI-powered tools without having to enter their credit card details to use an API of those who want to build their centralized monopolies while keeping us in the dark.

Moving forward, balancing healthcare innovation with personal freedoms is crucial. The dream of a decentralized, user-centric healthcare system is possible, but only if we guard against the encroaching shadow of totalitarian surveillance, get literate and start building.