Tech Thread
- Thread starter MindzEye
- Start date
MindzEye
Wayward Ditch Pig
When all of this is over, the only thing that is going to be left are models no better than what we have now, that can run on personal devices. Basically Deepseek. Where you choose the model you have the hardware to run and it can take the place of basic search, a research assistant who gets 30% wrong, and does some inelegant coding.
All of this hyperscaled bullshit will go down as the biggest waste of time and money in human history.
MindzEye
Wayward Ditch Pig
That's probably pretty easy to screen for though. They already do a good job of identifying AI generated text from non. Putting that in as a condition of accepting the training data probably isn't a huge challenge.
The issue right now is that these hyperscalers don't want to put in hand brakes like that on increasing model size because they thought that if they just made the models fucking huge they would get better.
zeke
Well-known member
do they tho
MindzEye
Wayward Ditch Pig
Yes. Even in types of writing where the AI is trying to mask it's AI'ness, specific AI designed specific for this task is very effective at identifying the content as AI generated. When it comes to things like news and AI slop, the AI doing the generating isn't trying to hide it's tracks like they are for generating things like University papers.
Identifying artificial intelligence-generated content using the DistilBERT transformer and NLP techniques - Scientific Reports
Natural language processing (NLP) has evolved significantly with the emergence of large language models (LLMs), leading to the rapid growth of artificial intelligence-generated content (AIGC). This expansion raises critical challenges in ensuring content authenticity and preventing the spread of...
www.nature.com
It would be pretty easy to incorporate this into the models as they're fed training data.
MindzEye
Wayward Ditch Pig
Even that number is hilariously high. SpaceX is 3 companies rolled into 1. It's a small, 10 million subscriber telecom that is a pretty good business unto itself but structurally has huge cost to deliver it's service with satelites only lasting 4-5 years before requiring replacement where traditional telecoms put fibre in the ground once and it lasts decades. It's not a durable business model and can't survive downturn. It's a government subsidized rocket for hire company that is somewhat profitable but attached to Elon it could easily die if any Democrat administration wanted to pump the money into revitalizing NASA instead, and then it's a money burning tire fire of a 2nd tier AI company that swallows up every penny the other two make and then some.
The numbers simply aren't real anymore. It's all a fucking grift.
zeke
Well-known member
Yes. Even in types of writing where the AI is trying to mask it's AI'ness, specific AI designed specific for this task is very effective at identifying the content as AI generated. When it comes to things like news and AI slop, the AI doing the generating isn't trying to hide it's tracks like they are for generating things like University papers.
Identifying artificial intelligence-generated content using the DistilBERT transformer and NLP techniques - Scientific Reports
Natural language processing (NLP) has evolved significantly with the emergence of large language models (LLMs), leading to the rapid growth of artificial intelligence-generated content (AIGC). This expansion raises critical challenges in ensuring content authenticity and preventing the spread of...
View attachment 34755
It would be pretty easy to incorporate this into the models as they're fed training data.
hmm. i've heard otherwise.
but either way, given that they think AI output is generally good, and given that they desperately need more data, are they actually going to try to filter out AI output from their training data?
MindzEye
Wayward Ditch Pig
"They" don't think that though. There's a whole ass computer sciences industry doing research on these models right now, don't get that confused with the hyperscaling industry whose existence is dependent on all of this ridiculous growth paying off. Are OpenAI, Anthropic, Google, Microsoft, Elon, etc going to do anything different than what they've done? No. But it's fairly trivial for someone who has the knowledge, right now, today to take the biggest deepseek model available, plug in the code from one of those research projects trained to identify AI content, and then run their own instance that can filter (most) AI generated material out of it's own training databases.
I think you're mistaking the difference between it being possible, and the big model makers to be willing to implement it. They're not going to. But you know who will? Deepseek, or the various teams working off of Deepseek models to make new shit right now. Apple when they start rolling our a next gen of apple silicon that can run custom models locally & easily with low system overhead, etc, etc.
The hyperscaled industry is going to die, but deepseek has already shown us the future of the technology itself. Lightweight on device models that you can custom train to whatever you want it to specialize in. Want to turn yours into a proprietary analytics driven hockey player scouting monster? Cool, get API access for as many free and paid analytics data sets as you can (macro and microstat) and work with it to generate custom algos with custom stat weightings for pro and minor league scouting and just pull reports off of the front end. The future of this tech is bottom up, not top down. It will still have it's drawbacks (the halucination issue is not going away, it's a feature of the tech, not a bug) but a dystopian future of AI's training on bad AI data making AI's more useless and then the next generation enshittifying further and further isn't likely. The hyperscale industry won't exist in the future to make it happen.
I am a full on Ed Zitron level skeptic of the AI industry as it exists in the US right now. That shit is fairy dust that is going to tank the economy and cause a historic recession when it goes. But the technology itself run locally on the type of hardware accessible off of the shelf at any computer store, where anyone interested in it can take a basic model and train it themselves? Some neat shit is going to come out of that. You can bet that when someone like Chayka refers to AI usage in our front office, he's not talking about using ChatGPT to vet trade ideas. He's talking about this.
Last edited:
CH1
The Artist Formerly Known as chiggins.
MindzEye
Wayward Ditch Pig
6) Apple is smarter than everyone involved and has already decided what the future of the technology is going to be, because they control a walled garden with 1.6 billion tech forward users in it. The rest of the incumbents are so pot committed to their current course (by way of inflated asset prices) that they either can't see it, or won't see it until it's too late. Apple is moving towards personal, on device models.
CH1
The Artist Formerly Known as chiggins.
Sometimes being late (or not the early bird) pays off. Gives you more time to think.
Apple was late to phones too
