I am not sure why you go on the subject of English speaking world etc.
Anyway, the models you tested with that query, which I am not sure why we think is a good benchmark, are local models running on a wireless device or they use datacenter and only convey the text back and forth?
I'm fairly sure Siri still sends user voice samples to a data center. At least for a while, it used to use multipath TCP to decrease latency over multiple available network connections if I'm not misremembering.
Some modern Apple devices support "local Siri", but it's a limited subset of both voice recognition performance and capabilities.
Your usage of Siri today (probably on an old version of iOS) frankly has nothing to do with the article we are discussing. Sorry to say this but it is going to take time. Comparing the performance of a chatgpt running in a big data center with a model running locally on a phone device... give it a few years.
People have been giving Siri a few years for a decade now. Siri used to run in a data center (and still does for older hardware and things like HomePods) and it has never supported compound queries.
Siri needs to be taken out back and shot. The problem with “upgrading” it is the pull to maintain backwards compatibility for every little thing Siri did, which leads them to try and incorporate existing Siri functionality (and existing Siri engineers) to work alongside any LLM. Which leads to disaster, and none of it works and just made it all slower. They’ve been trying to do an LLM assisted Siri for years now and it’s the most public facing disaster the company has had in a while. Time to start over.
As a user, I'd gladly opt into a slightly less deeply integrated Siri that understands what I want from it.
Build a crude router in front of it, if you must, or give it access to "the old Siri" as a tool it can call, and let the LLM decide whether to return its own or a Siri-generated response!
I bet even smaller LLMs would be able to figure out, given a user input and Siri response pair, whether the request was resonably answered or whether the model itself could do better or at least explain that the request is out of capabilities for now.
> Build a crude router in front of it, if you must, or give it access to "the old Siri" as a tool it can call, and let the LLM decide whether to return its own or a Siri-generated response!
Both of these approaches were tried internally, including even the ability for the LLM to rewrite siri-as-a-tool's response, and none of them shipped, because they all suck. Putting a router in front of it makes multi-turn conversation (when Siri asks for confirmation or disambiguation) a nightmare to implement, and siri-as-a-tool suffers from the same problem. What happens when legacy siri disambiguates? Does the LLM try to guess at an option? Does it proxy the prompt back to the user? What about all the "smart UI" like having a countdown timer with Siri saying "I'll send this" when sending a text message? Does that just pass through? When does the LLM know how/when to intervene in the responses the Siri tool is giving?
This was all an integration nightmare and it's the main reason why none of it shipped. (Well, that and the LLM being underwhelming and the on-device models not being smart enough in the first place. It was just a slower, buggier siri without any new features.)
The answer is that they need to renege on the entire promise of a "private" siri and admit that the only way they can get the experience they want is a _huge_ LLM running with a _ton_ of user context, in the cloud, and don't hinder it all with backwards compatibility with Siri. Give it a toolbox of things it can do with MCP to your device, bake in the stock tools with LoRA or whatever, and let it figure out the best user experience. If it's a frontier-quality LLM it'll be better than Siri on day one, without Apple having to really do anything other than figure out a good system prompt.
The problem is, Apple doesn't want to admit the whole privacy story is a dead-end, so they're going to keep trying to pursue on-device models, and it's going to continue to be underwhelming and "not meeting our quality bar", for the foreseeable future.
Very good details on why just bolting on an LLM isn't that trivial I hadn't really considered before, thank you!
But regarding Apple not wanting to admit that client side compute isn't enough: Haven't they essentially already done that, with Private Cloud Computing and all that? I believe not even proofreading and Safari summarization work fully on-device, at least according to my private compute privacy logs.
> Your usage of Siri today (probably on an old version of iOS) frankly has nothing to do with the article we are discussing.
Yes, but isn't that infuriating? The technology exits! It even exists, as evidenced by this article, in the same company that provides Siri!
At least I feel that way every time I interact with it – or for that matter my Google Home speaker, ironically made and operated by the company that invented transformer networks.
Hello, the self deport is for illegal aliens. There is no concern for who is in the USA with a valid visa to do science. In fact, the longest scientists and highly qualified people from Europe had to wait for a green card was under the Obama administration.
So much funding is being paused and cut though, who's to say if your field of study will still have federal funding in 1,2,4 years? This lack of funding stability is making US academia way less appealing.
Unlike the author, back in the day I would have preferred a 486DX50 to the DX2-66. 50MHz bus interface (including to the graphics card) instead of 33MHz
My first job was AutoCAD drafting on a DX50 with 16MB. Quite high specced in the early 90s. Not sure I would've noticed the difference compared with a DX2 though.
What are you talking about, why do you think that autistics are treated better in Europe, Africa, Asia? Also, people do not "hate" them, people in general hate everybody, don't play the victim
There are tens of millions of people doing a repetitive work every day, instead of being entrepreneurs. Just let them be, not everybody needs to dedicate their existence to maximizing their career opportunities, at any level.
As someone who's research goes against the grain, I just request the same. I have no problem with people maintaining the course and doing the same thing. In fact /most/ people should probably be doing this. BUT the system discourages going of course, exploring "out of bounds." The requests of these people have always been "let me do my thing."
Just make sure "let them be" applies in both directions
Most people have great pleasure in owning and driving a nice car. The ones that do not are a minority. There are many reasons why cars (and trucks) are desirable and sought after possessions. It would be a logical fallacy that because you currently do not want one, people should build major projects with no regard to car parking.
Too many people though love cars. Some cities are banning cars. Of course buses, trains, delivery vehicles still exist.
Wide open places like US and Canada sure but many places are super crowded. Southern China, Lagos Nigeria, Tokyo, Mexico City. They're all implementing trains for a reason. There is just no way for cars to move if packed into a small area with hundreds of thousands of people.
In case you're being serious, you rent a car or take an Uber. A metropolis might as well be defined by its lack of car ownership.
Even when you're renting or being driven, you go as far as you can via train. (The Metro-North goes to an Appalachian trailhead [1].) It's safer, simpler, cheaper and nobody has the deadweight loss of being unpaid chauffeur.
Hrmm, so you do need a car after all. It's really amusing listening to people from the Bay Area pass down edicts for how everyone else should use transportation, given that their decrees are a complete non-starter in well over 99% of the United States. Calling it "out of touch" would be a bit generous.
> so you do need a car after all. It's really amusing listening to people from the Bay Area
The Bay Area is car centric--most people have a driver's license. I live in Wyoming and lived in New York. You need a car in Wyoming. You don't in Manhattan. High rises make sense in the latter. They'd be stupid here.
If you're dense enough to make high rises necessary, you're too dense for cars to make sense. Parking for high rises is a dead giveaway the project is being pursued for appearances. (San Francisco's skyscrapers are dominated by commercial buildings that double as advertising.)
You didn't ask how to go hiking in the mountains, you asked how to "leave city limits". but you could still bike, go with friends, or rent a car for a day if no buses go where you want. Hiking locations tend to have lots of tourists so there would likely be bus routes, however.
Sure, that's true - though I can see a couple dozen of them out of the window behind my desk, and there'd be lots more if I were on the other side of the building - but I was really just replying to the previous commenter's mockery of the idea that one could go to the mountains via bus. It's not only possible, it's routine!
Bullshit, I'm from the Puget Sound and it hardly covers any of the good trails in the Cascades, nor could it do so practically if service were ever expanded.
Yeah, and when my grandpa was a kid, instead of a 20 minute drive to the neighboring town, it literally took them three days on foot because of all the hills, sloughs, and rivers. Cars and their associated infrastructure are a modern miracle, and the luddites opposed them are extremely foolish.
Are you claiming that an occasional need for a car dictates that every person needs their own dedicated full time car and that all dwellings need associated parking?
Excuse me but what do you need 10Gbps internet for? What is the use case? I get the cheapest thing that Comcast provides (50Mbps) and it seems to be enough for the 5 of us at home, everybody always streaming, gaming etc. I have an old asus router I got used in 2015 that I repaired with zip ties otherwise it powers off.
At work we have 'infinite' internet (on a class A IP block) and I do not feel any difference in browsing or streaming (obviously I do if I need to transfer a file)
The use-case is because it's so cheap, why bother with anything less.
Currently, I can get 1Gbps Internet for $15, while the cheapest package is 200Mbps for $5. I expect they'll offer 10Gbps in my area in the next few years for the same cost as the 1Gbps now.
Still, at that speed, the router CPU can actually become the bottleneck, and OpenWRT currently has pretty poor support for hardware accelerated routing.
> at that speed, the router CPU can actually become the bottleneck
I was pretty worried about that, but the machine I got for my OpenWrt router is a Lenovo ThinkCentre Tiny with a Core i3-8100T CPU ($80 used with SSD/RAM etc included), and it's seems like it's way overkill, even at full bore the CPU usage appears to be negligible. Power usage is the same as the ISP router at about 17W idle, 24W routing 10 gbit, and that's with a 10 Gbase-T SFP+.
It helps that with a 10 Gbps pipe you don't need to run any kind of fancy QoS algorithms or anything...
The price is the same as 1 Gbps ($30/mo) and I built the router out of $100 of parts, so it's kind of why not? It's nice when there are 10 GB updates to download, running off-site backups, and I can host whatever stuff I want out of my home instead of paying for a VPS on someone else's machine and worrying if it doesn't have enough RAM or something.
With "modern" developing you send whole operation systems images back and forth (docker) or you download AI models, statistics, etc that are many hundred GB. It's nice only having to wait seconds rather then hours.
A 10Gbe line usually does not mean dedicated 10Gbe internet exchange, it usually means many people are sharing the same 10Gbe line, so you do not always get the full bandwidth.
