It's funny you say that... I just finally tried it this morning, since it seems more polished now. There's no way to disable providers, so I'm stuck scrolling through over 100 models I will never use, to find the 12 I will use. So also crappy UX!
You've just flashed a future before my eyes where now the IT security team is forcing 50k tokens of security prevention context mandatorily into every prompt we issue. Harks back to the days when half your system memory and CPU was devoted to the continuously running virus checker.
Availability through Bedrock has been a major driver in use of Anthropic in my org. And I am betting there is actual margin in it as well.
I wonder if this is directly linked to the split up with Microsoft. Just from my anecdata, OpenAI is getting completely ignored in serious enterprise deployments because what they offer on Azure sucks and there is no other corporate friendly way to get it. They probably saw themselves getting destroyed in enterprise and realised it was existential to be able to compete with Anthropic on AWS.
May be very dependent on country/region, but in Germany where Azure is quite prevalent in mid-market companies (due to heavy historic reliance on Microsoft in general), OpenAI + Azure seems to have worked as a great synergy. Few customers I've worked are even trying to reach for other providers, as the OpenAI models are available for them and promoted by Azure.
ur probably right on the margin. Anthropic doesn't break it out, but enterprise spend on Bedrock is the highest quality revenue in the AI stack right now. Itzs sticky, multi-year, embedded in existing AWS commits. OpenAI was watching that compound while stuck on Azure
It might be just our region, but for a long time we couldn't access current frontier models at all. Only old GPT4 level models. Meanwhile, Anthropic is rolling out access to every model within 24 hours of announcement to Bedrock.
Not sure how much Azure OAI has changed, but when I last used it 2-3 years ago, it was just a farce to get you using provisioned throughput. The throughput quotas were small, the process to request more was bureaucratic, and the Azure SAs were
It was also very clear the OAI and MS teams held each other in contempt (not relevant, but was interesting and grew in the immediate aftermath of the Altman drama).
So why were we using it? OpenAI don’t really have an enterprise go to market, bedrock still relied on Claude 2, and we weren’t willing to YOLO on clickthroughs.
Once Claude 3 came out, we jumped ship. That sucked too, although I hear it’s gotten better though.
I have been out of openai azure deployments for a whole,1year+, but we had often spikes in latency, escalated to the Head of Azure Europe, and still no official clues about them, meanwhile they were trying to get us in some kind of collaboration announcements. And it was the only reasons we had a few meetings with that guy.
So yeah Azure sucked ass and plenty of outages or latency, like 3min for first byte while usually it was max 30sec to 1min, if not even faster (memory is a bit fussy)
Pandoc lives in a different tier because it gives you arbitrary filters so you can do any transformation you want on the intermediate JSON format. And it converts anything to and from that JSON format. So I prefer Pandoc based systems because anything the tool doesn't do that you want is probably implementable with a simple inline filter.
I agree with your preference, also largely because of filters. But note that the intermediate format is Pandoc’s internal abstract syntax tree, not JSON (https://lwn.net/Articles/1064692/).
The older filter mechanism acted on a JSON serialization of the AST, but the current recommendation is to use Lua filters that work with the internal AST directly.
> all of those choices have been made and agreed upon
Have they really? I have a few apps deployed on k8s and I feel like every time I need something, it turns out it doesn't do that and I'm into some exotic extension or plugin type ecosystem.
Something as simple as service autoscaling (this was a few years ago) was an adventure into DIY. Moving from google cloud to AWS was a complete writeoff almost - just build it again.
I'm sure it captures some layer of abstraction that's useful but my personal experience is it seems very thin and elusive.
This, and because of that, claiming your app "runs in kubernetes" is completely meaningless.
Concretely: Take your app. With one button click, or apt-get install ??? on all your machines, configure k8s. Now, run your app.
The idea that this could work has been laughable for any k8s production environment I've seen, which means you can't do things like write automated tests that inject failures into the etcd control plane, etc.
(Yes, I know there are chaos-monkey things, but they can't simulate realistic failures like kernel panics or machine reboots, because that'd impact other tenants of the Kubernetes cluster, which, realistically, is probably single tenant, but I digress..)
If your configuration is megabytes of impossible to understand YAML, and is also not portable to other environments, then what's the point?
(I understand the point for vendors in the ecosystem: People pay them for things like CNI and CSI, which replace Linux's network + storage primitives with slower, more complicated stuff that has worse fault tolerance semantics. Again, I digress...)
> If your configuration is megabytes of impossible to understand YAML, and is also not portable to other environments, then what's the point?
If almost all your configuration is about getting Kubernetes set up, and not about your application setup inside Kubernetes, there probably isn't a point. But being able to use roughly the same config inside different Kubernetes is quite good.
But I've never seen portable kubernetes configs (except for vendor software that probably wouldn't be needed outside of kubernetes).
If you just tell kubectl to dump your pod configs, then load them on some other cluster, that definitely won't work.
If you use the management software that generated the pod setup somewhere else, that probably won't work either because the somewhere else is going to be missing the CSI and CNI you targeted. Even if those match, it'll be missing the CRDs. God help you if you want to run two programs on one Kubernetes, and there's a CRD versioning conflict in their two dependency sets.
> Moving from google cloud to AWS was a complete writeoff almost - just build it again.
Yep. Kubernetes is not just kubernetes when moving between clouds, it becomes a very opinionated product (for better or worse) with lots of vendor addons. Could someone that is familiar with one pick up on the other? Sure! But there are gotchas. And then kubernetes on prem adds the hardware lifecycle piece, and potential data locality issues, etc.
There are differences across vendors, but there’s a way to build with k8s where the benefit far outweighs the cost.
We run a bunch of services in two very different cloud vendors (one of which used to be DIYed with kubeadm), and also on dev machines with k3s. Takes a while to figure this out and to draw the kustomize boundaries in the right place, but once you do, it’s actually really nice.
Two things work in our favor:
- we’ve been at this for around 8 years, so we didn’t have to deal with all the gotchas at once
- we aggressively avoid tech that isn’t universal (so S3 is OK, but SQS or DynamoDB is not; use haproxy instead of ingress controllers; etc)
> Kubernetes is not just kubernetes when moving between clouds, it becomes a very opinionated product (for better or worse) with lots of vendor addons.
I think this is gradually getting better. Networking with Gateways is better than with Ingress in this sense. Things like autoscaling groups need to get better, as they are (or were a couple of years ago) very bespoke.
I wouldn’t really call it “DIY” per se, k8s has the resource API and you can create whatever scaling policies you want to with it, but I do see how that’s not obvious when it’s advertised as ‘batteries included’
I think you do a good job capturing an actual microcosm of the real problems here at an emotional level - why people "hate" it.
(a) loss of fulfillment (b) lower quality of output and nobody will care so the world will just "degrade" and (c) a perceived lack of autonomy ("forcing", "you can't opt out") around how adoption itself is executed
It is hazardous to swim against that tide currently from a career perspective - people rapidly categorise you as generically anti-AI even if you try to express a reasonable nuanced view. It's pretty toxic.
there are plenty of people who basically believe this is the end of the human economy - there will be nothing left that isn't done by AI in the future. Even the bits left that humans do will be human facades on AI driven activity (like your hairdresser will be viewing you through AI powered glasses using AI powered scissors etc).
So from that point of view you can indeed look at it as the entire value of the economy should be invested into AI companies.
But let's be real, the AI output today is largely complete and utter shit. Like, really, without careful promoting and oversight, it's not fun to read, it's not smart, it's not accurate, it's not thorough, and it's certainly not yet over the massive show stopping hurdle that is lying and hallucinating all the time.
To me the bigger point is, the nature of humans is they seek out signals of value, and we will rapidly hone in on whatever unique attributes still differentiate something from being AI generated and those will become what people value. Or to put it another way, no matter how good AI gets, "shit" will be redefined to that.
Already in posting here you have to actively avoid using language that could be mistaken for AI generated text or people instantly devalue your comment if not outright call you on it and try to have your account banned etc. It's not about the content of the comments - it's because they don't think AI content carries value.
Which is all to say IMHO a surprisingly large slab of the economy is going to resist AI automation, because people will instinctively reject anything that IS automated as low value (because it actually will be, in the long run).
It's usually drastically better than humans. Most humans can't code. If you are comparing it to specialists then you are already admitting they are on their way to replacing us.
It’s meant to either act as a complement and/or substitute for humans. So it should be measured against those who get paid. Otherwise what is the firm paying for..?
That is ultimately where it is headed and has been headed for over 100 years now.
The question is when will we get there.
If the answer is tomorrow, money means nothing and none of these investments matter. If the answer is 30 years, well lots of money to be made up until the inflection point of machines being able to design, build, and repair themselves.
You heard AI powered scissors and thought that's where we're heading? I think you'd have to be totally divorced from the average person to believe this.
Meanwhile people are still begging car manufacturers to stop locking their glove box behind a touch screen. Or how about a TV that isn't loaded with crappy software that makes it unusable after 2 years. There's a reason we don't put tech in everything.
Well, fortunately (from a certain point of view), it doesn't really matter what people beg for as long as they need the thing anyway and you and your competitors all agree not to give them what they want.
[0] https://pi.dev/
reply