The experts in MoEs aren't specialized in any meaningful task sense. From level of what we would think as tasks MoEs are selected essentially arbitrarily per token and per block.
It’s unsupervised, yes, but “unspecialized in any meaningful task sense” is incorrect, that’s the whole point. It’s just not in the sense of “this is a legal expert, this is a software developer”.
Optimal expert separation depends on the goal and can be pretty arbitrary, for example DeepSeek v4 separates them more or less by domain if I remember correctly.
"You could argue that I’m abusing 414 URI Too Long. I respond that it’s funnier this way. Other options I considered were:
400 Bad Request, the generic client error code, which is correct but boring;
402 Payment Required, and honestly if you want to pay me to make a particular URL with query string work, I’m open to it;
404 Not Found, but it’s too likely to have side effects, and it doesn’t convey the idea that the request was malformed, which is what I’m going for; and
303 See Other with no Location header, which is extremely uncommon these days but legitimate. Or at least it was in RFC 2616 (“The different URI SHOULD be given by the Location field in the response”), but it was reworded in 7231 and 9110 in a way that assumes the presence of a Location header (“… as indicated by a URI in the Location header field”), while 301, 302, 307 and 308 say “the server SHOULD generate a Location header field”. Well, I reckon See Other with no Location header is fair enough. But URI Too Long was funnier."
I don't think it's an abuse, RFC9110 defines 414 as a response for "refusing to service the request because the target URI is longer than the server is willing to interpret". Since adding a query string involves only adding characters, this seems fine; there's no stipulation as far as I can tell that all pages a server hosts must adhere to the same length. I'd be curious if any well-known clients interpret it that way though, and make caching decisions based on it. As far as I know, they shouldn't.
Obviously it's against the spirit of the thing, but I don't think it's wrong per-se.
I reckon it is still an abuse. I am willing to interpret longer target URIs… so long as they don’t contain a question mark. /no-query-strings is longer than /?.
Opinions vary on how good an idea the robustness principle is. That is why, for example, the XML standard requires a conforming validator to throw an error on invalid XML.
In our modern world, the robustness principle has become an invitation to security bugs, and vendor lock-in. Edge cases snuck through one system on robustness, then trigger unfortunate behavior when they hit a different system. Two systems tried to do something reasonable on an ambiguous case, but did it differently, leading to software that works on one, failing to work on the other.
I generally agree, but I don't think XML is the best example. Getting HTML out of XML is considered to have been the right move isn't it? I was pro-XHTML2 at the time but in retrospect, have we suffered much for not sending webpage validation errors to end users?
Once people have gotten used to not having to conform, forcing them to conform is an uphill battle. Doubly so when, as happened with Microsoft and IE, the vendors would like to encourage vendor lock-in. The only time to reasonably do it is at the start.
That said, we are paying a huge complexity cost due to our efforts to allow nonconforming pages. This complexity is widely abused by malicious actors. See, for instance, https://cheatsheetseries.owasp.org/cheatsheets/XSS_Filter_Ev... for ways in which attackers try to bypass security filters. A lot of it is only possible because of this unnecessary complexity.
But, this is robust? I mean it's pretty clearly stating that you are visiting an unsupported URL. It provides direction on what to do about it to the user. It does not crash the browser or the server. In pretty much every dimension this is highly robust.
The robustness principle is itself bad manners, in plenty of contexts. If I deliver packages by throwing them at the customer, I really want a customer to tell me "hey, don't throw packages at me!" before I attempt to lob something fragile and breakable, or something heavy at someone fragile and breakable. Otherwise, how am I supposed to learn that I'm doing anything wrong?
Seems to be very critical of western, and especially American, foreign policy. Reasonably well argued and factual, although a bit edgy at times. A decent read.
I don't doubt this happens, but given all the wolf crying about clandestine Russian operations, it's hard to assess what the scale and influence of these are. Especially as this is based on an analysis by Atlantic Council, which is essentially a NATO think tank.
This will probably read to many as me being a useful idiot for Putin or something. And maybe I am, hard to say definitely.
OK, those are interesting choices that are outside of the realm of stuff that I was thinking about. What I was thinking about is that the Russians have been working the American people via the media for decades.
What public state speculation about Russian interference in anything ever was substantiated?
As far as I can tell, nothing that has been said about Russian intelligence operations in the West (over the past decade or so) has ever been substantiated. That's why everybody started blaming every single problem or disagreement in the West on Russia, because you wouldn't be asked to or expected to be able to substantiate it.
I've been called Russian or Chinese more times since 2015 than I've ever been called anything else other than my name. I was usually called that by people when I was denying something that those same people now say nobody ever really believed or insisted was true.
Most people lack principles and act purely emotionally. It is wicked and evil and vile if Russia does something because it is Russia doing it. It is good and right and true if “Western” powers do a thing because it is Western powers doing it. To a principled observer, they’re all evil regardless of which country is doing the thing.
> It tried to build something simple and while it got the job done the thinking displayed did not fill me with confidence. It was pages and pages of "actually no", "hang on", "wait that makes no sense". It was like the model was having a breakdown.
It has been probanly trained to assess its own "thoughts" regularly and outputs those for the assesment results. I wouldn't worry much about the reasoning text contents, and it's nice to have them in contrast to the closed model "summaries", so it's easier to see what's going on.
If I understand this correctly based on a quick read, it argues that subjective experience arises at the (or in the) "alphabetization" process where continuous physical states (e.g. voltage) are mapped to discrete logical states (roughly like e.g. a bit) or "concepts" (figure 2).
Per this reading, implementing something in ASIC would make it have (a different) experience, as opposed to CPU/GPU. Not sure what would be the case for FPGAs.
It also seems to rely on the classical "GOFAI" idea of symbol manipulation, and e.g. denies experience that isn't discretizable into concepts. Or at least the system producing such concepts seems to be necessary, not sure if some "non-conceptual experiences" could form in the alphabetization process.
It reads a bit like a more rigorous formulation of the Searle's "biological naturalism" thesis, the central idea being that experience can not be explained at the logical level (e.g. porting an exact same algorithm to a different substrate wouldn't bring the experience along in the process).
reply