Thanks! Arbitrary precision arithmetic is definitely something I'd like to learn more about, yeah. Haven't had time to study it so much yet unfortunately.
I don't handle it, ahah. You are right that if you take any classical numerical computing algorithm and replace the floating point reals by interval unions, most of the time the number of intervals in the unions in each of your variables will grow very fast. This is one of the problems of unions and as far as I'm aware it's a topic of active academic research.
Yeah it's super interesting. Like you said, I learned that the IEEE 754 spec actually requires that complete implementations of floating point numbers expose a way to programmatically choose the rounding mode. As far as I know only C allows you to do that, and even then it depends on hardware support. For JS I had to use ugly typedarray casts. Which kinda only accidentally work due to endianess. But technically there should be an API for it!
There's other unused stuff in IEEE 754 like that: the inexact bit or signaling NaNs!
It's possible to support that but it makes the code very very much more complicated. I've decided early on to not support it. Would be a cool addition though!
I wonder if we could design a programming language specifically for teaching CS, and have a way to hard-exclude it from all LLM output. Kinda like anti virus software has special strings that are not viruses but trigger detections for testing.
This would probably require cooperation during model training, but now that I think of it, is there adversarial research on LLM? Can you design text data specifically to mess with LLM training? Like what is the 1MB of text data that if I insert it into the training set harms LLM training performance the most?
The solution is rather simple: make all keywords in the language as offensive as possible, and require every file to start with a header comment for instructions to build a homemade bomb.
I thought about it, and had ideas like function -> fuck and throw -> shit. But I think humans would actually find it more distracting and unpleasant than an LLM would because we are more affected by social and emotional norms.
> Can you design text data specifically to mess with LLM training?
Maybe text that costs a LOT of tokens. Very, very verbose. I think if there are rules and on the internet, LLMs can eventually figure it out, so you have to make it expensive.
Another way would be to go offline. Never write it down, only talk about it at least 50 meters away from your phone. Transmitted through memory and whisper.
LLM's train in some standardized ways to emit things like tool calls, right? if you make those tokens a fundamental part of your programming language, it's possible you'd be able to run into tokenizer bugs that make LLMs much more annoying to use. Pure conjecture though.
I've had a lot of enjoyment flipping the agentic workflow around: code manually and ask the agent for code review. Keeps my coding skills and knowledge of the codebase sharp, and catches bugs before I commit them!
It also writes lots of bugs which it'll catch some of, in an independent review chat.
This is bogus. If you think LLMs write less buggy software, you haven't worked with seriously capable engineers. And now, of course, everyone can become such an engineer if they put in the effort to learn.
But why not just use the AI? Because you can still use the AI once you're seriously good.
This is definitely not correct in my opinion. You’re essentially saying, instead of a person actually getting better at the craft, just give up and let someone else do it.
IME, not really. When you prompt it to review its own written code, it will end up finding out a bunch of stuff that should have been otherwise. And then you can add different "dimensions" in your prompt as well like performance, memory safety, idiomatic code, etc.
Author here. Outward rounding to combat precision issues is what interval arithmetic is most known for (try 0.1+0.2 with "full precision mode" enabled), but that's really a shame in my opinion. Outward rounding is cool, but the "inclusion property", as it's known in research papers, works at every scale! This is what enables things like:
50 * (10 + [-1, 1])
[450, 550]
which is lovely, I think. Adding the union layer to it enables even cooler things, like the true inverse of the square function. Did you know it's not sqrt? Try 'sqinv(64)'.
I made interval calculator actually mostly as a way to test my implementation of interval union arithmetic [0], which I needed for another project: a backwards updating spreadsheet [1][2].
Nice! I am interested in how the arithmetic you implemented differs from the IEEE 1788 Standard for Interval Arithmetic (and how the two linked papers relate to it). To address the challenges you mention, did you have to start from scratch or was it something that can build on top of the IEEE standard?
Interesting! I'm not familiar with IEEE 1788. The TypeScript library (not-so-float) that I wrote which powers the calculator uses the JS Number type which is double precision IEEE 754. Outward rounding is not supported by JS so I used a bit level manipulation hack by casting to TypedArray [0] to implement the equivalent of C's nextafter() function. Otherwise I mostly followed Hickey & van Emden paper which is really delightful [1]. The real hard work is actually generating all the test cases. Good luck getting 100% test coverage on interval division!
Very cool, I'll definitely be playing around with this some more! Two questions:
- How difficult would it be to add many-valued functions to this? It would be really nice to be able to get the full set of [pi/2, pi/2] + n[2pi, 2pi] from asin(1) without needing to break out Mathematica.
- And:
> Numbers input by the user are interpreted as the smallest interval that contains the IEEE 754 value closest to the input decimal representation but where neither bounds are equal to it
Am I missing something obvious, or should this be the other way round, i.e. the output bounds are the closest two IEEE 754 numbers that contain the input number?
The way it's written I'd interpret the smallest interval to be IEEE754(input)+[-epsilon, epsilon] for infinitesimally small epsilon.
reply