I still don't understand it, yes it's a lot of data and presumably they're already shunting it to cpu ram instead of keeping it on precious vram, but they could go further and put it on SSD at which point it's no longer in the hotpath for their inference.
>I cannot find it there so this must be a false memory
It's also infamously the subject of a Von Neumann joke
>Two bicyclists start twenty miles apart and head toward each other, each going at a steady rate of 10 m.p.h. At the same time a fly that travels at a steady 15 m.p.h. starts from the front wheel of the southbound bicycle and flies to the front wheel of the northbound one, then turns around and flies to the front wheel of the southbound one again, and continues in this manner till he is crushed between the two front wheels. Question: what total distance did the fly cover ? The slow way to find the answer is to calculate what distance the fly covers on the first, northbound, leg of the trip, then on the second, southbound, leg, then on the third, etc., etc., and, finally, to sum the infinite series so obtained. The quick way is to observe that the bicycles meet exactly one hour after their start, so that the fly had just an hour for his travels; the answer must therefore be 15 miles. When the question was put to von Neumann, he solved it in an instant, and thereby disappointed the questioner: "Oh, you must have heard the trick before!" "What trick?" asked von Neumann; "all I did was sum the infinite series
I don't think fil-c is a drop in C replacement, there are things you can do in C such as certain types of pointer abuse that fil-c prohibits (e.g. cast pointer to int, then back). It's probably easier to port an existing C project to Fil-C than to rewrite it entirely though.
There's some irony there in that the whole maps fiasco lead to firing of Forstall which allowed Ive to become head of design, which basically led to the current state of macOS design.
I do wish that some day someone will tell the story of what happened during that time. Maps was bad at launch yes, but it also wouldn't get better without people contributing more data, and the fact that it took a decade to slowly improve implies that there's nothing anyone could have done to get it right "off the bat". It still feels to me Forstall was set up as the fall guy, especially considering no one was fired for antennagate.
Reportedly, Forstall wasn’t liked by the other senior execs but was kept “safe” as Jobs’ protégé, they thought alike and shared the love for skeuomorphism design. Ive in particular disliked Forstall, and Tim Cook made a choice.
He produces Broadway shows these days. Never say never but that kind of thing screams an “I’ve got all the cash I need, now I’m following my passions” mindset. You certainly don’t do it for the money…
The more abstract the “wealth” becomes the less it means in practical terms. The class dynamics around money mostly has to do with the State actively preserving and protecting claims over assets. If that same wealth becomes sufficiently concentrated with an overclass that it leeches away the competence and legitimacy of the State, then the underclass has other means of correcting the gap and establishing a more sustainable equilibrium.
The traditional means of reestablishing equilibrium are becoming more and more infeasible as state defenses and tactics improve. We are rapidly approaching a time when the asymmetric attacks on state protections traditionally used are less effective than the information asymmetry that the state can enforce. Hong Kong is a great example of these defenses leveraged effectively.
It’s possible, but I know people felt the same during prior technological revolutions like the advent of broadcast media (which fascist movements took too with great enthusiasm). I think people are clever, we learn from every failure and adapt.
you hit the nail on the head. the less wealthy a government the more poor its poorest citizens are because it doesn’t have the money to invest in their wellbeing. the solution has and always will be taxes
> Maps was bad at launch yes, but it also wouldn't get better without people contributing more data, and the fact that it took a decade to slowly improve implies that there's nothing anyone could have done to get it right "off the bat".
Absolutely.
Was the choice to release way way way too early the right choice in the end? Needed telemetry, or even more time, to beat Google? Also taking the data from Google must have had significant ramifications.
Hm interesting, this seems to examine Anthropic's prior work.
If I understood the paper right, during post-training (e.g. DPO) models learn the correct "shape" of responses. But unlike SFT they're also penalized for going off-manifold so this incentivizes development of circuits that can detect off-manifold responses (you can see this clearly with RLVR perhaps, where models have a "but wait" reflex to steer themselves back in the correct direction) [^1]. Since part of the training is to be the archetypical chatbot assistant though, when combining with anti-jailbreak training this usually gets linked into "refusal" circuits.
One hypothesis might be that the question itself is leading. I.e. models will by default respond "no" to "are there any injected thoughts", just as they would to "are you conscious" or "do you have feeligns", because of RLHF that triggers refusal behavior. Then injection provides a strong enough signal that ends up "scrambling" this pathway, _suppressing_ the normal refusal behavior and allowing them to report the injection. (Describing the contents of the injected vector is trivial either way, as the paper notes the detection is the important part).
The interesting thing is that ablating away refusals doesn't actually change the false positive rate though, so instead the above hypothesis of injections overriding a default refusal doesn't fit. Instead there really does seem to be a separate "evidence carrier" detector sensitive to off-manifold responses, which just so happens to get wired into the "refusal circuits" but when "unwired" via ablation allows the model to report injections.
I guess what's not clear to me though is whether this is really detecting _injection_ itself. Wouldn't the same circuits be triggered by any anomalous context? It shouldn't be any surprise that models can detect models anomalies in input tokens (after all LLMs were designed to model text), so I don't see why anomalies in the residual stream would be any different (it's not like a layer cares whether the "bread" embedding was injected externally or came through from the input token).
In theory the case of "anomalous input context" versus "anomalous residual via external injection" _can_ be distinguished though, because there would be a sort of "discontinuity" in the residual stream as you pass through layers, and the hidden state at token i depth n feeds into that of token i+1 depth n+1, you could in theory create a computational graph that could detect such tampering.
I think the paper sort of indirectly tested this in section 3.2 "SPECIFICITY TO THE ASSISTANT PERSONA"
>In contrast, the two nonstandard roles (Alice-Bob, story framing) induce confabulation. Thus, introspection is not exclusive to responding as the assistant character, although reliability decreases outside standard roles.
Which does seem to imply that as soon as you step out of distribution to things like roleplay that RLHF specifically penalized, the anomaly detectors start firing as well.
[^1] I think this is also related to how RLHF/DPO are sequence-level optimizations, with a notion of credit assignment. And optimizing in this way results in the model having a notion of whether the current position in the rollout is "good" or not.
Masked language modeling has been compared loosely to text diffusion [1], so the paper's title claim may be loosely true in some sense even if it's misleading.
Maybe for very high-level intuition, it's vaguely similar to other randomized algorithms that you want to minimize the worst-case on expectation and the easiest way to do so is to just introduce randomness (think quicksort, which is N^2 worst case with a badly chosen pivot). Your idea of there being an optimal distance is similar to the concept of "derandomization" maybe, e.g. in Quicksort there are deterministic pivot selection algorithms to avoid the worst case. But all of those require much more effort to compute and require an algorithm whose output is a function of the input data. Whereas randomly picking a pivot or randomly creating express lanes is simpler and avoids the data dependency (which is important since unlike sorting, the data isn't fixed ahead of time).
I still don't understand it, yes it's a lot of data and presumably they're already shunting it to cpu ram instead of keeping it on precious vram, but they could go further and put it on SSD at which point it's no longer in the hotpath for their inference.
reply