If it’s simply an engineering feat I agree. Very, very, very often people tend to mix up science being done in technology spaces with engineering. Just because it’s being done in software and perhaps even without rigor doesn’t mean you’re not doing something novel and quite different at a fundamental perspective than someone else. Sometimes it could just be luck of the draw in an implementation approach that gets you there but that’s not always the case.
I’ve worked in scientific computing for awhile now and there are countless subtle decisions often done in the implementation phase that, from a theoretical perspective, aren’t definitively answered by the science working behind the scenes. There’s often a gap in knowledge and implementers either hit those gaps by trial and error, luck, or insight. That’s my opinion, at least. So I don’t think it’s always “just an implementation problem” some will claim, as if the science is well understood and solved. Perhaps it is, but from my experience that tends to not be the case.
Yep I agree with that. I will say though that people, teams, and even entire companies (like what happened at Inflection) get poached everyday so maintaining a moat that way is tough. Also, even though it could happen in the future, is OpenAI's lead due to a moat of scientists with ideas so novel that no other AI company can compete? Certainly not because even though ChatGPT took the world by storm, numerous other companies built LLMs in a very short time span that now perform at very similar levels, both subjectively and based on benchmarks.
This practice supercedes modern science and was originally called trade secrets.
And these are generally "defeated" through spionage. 1 billion $ let's you do a lot, including completely legal things such as poaching someone that has a basic understanding of the secret sauce and then copying that from the abstract description
This 100%. And LLMs and the applications around them (even something as simple seeming as ChatGPT) have more subtle decisions than any other type of software that I've ever seen. Everyone claims there isn't a moat, I bet there is.
I’ve worked in scientific computing for awhile now and there are countless subtle decisions often done in the implementation phase that, from a theoretical perspective, aren’t definitively answered by the science working behind the scenes. There’s often a gap in knowledge and implementers either hit those gaps by trial and error, luck, or insight. That’s my opinion, at least. So I don’t think it’s always “just an implementation problem” some will claim, as if the science is well understood and solved. Perhaps it is, but from my experience that tends to not be the case.