This feels like the sort of advice that increases the variance of outcomes more than the mean, if that makes any sense. It'll help some people, but also really won't help others. It's like... You'd need some serious disregard for the rest of the physics community to think you can figure out a reactionless drive. This is great if you actually can, but otherwise it's going to lead you to failure.
In a lot of ways, getting advice from the outliers on the very edge of a field is like getting advice from lottery winners. That's not to say you shouldn't - but at the same time a lot of effort has to be made to separate the wheat from the chaff.
I think the advice here is "get as good as you can, get as smart as you can, and then, when you decide to set off on embarking on a calculated risk, try to forget all the noise around you and stay the course."
I feel like the problem is there's a lot of wannabes who skip the steps of actually gaining competence and then disregard the advice of others. That's not going to work. The first step is years of hard work. For instance, crank physicists who don't really understand graduate level physicists are not going to accomplish anything by disregarding the advice of others.
However, if you know what you're doing, don't second guess yourself when embarking down a risky path. It's probably not as dangerous as it may seem.
I think it would be very desirable to increase the variance of outcomes without significantly decreasing the mean.
The way research funding is structured now, researchers are incentivized towards taking small publishable steps without too much risk. This is needed of course, but it isn't going to bring about another Newton.
I think we desperately need a few more Newtons if we're to prosper in the next hundred years.
A research infrastructure that could support improving the variance of results without significantly impacting the mean would be an enormous boon to the world.
I often wonder how much this applies today. Academic science is currently heavily driven by funding of popular, low-risk research.
Also, without meaning any disrespect to Feynman or the advice, it's easy enough to say something like "disregard what others are doing" when you've won a Nobel. It's another thing to say this to people in just about any other inequivalent scenario, which is nearly everyone.
Anyone who wants high mean low variance career should become an accountant or a dentist: the payoff is vastly better, and it takes a comparable amount of intelligence.
This low variance attitude is why we can't have nice things. We have more "scientists" alive now than at any time in human history, and it is boring and useless with virtually no downstream results.
Part of the issue is that scientific fields are very punishing if you deliver negative results, thus the incentive to "shoot for the moon" is greatly lessened.
In fields like art, programming, and physics, you'd better be doing something weird. If you're doing something mainstream, the same thing thirty other people around the world are doing, you're all competing to make the same thing — paint the same painting, write the same text editor, prove the same theorem about black holes. Twenty-nine of you are going to get scooped by whoever is the hardest-working, the smartest, the best-supported institutionally, or whatever combination of those turns out to be the deciding factor. Your chance of being in that 97% who have totally wasted their efforts? 97%.
And if you're doing something really mainstream, like writing the next big client-side JavaScript framework, the one that will replace React, watch out! Your chances are a lot worse than that, because there are hundreds of thousands of people who fight with React every day and are frustrated with its shortcomings. Your chances are literally millions to one, unless you work at Microsoft or Google and have management support to beat those Facebook fuckers.
But if you're doing something offbeat, working on a problem† that's mainstream enough to be interesting if you're successful but not mainstream enough that dozens of people are already spending their weekends trying to solve it, you have a much better chance of finding a niche for your project. Maybe it's non-mainstream because people take for granted that it can't be solved (in which case they might be right, as with the reactionless drive — the objective here is to be weird in your project goal, not your epistemology); maybe because they don't understand why it would be important to solve it ("Where's the market?"), and you do; maybe, as with OpenSSL, it's an important problem, but there's no way to get paid for solving it.
A thousand hackers writing a thousand versions of the same library in the same way are only epsilon more productive than one hacker. A thousand hackers writing a thousand different libraries are almost a thousand times as productive.
Winning the lottery? Well, that's pretty much out of your control — but if you do decide to waste your money on the lottery, don't pick a number lots of other people are picking. Then you'll have to split the already-improbable winnings N ways.
But what if you're determined to solve a mainstream problem anyway, one where a thousand people are also trying to solve it? Then you need all the outcome variance you can get! If, of those thousand hackers, 500 are using a very conservative approach that is guaranteed to solve the problem with some quality metric 10 ±1 in 26 weeks ±2 weeks (these being the standard deviations, not some kind of 95% confidence interval), while the other 500 are using all kinds of wild approaches that give them quality metric exp(ln(4)±ln(4)) in time exp(ln(52)±ln(4)) weeks, it's pretty much guaranteed that the "winner" is going to be someone with a totally insane approach that hacked together a library with quality 49 in only 14 weeks. It's not going to be one of the 26-week plodders, because 14 weeks is 12 standard deviations out on their distribution (vs. 0.95 out on the crazy hackers' distribution), and quality 49 is 39 standard deviations out (vs. 1.3 out on the crazy hackers' distribution).
In fact, it's even worthwhile to sacrifice expectation to get higher variance in these situations. If your only hope of winning is to beat everyone else in a single round, you should do whatever will increase your minuscule chance of a home run, regardless of how it affects your chances of striking out.
It's still not socially optimal for people to behave this way — note that here you have 1000 hackers whose aggregate productivity is only about 20× the productivity of an average plodder — but if you've gotten suckered into competing for a mainstream niche, that's the way to play the game.
All of the above is for the simplified situation where you're working on a project by yourself. In a teamwork situation, the relevant actor is your team, not you individually. Do not write your code in Clojure if the rest of the team is working in Ruby. Do not try to solve only problems that nobody else on the team thinks are important.
And this advice definitely does not apply to a situation where doing the same thing someone else already did is valuable. If you're making a sandwich, there's no reason it needs to be different from the sandwich someone else is making across the street. They're two different sandwiches. If someone eats the sandwich across the street, it's gone and it can't feed your customer. They're going to be happy if you make them a sandwich they like, even if it's a little worse than the sandwich across the stret; even if there are many just like it, this sandwich is theirs. This is very different from the situation in software, where one sandwich feeds everybody in the world at once, except people with celiac disease. Nobody is going to be happy that you wrote a web browser from scratch for them instead of just installing Firefox.
† I recognize that a painting is not "solving" a "problem", but many of the same principles apply.
I don't know! That sounds like a worthwhile project to me; I'm on one email mailing list which only comes out in a nightly digest in order to remove the incentive for instant responses and promote thoughtful response instead. Harvard's Rotisserie system (http://h2oproject.law.harvard.edu/) was another effort in that direction. And I was also on Slashdot starting around 1997, and I observed the incentive structure of its commenting system pushing discussions in the opposite direction — with commenters racing to make the first comment in a new comment thread, even if it was just "First post", before the voting system was introduced.
The biggest difficulty I've had with such things is a sort of Gresham's Law of attention: because Twatter and Fecebutt (and IRC) offer the possibility of instant reactions, I have a strong temptation to spend time on them instead of in more thoughtful, slower forms of interaction. The wild success of Fecebutt and, even more so, Zynga, suggests I'm not the only one.
I hadn't heard about Gresham's Law. Thanks a lot for that. And I definitely hadn't heard about Rotisserie:
> The Rotisserie implements an innovative approach to online discussion that encourages measured, thoughtful discourse in a way that traditional threaded messaging systems cannot. In contrast to the completely asynchronous, broadcast-to-broadcast mode of existing threaded messaging systems, the Rotisserie adds structure to both the timing and the flow of the discussion.
Thanks for that too. That's exactly the communication platform I made/am making. For me there's also the "human" element as described in Time, Work-Discipline, and Industrial Capitalism [0]. Like how "human time" is not "modern life time," but we're still humans so wouldn't it be nice to have a space where we communicate in human time. But yes Gresham's Law... hmmm...
Rolled it out a bit as “email” actually but that proved confusing for people: “email” carries a lot of baggage, social and technical. Why do you say email? It’s instant.
It depends. If you're hacking on free software, as you should be, then after the first iteration, everyone can start using the clearly-much-better thing that is already working, rather than wasting months finishing their own inferior versions. And the person who wins the race that time around may not be the same one who won the first time.
The other thing is that there are distributions like the Cauchy distribution that are so heavy-tailed that they don't even have a mean†, or even a variance. The Law of Large Numbers doesn't apply to them at all! And even for more ordinary heavy-tailed distributions that do have means, like the lognormal distribution (relevant here since it's empirically the distribution of how much we misestimate tasks by) the Law of Large Numbers takes a lot more than "a few times" to start making the distribution of the sum look normal. Try it! Pop open Jupyter and convolve the lognormal distribution with itself a few times! How long does it take before it even starts to look Gaussian? How long does it take before it still looks Gaussian in a log-linear plot?
†In your comment you said "mean and expectation", but those are the same thing. Possibly you don't know anything about statistics yet? If so, welcome! You're one of today's lucky ten thousand!
I don't think this was about disregarding everyone around all the time. This was something that helped Feynman to move in that particular situation when he was weighted down by other peoples expectations and what not.
The story is at the point where he is already honored, known and has "outstanding popular touch". That is when he read a book and calls the wife in the middle of the night to say: "I think I’ve figured it out. Now I’ll be able to work again!"
This does not mean that people should disregard others all the time routinely. It means that there are situations where you have to disregard others expectations, advice and what not.
That is where expertise, experience and actual knowledge comes to play. Which Feynman at that moment possessed. It is also relative, you can be both too much accommodating and too much disregarding. One person can move between the two extremes over his own life.
The more experienced I get at programming, the more I find that disregarding others is the correct choice. I have a background and intuition informed by 'common wisdom', but enough confidence to go against it when it seems to make sense without second guessing myself.
Well, at least sometimes. I am getting better at it.
For a master of his domain like Feynman that affect would surely have been amplified. He can back himself to go on a tangent, against the backdrop of the backdrop of the fundamentals of his field.
I had opposite development. At first, those who disregarded others looked like geniuses and heroes. Over time, I started to see damage they caused. Well timed doing your thing is indeed good thing, but those who routinely disregard others slow overall speed of the whole team down.
That depends on how the team compares to the mean of (larger) teams. Teams that are grown by weak engineers who never tried to improve their skills through cross polination are examples. You cant fix the whole situation, but you can find a few very smart ones to go against "common" wisdom
Because common wisdom may be common to your small team, and not at all to a more skilled set of engineers.
An approach that increases variance + survivor bias = a number of exalted recommendations: "for that brilliant person it worked like a charm, despite everyone's doubts!"
Obviously when the same approach went someone to a spectacular ruin, that someone is an obvious crackpot who insisted on a wrong thing despite everyone's opinion. It's not worth a story, so nobody mentions these losers.
> increases the variance of outcomes more than the mean
So, it makes sense to apply it in circumstances where worse outcomes cost you the same as just bad ones, but best outcomes bring in much more than ordinary good ones.
In a lot of ways, getting advice from the outliers on the very edge of a field is like getting advice from lottery winners. That's not to say you shouldn't - but at the same time a lot of effort has to be made to separate the wheat from the chaff.