I’ve come full circle on this topic through grad school and dissertation and working in product companies.
I think “fundamental research” is not a thing, except possibly in mathematical theory. There is only incremental research + good salesmanship.
Research results should show up at a predictable regular pace, or else the result of that effort should be better & deeper explanation of a negative result which is also valuable.
If you’re sinking 2 years of R&D costs on something and you’re not getting the positive result you wanted and the negative results are not incrementally adding up to a clearer and clearer diff between the state you’re in and the state you’re trying to get to, then it is wasted money and the researcher isn’t being effective in their job.
I really think even difficult research tasks need to be rescoped and broken up into a series of incremental challenges, each of which has a known way to address it. You must treat it with reductionist dogma and eventually you’ll keep breaking it down into constituent parts that have known solutions until you hit on the novel problems to solve and it will be at a level of scope small enough that you can infer the solution from existing methods.
That’s all there is in the world. There’s no miracle cure for cancer or climate change or aging or social inequality. Let alone random business problems. There’s just a big bunch of little tiny problems with boring solutions that get all glued together into bigger messes that are hard to figure out. You can try smashing with a hammer or basically scaling the hammer up or down, that’s about it.
That seems like a blanket assertion. It certainly seems like the most "manageable" mode of operating, but looking back at human history it seems like the "big breakthroughs" rarely happened that way.
The model of scientific research as harvesting the tail events (low probability & massive payoff) seems quite incompatible with what you've said.
That said, you probably think the way you do because of your experiences. Can you articulate that better?
> Research results should show up at a predictable regular pace, or else the result of that effort should be better & deeper explanation of a negative result which is also valuable.
To make a slightly more provocative claim, I think that this fetish for predictable/steady research progress is one of the primary causes of the problem discussed in the article. While papers can be generated steadily, piling on details doesn't necessarily make insight. (To quote Alan Kay: "A change of perspective is worth 80 IQ points")
PS: There's an apt SMBC comic (Aaargh, I'm unable to find right now!) where researchers keep digging a tunnel linearly, and declare the field dead, while there is a big gold mine slightly off to the side.
I disagree. The perception that you need big discrete jumps forward in progress is what leads to the problem of the article. If instead you directly incentivize breaking a problem up into incremental goals from the start, and never pursue giant jumps of progress at all, then you can get it.
The problem is when people have been wasting time on big leaps of progress and then feel pressured to deliver something when they haven’t been pursuing incremental progress all along.
I disagree. In order to make real scientific Breakthrough it is necessary to take a leap in thinking.
Think special theory of relativity or quantum mechanics kind.
On the other hand:
> The problem is when people have been wasting time on big leaps of progress and then feel pressured to deliver something when they haven’t been pursuing incremental progress all along.
That’s why the most upvoted comment with Thompson’s quote is so true. You need to pay gifted people to do simple, predictably realisable things but give them lots of leisure time to pursue big leaps.
Many problems with the universities and academic system comes from the fact that granting system etc. sweats the small stuff not leaving enough leisure time to pursue big ideas.
Why do you think special relativity or quantum mechanics are “big leap” things? Those are huge systems of theories comprised of thousands of different experiments and mathematical results driven by thousands of different researchers, almost all of whom were pursuing very small scope, narrowly defined incremental gains of knowledge. Even modern quantum computing follows this pattern. People are not solving it with lots of leisure time. They are solving it within private companies as full time researchers or via industry grants, federal grants, university pipelines of incremental experiments, and focus on very straightforward scalability of known existing systems and so on.
If anything, examples like these highlight why the Thompson quote _is wrong_.
> Why do you think special relativity or quantum mechanics are “big leap” things?
It has been more than 90 years since they were invented. At the time they were proposed they've been unexpected and quite unwelcome.
I definitely agree that right now the progress is more linear, distributed and steady, but I wouldn't exactly call quantum computing (application of theory that has been created ~90 years ago) "big leap". I would rather compare it to development of hear engine from thermodynamics.
By big leaps I meant "paradigm-shifting" kind of things. These things cannot be expected to come from steady pipelines and projects with expected outcomes with anticipated results.
I have not seen an example of a singular thing that is “paradigm shifting.” Even with relativity, the overall set of concepts and results is paradigm shifting, but every individual contributor, including Einstein, pursued small specific incremental pieces and various postulations whose significance wasn’t determined until years latee through incremental experiments.
I think “fundamental research” is not a thing, except possibly in mathematical theory. There is only incremental research + good salesmanship.
Research results should show up at a predictable regular pace, or else the result of that effort should be better & deeper explanation of a negative result which is also valuable.
If you’re sinking 2 years of R&D costs on something and you’re not getting the positive result you wanted and the negative results are not incrementally adding up to a clearer and clearer diff between the state you’re in and the state you’re trying to get to, then it is wasted money and the researcher isn’t being effective in their job.
I really think even difficult research tasks need to be rescoped and broken up into a series of incremental challenges, each of which has a known way to address it. You must treat it with reductionist dogma and eventually you’ll keep breaking it down into constituent parts that have known solutions until you hit on the novel problems to solve and it will be at a level of scope small enough that you can infer the solution from existing methods.
That’s all there is in the world. There’s no miracle cure for cancer or climate change or aging or social inequality. Let alone random business problems. There’s just a big bunch of little tiny problems with boring solutions that get all glued together into bigger messes that are hard to figure out. You can try smashing with a hammer or basically scaling the hammer up or down, that’s about it.
edits: fixed typos