How to Be Good at Research
- Source
- x.com
- Author
- vivek
- Date
Context
Research judgment can be practiced by making predictions explicit before seeing results. In vivek's research essay, the proposed habit is to record a hypothesis, experimental setup and expected outcome, then compare the result with that expectation and update the explanation.
The same discipline applies when an experiment looks successful. The essay recommends strengthening the comparison method and removing components one at a time to find what actually caused an improvement. These are concrete ways to challenge a favored explanation: a better score matters less if it disappears against a stronger baseline or comes from a different component than expected.
- An absorbed problem hands you the conclusion without the reasoning. You know a famous lab cares about a direction but not why, not what they expect to find, not what would make them quit — so when they pivot, you learn about it a year late.
- Schulman splits research into two modes: read the literature and hunt for improvements, or pick an outcome you want to exist and reason backwards to experiments. The second manufactures originality, because a goal you care about drags you where no survey goes.
- Old material is underpriced because the field reruns its own past on a delay: is from 1991, LSTMs 1997, backprop went mainstream in 1986. Sutton's thousand-word bitter lesson predicts the field better than surveys ten times its length.
- Shannon's 1952 method for a stuck problem: shrink it until it is nearly trivial, solve the small version, then add the difficulty back one piece at a time.
- Read the paper, not the thread about it. The appendix is where the bodies are buried and the limitations section is usually the most honest paragraph in the document.
- mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
“a forecast plus a correction, repeated a few hundred times, is how every good model gets trained, including the one in your head.”
“the appendix is where the bodies are buried, and the limitations section is usually the most honest paragraph in the document.”
“honest statistics might be the rarest skill in ml, where a lot of published rigor is vibes with error bars.”
“a body of public writing also doubles as the strongest credential you can hold, because it's an unfakeable sample of how you think.”
Checking sign-in…
Loading comments…