Systems research and open problems

Begin with an observed bottleneck and competing explanations. If inference latency rises, possibilities include memory bandwidth, queueing, compilation churn, synchronization, or a workload shift. A research question should predict an observable difference between explanations. For example, if a cache policy helps by reducing repeated transfers, its advantage should shrink when reuse is removed. Define that counterfactual before building a large optimized implementation.

Separate the contribution from the evidence standard. A new measurement method needs validation against known cases; a new scheduler needs workload and quality comparisons; a correctness mechanism needs an explicit fault model and argument. A useful engineering synthesis may combine known mechanisms in a newly effective design. Describe that contribution honestly instead of implying that a practical integration proves a universal principle. State which assumptions are essential and which are conveniences of the prototype.

An ablation removes or changes one proposed component while keeping other relevant conditions comparable. If the full system improves but the ablation does too, the claimed explanation may be wrong. Interactions complicate one-at-a-time comparisons: batching and caching can help jointly even when each alone does little. Use factorial comparisons where feasible, report tuning effort, and avoid crippling the baseline through an unsuitable configuration.

Evaluate regimes where the idea should fail. A communication optimization should be tested on a fast network as well as a slow one; a queue policy should face bursts and skew; a numerical shortcut should face difficult inputs. These cases reveal the operating envelope rather than merely lowering an aggregate score. Include implementation complexity, resource consumption, and failure recovery in the tradeoff, because a small throughput gain may require unacceptable operational burden.

Several durable questions connect the module: how to allocate compute when data quality changes, when communication compression beats a strong baseline, how compilation adapts to varied shapes, and how energy accounting changes a deployment choice. These are research directions, not claims that one method is currently best. Narrow each to a workload, an intervention, and a measurable outcome. A small experiment that rejects a poor assumption can save more resources than a large benchmark campaign.

Prioritize by expected information and practical consequence. Estimate experiment cost, identify the decision that a positive or negative result would change, and reserve an independent confirmation budget. Publish limitations and negative results with enough context to be reusable. The end of a study should identify what remains unknown and the cheapest credible test that would reduce that uncertainty, while preserving the artifact and evidence needed to challenge the present conclusion.