Your NeurIPS 2025 Paper: 7 Costly Comment Mistakes

The NeurIPS 2025 deadline is looming. You’ve spent months refining your architecture, tuning hyperparameters, and your validation loss is finally something to be proud of. But as you bundle up your supplementary materials, there’s one crucial element that often gets treated as an afterthought: your code comments.

Think of your code as a second abstract. Reviewers, especially those dedicated to checking reproducibility, will look at it. Clear, insightful comments can build trust and illuminate your contribution. Poor comments, however, can sow confusion, signal sloppiness, and sink an otherwise great paper. Let's make sure your comments help, not hurt, your chances.

Why Comments Matter More Than You Think

In a commercial software environment, good comments are about maintainability. In an academic context like NeurIPS, the stakes are different—and arguably higher. Your comments serve three critical functions:

• Building Reviewer Trust: Clean, explanatory code shows diligence and a commitment to open science. It tells the reviewer you care about others understanding your work.
• Ensuring Reproducibility: Your paper might explain the theory, but the code is the ground truth. Comments clarify non-obvious implementation choices that are vital for anyone trying to reproduce your results.
• Future-Proofing Your Own Work: Six months from now, will you remember why you chose a specific tensor permutation or a weird-looking constant? Your future self will thank you.

The 7 Costly Comment Mistakes

Avoiding these common pitfalls will dramatically improve the quality and impact of your supplementary code.

1. The “What” Comment: Redundancy Hell

This is the most common sin. These comments simply state what the code is doing, paraphrasing the code itself. They add clutter without adding information.

The Bad Example:

# Set learning rate to 0.001
learning_rate = 0.001

# Loop over the dataset
for i, batch in enumerate(dataloader):
    # Move data to device
    batch = batch.to(device)

A reviewer can read this code. The comments are just noise. The code is self-documenting.

The Good Example:

# Set a conservative learning rate. We found higher rates led to instability in early training epochs.
learning_rate = 0.001

# Iterate over the dataset for one full epoch.
for i, batch in enumerate(dataloader):
    # Move data to the GPU to accelerate computation.
    batch = batch.to(device)

Notice the difference? The good comments explain the why behind the code. Why 0.001? Why move to the device? This is the context a reviewer needs.

2. The “Tombstone” Comment: Outdated Lies

A comment that is wrong is infinitely worse than no comment at all. “Tombstone” comments are artifacts of previous experiments that you forgot to update. They actively mislead the reader and destroy your credibility.

Imagine a reviewer reading a comment that says # Using AdamW optimizer as per the paper, but the code below it clearly initializes torch.optim.SGD(...). That single discrepancy casts doubt on your entire implementation.

The Fix: Before you zip up your code for submission, do a dedicated pass to read only the comments and verify they still match the code they describe. Treat it like proofreading your paper.

3. The “Novel” Comment: Cryptic Shorthand

Your code is not the place for inside jokes, obscure acronyms, or overly-clever remarks that only you and your lab-mate will understand. The goal is to make your work accessible to a broad audience of researchers, not to prove you have a witty personality.

The Bad Example:

# The Bob maneuver for tensor alignment. Don't ask.
x = x.view(B, T, C).transpose(0, 1)

The Good Example:

# Reshape and transpose the tensor from (Batch, Tokens, Channels) to (Tokens, Batch, Channels)
# to match the input format expected by PyTorch's TransformerEncoderLayer.
x = x.view(B, T, C).transpose(0, 1)

4. The “TODO” Graveyard: A Trail of Broken Promises

We all use // TODO or // HACK during development. But leaving them scattered in your submission code is like leaving scaffolding on a finished building. It signals to the reviewer that the work is incomplete, unstable, or contains known issues.

If a piece of code is truly a temporary workaround, explain why and what its limitations are. Don't just leave a hanging TODO.

The Bad Example:

# TODO: Fix this, it's slow
for i in range(len(data)):
    # ... some inefficient operation

The Good Example:

# Note: This data aggregation is currently implemented with a simple loop for clarity.
# A vectorized NumPy/PyTorch implementation would be more performant but less readable.
# For our dataset size, the performance impact is negligible.
for i in range(len(data)):
    # ... some inefficient operation

5. The “Banner” Blindness: Noise Over Signal

While well-intentioned, large ASCII-art banners, elaborate comment boxes around every function, and redundant file headers often do more harm than good. This kind of boilerplate creates “banner blindness,” where reviewers learn to ignore large comment blocks entirely, potentially missing the important information hidden within.

Keep your file headers concise and your function docstrings focused. Let the code breathe.

6. The “Implementation-Only” Comment: Missing the Big Picture

This mistake is subtle. The comments might be locally correct, but they fail to connect the code to the core ideas of your paper. Your supplementary material is an extension of your paper; the two should be in constant dialogue.

Create explicit links between your code and your paper’s concepts.

The Bad Example:

# Multiply query and key, then scale
attn_scores = torch.matmul(q, k.transpose(-2, -1)) / math.sqrt(d_k)

The Good Example:

# Calculate attention scores as described in Section 3.2, Equation 4 of our paper.
# The scaling factor sqrt(d_k) is crucial for stabilizing gradients.
attn_scores = torch.matmul(q, k.transpose(-2, -1)) / math.sqrt(d_k)

7. The “No Comment” Comment: The Void of Confusion

Finally, the most straightforward mistake: not writing comments at all. This is especially damaging in a few key areas:

• Tensor Manipulations: Any .view(), .permute(), or .transpose() on a multi-dimensional tensor needs a comment explaining the change in shape and the reason for it (e.g., # (B, C, H, W) -> (B, H, W, C) for TensorFlow channel-last format).
• Hyperparameters: Why was the dropout rate 0.3 and not 0.5? Why 4 attention heads? A brief comment justifying non-obvious choices is invaluable.
• Data Preprocessing: The steps to transform raw data into model-ready tensors are often complex and unique to a project. This section must be meticulously commented.

Here is a comparison table to summarize the philosophy:

A Better Philosophy for Research Code Comments

Shift your mindset. Don’t think of comments as a chore. Think of them as part of the research itself. Here are the principles to guide you:

• Comment the Why, Not the What. If the code is complex, rewrite the code to be simpler before explaining it with a comment. Reserve comments for the high-level reasoning.
• Write for a Competent Outsider. Assume the reader is a fellow researcher who knows the field but doesn't know your project's history. What would they need to know?
• Comments Should Tell a Story. Your code and its comments should narrate the journey from data to result, aligning with the story you tell in your paper.
• Treat Comments as an Asset. Good comments are an investment in reproducibility and the future impact of your work. They elevate your code from a script into a durable research artifact.

Conclusion: From Code to Research Artifact

Your NeurIPS paper is more than just the PDF. It's the full package of ideas, experiments, and implementation. By avoiding these seven costly mistakes, you ensure your code's comments reinforce your research, build confidence with reviewers, and pave the way for others to build upon your brilliant work. Good luck!