BetterOffice / evals

v2 · 2026-10-15 · synthetic

PPTX tasks

gpt-6-luna

PPTX results for gpt-6-luna
TaskGeneric file toolsBetterOffice MCP
Change a slide title

pptx-01-change-title

Task prompt

In input.pptx, change the title on slide 1 from Quarterly update to Quarterly results. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Fail

Run 1 · 2026-10-13

2 tool calls · 9.96 seconds

The model assigned shape.text to rebuild the title paragraph, dropping its paragraph and run formatting. Cause: model.

Checker message

slide 1.shape 1.paragraphs[1].props: count differs (4 expected, invalid found)

Pass

Run 1 · 2026-10-14

5 tool calls · 24.63 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Update a reported figure

pptx-02-update-revenue

Task prompt

In input.pptx, on slide 2 change Revenue: $1.2 million to Revenue: $1.5 million. Leave the margin and retention figures intact. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

2 tool calls · 24.05 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

7 tool calls · 22.95 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Replace a bullet list

pptx-03-replace-bullets

Task prompt

In input.pptx, replace the three bullets on slide 2, Next steps, with these three bullets in order: Finish the pilot; Collect feedback; Launch in October. Keep the existing bullet formatting. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

2 tool calls · 18.98 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

7 tool calls · 25.75 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Rename a project in slide text

pptx-04-replace-term

Task prompt

In input.pptx, replace Atlas with Beacon everywhere in the visible slide titles and bullets. Leave speaker notes as they are. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

4 tool calls · 21.97 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

11 tool calls · 28.91 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Finalize two slide titles

pptx-05-finalize-titles

Task prompt

In input.pptx, change slide 2 title Draft results to Results and slide 3 title Draft next steps to Next steps. Leave mentions of those phrases in bullets and speaker notes unchanged, and keep both slides in their current positions. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

2 tool calls · 29.40 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

7 tool calls · 19.41 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Remove an appendix slide

pptx-06-delete-appendix

Task prompt

In input.pptx, delete slide 3, titled Appendix. Keep the other slides in their current order. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

2 tool calls · 12.32 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

5 tool calls · 18.42 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Reorder a presentation

pptx-07-reorder-slides

Task prompt

In input.pptx, arrange the slides in this order by title: Overview, Agenda, Results, Next steps. Keep each slide's content and speaker notes attached to that slide. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

3 tool calls · 21.30 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

8 tool calls · 33.42 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Write speaker notes

pptx-08-speaker-notes

Task prompt

In input.pptx, replace the speaker notes on slide 2, Results, with exactly two paragraphs: Pause for questions after the chart. and Ask which metric needs a follow-up. Put each sentence in its own paragraph. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

2 tool calls · 27.24 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

6 tool calls · 19.46 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Add a positioned text box

pptx-09-position-textbox

Task prompt

In input.pptx, add a text box on slide 1 containing Internal draft. Place its top-left corner 0.5 inches from the left edge and 5 inches from the top edge, with width 3 inches and height 0.5 inches. Use 18-point text in #666666. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Pass

Run 1 · 2026-10-13

2 tool calls · 11.13 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Pass

Run 1 · 2026-10-14

5 tool calls · 26.72 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Insert a slide with bullets and notes

pptx-10-insert-slide

Task prompt

In input.pptx, insert a new slide immediately after slide 2, Agenda. Give it the title Pilot plan and three separate bullets in this order: Start with ten users; Collect feedback daily; Report results in two weeks. Add exactly this speaker note to the new slide: Keep the pilot to two weeks. Save the result as output.pptx, leaving input.pptx unchanged and preserving everything else.

Fail

Run 1 · 2026-10-13

8 tool calls · 52.57 seconds

python-pptx created an extra notes master and theme when the model added speaker notes to the new slide. Cause: library.

Checker message

layouts and masters: fields differ

Pass

Run 1 · 2026-10-14

9 tool calls · 35.73 seconds

Checker message

exact requested content, formulas, formatting, order, notes, and preserved content verified

Reading these results

Tool calls count different things across arms.

  • Generic file tools: Shell commands and file edits; one command can run a whole script.
  • BetterOffice MCP: BetterOffice MCP calls.

Each arm ran in separate batches (Generic file tools: 2026-10-13; BetterOffice MCP: 2026-10-14). Times are not comparable across arms.