What this research grades
This research note grades one forecast for one reef on one day: Macaronis, a left-hand reef in the Mentawai Islands, Indonesia, on Sunday 13 September 2026. The call was sent to the operator at the break two days in advance, with a written list of what would make it wrong, and it is scored here against the model at the corrected reef cell, win or lose.
Two things happened before the call was made. The operator corrected where AYON Ecosystem thought the reef was, and the operator's own rules for the break were written into the AYON Wave spot record. Both change how the grade should be read, so they come first.
The point sat 8.3 km from the reef, in a different model cell
The operator received AYON Ecosystem's free forecast widget after an outreach e-mail on 4 September 2026. They asked for it to carry the break's name, then asked for proof of accuracy against Surfline and Windy, and whether the forecast was a subscription. The answers were that the forecast is free, since AYON Ecosystem charges for the school operating layer and not for the weather, and that AYON Ecosystem would not claim to beat Surfline.
On 10 September the operator sent the exact reef position. AYON Ecosystem's point for the break sat 8.3 km to the south-east of it, and the two positions resolve to different marine model cells: -2.875/100.042 for the old point, -2.792/99.958 for the reef.
Most hours the difference between the cells is small. At times it reaches 0.40 m in height, 2.55 s in period and 12° in direction, which at a reef that cares about a ten-degree window is not small. The record now carries the reef at -2.783948, 99.972289.
The operator's rules, encoded, and why the record still says unverified
With the position came the operator's rules, in their words. Best swell is "SW around 210 to 220 degrees. S swell also works but the wave gets shorter and heavier at the takeoff". On wind, the break "handles almost every direction except NW, which is our devil wind and shuts it down".
Two edits followed in commit ccfbd8d on 10 September. The swell window was recentred to 195–225°, so that a south swell still scores through the taper instead of being cut off at the edge, which is how the operator describes it working. The offshore wind bearing was set to 135°, the exact reciprocal of north-west, so the devil wind sits at the bottom of the wind score.
The record's verified flag stays false, because in AYON Ecosystem's evidence vocabulary the operator's e-mail is a field observation: one timestamped human report with legitimate provenance. Verified means evidence reviewed against source and context, with repeat observations. One e-mail is a single observation, not yet reviewed against source or repeated, and no one from AYON Ecosystem has reviewed the reef in the water. The operator's reply on 11 September, "That is a level of care we did not expect from a free tool", is welcome and changes nothing about the flag.
The operator asked for one thing on the product: swell direction and wind direction on the widget. That shipped, on the widget and in the app spot card, as a line of the form "2.8 m · 13 sec · SSW 209°".
A call sent on Friday, with its own miss condition
On 11 September AYON Ecosystem sent the operator a call for Sunday 13 September: peak around 2.2 m at 14 s from 213°, wind light ESE to SE at 7–11 kn, and no north-west wind for seven days. The message carried an explicit written list of what would make the call wrong. The operator said they would grade it against Surfline and Windy.
Before Sunday, live model runs began arriving with direction at 195–209°, south of the operator's window. Rather than adjust the call, AYON Ecosystem wrote to the operator: "if it stays south of 210 through Sunday … that's a miss on our side and I'll say so." The call was left as sent.
How Sunday scored
The grade is against Open-Meteo at the corrected reef cell on 13 September. It is a modeled verification, not a beach observation. It is kept here as published on 13 September, because it contains an error of AYON Ecosystem's own that the operator's report exposed the next day; the correction follows below, and the record of what was published first stays.
Four hits, two misses. The peak day was right, and 06:00 on Sunday was the true peak of the whole 11–16 September window. The height was under-called by about 30%: 2.86 m arrived against 2.2 m called. The direction stayed south of the operator's 210–220° window all day, 194–202° with 201° at the peak, a miss of about 12° that had been declared before Sunday.
- Peak dayHit
- Called
- Sunday 13th
- Model at the reef cell
- Sunday 13th, peak 06:00 — the true peak of the 11–16 Sep window
- Peak heightMiss
- Called
- ~2.2 m
- Model at the reef cell
- 2.86 m — under by ~30%
- PeriodHit
- Called
- 14 s
- Model at the reef cell
- 15 s at peak, 12–15 s across the day
- DirectionMiss
- Called
- 213° (inside the operator's 210–220° window)
- Model at the reef cell
- 194–202°, peak 201° — south of the window all day, ~12° off
- WindHit
- Called
- Light ESE/SE, 7–11 kn
- Model at the reef cell
- 3–10 kn, 117–169°, drifting SSE/S in the afternoon — speed hit, direction wider
- No NW wind for 7 daysHit
- Called
- None
- Model at the reef cell
- Zero NW hours
Both misses point the same way, and one answer decides what they mean
The two misses are not independent. The swell came in bigger and further south than called. By the operator's own description, a south swell at Macaronis "gets shorter and heavier at the takeoff". So the model, at its own cell, is describing a Sunday that should have broken shorter and heavier than the call implied.
That leaves one question with two very different answers, and it has been sent to the operator: did it actually break shorter and heavier on Sunday? If yes, the model at the reef cell carries a measurable direction bias that can be corrected in the record. If no, AYON Ecosystem's read of this reef is wrong at a level deeper than a few degrees, and the correction is to the understanding, not to a number.
At publication the answer had not been received. It arrived on 14 September, and it was no.
The operator's report: it did not break short and heavy, and the wind was twice the call
The operator's report came on 14 September, and it reversed the reading. The wave did not break shorter and heavier. The direction was good for Macaronis, "more southwest than your reading showed", close to Surfline's call of around 218° and solidly inside the 210–220° window. Faces were 4–5 ft, maybe 6 ft on the bigger sets, smaller than a 2.86 m call would suggest at this reef. And the wind was SSE to S at 15–20 kn, much stronger than the 3–10 kn in AYON Ecosystem's scorecard, a pattern the operator says has been consistent since mid-July.
The direction reversal sent AYON Ecosystem back to the data, and the error was AYON Ecosystem's. The 194–202° and 2.86 m that the 13 September grade scored against were whole-sea figures, swell plus wind chop, pulled south and up by a 0.5–0.7 m sea raised by the south-south-east wind. A swell call is scored against the swell partition, and that was not done. The swell partition at the reef cell for 13 September reads 206–210° in the MFWAM wave model and 205–211° in GWAM, at 2.4–2.7 m and 14 s through the morning peak; ECMWF's wave model, which publishes only the whole sea, reads 206–210° at 2.2–2.4 m. The 213° call was inside the window, within a few degrees of both models' swell and of Surfline's 218°. Direction was a hit, and the published miss was a grading error, not a model error.
Size is not scored, neither as a hit nor as a miss. The 2.86 m was never the call; it was the whole-sea height the grade wrongly used. The call was 2.2 m of deep-water swell, and the operator reports faces of 4–6 ft, which is a different quantity, measured by eye. No instrument sat between the two, so the row carries no verdict.
Wind is the real miss, and it is not softened. Four atmospheric models, the blended model, ECMWF IFS, GFS and ICON, put the 10 m wind at the reef cell at 3–12 kn sustained through the day, from ESE in the morning turning to SSE and S in the afternoon, 117–173°. Gusts ran 11–24 kn, and only the gusts come near the 15–20 kn the operator reports: 15–18 kn in ECMWF IFS and 17–18 kn in ICON through the afternoon, 19–24 kn in the blended model. The models had the direction and about half the speed. Since the operator says the pattern has held since mid-July, this reads as a systematic under-read of the south-east trade wind at this channel by global models whose cells are tens of kilometres wide, not as a bad day.
- Peak dayHit
- Called
- Sunday 13th
- Operator's report and model swell partition
- Not disputed by the operator; peak 06:00–07:00 at the reef cell
- Peak heightNot scored
- Called
- ~2.2 m
- Operator's report and model swell partition
- Faces 4–5 ft, maybe 6 ft on sets, by eye; swell partition 2.4–2.7 m. The 2.86 m was the whole sea, not the swell and not the call. No instrument, no verdict
- PeriodHit
- Called
- 14 s
- Operator's report and model swell partition
- Swell partition 14 s through the morning peak; not disputed
- DirectionHit
- Called
- 213° (inside the operator's 210–220° window)
- Operator's report and model swell partition
- Broke as SW, about 218° on Surfline; swell partition 205–211°. The published 12° miss was AYON Ecosystem's grading error
- WindMiss
- Called
- Light ESE/SE, 7–11 kn
- Operator's report and model swell partition
- SSE to S at 15–20 kn, consistent since mid-July; four models 3–12 kn sustained. About half the speed
- No NW wind for 7 daysHit
- Called
- None
- Operator's report and model swell partition
- None reported; zero NW hours in the models
What the grading error was, and what changes
The rule that was broken is simple. A swell call is scored against the swell partition, the long-period energy from the source, and never against the whole sea, which adds the short chop the local wind raises. On a trade-wind day at Macaronis the two differ by 6–13° and by about 0.3 m, and the chop sits on the south side. The 13 September grade used the whole sea. From now on, every swell call in this library is scored against the swell partition, and the Trestles locks follow the same rule.
The same error sat on the public widget. The card's "Swell from" line read the whole-sea direction, not the swell partition, so a reader at Macaronis saw a bearing 6–13° south of the swell. Commit 34efd4e on 14 September makes the line read the swell partition, with the whole sea only as a fallback where a model has no partition for the cell. At the time of the fix the reef cell read 195° for the whole sea and 208° for the swell. The AYON Wave app already read the partition and needed no change.
The wind miss is now in the Macaronis record, as an observation and not as a correction. Two questions went to the operator on 14 September and were answered on 15 September. The 15–20 kn is felt wind, estimated by their team at the takeoff and not read from any instrument. And since mid-July it has been strong and consistent through most of the day, with some early mornings lighter before building. That is the shape a systematic under-read by 25 km grid cells would produce, and it is specific enough to write down without dressing it up as more precise than it is.
On 16 September the spot record gained a wind field note, labelled OBSERVED and dated, carrying the operator's estimate, the mid-July onset, the lighter dawn and the build through the day, with the four models' 3–12 kn recorded beside it. It is a standing caveat and not a scoring input. A felt estimate is not a calibrated correction, and letting one silently overwrite the live forecast number would be its own kind of dishonesty. The verified flag stays false: one operator, no instrument, and nobody from AYON Coastal Intelligence has been at this reef.
What the grade is, and what it is not
It is a model scored against a model at the same cell, after the position was corrected. That answers whether the call held against the best available estimate at the reef; it does not say what a person on the reef saw. The operator's report, which came on 14 September, is the only beach-level evidence in this test, and it is a field observation, not a verification. It reversed one verdict and created another.
The record stays unverified. The direction miss was pre-declared, and it turned out to be AYON Ecosystem's own grading error rather than the model's; the wind miss was declared by no one at AYON Ecosystem and found by the operator. Both grades are published, the first one uncorrected and the second one beside it, because that was promised to the operator in writing before the swell arrived, and a verification that reports only when it wins is not one.
This is research. Estimates are models, not guarantees; a model is never an observation, and a machine estimate is never “confirmed” — only a human reviewing it in the field makes it verified. Nothing on this page is an operational-capability claim, a safety promise, or a prediction of a specific event at a specific place.
References & data
- 1.Open-Meteo Marine API — Verification source: hourly whole-sea and swell-partition height, period and direction at the corrected reef cell -2.792/99.958 for 13 September 2026, from the MFWAM, GWAM and ECMWF wave models, and 10 m wind and gusts from the blended, ECMWF IFS, GFS and ICON models. Modeled estimates, not measurements.
- 2.Operator field report, e-mail 2026-09-10 — Reef position and local rules from the operator at the break, quoted in the article. A field observation; the operator is not named.
- 3.Operator field report, e-mail 2026-09-14 — The Sunday report from the operator at the break: how the wave broke, face height and wind, quoted in the article. A field observation; the operator is not named.
- 4.Operator field report, e-mail 2026-09-15 — The answers to the two wind questions: felt at the takeoff rather than instrument-read, and strong through most of the day since mid-July, lighter some early mornings before building. A field observation; the operator is not named.
- 5.AYON Wave spots.ts — Macaronis record (commit ccfbd8d) — The 10 September 2026 edits: reef position -2.783948, 99.972289; swell window 195–225°; offshore wind bearing 135°; verified flag left false.
- 6.AYON Wave spots.ts — Macaronis wind field note — The 16 September 2026 addition: the operator's felt-wind estimate recorded as OBSERVED against the four models' 3–12 kn, as a standing caveat and not a scoring input; verified flag left false.
- 7.AYON Ecosystem website widget — commit 34efd4e — The 14 September 2026 fix: the widget's "Swell from" line reads the swell partition instead of the whole sea, with the whole sea as a fallback only where no partition is published.