2024 Japan Weather Association’s Forecast Accuracy Verification Results

2025.04.16

JWA, guided by its corporate mission of “Creating a society harmonized with the natural world”, promotes efforts by leveraging the “Power of Weather” to address societal challenges and create new value.

High accuracy weather forecasting is the key to creating benefits for people and society with the ‘Power of Weather’.

In 2023, JWA developed the integrated weather forecasting model “JWA Blend of Models*”, providing high-precision, high frequency, and high resolution forecasts for its various weather services.

* For more information about “JWA Blend of Models,” please click here: JWA Blend of Models (only available in Japanese)

To demonstrate the high accuracy of the “JWA Blend of Models,” JWA has been publishing its accuracy verification results since 2023. (2023 JWA’s Forecast Accuracy Verification Results)

In 2024, following the evaluation conducted in 2023, JWA assessed the “accuracy rate of precipitation forecasts on the day”, and introduced the evaluation of the “accuracy rate of hourly weather forecasts for the following day”.

At JWA, we decided to verify the “accuracy rate of hourly weather forecasts for the next day” because we believe that accurate weather forecasts should not only predict whether it will rain at some point during the day at a specific location but also provide precise hourly forecasts, such as “if rain is predicted to start at 2PM, it should actually begin at that time.”

Weather forecasts consist of various elements, and as a result, there are multiple indicators used to measure forecast accuracy. At JWA, we believe that enhancing the accuracy of essential information is crucial, and we remain committed to continuously improving forecast precision.

1. Summary of 2024 Results

• The “JWA Blend of Models” had a higher accuracy rate than the Japan Meteorological Agency (JMA)’s forecasts in both metrics: the “accuracy rate of precipitation forecasts on the day” and the “accuracy rate of hourly weather forecasts for the following day.”
• The “JWA Blend of Models” also exhibited smaller forecast errors and higher accuracy in predicting the highest and lowest temperatures from the following day up to seven days ahead, compared to the JMA’s forecasts.

2. 2024 Verification Results of the Accuracy Rate of Precipitation Forecasts on the Day

2-1. Methodology

• Period: From January to December 2024
• Used Forecasts: Nationwide weather forecasts announced at 5 AM every day
• Method: This analysis focused on whether rain actually occurred on days it was forecasted.
  It compared the “accuracy rate of precipitation occurrence on the day based on the forecast announced at 5 AM” (hereafter referred to as “accuracy rate of precipitation forecasts on the day”) between JMA and JWA predictions.

2-2. Results

“JWA Blend of Models” achieved an accuracy rate of 89% for the “accuracy rate of precipitation forecasts on the day” throughout 2024.

When compared to the JMA’s monthly accuracy rate for the same metric—calculated using the same criteria and publicly available data—the accuracy rate of “JWA Blend of Models” was equal to or higher than JMA’s in 11 out of 12 months, except for August.

The “accuracy rate of precipitation forecasts on the day” evaluates whether total precipitation of 1mm or more occurs between 5 AM and 12 AM on a given day, based on the forecast issued at 5 AM.

Therefore, even if the timing of rainfall at a certain location differs between the forecast and the observation, as long as it rains (or does not rain) at some point during the day, the forecast is considered accurate. This means that this indicator tends to yield higher accuracy results.

* Calculation Method for the “Accuracy Rate of Precipitation Forecasts on the Day”

3. 2024 Verification Results of the Accuracy Rate of Hourly Weather Forecasts for the Following Day

3-1. Methodology

• Period: January to December 2024
• Forecasts Used: Next-day weather forecasts issued at 5:00 PM
• Locations: 141 locations in the representative cities throughout Japan
• Method: This analysis evaluated whether the hourly weather conditions (clear, cloudy, rain, sleet (rain or snow), snow) for the following day were accurately forecasted.
  The forecasts issued at 5PM on the previous day were compared with the actual weather conditions, assessing accuracy between JMA and JWA predictions.

3-2. Results

The “JWA Blend of Models” achieved a 70% accuracy rate for the “hourly weather forecasts for the following day” throughout 2024.

When compared to the JMA’s monthly accuracy rates, the “JWA Blend of Models” outperformed JMA forecasts in every month.

4. Calculation method for the accuracy rate of hourly weather forecasts for the following day

The “accuracy rate of hourly weather forecasts for the following day” is determined by comparing the next-day weather forecasts from the “JWA Blend of Models” and the JMA’s “Three-hourly Forecasts” with the actual weather conditions, based on the weather icons from the JMA’s “Estimated weather distribution.”

The forecasts and actual conditions are compared hourly, accuracy is assessed based on whether the forecasts were correct or incorrect.

4-1. JMA’s Forecast: “Three-hourly Forecasts”

This 5 km mesh and 3-hourly weather forecast, available on the JMA’s website, provides information on weather conditions; temperature, precipitation amount, snowfall amount, and the highest and lowest temperatures until 12:00 AM the following day.

The forecast is issued daily at 5 AM, 11 AM, and 5 PM.

For this verification, comparisons were made on an hourly basis, therefore, the “three-hourly forecasts” were converted into hourly forecasts.

Since the “weather” in the three-hourly forecasts represents the predominant weather condition for that time period, adjustments were made accordingly.
For example, if the forecast for 12 PM predicted rain, then the forecasts for 10 AM, 11 AM, and 12 PM were treated as indicating rain.

4-2. Actual Weather: JMA’s “Estimated weather distribution”

The “Estimated weather distribution” includes the estimated data on actual weather conditions calculated by the JMA based on observation data from AMeDAS and weather satellites.

Weather information is updated hourly and categorized into five types- clear, cloudy, rain, sleet (rain or snow) and snow- on a 1 km mesh grid.

Since AMeDAS, which conducts meteorological observations across Japan, does not record direct weather conditions, this verification used the JMA’s “Estimated weather distribution” as the actual weather data.

4-3. Details for Verification

• Time of forecast issuance: Forecast for the next day issued at 5:00 PM
• Time frame for verification: Hourly weather data from 1:00 AM to 12:00 AM the following day
• Locations for comparison: 141 locations in the representative cities throughout Japan

4-4. Weather Evaluation

The accuracy rate is calculated by comparing the actual weather conditions from the estimated weather distribution with the forecasted weather icons to determine whether the weather forecasts were accurate.

Please note that both the JMA’s estimated weather distribution (used as the actual weather data) and the three-hourly forecasts (used as the JMA’s forecasts) describe weather conditions using five categories: “clear,” “cloudy,” “rain,” “rain or snow,” and “snow.”

Among these, “rain or snow” indicates either rain or snow. However, since this forecast category does not exist in the “JWA Blend of Models”, “sleet” is verified as “rain or snow.”

5. 2024 Forecast Accuracy for the Highest and Lowest Temperatures

5-1. Methodology

• Period: January 1st to December 31st, 2024
• Forecasts Used: Daily highest and lowest temperature forecasts for the following day up to 7 days ahead, based on the data issued at 11:00 AM
• Actual Temperatures Data: JMA’s AMeDAS observations (daily highest and lowest temperatures)
• Locations: 56 meteorological stations
• Method: This analysis focused on Root Mean Squared Error (RMSE), a key indicator that measures the magnitude of errors between predicted and observation values.
  The accuracy of JMA’s and JWA’s forecasts was evaluated by comparing the differences between the predicted and actual highest and lowest temperatures for the following day up to seven days ahead, based on forecasts issued at 11 AM the previous day.

5-2. Results

For all forecast periods from the following day up to seven days ahead, the temperature forecast error (RMSE) of the “JWA Blend of Models” was smaller than that of the JMA’s forecast.

Since RMSE represents the difference between the predicted and actual temperatures, a lower RMSE value indicates a more accurate temperature forecast.

* For more information about JWA’s forecast technology, please click here: TECHNOLOGY

6. Weather Forecast Services by Japan Weather Association

Weather is one of the few areas where future conditions can be predicted using physical models. This predictive ability is highly valuable for business as weather impacts many industries. It’s estimated that one-third of all global industries face weather-related risks*1, highlighting the strong link between weather and business.

Introducing services that leverage JWA’s continuously improving weather forecasts for business applications.

• Weather Data API
  JWA’s Web API provides access to both weather forecasts and historical weather data.
  By obtaining weather data via API, you can make use of it in a variety of ways, such as displaying it on apps and web services, analyzing your own data with weather data, and building forecast models.
• Product Demand Forecasting
  By analyzing highly accurate weather forecast data and product data, JWA predicts future demand.
  With precise product demand forecasting, businesses can take approaches that do not only rely on experience or intuition to increase sales and reduce costs.

For inquiries, contact us.

*1 Dutton, J. A., Bull. Amer. Meteor. Soc., 83, 1303-1311 (2002)