Amount of Hot Days and Cold Nights

Figure. Annual Number of hot days and cold nights at Koror. Hot days are defined as days above the 90th percentile for that same calendar day (e.g., January 15th) from the 1960–1990 period, while cold nights are defined as days below the 10th percentile for that same calendar day in the 1960–1990 period. The solid black lines represent statistically significant trends (p < 0.05).

Setup

First, we need to import all the necessary libraries. Some of them are specifically developed to handle the download and plotting of the data and are hosted at the indicators set-up repository in GitHub

Show code cell source Hide code cell source

import warnings
warnings.filterwarnings("ignore")

import os
import os.path as op
import sys

from myst_nb import glue 

import numpy as np
import pandas as pd
from datetime import datetime

sys.path.append("../../../../indicators_setup")
from ind_setup.plotting_int import plot_timeseries_interactive, fig_int_to_glue

sys.path.append("../../../functions")
from data_downloaders import GHCN, filter_by_time_completeness
from temp_func import exceedance_rate_for_base_period, exceedance_rate_for_outbase_period

Define location and variables of interest

country = 'Palau'
update_data = False
path_data = "../../../data"
path_figs = "../../../matrix_cc/figures"

Observations from Koror Station

The data used for this analysis comes from the GHCN (Global Historical Climatology Network)-Daily database.
This a database that addresses the critical need for historical daily temperature, precipitation, and snow records over global land areas. GHCN-Daily is a composite of climate records from numerous sources that were merged and then subjected to a suite of quality assurance reviews. The archive includes over 40 meteorological elements including temperature daily maximum/minimum, temperature at observation time, precipitation and more.

https://www.ncei.noaa.gov/data/global-historical-climatology-network-daily/doc/GHCND_documentation.pdf

Show code cell source Hide code cell source

if update_data:
    df_country = GHCN.get_country_code(country)
    print(f'The GHCN code for {country} is {df_country["Code"].values[0]}')

    df_stations = GHCN.download_stations_info()
    df_country_stations = df_stations[df_stations['ID'].str.startswith(df_country.Code.values[0])]
    print(f'There are {df_country_stations.shape[0]} stations in {country}')

Show code cell source Hide code cell source

if update_data:
    GHCND_dir = 'https://www.ncei.noaa.gov/data/global-historical-climatology-network-daily/access/'
    id = 'PSW00040309' # Koror Station
    dict_min = GHCN.extract_dict_data_var(GHCND_dir, 'TMIN', df_country_stations.loc[df_country_stations['ID'] == id])[0][0]
    dict_max = GHCN.extract_dict_data_var(GHCND_dir, 'TMAX', df_country_stations.loc[df_country_stations['ID'] == id])[0][0]
    st_data = pd.concat([dict_min['data'], (dict_max['data'])], axis=1).dropna()
    st_data['DATE'] = st_data.index
    st_data['DAY'] = "2024-" + st_data['DATE'].dt.strftime('%m-%d')
    st_data['DAY'] = pd.to_datetime(st_data['DAY'], format='%Y-%m-%d')
    st_data.index = range(len(st_data))
    st_data.to_pickle(op.join(path_data, 'GHCN_surface_temperature_hotdays.pkl'))
else:
    st_data = pd.read_pickle(op.join(path_data, 'GHCN_surface_temperature_hotdays.pkl'))

st_data.index = pd.to_datetime(st_data.DATE)

# Inclusion criteria: remove months/years with less than 75% of data

df_filt, removed_months, removed_years = filter_by_time_completeness(
    st_data,
    month_threshold=0.75,
    year_threshold=0.75
)
print(f"Removed {removed_months.shape[0]} months due to insufficient data: {removed_months.index.tolist()}")
print(f"Removed {removed_years.shape[0]} years due to insufficient data: {removed_years.index.tolist()}")

st_data = df_filt #replace data with filtered data
st_data_daily = st_data.copy()

Removed 14 months due to insufficient data: [(2018, 8), (2019, 1), (2020, 3), (2022, 7), (2022, 10), (2022, 11), (2022, 12), (2023, 1), (2023, 6), (2023, 8), (2023, 12), (2024, 2), (2024, 12), (2025, 1)]
Removed 4 years due to insufficient data: [2019, 2022, 2023, 2025]

Analysis

exceed_rates_TMAX = exceedance_rate_for_outbase_period(st_data, "TMAX")
exceed_rates_TMIN = exceedance_rate_for_outbase_period(st_data, "TMIN")

TMAX_dict = dict(zip(exceed_rates_TMAX['DAY'], exceed_rates_TMAX['THRESHOLD']))
TMIN_dict = dict(zip(exceed_rates_TMIN['DAY'], exceed_rates_TMIN['THRESHOLD']))

df_exceed = st_data.copy()
df_exceed['THRESHOLD_TMAX'] = df_exceed['DAY'].apply(lambda day_value:TMAX_dict.get(day_value))
df_exceed['HOT_DAY'] = df_exceed[['TMAX',"THRESHOLD_TMAX"]].apply(lambda x: x["TMAX"] > x["THRESHOLD_TMAX"],axis=1)

df_exceed['THRESHOLD_TMIN'] = df_exceed['DAY'].apply(lambda day_value:TMIN_dict.get(day_value))
df_exceed['COLD_NIGHT'] = df_exceed[['TMIN',"THRESHOLD_TMIN"]].apply(lambda x: x["TMIN"] < x["THRESHOLD_TMIN"],axis=1)

df_exceed['YEAR'] = pd.DatetimeIndex(st_data['DATE']).year

out_of_base_hot = {}
out_of_base_cold = {}
for x in df_exceed["YEAR"].unique():
    if x > 1990:
        out_of_base_hot[x] = df_exceed[df_exceed["YEAR"] == x]['HOT_DAY'].mean()
        out_of_base_cold[x] = df_exceed[df_exceed["YEAR"] == x]['COLD_NIGHT'].mean()

Here we are generating the count of hoy days and cold nights. A day is measured as a hot day (cold night) if it is over (below) the 90th (10th) percentile for that same day in the period 1960-1990.

ex_cold, all_cold = exceedance_rate_for_base_period(st_data, "TMIN")
ex_hot, all_hot = exceedance_rate_for_base_period(st_data, "TMAX")
all_hot = ex_hot|out_of_base_hot
all_cold = ex_cold|out_of_base_cold
cold_bar = sum(ex_cold.values()) / len(ex_cold)
hot_bar = sum(ex_hot.values()) / len(ex_hot)

hot_anom = {}

for x in all_hot:
    hot_anom[x] = 100*(all_hot[x]-hot_bar)

cold_anom = {}
for x in all_cold:
    cold_anom[x] = 100*(all_cold[x]-cold_bar)

df_cold_anom = pd.DataFrame.from_dict(cold_anom, orient='index', columns=['Perc_Anom'])
df_cold_anom.index = pd.to_datetime(df_cold_anom.index, format='%Y')

df_hot_anom = pd.DataFrame.from_dict(hot_anom, orient='index', columns=['Perc_Anom'])
df_hot_anom.index = pd.to_datetime(df_hot_anom.index, format='%Y')

Plotting

Cold Nights

dict_plot = [{'data' : df_cold_anom*3.6525, 'var' : 'Perc_Anom', 'ax' : 1, 'label' : 'Cold Nights'},]
fig, TRENDS = plot_timeseries_interactive(dict_plot, trendline=True, figsize = (25, 12), return_trend = True, label_yaxes = 'Number of days/year')

Hot Days

dict_plot = [{'data' : df_hot_anom*3.6525, 'var' : 'Perc_Anom', 'ax' : 1, 'label' : 'Hot Days', 'color':'red'}]
fig, TRENDS = plot_timeseries_interactive(dict_plot, trendline=True, figsize = (25, 12), return_trend = True, label_yaxes = 'Number of days/year')

Cold Nights and Hot Days

The following plot shows how many days a year the temperature is over (below) the 90th (10th) percentile

dict_plot = [{'data' : df_cold_anom*3.6525, 'var' : 'Perc_Anom', 'ax' : 1, 'label' : 'Cold Nights'},
             {'data' : df_hot_anom*3.6525, 'var' : 'Perc_Anom', 'ax' : 1, 'label' : 'Hot Days'}]
fig, TRENDS = plot_timeseries_interactive(dict_plot, trendline=True, figsize = (25, 12), return_trend = True, label_yaxes = 'Number of days/year')
fig.write_html(op.join(path_figs, 'F4_ST_hot_cold.html'), include_plotlyjs="cdn")

glue("trend_cold_decade", float(TRENDS[0]*10), display=False)
glue("trend_hot_decade", float(TRENDS[1]*10), display=False)
glue("trend_fig", fig_int_to_glue(fig), display=False)

Fig. Number of hot days and cold nights relative to 1961–1990 climatology at Koror. Hot days are defined as days above …10 and 90 , which corresponds to 32°C (90°F) nights. Cold nights are defined as days below 23.5°C/74°F. The solid black lines represent trends, which are statistically significant (p < 0.​​05).

The plot below shows the same information but measured as the % of time

dict_plot = [{'data' : df_cold_anom, 'var' : 'Perc_Anom', 'ax' : 1, 'label' : 'Cold Nights'},
             {'data' : df_hot_anom, 'var' : 'Perc_Anom', 'ax' : 1, 'label' : 'Hot Days'}]
fig, _ = plot_timeseries_interactive(dict_plot, trendline=True, figsize = (25, 12), return_trend = True, label_yaxes = '% of days/year')

annual_cold = df_cold_anom*3.6525
annual_hot = df_hot_anom*3.6525
annual_cold['Perc_Anom'] = np.where(annual_cold['Perc_Anom'] > 0, annual_cold['Perc_Anom'], annual_cold['Perc_Anom'])
annual_hot['Perc_Anom'] = np.where(annual_hot['Perc_Anom'] > 0, annual_hot['Perc_Anom'], annual_hot['Perc_Anom'])

Table

The final step is to generate a table summarizing different metrics of the data analyzed in the plots above

from ind_setup.tables import style_matrix, table_temp_13, table_temp_13b

style_matrix(table_temp_13(st_data, annual_hot, annual_cold, df_hot_anom, df_cold_anom, TRENDS))

Key Metrics Summary

Metric	Value	Year
Daily Maximum Temperature (°C)	35.000
Daily Minimum Temperature (°C)	20.600
Average number of hot days	26.199
Change in Average Annual Number of Hot Days	1.221
Average Annual Number of Hot days: 1961-1971	-9.379
Average Annual Number of Hot days: 2001-2011	67.099
Average Annual Number of Hot days: 2011-2021	32.951
Maximum number of hot days	162.000	1998
Minimum number of hot days	-17.000	2021
Average number of cold nights	-1.912
Change in Average Annual Number of Cold Nights	-0.258
Average Annual Number of Cold Nights: 1961-1971	0.868
Average Annual Number of Cold Nights: 2001-2011	-7.947
Average Annual Number of Cold Nights: 2011-2021	-15.775
Maximum number of cold nights	40.000	2000
Minimum number of cold nights	-20.000	2020

Hot days and cold nights

Now defined as the number of days over and above the 90th and 10th percentile of the maximum and minimum daily temperatures respectively

q90 = st_data.loc['1961':'1991'].TMAX.quantile(0.9)
q10 = st_data.loc['1961':'1991'].TMIN.quantile(0.1)
print('The 10th percentile of TMIN in the period 1961-1991 is:', q10)
print('The 90th percentile of TMAX in the period 1961-1991 is:', q90)

The 10th percentile of TMIN in the period 1961-1991 is: 23.3
The 90th percentile of TMAX in the period 1961-1991 is: 32.2

st_data_min = st_data[['TMIN']]
st_data_min = st_data_min.loc[st_data_min['TMIN'] < q10]
st_min_counts = st_data_min.groupby(st_data_min.index.year).count()
st_min_counts.index = pd.to_datetime(st_min_counts.index, format='%Y')
print(f'The average number of cold nights per year in the period 1961-1991 is: {st_min_counts.loc["1961":"1991"].mean().values[0]:.0f}')

st_data_max = st_data[['TMAX']]
st_data_max = st_data_max.loc[st_data_max['TMAX'] > q90]
st_max_counts = st_data_max.groupby(st_data_max.index.year).count()
st_max_counts.index = pd.to_datetime(st_max_counts.index, format='%Y')
print(f'The average number of hot days per year in the period 1961-1991 is: {st_max_counts.loc["1961":"1991"].mean().values[0]:.0f}')

The average number of cold nights per year in the period 1961-1991 is: 32
The average number of hot days per year in the period 1961-1991 is: 17

dict_plot = [{'data' : st_min_counts, 'var' : 'TMIN', 'ax' : 1, 'label' : 'Cold Nights below 10th percentile'},
             {'data' : st_max_counts, 'var' : 'TMAX', 'ax' : 1, 'label' : 'Hot Days over 90th percentile'}]
fig, TRENDS_prctiles = plot_timeseries_interactive(dict_plot, trendline=True, figsize = (25, 12), return_trend = True, label_yaxes = 'Number of days/year')
fig.write_html(op.join(path_figs, 'F4_ST_hot_cold_percentiles.html'), include_plotlyjs="cdn")

style_matrix(table_temp_13b(st_data, st_max_counts, st_min_counts, TRENDS_prctiles))

Key Metrics Summary

Metric	Value	Year
Daily Maximum Temperature (°C)	35.000
Daily Minimum Temperature (°C)	20.600
Average number of hot days	36.250
Change in Average Annual Number of Hot Days	1.125
Average Annual Number of Hot days: 1961-1971	8.600
Average Annual Number of Hot days: 2001-2011	72.727
Average Annual Number of Hot days: 2011-2021	46.778
Maximum number of hot days	164.000	1998
Minimum number of hot days	1.000	1956
Average number of cold nights	29.232
Change in Average Annual Number of Cold Nights	-0.407
Average Annual Number of Cold Nights: 1961-1971	31.545
Average Annual Number of Cold Nights: 2001-2011	15.182
Average Annual Number of Cold Nights: 2011-2021	7.000
Maximum number of cold nights	70.000	2000
Minimum number of cold nights	2.000	2018