Discretizing time to improve econometric analysis

Processing a new variable to distinguish the periods of the 2008 financial crisis improves the statistical relationship between Mortgage Rates (MR) and Inflation (CPI).

Scatterplot depicting CPI and MR relationships by phase

If we hadn’t distinguished the periods, we would have to conclude that there is no relationship between MR and CPI.

Weekly relationship between CPI and MR

The relationship depicts a pattern more similar to a dragon than any kind of linear relationship.

However, creating a new feature is not the only transformation we have applied to the data.

Questions

  1. Why does resampling the time from weeks to months improve the correlation analysis?
  2. How much does the \(R^2\) score improve after adding the period feature?
  3. Why did the MR decrease when the CPI increased before the crisis?

Data

For each week between 2007-06-01 and 2009-06-01, we have the Consumer Index Price (CPI), also known as the Inflation, and the Mortgage Rates (MR).

Download the dataset and practice with this tutorial.

import pandas as pd

df = pd.read_excel('CPI_MR_2008.xlsx', index_col=0)
df

Weekly CPI and MR

The goal is to measure the influence of the CPI on the MR.

In other words, how much will the MR increase if the CPI increases by 1%?

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to datons.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.