modelnewton module
Created on Fri Jun 19 19:49:50 2020
@author: IBH
Module which handles model differentiation, construction of jacobi matrizex, companion matrices, eigenvalues and creates dense and sparse solving functions.
- class modelnewton.diff_value_base(var: str, pvar: str, lag: int, var_plac: int, pvar_plac: int, pvar_endo: bool, pvar_exo_plac: int)[source]
Bases:
objectclass define columns in database with values from differentiation
- var: str
- pvar: str
- lag: int
- var_plac: int
- pvar_plac: int
- pvar_endo: bool
- pvar_exo_plac: int
- class modelnewton.diff_value_col(var: str, pvar: str, lag: int, var_plac: int, pvar_plac: int, pvar_endo: bool, pvar_exo_plac: int)[source]
Bases:
diff_value_baseThe hash able class which can be used as pandas columns
- class modelnewton.diff_value(var: str, pvar: str, lag: int, var_plac: int, pvar_plac: int, pvar_endo: bool, pvar_exo_plac: int, number: int = 0, date: any = 0)[source]
Bases:
diff_value_baseclass to contain values from differentiation
- number: int = 0
- date: any = 0
- class modelnewton.newton_diff(mmodel, df=None, endovar=None, onlyendocur=False, timeit=False, silent=True, forcenum=False, per='', ljit=0, nchunk=None, endoandexo=False)[source]
Bases:
objectClass to handle Newton solving for un-normalized or normalized models, i.e., models of the form:
0 = G(y,x) y = F(y,x)
This class provides functionalities to differentiate model equations, analyze eigenvalues and eigenvectors, and provide utilities for solving dynamic systems through various approaches.
Provided Functions: - eigenvector_plot: Plot eigenvectors for a specified period. - eigplot: Plot eigenvalues for a specific period in polar coordinates. - eigplot_all: Plot all eigenvalues for specified periods in polar coordinates. - eigplot_all0: Alternative method to plot all eigenvalues in polar coordinates. - get_df_comp_dict: Get a dictionary of DataFrames representing companion matrices. - get_df_eigen_dict: Get a dictionary of DataFrames containing eigenvalues and eigenvectors. - get_diff_df_1per: Get a DataFrame of derivatives for one period. - get_diff_df_tot: Get a DataFrame of stacked Jacobian matrices for the entire period range. - get_diff_mat_1per: Get a dictionary of sparse matrices representing the Jacobian for one period. - get_diff_mat_all_1per: Get a dictionary of all derivative matrices for one period, including all lags. - get_diff_mat_tot: Get a sparse matrix representing the stacked Jacobian for the entire period range. - get_diff_melted: Get a “melted” DataFrame of derivatives suitable for creating sparse matrices. - get_diff_melted_var: Get a melted DataFrame of derivatives including variable information. - get_diff_values_all: Get all derivative values in a structured format. - get_diffmodel: Generate a model which calculates the partial derivatives of the original model. - get_eigen_jackknife: Perform a jackknife analysis by computing eigenvalues with each variable excluded one at a time. - get_eigen_jackknife_abs: Compute the absolute values of the largest eigenvalues from the jackknife analysis. - get_eigen_jackknife_abs_select: Summarize the absolute largest eigenvalues for a specific year from the jackknife analysis. - get_eigen_jackknife_df: Convert jackknife eigenvalue data into a DataFrame. - get_eigenvalues: Calculate and return the eigenvectors based on the companion matrix for a dynamic system. - get_feedback: Static method to return feedback on the max absolute eigenvector and its sign. - get_solve1per: Get a solving function for one period using precomputed Jacobian matrices. - get_solve1perlu: Get a LU decomposition-based solving function for one period. - get_solvestacked: Get a solving function for the stacked system of equations. - get_solvestacked_it: Get an iterative solving function for the stacked system using a specified solver. - modeldiff: Differentiate model equations with respect to endogenous variables. - show_diff: Display expressions for differential coefficients for specified variables. - show_diff_latex: Display LaTeX-formatted differential expressions and possibly their values. - show_stacked_diff: Show selected rows and columns of the stacked Jacobian as a DataFrame.
- Parameters:
mmodel (TYPE) – Model to analyze.
df (TYPE, optional) – Dataframe. if None mmodel.lastdf will be used
endovar (TYPE, optional) – if set defines which endogeneous to include . Defaults to None.
onlyendocur (TYPE, optional) – Only calculate for the curren endogeneous variables. Defaults to False.
timeit (TYPE, optional) – writeout time informations . Defaults to False.
silent (TYPE, optional) – Defaults to True.
forcenum (TYPE, optional) – Force differentiation to be numeric else try sumbolic (slower) Defaults to False.
per (TYPE, optional) – Period for which to calculte the jacobi . Defaults to ‘’.
ljit (TYPE, optional) – Trigger just in time compilation of the differential coiefficient. Defaults to 0.
nchunk (TYPE, optional) – Chunks for which the model is written - relevant if ljit == True. Defaults to None.
endoandexo (TYPE, optional) – Calculate for both endogeneous and exogeneous . Defaults to False.
- Returns:
None.
- modeldiff()[source]
Differentiate relations for self.enovar with respect to endogeneous variable The result is placed in a dictory in the model instanse: model.diffendocur
- show_diff(pat='*')[source]
Displays espressions for differential koifficients for a variable if var ends with * all matchning variables are displayes
- show_stacked_diff(time=None, lhs='', rhs='', dec=2, show=True)[source]
- Parameters:
time (list, optional) – DESCRIPTION. The default is None. Time for which to retrieve stacked jacobi
lhs (string, optional) – DESCRIPTION. The default is ‘’. Left hand side variables
rhs (TYPE, optional) – DESCRIPTION. The default is ‘’. Right hand side variabnles
dec (TYPE, optional) – DESCRIPTION. The default is 2.
show (TYPE, optional) – DESCRIPTION. The default is True.
- Return type:
selected rows and columns of stacked jacobi as dataframe .
- get_diff_melted(periode=None, df=None)[source]
returns a tall matrix with all values to construct jacobimatrix(es)
- get_diff_mat_tot(df=None)[source]
Fetch a stacked jacobimatrix for the whole model.current_per
Returns a sparse matrix.
- get_diff_melted_var(periode=None, df=None)[source]
makes dict with all derivative matrices for all lags
- get_diff_values_all(periode=None, df=None, asdf=False)[source]
stuff the values of derivatives into nested dic
- get_eigenvalues(periode=None, asdf=True, filnan=False, silent=False, dropvar=None, dropvar_nr=0, progressbar=False)[source]
Calculate and return the eigenvectors based on the companion matrix for a dynamic system.
This method computes the eigenvectors of a dynamic system represented by a companion matrix derived from Jacobian matrices for different periods. The computation involves handling missing values, optionally filling NaN values with zero, and the ability to drop specific variables from the calculation.
Parameters: - periode (optional): The period for which the eigenvectors are to be calculated. If None, defaults to the entire range. - asdf (bool, optional): Determines the format of the matrices (DataFrame or sparse matrix). Defaults to True (DataFrame). - filnan (bool, optional): If True, fills NaN values in the Jacobian matrices with zero. Defaults to False. - silent (bool, optional): If False, prints detailed information about NaN values and other relevant details during the computation. Defaults to False. - dropvar (optional): Specifies variables to be dropped from the calculation. If None, no variables are dropped. Defaults to None. - dropvar_nr (int, optional): The number of variables to drop. Defaults to 0.
Returns: dict: A dictionary with keys as dates and values as eigenvectors for each period, derived from the companion matrix of the system.
The function performs several steps: - Computes the Jacobian matrices for the given period. - Handles NaN values based on the ‘filnan’ parameter. - Optionally drops specified variables from the calculation. - Constructs the companion matrix from the modified Jacobian matrices. - Calculates the eigenvectors from the companion matrix.
Note: The companion matrix is crucial in analyzing the stability and dynamics of the system. The eigenvectors provide insights into the system’s behavior over time.
- get_eigen_jackknife(maxnames=200000, periode=None, progressbar=True)[source]
Compute and cache eigenvalues for matrices with each variable excluded one at a time, up to a maximum number.
This function uses the jackknife technique to evaluate the impact of each variable on the system’s stability by excluding each variable one by one from the eigenvector calculation. It caches the results for efficient repeated access.
Parameters: - maxnames (int, optional): The maximum number of variables to consider for exclusion in the jackknife process. Defaults to 20.
Returns: dict: A dictionary where keys are the names of variables excluded (or ‘ALL’ for no exclusion) and values are the corresponding eigenvectors.
Note: The function is computationally intensive and can take significant time for larger systems.
- get_eigen_jackknife_df(maxnames=200000, progressbar=True, periode=None, filecache=True, refresh=False)[source]
Convert the eigenvalue data obtained from a jackknife analysis into a pandas DataFrame, including additional columns for the absolute length, real, and imaginary parts of the eigenvalues.
The jackknife analysis is performed by computing and caching eigenvalues for matrices with each variable excluded one at a time, up to a specified maximum number. This method uses the jackknife technique to evaluate the impact of each variable on the system’s stability by excluding each variable one by one from the eigenvector calculation. The results are then flattened and transformed into a DataFrame for further analysis.
Parameters: - maxnames (int, optional): The maximum number of variables to consider for exclusion in the jackknife process. Defaults to 200,000. - progressbar (bool, optional): If True, displays a progress bar during the computation of eigenvalues. Defaults to True.
Returns: pandas.DataFrame: A DataFrame containing the eigenvalues with additional columns for length, real, and imaginary parts. Each row represents an eigenvalue for a specific variable exclusion, year, and index.
Note: - The function is computationally intensive and can take significant time for larger systems. - A progress bar can be displayed for monitoring the computation progress. - The function is especially useful for detailed analysis and visualization of the eigenvalues obtained from the jackknife analysis.
- get_eigen_jackknife_abs(largest=20, maxnames=200000)[source]
Compute the absolute values of the largest eigenvalues from the jackknife eigenvalue analysis.
This function calculates the absolute values of the largest eigenvalues for each set of eigenvalues obtained from the get_eigen_jackknife method. It focuses on the largest eigenvalues to understand the most significant influences on the system’s stability.
Parameters: - largest (int, optional): The number of largest eigenvalues to consider. Defaults to 20. - maxnames (int, optional): The maximum number of variables to exclude in the jackknife process. Defaults to 20.
Returns: dict: A dictionary with the absolute values of the largest eigenvalues for each variable exclusion scenario.
Note: This method helps in identifying the most impactful variables on the system’s stability by focusing on the largest eigenvalues.
- static jack_largest_reduction(jackdf, eigenvalue_row=0, periode=None, imag_only=False)[source]
Identifies the largest reduction in eigenvalue magnitude for a specified period and optionally focuses on eigenvalues with imaginary parts if imag_only is True.
Parameters: - jackdf (DataFrame): A DataFrame containing jackknife analysis results,
including eigenvalues, their real and imaginary parts, and descriptions of exclusions.
eigenvalue_row (int, optional): The row index of the eigenvalue to analyze. Defaults to 0, which typically represents the largest magnitude eigenvalue.
periode (int/str, optional): The specific period (year) to analyze. If None, the function processes the first year found in the DataFrame. Defaults to None.
imag_only (bool, optional): If True, only considers eigenvalues with non-zero imaginary parts for analysis. Defaults to False.
Returns: DataFrame: A sorted DataFrame with the nth largest length (eigenvalue magnitude) for each excluded variable or condition, including the year, exclusion identifier, length (magnitude of the eigenvalue), description of the exclusion, and the real and imaginary parts of the eigenvalue. The row for ‘excluded == “NONE”’ is moved to the front.
Raises: Exception: If the specified period is not found in the DataFrame’s years.
- jack_largest_reduction_plot(jackdf, eigenvalue_row=0, periode=None, imag_only=False)[source]
Creates an interactive Plotly plot to visualize the reduction in eigenvalue magnitude across different exclusions, highlighting the ‘NONE’ exclusion category with a distinct color and displaying detailed information on hover. Optionally focuses on eigenvalues with imaginary parts if imag_only is True.
Parameters: - jackdf (DataFrame): A DataFrame containing the results of a jackknife analysis, including eigenvalues and their
descriptions. The DataFrame is expected to have at least the columns ‘excluded’, ‘length’, ‘excluded_description’, ‘realvalue’, and ‘imagvalue’.
eigenvalue_row (int, optional): Specifies the row index of the eigenvalue to analyze. Defaults to 0, which typically corresponds to the largest magnitude eigenvalue.
periode (int/str, optional): The specific period (year) to analyze. If None, the function processes data without filtering by period. Defaults to None.
imag_only (bool, optional): If True, only considers eigenvalues with non-zero imaginary parts for analysis. Defaults to False.
The function processes the input DataFrame to highlight the ‘NONE’ category in red and all other categories in blue. It then creates a Plotly FigureWidget to plot these data points as markers on a scatter plot. The y-axis tick labels are hidden to emphasize the data points rather than the categorical labels.
An HTML widget is used to display detailed information about a data point (exclusion description, length, and imaginary part of the eigenvalue) when the user hovers over it. This interactive feature provides a deeper insight into the impact of each exclusion on the eigenvalue magnitude.
Displays: - An interactive Plotly scatter plot within the Jupyter notebook. - A dynamic HTML widget that updates with detailed information about the hovered data point.
- get_eigen_jackknife_abs_select(year=2023, largest=20, maxnames=200000)[source]
Select and summarize the absolute largest eigenvalues for a specific year from the jackknife analysis.
This function focuses on a specific year and extracts the sum of the absolute largest eigenvalues obtained from the get_eigen_jackknife_abs method. It helps in understanding the aggregate impact of variable exclusions on the system’s stability for a particular year.
Parameters: - year (int, optional): The specific year to focus on. Defaults to 2023. - largest (int, optional): The number of largest eigenvalues to consider. Defaults to 20. - maxnames (int, optional): The maximum number of variables to exclude in the jackknife process. Defaults to 20.
Returns: pandas.Series: A series sorted by the sum of the absolute largest eigenvalues for each variable exclusion scenario in the specified year.
Note: This method is useful for temporal analysis of the system’s stability, focusing on the contributions of each variable in a specific year.
- eigenvalues_show(eig_dic)[source]
Generates and displays a polar plot of eigenvalues and their corresponding eigenvectors for a selected year from a dictionary of DataFrames. Each DataFrame contains eigenvalues and eigenvectors of the companion matrix for that year. The first row of the DataFrame consists of eigenvalues, and the subsequent rows contain the corresponding eigenvectors. The user can interact with the plot via a dropdown for year selection, a slider for eigenvalue selection, and a button to toggle additional plot details. The polar plot dynamically updates to reflect the selected eigenvalue, displaying its magnitude and phase. Additional information about the selected eigenvector is also displayed, facilitating a detailed temporal analysis of the eigenvalues.
Parameters: - eig_dic (dict): A dictionary where keys are years (or time periods) and values are DataFrames
containing the first row as eigenvalues and the subsequent rows as the corresponding eigenvectors of the companion matrix for each year.
Side Effects: - Displays interactive widgets including a dropdown for year selection, a plot output area,
and a slider for selecting specific eigenvalues. Additionally, displays textual information about the selected eigenvalue and its eigenvectors.
Utilizes Plotly for generating the polar plot and ipywidgets for interactive controls.
The method defines and uses several inner functions to handle events like year change, eigenvalue selection, and other interactions.
Returns: - None. The method’s primary function is to display interactive widgets and plots within a Jupyter
notebook environment.
Note: - This method is designed for use within a Jupyter notebook as it relies on IPython.display
for rendering and ipywidgets for interactivity.
The actual plotting and widget setup are accomplished through several nested functions within this method, making use of closures and nonlocal variables for state management.