library("rjd3x13")
mod <- x13(my_series)
mod$result$finalPart 1: Seasonal Adjustment and Production
The Statistical Process
Seasonal adjustment removes periodic movements from the data series that mask cyclical variations and long-term trends. For this reason, most economic indicators are published seasonally (and calendar) adjusted (SA). Seasonal adjustment requires estimating a seasonal component that captures all periodic intra-annual movements in order to remove it from the raw series \(Y_\text{CVS} = Y-S\). To do this, the raw series is broken down into three unobservable components: seasonality, trend, and irregularity (\(Y=S+T+I\)), where the trend \(T\) represents long-term movements and \(I\) represents noise generated by exceptional events (strikes, weather conditions, short-term shocks, etc.) and all measurement errors. The algorithms most commonly used today by European national statistical institutes and central banks are Tramo-Seats (Gomez and Maravall 1996) and X13-Arima (US-Census-Bureau 2015). The latter is used (almost) exclusively by the French public statistics service. These methods proceed in two stages: the raw series is not directly decomposed but first linearized, by (permanently) removing calendar effects and (temporarily) removing outliers using Reg-Arima modelling.
Objectives
The main objective is, of course, to produce seasonaly adjusted (SA) series, particularly with a view to publication. However, to ensure the quality of the seasonal adjustment, a broader output is necessary. The process must therefore allow the diagnostics to be viewed (seasonality tests, residual seasonality tests, and residual calendar effects for the seasonally adjusted series, etc.) and the parameters to be adjusted in light of the problems identified by these diagnostics. Intermediate series, in addition to the final SA series, such as unobservable components, the linearized series, and seasonal factors (including forecast factors), are often useful.
Steps
Seasonal adjustment begins with a seasonality test to ensure that this operation is necessary for the series in question. If this is not the case, the raw series will also be published as SA. In practice, this test is often performed at the end of the seasonal adjustment process, which allows it to be performed on the linearized series (from which the \(S\) factors will be extracted) and to always keep the same input data set, excluding only the series that are not to be seasonally adjusted at the dissemination stage.
A correction for calendar effects must often be implemented or at least tested . The first estimation (pre-adjustment and decomposition) can then be made, probably with a largely automatic specification, and a quality report can be generated to improve the quality of the seasonal adjustment. We refer here to a quality report and not simply to diagnostics, as the former consists of formatting the diagnostics and summarizing them in a score that allows a βselective editingβ approach to prioritize manual expertise in the series. It is generally necessary to quantify the revisions and the numerical impact of the choice of parameters, which will influence the choice of final parameters. Once these elements have been reviewed, a second estimation can be made, and the process may be repeated until a final set of parameters is reached.
Table 1: The steps
| Domain | Operations |
|---|---|
| Correction of calendar effects | Creation of a reference calendar |
| Generation of regression sets | |
| Selection of the best set for each series | |
| Customization of specifications | Assignment of calendar regressors |
| Assigning outliers | |
| Other parameters | |
| Estimation | Preliminary estimation |
| Output generation | |
| Quality Control | Quality Report |
| Manual fine tuning | |
| Final estimation | |
| Final output generation |
The production process
Properties
Production inherently requires the automation of as many operations as possible (definition of specifications, estimation, production of a quality report, application of revision policies), but also, in the case of seasonality adjustment, the visualization of diagnostics and graphs in an environment that allows for rapid manual adjustment of parameters.
The process can be divided into three distinct phases based on their objectives and the type of intervention they require: implementation of a production process, annual campaigns, and infra-annual campaigns.
Set-up
The process must be initialized by making medium-term choices that will not, in principle, be called into question for a period of approximately five years. User requirements are then specified: the variables of interest to be seasonally adjusted are selected, as well as the level of the nomenclature and the direct (seasonal adjustment at each level) or indirect (aggregation of seasonally adjusted series) approach. See Guidelines chapter 5.4 for more details. The relevance of a calendar effect correction must be tested. This is also the time to determine the length of the estimation period (Guidelines, Chapter 8) and the depth of revisions during publication (Guidelines, Chapter 6). This is also when the algorithm and software to be used should be chosen.
Annual and infra-annual reviews
Approximately once a year, the producer updates the statistical parameters based on new raw data (Guidelines, Chapter 6). One method is to compare current models with an automatic re-estimation, as described in Part 3.
During infra-annual production, the goal is to minimize technical revisions. It is recommended that you review only a limited number of parameters, and you may even use the seasonal coefficients predicted during the last annual campaign. The focus should be on revisions and on the appearance or disappearance of outliers in the recent period.
(References: revision policies, controlled current (ref): Part 3)
The rest of this article presents a procedure and the tools needed to implement the three phases described above. As part of harmonizing statistical practices and tools, Eurostat has recommended JDemetra+ to member countries of the European Statistical System for seasonal adjustment since 2015. We will therefore use this software. The following section briefly presents its main features relating to the production of seasonally adjusted series.
JDemetra+: a tool for seasonal adjustment
JDemetra+ is an open-source library of algorithms for time series analysis. These are accessible via a graphical user interface (GUI), which can be enhanced with [plug-ins π] (https://jdemetra-new-documentation.netlify.app/t-plug-ins) and an ecosystem of R packages, the rjdverse π. In addition to seasonal adjustment, they also enable nowcasting, benchmarking, temporal disaggregation, trend estimation, and revision analysis. The development and promotion of JDemetra+ are co-financed by Eurostat through successive grants and currently within the [COSA π]](https://jdemetra-new-documentation.netlify.app/#COSA-abstract) project.
Seasonal adjustment is the area for which JDemetra+ is best known. The software was initially developed as a simple graphical interface, based on the original programs for the two algorithms X12-arima and Tramo-Seats. The aim was to provide users with a user-friendly view of the results, diagnostics, and parameters, and to harmonize this for both algorithms. The software was initially called [Demetra π]](https://github.com/cosa-tsa-shop/Resources-Logistics/tree/main/Historical%20Documents), the goddess of agriculture and the seasons. When the Fortran programs of the US Census Bureau and the Bank of Spain were rewritten in Java, Demetra became JDemetra+ (Grudkowska et al. 2013).
Algorithms
JDemetra+ now offers additional algorithms for seasonal adjustment, but only the two historical ones are really designed for mass data production, as they allow parameters to be stored in a workspace and use an automation module called Cruncher. In our examples, we will refer to X13-Arima, but all recommendations remain valid with Tramo-Seats.
Table 2: Seasonal adjustment algorithms
| Algorithm | GUI | Packages R | Suitable for production |
|---|---|---|---|
| X13-ARIMA | β | {rjd3x13} | yes |
| Tramo-Seats | β | {rjd3tramoseats} | yes |
| X12+ | β | {rjd3x11plus} | not yet |
| STL | β | {rjd3stl} | not yet |
| Basic Structural Models | β | {rjd3sts} | not yet |
All of the above algorithms can be used to seasonally adjust monthly, quarterly, bimonthly, four-monthly, and biannual data. They also have extensions for high-frequency (infra-monthly) data, as detailed in Webel and Smyk (2024).
Tools
Graphical User Interface (GUI)
JDemetra+ is available as a [Graphical User Interface π]](https://jdemetra-new-documentation.netlify.app/t-gui-overview) designed to facilitate manual fine-tuning through an organized presentation of results and parameters. Operations are mainly performed by clicking buttons, which requires auxiliary automation tools. Using the GUI and the Cruncher also requires organizing data into a specific structure: the workspace π.
Cruncher
The Cruncher avoids opening the interface and clicking when launching or refreshing an estimation. It is a command line executable module that we will use later with two R packages, rjwsacruncher for estimations and exporting results, and JDCruncher for formatting diagnostics. Another advantage of the Cruncher is its speed: it can be run on hundreds or thousands of series and produce results in a reasonable amount of time. The installation procedure and main functions are described in the documentation π.
R Packages
A family of packages, rjdverse π provides direct access to JDemetra+ algorithms in R and automates operations related to the use of workspaces. The rjd3production package offers macro functions corresponding to complete stages in the production process.
Table 3: R packages for the production of seasonally adjusted series
| Package | Useful functions for production |
|---|---|
| {rjd3toolkit} π | Generation of calendar regressors, Customization of specifications, Tests |
| {rjd3x13} π | Seasonal adjustment (Reg-Arima pre-adjustment and X11 decomposition) |
| {rjd3tramoseats} π | Seasonal adjustment (TRAMO pre-adjustment and SEATS decomposition) |
| {rjwsacruncher} π | Workspace update and output export |
| {JDCruncheR} π | Create a quality report for the workspace based on the outputs |
| {rjd3production} π | Wrapper with macro functions for each production step |
| {rjd3workspace} π | Workspace creation and modification |
| {rjd3providers} π | Input data management |
The installation procedures for all those tools are to be found in the documentation π.
Data structures
In the following paragraphs, we briefly describe the data structures and concepts specific to JDemetra+, which will be useful in the following sections.
Input series
The JDemetra+ interface accepts different types of data formats that can be imported from the menus in the providers tab.
- CSV files (specifying the field separator, date format, decimal separator, etc.)
- Excel files (only *.xlsx in version 3.x)
- XML files
- SDMX data sources
- ODBC or JDBC databases
All series, whether seasonally adjusted data or auxiliary variables, are imported in the same way. For estimations, R packages work with TS class objects.
Outputs
JDemetra+ generates different types of outputs:
- Series (seasonally adjusted, seasonal factors, linearized series)
- Parameters (specification information)
- Diagnostics (information on the quality of the fit)
The graphical interface allows you to view these outputs and export them using drop-down menus π.
The Cruncher avoids menu navigation by offering an identical automated export. The parameters and diagnostics are grouped into a single data file where the identifier is the name of the series. The exported file is called demetra_m.CVS, where βmβ stands for matrix.
If the estimation is done in R, for example with {rjd3x13}, a list of lists groups the series, diagnostics, and parameters.
Specifications
A specification is a set of parameters and constraints specific to X-13 or Tramo-Seats that enable a series to be seasonally adjusted. Each algorithm has a set of default specifications π that the user can customize. Thus, a specification will be attached to each series, stored in the workspace or in the R object.
More specifically, version 3 of JDemetra+ allows to manipulate three versions of the same initial specification: domainSpec (or reference spec in the GUI), estimationSpec, and pointSpec. The distinction between these three stages becomes useful when applying an estimation or refresh policy with the GUI or Cruncher during annual or infra-annual campaigns.
domainSpec: set of constraints within which the estimation will be performed; if the user sets parameters here, they will never be deleted.estimationSpec: contains all the parameter choices resulting from the estimation, for example:pointSpec: complete result of the estimation (coefficients, length and weight of filters, etc.) allowing the estimated series to be retrieved from the raw series.
The display and modification by the user of these three types of specifications, as well as an example of use cases, are presented in the appendices π.
This definition and the behavior of refresh policies is different from JDemetra+ version 2. Details are provided in the appendices π.
Workspaces
A workspace is a data structure specific to JDemetra+, consisting of a set of XML files organized into folders (image and link to doc) containing, among other things:
- External variables (calendar regressors, etc.)
- Custom calendars for the workspace
- Specifications (seasonal adjustment models)
- Multi-documents, also known as SA-Processing
At the first level, a workspace is the combination of a directory (containing XML files) and a master XML file that defines the structure of the folder and its contents. Both have the same name and must be located together in the same directory.
A workspace contains SA-Processing, the equivalent of subdirectories specific to JDemetra+, where SA-items are created. These are sub-structures containing all the information relating to a series (raw data, links to it, and the three types of specifications seen in the previous section). The workspace does not store results, but rather all the parameters needed to regenerate them. We complete this description in appendices π.
The following section details the steps involved in building a production process by combining the tools described above.