Production deployment of time series forecasting

Time series analysis is as vast a topic as time itself, and as such, there is far too much information to cover in one article. This article is the second part in a series to go over the many intricacies of time series analysis. Read part one, Incorporating time series analysis into your business, to learn more.
Many businesses utilize time series analyses to assist with their operational planning, such as conducting quarterly sales forecasts to fine-tune sales targets. In other cases, time series analyses is an exploratory exercise that informs a specific business decision, such as forecasting customer growth to gauge upcoming customer service hiring needs. Both scenarios are examples of manual, pre-planned time series analysis, which usually require advance planning and dedicated time from a data scientist. The data scientist spends a significant amount of effort to collect and cleanse data, tune the model and generate and present the results. Ultimately, this means decision makers must do a lot of one-off work because analysts will have to re-calibrate all of the data to repeat the process manually. Lacking access to data when needed can hinder a business’s ability to properly preparing for the future. Establishing steps to eliminate the need for manual processes and go beyond pre-planned one-off analyses is vital for a business to respond quickly to market changes and adapt.

Building a data pipeline

Establishing a data pipeline is the first step to getting the right data. A data pipeline must capture the variables of interest, while also arriving on time in a consistent format. The foundation for a relevant and accurate data analysis includes timely and high quality data. A data pipeline takes data from one or more sources and transforms that content to develop a new data set that can be leveraged for specific analysis. For example, sales figures can be uploaded and processed by a service to be loaded into a dataset. Once the data set is established, a business can examine trends in their data to better guide future sales decisions as it relates to their products across all aspects of their business. Services such as Amazon Web Services (AWS) Glue and Azure Data Factory, as well as open source systems such as Apache Airflow or Luigi, are common choices for building a data pipeline. By implementing a data pipeline, a business can build their foundation and enable on-demand forecasts to be produced by guaranteeing that inputs are available.