Summary

Summary Bridging the Gap Between Knowledge and Data Driven Techniques for Time Series Forecasting .

0 purchase

Course
Bridging the Gap Between Knowledge and Data Driven

Institution
Bridging The Gap Between Knowledge And Data Driven

1 Introduction Recent advances in computational hardware have made it possible to achieve state-of-the-art performance in various domains, by utilizing DNNs, ranging from image classification [21], playing board games [15], natural language processing [6] to speech recognition [8]. As a result, th...

[Show more]

Preview 2 out of 11 pages

View example

Uploaded on October 17, 2024
Number of pages 11
Written in 2024/2025
Type Summary

1introduction recent advances in computational

Course
Bridging the Gap Between Knowledge and Data Driven

$14.49

Add to cart

Add to wishlist

100% satisfaction guarantee
Immediately available after payment
Both online and in PDF
No strings attached

Bridging the Gap Between Knowledge
and Data Driven Techniques for Time
Series Forecasting .
TU Kaiserslautern, Kaiserslautern, Germany
1

2 German Research Center for Artificial Intelligence (DFKI) GmbH, Kaiserslautern
muhammad ,
3 School of Electrical Engineering and Computer Science (SEECS), National University of
Sciences and Technology (NUST), Islamabad, Pakistan
4 Deep Learning Laboratory, National Center of Artificial Intelligence, Islamabad, Pakistan

Abstract. Artificial Intelligence (AI) can roughly be categorized into two
streams, knowledge driven and data driven both of which have their own
advantages. Incorporating knowledge into Deep Neural Networks (DNN),
that are purely data driven, can potentially improve the overall performance
of the system. This paper presents such a fusion scheme, DeepEX, that
combines these seemingly parallel streams of AI, for multistep time-series
forecasting problems. DeepEX achieves this in a way that merges best of
both worlds along with a reduction in the amount of data required to train
these modelsnow. This direction has been explored in the past for single step
forecasting by opting for a residual learning scheme. We analyze the
shortcomings of this simple residual learning scheme and enable DeepEX to
not only avoid these shortcomings but also scale to multi-step prediction
problems. DeepEX is tested on two commonly used time series forecasting
datasets, CIF2016 and NN5, where it achieves competitive results even
when trained on a reduced set of training examples. Incorporating external
knowledge to reduce network’s reliance on large amount of accurately
labeled data will prove to be extremely effective in training of neural
networks for real-world applications where the dataset sizes are small and
labeling is expensive.

Keywords: Deep Neural Networks · Knowledge Incorporation · TimeSeries ·
Residual Learning.

1 Introduction

Recent advances in computational hardware have made it possible to achieve state-of-
the-art performance in various domains, by utilizing DNNs, ranging from image
classification [21], playing board games [15], natural language processing [6] to speech
recognition [8]. As a result, there is heightened interest, both academically and
industrially, in DNNs with deep learning being listed at the top of Gartner hype cycle
for emerging technologies [5]. This increased interest coupled with advances in

, 2 Chattha et al.

hardware has paved the way for the development of more sophisticated DNN
algorithms, which may contain millions of parameters to train and optimize. Version of
NASNet-A [21] model, for example, with highest accuracy on ImageNet dataset contains
around 88.9M parameters. Optimizing such a huge number of parameters is a challenge
itself and requires equivalently bigger training dataset that allows the model to extract
enough features to train its parameters. As a result, these models perform exceptionally
well in domains where ample data is available but in data scarce domains, these model
suffer as they can easily overfit. This is even more so true for time-series domain, where
scantiness of data is further compounded by the fact that time-series often do not have
enough features for deep networks to work with. Leveraging information present in
the form of knowledge can be particularly useful here. These techniques, especially the
statistical ones, have shown considerable success in time-series domain which is
evident from results of forecasting competitions, like M3 [11], M4 [12] and NN5 [16],
which were dominated by statistical based techniques.
In contrast to DNNs, humans tend to rely on their knowledge while solving
problems. This knowledge is acquired not only from problem specific examples
but also from other sources, like education and experiences [10]. However, the
very notion of ”knowledge” is tricky to explain and equivalently difficult to collect
and store in a form that is understandable or transferable to a computing program.
Knowledge-Based Systems (KBS) aims to store such knowledge expressed in the
form of logic rules or some other declarative language which can then be used to
find solution to complex problems [19]. Similarly, there are statistical methods
that are based on strong logical reasoning, like Auto-Regressive Integrated Moving
Average (ARIMA), that do perform exceptionally well in their respective domains
and are used by many experts to aid them in decision-making process. . .
In fact, complementing DNNs with expert knowledge or some form of extra
knowledge has been actively researched upon [20,3,9]. Most of the work in the
literature, although improves performance of the DNNs but adds extra
dependency on the network on quality of expert information used [17,9]. The focus
of this work is to combine knowledge driven and data driven streams in a way that
retains advantages of both while suppressing their individual disadvantages.
Specifically, we aim to reduce the dependency of DNNs on the data by leveraging
information contained in the knowledge stream. Finding state-of-theart
knowledge-based system or DNN model is not the focus here, but instead, the goal
is to devise a knowledge incorporation scheme that bridges the gap between data
and knowledge driven approaches and combines their strengths. Chattha et al.
(2019) [4] recently introduced Knowledge Integrated Neural Network (KINN), a
residual framework that combined information contained in the knowledge
stream with the data stream in order to reduce the dependence of DNN on large
amount of accurately labeled data. However, KINN [4] failed to produce acceptable
results on benchmark time-series datasets. KINN particularly suffered when
dealing with time-series that encapsulated significant trend variation. This
resulted in poor performance on more sophisticated time-series datasets. In this
paper, we present DeepEX, that not only addresses the shortcomings of KINN [4],
but also strengthens the network allowing information in the two streams to

The benefits of buying summaries with Stuvia:

Guaranteed quality through customer reviews

Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.

Quick and easy check-out

You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.

Focus on what matters

Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!

Frequently asked questions

What do I get when I buy this document?

You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.

Satisfaction guarantee: how does it work?

Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.

Who am I buying these notes from?

Stuvia is a marketplace, so you are not buying this document from us, but from seller antonysila275. Stuvia facilitates payment to the seller.

Will I be stuck with a subscription?

No, you only buy these notes for $14.49. You're not tied to anything after your purchase.

Can Stuvia be trusted?

4.6 stars on Google & Trustpilot (+1000 reviews)

68175 documents were sold in the last 30 days

Founded in 2010, the go-to place to buy study notes for 15 years now

Start selling

Seller

Summary ·