A Python Library to Generate a Synthetic Time Series Data

Synthetic data is widely used in various domains. This is because many modern algorithms require lots of data for efficient training, and data collection and labeling usually are a time-consuming process and are prone to errors. Furthermore, some real-world data, due to its nature, is confidential and cannot be shared.

Some methods, such as generative adversarial network¹, are proposed to generate time series data. However, GAN is hard to train and might not be stable; besides, it requires a large volume of data for efficient training.

This article will introduce the tsBNgen, a python library, to generate synthetic time series data based on an arbitrary dynamic Bayesian network structure.

Although tsBNgen is primarily used to generate time series, it can also generate cross-sectional data by setting the length of time series to one.

The following is a list of topics discussed in this article.

Introduction
Features
Instruction
Example
Conclusion

Introduction

tsBNgen is a python package released under the MIT license to generate time series data from an arbitrary Bayesian network structure. Bayesian networks are a type of probabilistic graphical model widely used to model the uncertainties in real-world processes. Dynamic Bayesian networks (DBNs)are a special class of Bayesian networks that model temporal and time series data.

Bayesian networks receive lots of attention in various domains, such as education and medicine. For example, in², the authors used an HMM, a variant of DBN, to predict student performance in an educational video game. One significant advantage of directed graphical models (Bayesian networks) is that they can represent the causal relationship between nodes in a graph; hence they provide an intuitive method to model real-world processes. This statement makes tsBNgen very useful software to generate data once the graph structure is determined by an expert. In a sense, tsBNgen unlike data-driven methods like the GAN is a model-based approach.

To learn more about the package, documentation, and examples, please visit the following GitHub repository.

manitadayon/tsBNgen

I recently created a series of YouTube videos on to use this package. The videos are also on my GitHub page and you can access them here:

I try to go over the Bayesian network and tsBNgen package in detail. If you have any questions or comments, you can ask me on my YouTube channel.

Features

Following is the list of supported features and capabilities of tsBNgen:

Easy and simple interface.
Support for discrete, continuous, and hybrid networks (a mixture of discrete and continuous nodes).
Support for discrete nodes using multinomial distributions and Gaussian distributions for continuous nodes.
Supports arbitrary loopback (temporal connection) values for temporal dependencies.
Easy to modify and extend the code to support the new structure.
The model-based approach, which can generate synthetic data once the causal structure is known.

Instruction

To use tsBNgen, either clone the above repository or install the software using the following commands:

pip install tsBNgen

After the software is successfully installed, then issue the following commands to import all the functions and variables.

from tsBNgen import * 
from tsBNgen.tsBNgen import *

This is all you need to take advantage of all the functionalities that exist in the software.

Example

Before going over some examples, let me define the following parameters, which will be used throughout this section. Note: The following description, tables (as a form of an image), and images are obtained from this paper by the author³.

Example 1

Assume you would like to generate data for the following architecture in Fig 1, which is an HMM structure.

The top layer nodes are known as states, and the lower ones are called the observation. In HMM, states are discrete, while observations can be either continuous or discrete. The following tables summarize the parameters setting and probability distributions for Fig 1.

Conditional Probability Distribution for all the nodes at time t (Photo by the Author)

In Table 1, T refers to the length of time series, N refers to the number of samples, and loopback determines the length of the temporal connection. For example, a loopback value of 1 implies that a node is connected to some other nodes at a previous time.

Note: tsBNgen can simulate the standard Bayesian network (cross-sectional data) by setting T=1.

Architecture 1 with the above CPDs and parameters can easily be implemented as follows:

import time
START=time.time()
T=20
N=1000
N_level=[4]
Mat=pd.DataFrame(np.array(([0,1],[0,0]))) # HMM
Node_Type=['D','C']

CPD={'0':[0.25,0.25,0.25,0.25],'01':
{'mu0':20,'sigma0':5,'mu1':40,'sigma1':5,
'mu2':60,'sigma2':5,'mu3':80,'sigma3':5}}

Parent={'0':[],'1':[0]}

CPD2={'00':[[0.6,0.3,0.05,0.05],[0.25,0.4,0.25,0.1],[0.1,0.3,0.4,0.2],
[0.05,0.05,0.4,0.5]],'01':{'mu0':20,'sigma0':5,'mu1':40,'sigma1':5,
'mu2':60,'sigma2':5,'mu3':80,'sigma3':5
}}

loopbacks={'00':[1]}
Parent2={'0':[0],'1':[0]}
Time_series1=tsBNgen(T,N,N_level,Mat,Node_Type,CPD,Parent,CPD2,Parent2,loopbacks)
Time_series1.BN_data_gen()
FINISH=time.time()
print('Total Time is',FINISH-START)

The above code generates a 1000 time series with length of 20 correspondings to states and observations. Observations are normally distributed with a particular mean and standard deviation. The states are discrete (hence the ‘D’) and take four possible levels determined by the _Nlevel variable. loopbacks is a dictionary in which each key has the following form: node+its parent. Since in architecture 1, only states, namely node 0 (according to the graph’s topological ordering), are connected across time and the parent of node 0 at time t is node 0 at time t-1; therefore, the key value for the loopbacks is ’00’ and since the temporal connection only spans one unit of time, its value is 1. Node_Type determines the categories of nodes in the graph. For example in this example, the first node is discrete (‘D’) and the second one is continuous (‘C’). Mat represents the adjacency matrix of the network.

The total time to generate the above data is 2.06 (s), and running the model through the HMM algorithm gives us more than 93.00 % accuracy for even five samples. Now let’s take a look at a more complex example. From now on, to save some space, I avoid showing the CPD tables and only show the architecture and the python code used to generate data.

Example 2

Example 2 refers to the architecture in Fig 2, where the nodes in the first two layers are discrete and the last layer nodes(u₂) are continuous.

Based on the graph’s topological ordering, you can name them nodes 0, 1, and 2 per time point. Let’s say you would like to generate data when node 0 (the top node) takes two possible values (binary), node 1(the middle node) takes four possible values, and the last node is continuous and will be distributed according to Gaussian distribution for every possible value of its parents. The following python codes simulate this scenario for 2000 samples with a length of 20 for each sample.

T=20
N=2000
N_level=[2,4]
Mat=pd.DataFrame(np.array(([0,1,1],[0,0,1],[0,0,0])))
Node_Type=['D','D','C']

CPD={'0':[0.6,0.4],'01':[[0.5,0.3,0.15,0.05],[0.1,0.15,0.3,0.45]],'012':{'mu0':10,'sigma0':2,'mu1':30,'sigma1':5,
 'mu2':50,'sigma2':5,'mu3':70,'sigma3':5,'mu4':15,'sigma4':5,'mu5':50,'sigma5':5,'mu6':70,'sigma6':5,'mu7':90,'sigma7':3
}}

Parent={'0':[],'1':[0],'2':[0,1]}

CPD2={'00':[[0.7,0.3],[0.2,0.8]],'011':[[0.7,0.2,0.1,0],[0.6,0.3,0.05,0.05],[0.35,0.5,0.15,0],
[0.2,0.3,0.4,0.1],[0.3,0.3,0.2,0.2],[0.1,0.2,0.3,0.4],[0.05,0.15,0.3,0.5],[0,0.05,0.25,0.7]],'012':{'mu0':10,'sigma0':2,'mu1':30,'sigma1':5,
  'mu2':50,'sigma2':5,'mu3':70,'sigma3':5,'mu4':15,'sigma4':5,'mu5':50,'sigma5':5,'mu6':70,'sigma6':5,'mu7':90,'sigma7':3
}}
Parent2={'0':[0],'1':[0,1],'2':[0,1]}
loopbacks={'00':[1],'11':[1]}
Time_series2=tsBNgen(T,N,N_level,Mat,Node_Type,CPD,Parent,CPD2,Parent2,loopbacks)
Time_series2.BN_data_gen()

As the above code shows, node 0 (the top node) has no parent in the first time step (This is what the variable Parent represents). This is sometimes known as the root or an exogenous variable in a causal or Bayesian network. Node 1 is connected to node 0 and node 2 is connected to both nodes 0 and 1. To represent the structure for other time-steps after time 0, the variable Parent2 is used. This says node 0 is connected to itself across time (since ’00’ is [1] in loopbacks then time t is connected to t-1 only). Node 1 is connected to node 0 for the same time and to node 1 in the previous time (This can be seen from the loopback variable as well).

Since tsBNgen is a model-based data generation then you need to provide the distribution (for exogenous node) or conditional distribution of each node. If you would like to generate synthetic data corresponding to architecture with arbitrary distribution then you can choose CPD and CPD2 to be anything you like as long as the sum of entries for each discrete distribution is 1. For example, the CPD for node 0 is [0.6, 0.4]. You can change these values to be anything you like as long as they are added to 1.

Example 3

Example 3 refers to the architecture in Fig 3, where the nodes in the first two layers are discrete and the last layer nodes(u₂) are continuous.

Assume you would like to generate data when node 0 (the top node) is binary, node 1(the middle node) takes four possible values, and node 2 is continuous and will be distributed according to Gaussian distribution for every possible value of its parents. The following python codes simulate this scenario for 1000 samples with a length of 10 for each sample.

T=10
N=1000
N_level=[2,4]
Mat=pd.DataFrame(np.array(([0,1,0],[0,0,1],[0,0,0])))
Node_Type=['D','D','C']
CPD={'0':[0.5,0.5],'01':[[0.6,0.3,0.05,0.05],[0.1,0.2,0.3,0.4]],'12':{'mu0':10,'sigma0':5,'mu1':30,'sigma1':5,
    'mu2':60,'sigma2':5,'mu3':80,'sigma3':5}}
Parent={'0':[],'1':[0],'2':[1]}

CPD2={'00':[[0.7,0.3],[0.3,0.7]],'0011':[[0.7,0.2,0.1,0],[0.5,0.4,0.1,0],[0.45,0.45,0.1,0],
[0.3,0.4,0.2,0.1],[0.4,0.4,0.1,0.1],[0.2,0.3,0.3,0.2],[0.2,0.3,0.3,0.2],[0.1,0.2,0.3,0.4],[0.3,0.4,0.2,0.1],[0.2,0.2,0.4,0.2],
 [0.2,0.1,0.4,0.3],[0.05,0.15,0.3,0.5],[0.1,0.3,0.3,0.3],[0,0.1,0.3,0.6],[0,0.1,0.2,0.7],[0,0,0.3,0.7]],'112':{'mu0':10,'sigma0':2,'mu1':30,'sigma1':2,
    'mu2':50,'sigma2':2,'mu3':60,'sigma3':5,'mu4':20,'sigma4':2,'mu5':25,'sigma5':5,'mu6':50,'sigma6':5,'mu7':60,'sigma7':5,
   'mu8':40,'sigma8':5,'mu9':50,'sigma9':5,'mu10':70,'sigma10':5,'mu11':85,'sigma11':2,'mu12':60,'sigma12':5, 
    'mu13':60,'sigma13':5,'mu14':80,'sigma14':3,'mu15':90,'sigma15':3}}

Parent2={'0':[0],'1':[0,0,1],'2':[1,1]}
loopbacks={'00':[1], '01':[1],'11':[1],'12':[1]}

Time_series2=tsBNgen(T,N,N_level,Mat,Node_Type,CPD,Parent,CPD2,Parent2,loopbacks)
Time_series2.BN_data_gen()

In the same way, you can generate time series data for any graphical models you want. This is a wonderful tool since lots of real-world problems can be modeled as Bayesian and causal networks.

For more examples, up-to-date documentation please visit the following GitHub page.

manitadayon/tsBNgen

If you would like to know more make sure you watch my recent videos on my YouTube channel.

Bonus: If you would like to see a comparative analysis of graphical modeling algorithms such as the HMM and deep learning methods such as the LSTM on a synthetically generated time series, please look at this paper⁴.

Conclusion

In this article, I introduced the tsBNgen, a python library to generate synthetic data from an arbitrary BN. The features and capabilities of the software are explained using two examples. For more up-to-date information about the software, please visit the GitHub page mentioned above.

Reference

[1] M. Frid-Adar, E. Klangand, M. Amitai, J. Goldberger, H. Greenspan, Synthetic data augmentation using gan for improved liver lesion classification(2018), IEEE 2018 15th international symposium on biomedical imaging.

[2] M. Tadayon, G. Pottie, Predicting Student Performance in an Educational Game Using a Hidden Markov Model(2020), IEEE 2020 IEEE Transactions on Education.

[3] M. Tadayon, G. Pottie, tsBNgen: A Python Library to Generate Time Series Data from an Arbitrary Dynamic Bayesian Network Structure (2020), arXiv 2020, arXiv preprint arXiv:2009.04595.

[4] M. Tadayon, G. Pottie, Comparative Analysis of the Hidden Markov Model and LSTM: A Simulative Approach (2020), arXiv 2020, arXiv preprint arXiv:2008.03825.