Author: Davis Chu

In DolphinDB, we can import historical data into a stream table in chronological order as “real-time data” so that the same script can be used both for backtesting and real-time trading. Regarding streaming in DolphinDB, please refer to DolphinDB Streaming Tutorial.

This article introduces the functions replay and replayDS and then demonstrates the process of data replaying.

DolphinDB is a high-throughput distributed time-series database, specially designed for big data analysis scenarios in industrial IoT (IIOT) and finance.

Data analysis in DolphinDB is often conducted based on the basic product information (such as the SKU, size, and storage of a device) from an external business platform, which conventionally runs on a relational database. To associate external product information with the time series data in DolphinDB, you can use the cached tables to save the external data fetched through plugins (MySQL, MongoDB, CSV file, etc.) in memory with regular synchronizations.

OHLC bars can be efficiently calculated in various scenarios in DolphinDB. This tutorial will introduce how to calculate OHLC bars with historical data and real-time data. 

• With historical data

We will explain how to calculate OHLC bars with batch calculation for the following scenarios:

For high-frequency data in financial markets, each record typically holds the information of a stock at a specific timestamp. We often need to rearrange a column (or the calculation results involving multiple columns) into a matrix or table with the timestamps as row labels and security IDs as column labels. This operation (referred to as “pivoting”) can be achieved with the SQL pivot by keyword or the pivot function in DolphinDB. The result can be used in vectorized operations for optimal performance. 

1. Calculating Pairwise Correlations of Stock Returns

In pairs trading and hedging, we often need to calculate the pairwise correlations of multiple securities. Traditional databases are not able to perform such complex calculations. Using statistical software would require data migration between systems, which can be very time-consuming with a large amount of data. In DolphinDB, pairwise correlation can be calculated with the help of SQL pivot by clause.