Memory usage is an important metric for Arctype. Developers expect their tools - especially UIs - to be fast and smooth. Recently we improved our memory management algorithms using IndexedDB, Socket.IO, and Typescript. This post takes you inside our reason for the changes, design considerations, implementation, and impact.
Prior to this work, querying a database from Arctype was simple. We would grab every row up to 10,000 and load them into memory. We would display a fraction of rows in the results pane, the spreadsheet view, or a chart. When you needed more rows, we would throw out the first 10,000 and fetch another batch.