Apache arrow is a cross-platform binary format for in-memory data storage. Being much more compact than JSON it is suitable for larger datasets as well as for interoperability with other software that generates apache-arrow format files:
To use Apache Arrow data with Vega-Lite (or Vega) you need to link to the vega-loader-arrow and apache-arrow javascript packages in the head of your HTML along with the normal links to the vega and vega-lite runtimes:
<script src="https://cdn.jsdelivr.net/npm/apache-arrow"></script> <script src="https://cdn.jsdelivr.net/npm/vega-loader-arrow"></script>
Data source contains 10,000 flight records as a binary arrow file.
arrow1 : Spec arrow1 = let data = dataFromUrl "https://gist.githubusercontent.com/domoritz/0f53a5abde95564c36dfaac623a7a922/raw/cce3719b853e25d5dfff97a270283ba83af3c0e6/flights-10k.arrow" [ arrow ] enc = encoding << position X [ pName "DEP_TIME", pMType Quantitative ] << position Y [ pName "ARR_DELAY", pMType Quantitative ] << color [ mName "DISTANCE", mMType Quantitative ] in toVegaLite [ width 500, height 300, data, enc [], circle [ maSize 4 ] ]
10,000 random samples from a 1.5 million record scrabble dataset that is 157Mb in CSV format and 47Mb in arrow format.