Program

AI is evolving beyond traditional text-based analysis, now encompassing multi-modal data such as images, video, and audio. This evolution exponentially increases the scale and scope of data available for analysis and requires a new Data Platform for exabyte-scale data. Scalability, Complexity and cost have historically been significant barriers to the widespread adoption of data and AI technologies. However, the emergence of new, more scalable and resilient architectures is rapidly dismantling these obstacles, unlocking a multitude of opportunities for simplification and innovation.

In this talk I will focus on the algorithms and data structures we’ve implemented in the VAST Data Platform, a unified global datastore, database, and AI computing engine. I will explore practical algorithms and data structures that break traditional trade-offs between speed, cost, and resiliency. Key topics will include similarity compression, error correction codes, and scalable data structures designed to optimize high performance and scalable file system, database queries and AI pipelines.

There are many problems in hardware that can be succinctly described with the reinforcement learning (RL) framework. In such problems an agent needs to make a decision has potentially long-term consequences. While in general RL algorithm are too complex to implement in real-time hardware, we describe the tradeoffs needed to facilitate effective and efficient implementation of RL algorithms in hardware. I will focus on practical algorithms we have implemented at Nvidia for network congestion control, cache prefetching aggressiveness, and online configuration selection. I will then describe the challenges in building practical real world RL-based systems such as lack of stationarity, stability and safety requirements, etc.

This talk gives a glimpse onto a decade of innovation which has been sparked by blockchains in scaling BFT Consensus.

In a nutshell, a Consensus system allows a network of nodes to act in coordinated manner despite both benign outages and security (“Byzantine”) threats. For over four decades, experts in the field have been looking for Consensus solutions that can perform well in practice.

In 2018, HotStuff transformed the landscape of practical BFT consensus solutions by introducing two key contributions, linearity and streamlining: HotStuff is the first BFT protocol that achieves linear communication complexity even when a fault occurs. Additionally, the HotStuff protocol is modular, consisting of a recurring broadcast module that can be pipelined, such that each broadcast extends the sequence of decisions by one.

HotStuff was adopted by the famous Diem(Libra) project to drive global payment rails backed by a consortium-operated blockchain. In the end, Diem did not launch, but the technology behind it illuminated and fertilized the field.

The talk overviews solutions enabled by HotStuff to open problems which emerged under a variety of settings, underscoring insights from the blockchain arena that led to new approaches.