What are the most challenging issues regarding data today? The huge increase in the volume and variety of data, and the customer’s call for immediate insights into fresh data. The whole world is moving to data with scalability and insights in real-time. With all that happening together, how should you design an effective data processing architecture? Can we combine transactional and analytical workloads? How can we make this work in the real world? Or is this even possible? Max Liu, the Founder and CEO of PingCAP hits the nail right on the head in his session.
In this talk, Matei, the co-founder of Databricks explains how the analytics stack is moving to real-time. He also introduces the lakehouse system and streaming in a real-world enterprise, and discusses in detail how they power their control plane with a powerful HTAP database: TiDB.
Tim Tully, a partner of Menlo Ventures, hosts an inspiration panel featuring PingCAP CTO Ed Huang and technology leaders of AWS, StarTree and Catalyst. They discuss industry trends, rising technology, and why it is high time to consolidate OLTP and OLAP. Enjoy.
Users share stories of building robust transaction processing and analytical capability in the real-time HTAP environment, to support efficient business growth.
Data analytics is a crucial factor in business success. However, most companies, due to a lack of data management capability, analyze less than 25% of their data. How can we better utilize data to enable more innovations and generate bigger values? Noel Yuhanna, Vice President, Principal Analyst from Forrester Research presents five key areas where data management will innovate in the coming years. Check out this speech to find out more.
Square Message is the messaging hub that aims to unify buyer-seller communication across the Block’s ecosystem. As their data volume quickly grew, the original data stack couldn’t scale, query latency increased, and schema migrations were shaky. After researching alternatives, they chose TiDB and thought it was an ergonomic alternative to sharded MySQL. In their session, Software Engineer Henry Qin and his team discuss why they went with TiDB, their experience with it, and what they’ve learned.
As the leading visual discovery engine for over 430 million MAU, Pinterest handled more than 1.3 million QPS. HBase, their critical storage backend could not serve their growing needs. They evaluated more than 10 storage backends, they replaced HBase with TiDB. What made TiDB the best candidate? What were the key wins after deploying TiDB? For more details, check out this speech given by Ankita Wagh, Senior Software Engineer at Pinterest.
Catalyst is a customer success platform that helps unify customer data and provides actionable data insights that drive retention and growth. These tasks are not easy, considering the challenges of multiple data sources, query latencies, and workload varieties. After a 9 day “hackhouse”, Catalyst decided to re-architect their data stack with TiDB, an HTAP solution, as their data serving layer. Andy Trimble, Principal Engineer at Catalyst has a great HTAP story to tell, with tips and insights from their journey.
Niantic is an American gaming company best known for developing the augmented reality mobile game. One of their games, Pokémon Go, has over 147 million monthly active users in its peak time. A game this large needs a database that is elastically scalable and that can quickly respond to highly concurrent queries. In his session, Xingkai Wang, Staff Software Engineer at Ninantic, will introduce WStore, a database service built upon TiKV, an open source distributed KV storage. WStore provides uninterrupted gaming to users around the world.
Flipkart is India’s largest e-commerce company with 400 M+ registered users and 10 M+ daily page visits. As their business grew, their original data stack couldn’t support the high data volumes and throughput. Flipkart searched for a distributed SQL solution with scalability, reliability, and efficiency. TiDB emerged as the best fit. In his session, Kaustav Chakravorty, Senior Architect from Flipkart will discuss TiDB and the benefits it has brought to his company.
Web3 is one of the most important trends in recent years. It’s booming; data volume is increasing and many types of Web3 applications are trying to monetize that data. None of this is easy. How can we overcome those challenges? One answer is HTAP. Watch this speech given by Thomas Yu, Co-Founder of KNN3 Network, and learn how TiDB, an open source HTAP database, fixes their pain points and supercharges their Web3 business.
The booming Non-fungible token (NFT) market has generated a massive volume of NFT that’s distributed across various blockchains. How can you collect and analyze that multi-chain data for useful insights? How do you choose the right database to store such massive data and support data analytics? Check out this speech by Cathy Ray, Marketing Manager at NFTScan, to learn how NFTScan handles terabytes of NFT data and how they chose their databases.
Bolt is a global logistics leader that serves over 100 million customers in more than 45 countries and regions. As their business grew rapidly, their MySQL database could no longer store, manage, and analyze the massive amount of data. This drove them to replace MySQL with TiDB. But why TiDB? How does TiDB fix their pain points? Watch this video to learn more about how Bolt replaced their database and see if you have similar headaches.
U-Next is a video on demand (VOD) streaming platform similar to Netflix or Amazon Prime Video. U-Next tops the market in content volume, and is in the top 3 in market share. This are facing several challenges including large data volume and no downtime for maintenance. In this video, you’ll see how they found the right solution to overcome their performance and availability issues, and how they received sustained community and enterprise support.
Engineering stories of designing and implementing HTAP platforms.
HTAP systems are in high demand: they offer less complexity, fresher data, and more consistent views. Then, what kind of HTAP systems are in the market? What’s the challenge of building an HTAP system? What is critical for HTAP systems? In his session, Mattias Jonsson, Senior Database Engineer from PingCAP, provides his insights on the state of the art for HTAP systems.
Apache Pinot is an open-source distributed OLAP database that provides real-time analytics. It is suited in contexts where fast analytics, such as aggregations, are needed on immutable data, possibly, with real-time data ingestion. What makes Pinot so fast? In his session, Mayank Shrivastava, Founding Engineer of StarTree, will walk you through the internal architecture and features of Pinot with use cases.
Modern systems and businesses typically require real-time decision making on transactional data, which calls for unifying transactions and analytics in the same platform. Traditional databases aren’t up to the task. Rajkumar Sen, Founder & CTO of Arcion believes that cloud-native distributed HTAP databases are the future. He will share Arcion’s solution, a cloud-native change data control (CDC) platform, which functions as a data bridge between legacy systems and modern ones.
HTAP is great, but do users really need it? Can it replace OLAP databases or OLTP databases? Not necessarily. Transactional and analytical workloads are processed separately for good reasons. In his session, Robert Hodges, CEO of Altinity, will share the criteria for using an HTAP database, and the ways to integrate OLTP and OLAP for your workloads.
As a modernized architecture, HTAP takes many forms. In their session, BP Yao, Senior Product Manager at AWS, and Rahul Chaturvedi, Senior Analytics Specialist at AWS will share how Amazon Redshift, the cloud data warehouse, fits in the HTAP architecture with real-life use cases.
As an emerging technology, HTAP provides both transactional processing and analytical processing in the same database without mutual interference. And it allows real-time insights into operational data. It sounds all fancy, but how good is an HTAP system? Sam Dillard, Principal Product Manager from PingCAP will introduce HATtrick, a benchmark tool specifically designed for HTAP systems by UW-Madison, with TiDB as a sample test case. In the second part of the talk, Kan Wu from UW-Madison will provide an overview of the ongoing work that UW-Madison and PingCAP are working together on automatic cache tuning for HTAP systems.
Tying HTAP to the data ecosystem - integrations, security, client, APIs, etc.
What does AWS think of HTAP? Why do they believe that HTAP will be the trend in the following years? What scenarios does HTAP database fit the most? Watch this speech given by Srinivas Kesanapally and Sri Raghavan from AWS, and learn industry leaders' views about HTAP.
What fueled the data explosion? How to cope with such a big size of data? Why is the HTAP system the future? Check out this fascinating talk given by Tim Tully, Partner at Menlo Ventures, and look back at the evolution of data systems from their "Big Bang" moment along a fantastic journey towards the adoption of the HTAP system.
Vercel is a powerful frontend platform that makes it easier to develop applications. TiDB Cloud is a fully-managed service of TiDB that's available on both AWS and Google Cloud Platform (GCP). The integration of Vercel and TiDB Cloud makes it much easier and faster for frontend teams to build their web applications and frees them from time-consuming configuration tasks. Check out this video to learn how you can leverage this integration to accelerate your app development process.
ProxySQL is an open-source, high-performance proxy for MySQL protocol. It helps you build, support, and improve the performance and reliability of both cloud-native MySQL infrastructures such as TiDB. By integrating ProxySQL with your TiDB clusters, your TiDB using experience can be greatly enhanced. What ProxySQL features are particularly useful for your TiDB clusters? How does ProxySQL integrate with TiDB? To learn more, check out this speech given by René Cannaò, the CEO of ProxySQL.
Social networks such as Facebook, TikTok, and Twitter generate massive data every second. This data is stored in large-scale, geo-distributed storage systems. TAOBench is a useful benchmark tool that can reproduce large-scale production workloads of those social networks. It can also help you research the underlying storage architectures of those large-scale social applications and evaluate and benchmark their storage alternatives. Wonder how different databases perform under TaoBench evaluation? What else can TAOBench do? To learn more, check out this speech given by Audrey Cheng, a PhD student at UC Berkeley and one of the main contributors to TAOBench.
Today, applications need instant access to all data from sensors to logs and events. This data needs to be stored, accessed, and joined with live transactional data. We can no longer wait until data has arrived in various systems to analyze it; we need real-time analytics. In his session, Timothy Spann, Developer Advocate from StreamNative, will share how they enable real-time analytics, ingest, and egress of HTAP data at scale through FLiP Stack, a combination of Apache Pulsar and Apache Flink.