From Bottlenecks to Breakthroughs: Moka BI’s TiDB Transformation

Moka, a leading HR SaaS provider, sought to enhance its business intelligence (BI) architecture to address performance bottlenecks, data silos, and the complexity of its system. After evaluating various database solutions, Moka selected TiDB to transform its BI system.

By implementing TiDB, Moka achieved significant improvements:

• Increased performance: Daily average QPS now exceeds 5K, with peak stability reaching 30K, ensuring stable system performance even under heavy loads.
• Real-time capabilities: With a near 1:1 read-write ratio, Moka can now perform real-time modeling without delays, providing users with real-time insights and reporting.
• Improved reliability: TiDB’s multi-replica architecture allowed for a 10-second recovery during a node crash, maintaining service continuity without impacting online operations.
• Seamless scalability: Expanding three TiKV storage nodes during the HR system migration did not disrupt business operations, showcasing TiDB’s smooth scaling and maintenance processes.
• Data unification: TiDB’s unified storage resolved previous data silos, enabling Moka to establish a preliminary data lake model for improved data access and governance.

These enhancements have simplified Moka’s system architecture, improved operational efficiency, and provided better real-time reporting and decision-making capabilities.

What is Moka? 

With the increasing emphasis on workforce analytics and real-time decision-making, HR SaaS platforms incorporating business intelligence (BI) systems are becoming more critical. These systems empower HR departments by offering actionable insights, predictive analytics, and real-time reporting that enhance recruitment processes and overall HR performance.

Founded in 2015, Moka is a leading HR SaaS provider dedicated to transforming the human resources landscape with innovative, data-driven solutions. Moka’s core offerings include intelligent recruitment and human resource management systems, serving mid-to-large enterprises. 

Moka’s client base includes renowned organizations across various industries, such as Ctrip, Bosch, and Geely. With over 2,000 clients, Moka is recognized for its ability to empower enterprises with strategic HR insights through advanced technology and exceptional user experiences. Moka BI, the company’s business intelligence system, is pivotal in enabling intelligent recruitment and HR management through real-time analytics and flexible reporting, giving businesses a competitive edge in talent acquisition and management.

The Challenge: Overcoming Data Silos and Scalability Issues

Before implementing TiDB, Moka BI faced several key challenges that hindered its ability to deliver an optimal HR analytics experience. These challenges revolved around data processing limitations, scalability issues, and operational inefficiencies:

• Inefficient Data Processing: Moka’s existing database infrastructure struggled to manage the increasingly large volumes of real-time data generated by its HR systems. This led to delayed reporting and analytics, preventing Moka from providing real-time insights to its clients. The target was to reduce data processing delays and ensure seamless real-time reporting for HR operations.
• Lack of Scalability: As Moka’s client base grew, the existing database system lacked the necessary scalability to handle the growing recruitment and HR data volume. The goal was to implement a solution that could seamlessly scale to accommodate at least a 50% increase in data volume without impacting performance or causing downtime.
• Data Silos and Fragmented Storage: Moka’s data was stored across multiple systems, resulting in operational inefficiencies and inconsistent data access. This siloed approach made it difficult to achieve unified data reporting, impacting decision-making and operational coordination. A critical success criterion was the creation of a unified data architecture to eliminate silos, allowing for integrated HR analytics across all platforms.
• Service Instability: Several instances of database node crashes impacted the stability of Moka’s services. Any future solution had to ensure high availability and automatic recovery to avoid disruptions in online services.

Original Data Architecture 

Moka BI’s original data architecture involved a legacy database system that was unable to support the growing demand for real-time analytics and dynamic data modeling. The siloed storage systems added complexity, hindering unified data access and delaying insights crucial for HR operations. While the architecture handled moderate loads, it struggled to efficiently process larger volumes of data from recruitment management and human resource systems. Additionally, the lack of a unified database for storage across departments limited collaboration and reporting capabilities.

The Solution: Scalable, Real-Time Data Management with TiDB  

To tackle Moka BI’s complex data processing needs, TiDB was selected after a rigorous evaluation of several database solutions. TiDB stood out for its MySQL compatibility, scalability, and ability to maintain high performance in real-time scenarios. The implementation addressed Moka BI’s key challenges in the following ways:

• Compatibility: TiDB’s full compatibility with MySQL protocols and its support for Canal-format data changes ensured seamless integration with Moka BI’s existing ecosystem. This included flawless downstream data synchronization using TiCDC, making the transition smooth with minimal disruptions.
• Decentralization and Performance: TiDB’s Shared Nothing architecture avoided performance bottlenecks. This architecture allowed for linear scalability, meaning Moka BI could continuously add nodes to handle increasing data loads without degradation in performance. Peak stability reached 30K QPS, showcasing TiDB’s ability to manage high query volumes efficiently.
• High Availability and Stability: TiDB met Moka BI’s high availability requirements. Multi-replica storage, synchronized by the Multi-Raft protocol, ensured service stability even in failure scenarios. For instance, TiDB recovered automatically within 10 seconds from a node crash without impacting online services. The solution supported multiple disaster recovery levels, with failure response times below 30 seconds.
• Usability and Scalability: The ease of scaling was critical to the project’s success. TiDB allowed for one-click scaling through TiUP, and the scaling process remained transparent to business operations. This was evident when Moka BI expanded three TiKV storage nodes without any disruptions during the migration of their human resource management system.
• Simplicity and Real-Time Analytics: TiDB simplified Moka BI’s data architecture, unifying storage across previously siloed systems. This led to establishing a data lake model, making it easier to manage and retrieve data. Furthermore, TiDB’s Hybrid Transactional and Analytical Processing (HTAP) capabilities, powered by TiKV and TiFlash, enabled Moka BI to handle both transactional and analytical workloads in real-time.

In the transition to TiDB, Moka BI’s architecture was transformed from a siloed system with limited scalability into a unified, high-performance data infrastructure. The migration to TiDB enhanced real-time data processing and resolved scalability issues, allowing the company to support increasing data volumes and more complex queries without compromising on performance.

Comparatively, TiDB’s architecture allowed significant improvements over the original system, particularly in handling concurrent queries and large datasets. The use of Flink for real-time data synchronization further enhanced Moka BI’s ability to model and process data in real-time with minimal latency.

The Results: Transformative Impact and Measurable Improvements 

Following the successful deployment of TiDB, Moka BI experienced substantial improvements across multiple operational metrics, clearly demonstrating the new system’s benefits.

• Increased Performance and Stability
  TiDB has significantly improved query performance. With an average QPS now exceeding 5K, and peak stability reaching 30K, the platform can easily handle large-scale operations. The improved read-write ratio of nearly 1:1 allows for real-time modeling without delays, enhancing the overall user experience.
• High Availability and Disaster Recovery
  TiDB’s multi-replica storage architecture ensures service stability even in the face of failure. When a node encounters an issue, TiDB automatically recovers within 10 seconds with no disruption to online services, showcasing its resilience and high availability.
• Seamless Scalability
  TiDB’s architecture facilitated seamless scaling. For instance, the expansion of three TiKV storage nodes as part of the human resource management system migration did not impact the online business. This flexibility ensures that Moka BI can grow without concerns about technical limitations.
• Elimination of Data Silos
  By unifying the storage of previously isolated data systems, TiDB has allowed Moka BI to establish a preliminary data lake, simplifying data access and improving the overall efficiency of business intelligence operations.
• Key Workload Metrics Comparison
  Compared to the previous system, Moka BI has experienced a sixfold increase in peak QPS, rising from 4K to 24K in single-company scenarios and a fourfold increase in multi-company scenarios, from 20K to 80K. This performance boost has ensured that the system remains stable even under heavy loads.

Future Plans

As Moka BI continues to utilize TiDB, the company increasingly recognizes its alignment with its business scenarios. Looking ahead, Moka BI has established the following plans for TiDB in 2023:

1. Unified Data Storage: Moka BI aims to achieve unified data storage by integrating TiSpark and TiKV to replace existing systems, including Hive, HBase, and offline data warehouses. This transition will streamline their data management processes.
2. HTAP Capabilities Exploration: The company will conduct in-depth research into TiFlash to leverage TiDB’s HTAP capabilities. This will enable Moka BI to support unified OLTP and OLAP query scenarios, simplifying their data architecture and reducing the complexity of data governance.
3. Company-wide Unified Query Platform: Based on the unifications above, Moka BI plans to establish a unified query platform that will support complex queries across various business departments. This platform will enhance their ability to consistently provide robust data output capabilities.

Conclusion 

TiDB has empowered Moka BI to optimize its data infrastructure, enhancing operational efficiency and driving informed decision-making. If you want to improve your data strategy and achieve similar results for your business, we encourage you to reach out. Our team is ready to discuss how TiDB can transform your operations and support your growth ambitions. Let’s unlock the full potential of your data together!