Tencent Cloud Big Data-WeData: An Integrated Data + AI Intelligent Platform

Introduction: An Integrated Solution to Address Challenges in Digital Transformation

SHENZHEN, China, April 28, 2025 /PRNewswire/ -- In the era where LLM driven intelligence prevails, enterprise data platforms face emerging challenges from LLM application and AI development. To name a few, efficiency loss owing to segmented AI/Data development process, data governance for heterogeneous data, traditional architecture that struggles to meet the needs for real-time data and rapid AI integration. Tencent Cloud WeData addresses these complexities through flexible architecture and product innovation, presenting an integrated data intelligence platform for enterprises to navigate these challenges and prosper in the age of AI.

4 Unique Aspects of the Integrated Data+AI Solution

• Unified Storage Layer: A unified Lakehouse storage system with a centralized catalog service that supports structured/unstructured data and model assets, enablingcross-engine queries .
• Unified Compute Layer: A serverless resource pool to integrate frameworks like Spark, Flink, and PyTorch, improving CPU/GPU resource utilization by 30%.
• Integrated Streaming-Batch-Lakehouse Engine: The Setats engine dynamically switches between streaming, batch, and incremental compute modes, merging real-time states with lake storage to reduce storage costs by 50%.

• Unified Catalog: A centralized catalog to manage structured/unstructured data, enabling seamless access across compute engines (Spark, Flink) and AI frameworks (TensorFlow, PyTorch).
• Global Asset View: Integration of metadata for tables, volumes, models, and applications, supporting semantic search across all assets.
• Federated Permissions: One single authorization to grant data access across engines like Spark and Ti-One, reducing policy propagation time from hours to minutes.
• Intelligent Semantic Enhancement: Leverages LLMs to auto-generate data description, perform intelligent asset classification and maintenance, and improve documentation completeness by 80%.

Built on the OneOps philosophy, our solution combines DataOps and AIOps to deliver three key capabilities:

• Full-Lifecycle Standardization: Automates enterprise CI/CD pipelines for end-to-end management of data and models.
• End-to-End Observability: Automatically captures lineage from raw inputs to AI model outputs, enabling rapid root-cause analysis, change impact assessment, real-time monitoring, and audit logging.
• Governance-by-Design: Embeds standardized development practices, AI-oriented data modeling, and enterprise semantic layers to systematically accumulate digital assets.

Ensures security across the entire lifecycle—data preparation, model training, and LLM applications:

• Data Security: Global access control, intelligent data classification/categorization, sensitive data masking/encryption, and audit trails.
• Model Security: Detects and defends against adversarial sample attacks during training; embeds watermarks to prevent model theft; blocks discriminatory/harmful AI-generated content during inference via rule engines.
• Application Security: API protection (e.g., unauthorized access prevention) and content moderation (e.g., forgery detection, sensitive content filtering).

#Tencent Cloud Big Data #WeData