Music Customer Tag Management System Architecture & Design
——For Music Streaming Applications (Deliverable Solution)
- Background & Business Pain Points Music streaming apps accumulate massive user behavior data: song playback, search, collection, playlist creation, sharing, subscription payment, in-app interaction, etc. Most platforms face the following common problems: 1.Identity fragmentation: Users log in across mobile APP, in-car audio, smart speakers, WeChat mini-programs; anonymous users convert to registered users, leading to scattered user profiles. 2.Unstandardized tags: Different teams (recommendation, operation, membership) define tags independently, causing inconsistent metrics (e.g., "active user" has multiple definitions). 3.Imbalance between real-time & offline computing: Pure offline tags cannot support real-time recommendation & instant marketing; pure real-time tags lack global accuracy for large-scale crowd segmentation. 4.Weak tag governance: No version management, data lineage tracing, lifecycle expiration or access permission control, leading to redundant tags and data compliance risks. 5.Difficult business integration: Tags cannot be directly invoked by recommendation engines, push systems, marketing tools, resulting in low business value conversion. This solution designs a closed-loop tag management system to solve the above pain points.
- Core Architecture Philosophy 2.1 Unified Music Identity (MID) Construct a globally unique Music ID (MID) as the master user identifier. Map device ID, mobile number, third-party OAuth ID (WeChat/Apple/Google), anonymous session ID to one MID. Support anonymous user → registered user identity merging Link user behavior across all terminals (mobile, car, smart device, PC) MID serves as the only association key for all tag data 2.2 Full-Cycle Tag Metadata Governance Every tag must be pre-defined with complete metadata before going online: Metadata Item Description Example Tag ID & Name Unique identifier & readable name tag_001_music_taste_pop Business Purpose Which business scenario it serves Personalized recommendation / Membership churn intervention Tag Category Factual / Statistical / Rule-based / Model Statistical tag Calculation Logic SQL / Flink rule / ML model inference logic Count play times of pop songs in last 30 days Update Mode Offline full / Real-time incremental / Hybrid Hybrid Update Frequency Daily / Hourly / Second-level Daily offline + real-time incremental Data Owner Team responsible for the tag Recommendation Algorithm Team Effective & Expiration Time Tag lifecycle Valid from 2026-11-01 to 2027-11-01 Retention Period How long historical snapshots are kept 180 days Data Lineage Source tables & upstream dependencies User play log table ods_user_play_log Permission Level Public / Internal / Sensitive (PII) Sensitive (requires approval)
2.3 Hybrid Computing Model: Offline Full Recomputation + Real-time Incremental Update
Offline pipeline: Generates stable baseline tags via batch computing. It is the source of truth for tag data. Used for large-scale audience segmentation, tag quality verification, historical trend analysis, and ML model training.
Real-time pipeline: Processes user behavior events with low latency, updates lightweight tags incrementally, and overwrites the offline baseline for online query.
Reconciliation mechanism: Periodically compare offline baseline and real-time incremental tag values, repair inconsistent data automatically.
Tag versioning: Each tag calculation generates a version number; downstream services can specify a tag version to ensure result reproducibility.
2.4 Four-Dimensional Tag Classification (Mapped to Music Business)
Tag Type Definition Music App Typical Examples Computing Mode
Factual Tags Objective attributes directly collected from source data, no complex calculation Gender, age, city, registration channel, device model, membership level (free/paid/VIP), subscription expiry date, registration time Real-time / Near real-time
Statistical Tags Aggregated metrics derived from user behavior over a sliding time window 30-day play count, average daily listening duration, top 3 favorite genres, favorite artists, playlist creation times, search frequency, weekly new song playback ratio Offline daily + real-time incremental
Rule-based Tags Tags defined by explicit business rules combining multiple conditions High-value potential user (VIP expired within 30 days + 30-day play >100 times + no re-purchase), Churn-risk user (no login in 7 days + listening duration dropped by 50%), Night owl listener (play peak 23:00–02:00), Indie music enthusiast Offline full / Real-time rule engine
Model Tags Tags generated by machine learning models Music taste embedding vector, churn probability score, price sensitivity level, genre preference score, ad tolerance level, potential paid conversion probability Offline batch inference / Online model serving
- Layered System Architecture The system adopts a five-layer architecture, from bottom to top: Layer 1: Data Source Layer Collect raw data from all touchpoints of the music app: User behavior logs: Play, pause, skip, search, collect, download, share, comment, playlist create/delete, radio tune-in User profile data: Registration info, bound phone, profile settings, minor mode status Transaction data: Subscription order, in-app purchase, coupon usage, refund records Content metadata: Song, artist, album, genre, language, release year, tag of songs Context data: Login time, geo location (optional, privacy-compliant), device type, network type, in-car / smart speaker terminal mark Data is collected via SDK, server logs, and binlog, then sent to Kafka as the unified data bus. Layer 2: Data Integration & Identity Resolution Layer Data Cleaning: Deduplication, invalid data filtering, format normalization MID Mapping Service: Map device ID / anonymous ID / third-party ID to unified MID; handle ID merging when anonymous users register Data Standardization: Unify behavior event formats, align dimension data (song ID → genre / artist) Layer 3: Tag Computing Layer Offline Computing Engine: Spark / Hive, executes full tag recalculation on a daily/weekly schedule Real-time Computing Engine: Flink, consumes Kafka behavior events, updates tags in near real-time Rule Engine: Executes rule-based tag logic, supports dynamic rule configuration without code release ML Model Service: Loads trained music taste / churn prediction models, executes batch & online inference Tag Version Controller: Manages tag version generation, snapshot creation, version rollback Data Quality Checker: Verifies tag coverage, null rate, outlier detection before tag release Layer 4: Tag Storage & Index Layer Storage Component Stored Content Purpose HBase / ClickHouse Detailed tag values for each MID (latest version + historical versions) Online point query & batch query Elasticsearch Inverted index of tag values Fast audience segmentation (filter users by tag conditions) MySQL / PostgreSQL Tag metadata, tag version info, data lineage, permission config Management console query HDFS Tag snapshots, historical backup, audit logs Disaster recovery & compliance audit Redis Hot user tag cache Low-latency real-time query (P99 < 20ms)
Layer 5: Service & Application Layer
Tag Query API: get_user_tags (single user), batch_get_user_tags (batch users), get_tag_metadata
Audience Segmentation Engine: Visual rule builder, supports AND/OR/NOT conditions, segment estimation, segment export, scheduled refresh
Tag Management Console: Web UI for tag creation, editing, version management, lifecycle configuration, data quality monitoring, usage statistics
Permission & Audit Service: RBAC access control, sensitive tag approval flow, full operation audit log
Data Export Service: Export audience ID lists to recommendation engine, push notification system, CRM, marketing platform
- End-to-End Data Pipeline Design 4.1 Offline Full Recomputation Pipeline (Daily Baseline) 1.Extract raw data from data warehouse (user logs, order data, content metadata) 2.Execute MID identity resolution to unify user IDs 3.Run Spark jobs to calculate statistical / rule-based / model tags 4.Data quality check: coverage rate > 95%, null value rate < 1%, outlier filtering 5.Write tag results to HBase & Elasticsearch, generate daily snapshot to HDFS 6.Tag version is marked as baseline, open to downstream services 4.2 Real-time Incremental Update Pipeline (Low Latency) 1.User behavior events (play, collect, subscribe) are sent to Kafka in real time 2.Flink job consumes events, updates incremental tag values (e.g., real-time play count, latest favorite genre) 3.Write updated tags to Redis (hot cache) and HBase 4.Overlay incremental results on the offline baseline for online query 5.Send change events to notify downstream recommendation & push systems 4.3 Hybrid Synchronization & Reconciliation Every 1 hour, run reconciliation tasks to compare offline baseline tags and real-time incremental tags If inconsistency exceeds threshold, trigger automatic repair (use offline baseline as priority) Keep full trace of tag change history for audit
- Business Scenario Implementation Details Scenario 1: Personalized Music Recommendation Required tags: Statistical tags (top genres, favorite artists), Model tags (music taste vector) Workflow: Recommendation engine calls get_user_tags API to fetch user tags → Generate daily recommendation playlist, radio station tuning, "similar to your favorites" list → User behavior feeds back to real-time pipeline, updates tags instantly → Recommendation results adjust dynamically Value: Improve recommendation CTR, increase user listening duration Scenario 2: Precision Marketing & Campaign Operations Required tags: Factual tags (city, device), Rule-based tags (high-value potential, churn-risk), Statistical tags (activity level) Workflow: Operators use console to build audience segment (e.g., pop music lover + free user + 30-day play >60min) → Segment engine filters MID list → Push system sends targeted in-app message / coupon → Tag system tracks campaign exposure & conversion, evaluates tag effectiveness Value: Improve marketing conversion rate, reduce invalid push annoyance Scenario 3: Membership Lifecycle Management Required tags: Factual tags (VIP level, expiry date), Model tags (churn probability), Rule-based tags (renewal-intent user) Workflow: Identify users with high churn probability via model tags → Trigger real-time intervention (discount coupon, exclusive playlist) → Track whether user renews subscription → Feed renewal result back to model for retraining Value: Increase VIP renewal rate, reduce membership churn Scenario 4: Content Curation & Music Supply Required tags: Statistical tags (genre preference distribution), Model tags (music taste cluster) Workflow: Content team queries audience segments (e.g., indie music enthusiasts) → Analyze segment size, listening habits → Create exclusive playlists / promote emerging artists → Recommendation engine pushes content to target segments → Measure playlist adoption rate via tag system Value: Optimize content supply, improve long-tail music exposure Scenario 5: Risk Control & Compliance Required tags: Factual tags (minor status, device), Rule-based tags (abnormal playback behavior) Workflow: Detect abnormal behaviors (e.g., batch skipping, machine playback) via rule tags → Trigger risk control interception → Minor protection mode automatically enabled based on age tag → Support user data deletion request (cascade delete all tags under MID for GDPR/CCPA compliance) Value: Reduce fraudulent traffic, meet data privacy regulations
- Permission, Security & Compliance Design 6.1 RBAC Permission Model Role Permission Scope Tag Administrator Create/edit/delete tags, manage metadata, configure lifecycle, assign permissions Tag Developer Develop tag calculation logic, debug pipelines, view data lineage Operations User Use tags for audience segmentation, export audience lists, view tag values (non-sensitive only) Auditor Read-only access to all tags, view audit logs, export compliance reports
6.2 Data Security
Encryption: TLS for data in transit; AES-256 for data at rest; field-level encryption for PII tags (age, phone, location)
Sensitive Tag Control: Access to sensitive tags requires secondary approval; access records are saved in audit log
Anonymization: For non-essential scenarios, use anonymized MID instead of raw user identifiers
6.3 Compliance (GDPR / CCPA / Local PIPL)
Support right to be forgotten: When user requests account deletion, cascade delete all tag data and historical snapshots of the corresponding MID
Tag retention period is enforced automatically; expired tags are purged without manual intervention
Full audit log records all tag query, modification, export, deletion operations (operator, timestamp, target MID scope)
- Capacity Planning & Disaster Recovery Storage Capacity: Estimated based on total MID count × average tag count per user × historical snapshot retention days; needs stress test verification Computing Resource: Offline Spark cluster resources are allocated according to daily tag recalculation data volume; Flink real-time cluster is scaled based on Kafka event QPS Performance Baseline: Single-user tag query P99 latency, batch query throughput, audience segmentation response time — all need to be verified via stress testing Disaster Recovery: ¡Offline tag baseline: RPO < 24 hours, RTO < 4 hours (batch job rerun) ¡Real-time tag service: Multi-replica deployment, automatic failover, RTO < 30 seconds ¡Regular snapshot backup to HDFS for disaster recovery
Phased Implementation Roadmap
Phase Duration Key Deliverables
Phase 1: Foundation Month 1–2 MID unified identity system, core factual & statistical tags, offline computing pipeline, basic tag management console
Phase 2: Real-time & Business Integration Month 3–4 Real-time incremental pipeline, rule-based tag engine, audience segmentation service, integration with recommendation & push systems
Phase 3: Intelligence & Governance Month 5–6 ML model tag service, full metadata governance, data quality monitoring, compliance & security hardening, disaster recovery deployment
Phase 4: Optimization & Scaling Ongoing Performance tuning, cost optimization, new scenario expansion, automated tag lifecycle managementAcceptance Metrics
Metric Category Key Metrics
Data Quality Tag coverage rate, tag accuracy rate, offline-real-time consistency rate, null value rate, outlier rate
System Performance Single-user tag query P99 latency, batch query QPS, audience segment generation time, real-time tag update latency
Business Value Recommendation CTR improvement, marketing conversion rate lift, VIP renewal rate increase, operational efficiency gain (manual segmentation time reduction)
Compliance & Security Audit log completeness, data deletion response time, encryption coverage rate, compliance audit pass rateConclusion
This Music Customer Tag Management System solves the core pain points of identity fragmentation, tag inconsistency and computing timeliness for music streaming apps. By combining Unified MID, full tag governance, hybrid offline & real-time computing, and scenario-driven tag classification, it provides a reliable, scalable and compliant foundation for personalized recommendation, precision operation, membership management, content curation and risk control.
All capacity, performance and disaster recovery indicators must be finalized through real business scale assessment and stress testing before production launch.
