AI & ML
CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais
arXiv cs.SD0 views
arXiv:2611.17573v1 Announce Type: cross Abstract: Cameroon Pidgin English (a West African Pidgin English variety) and Camfranglais are widely used in Cameroon, yet to our knowledge, neither has a dedicated spoken Natural Language Inference (NLI) resource. We introduce CamSpokenNLI, a pilot dataset of 114 scenario-matched spoken NLI items in each variety (228 in total), recorded by one native speaker from Buea. Each item pairs premise audio and text with a hypothesis, an NLI label, and English glosses. The author assigned labels, so the dataset is an exploratory resource rather than a mature benchmark. The NLI experiments use the reference text; the audio is evaluated separately. XLM-RoBERTa baselines with five-fold item-level cross-validation reach 44.7% on Pidgin and 46.2% on Camfranglais (chance 33.3%). Still, neither is statistically distinguishable from hypothesis-only input, so hypothesis wording may account for much of the accuracy. Under disjoint-item cross-variety transfer, where a test item's counterpart is never seen in training, accuracy is 40.1% (Pidgin to Camfranglais) and 45.6% (Camfranglais to Pidgin). A counterpart-leak audit raises this to 60-64%, illustrating how counterpart leakage can overstate transfer. Zero-shot Whisper transcription of the recordings performs poorly: the best word error rates are 66.6% for Pidgin and 45.5% for Camfranglais, and with automatic language identification, Whisper Large-v3 labels 93 of 114 Pidgin recordings as Yoruba or Swahili. Together, the results suggest that such resources need disjoint-item evaluation and hypothesis-only controls, and that speech recognition must be adapted to these varieties before spoken NLI can be tested end to end.
Source:https://arxiv.org/abs/2611.17573PreviouseDIAR: Training-Free Open-Speaker Streaming Diarization for Long Meetings and Large Archives via e-meansNextActivation Steering the Nan-Shan in Typhoon
Related posts
LVMH’s Media Division announces strategic partnership with OpenAI
0OpenAI Blog
Bharat Semi Raises $24M for AI Sensors That Think Where They See
0dev.to
Android 热更新实战:XopHotFix SDK 集成与补丁生成、发布完整教程
0dev.to

WeeklyReview#202646
0dev.to
evgmrf: An R Package For Gaussian Markov Random Field Extreme Value Models
0arXiv stat.CO
Limitations of nonreversible lifts for subgeometric diffusions
0arXiv stat.CO