AI/ML News & Innovations Hub

AI/ML news, top picks, and generated innovation digests.

★ Visit ai-karthik.com
426Sources
86241News Items
8Top Picks
428Blogs
successLast Run

NLP

200 articles tagged with this keyword, sorted by most recent first.

← All Keywords
LessWrong AI 2026-11-08 21:18 UTC Score 77.0 USR-0152-20261108-community-fo-c9e869de

We asked 100 AI researchers at EMNLP whether AI could be conscious, how they define consciousness, and how they engage with it

Before reading this post, please take 30 seconds to answer the 3-question survey yourself: [ survey link ] Introduction The possibility of AI consciousness, the use of AI for companionship or emotional support, and the trust we place in AI as coding tools raise personal interrogatives for most AI researchers, but these topics are often relegated to informal conversations. Apart from the few who write about it, it’s hard to get a sense of how much AI has permeated researchers’ personal lives, and how they feel about it. To gain a better understanding of this, we ran a quick, informal three-question survey, plus an optional fourth, open-ended question. We spoke to 107 EMNLP 2026 attendees in Budapest. 79 respondents were randomly-met researchers, collected during the opening reception and during the conference breaks. We also asked the same questions to 28 people working at industry booths, mostly recruiters and software engineers. The two groups gave quite different answers, with the industry group being way more willing to entertain the possibility of AI consciousness. Additionally, we also asked a variation of the same questions to 21 LLMs, including both self-hosted and closed source models. The post below presents the results of the survey, our recall of some of the conversations, and a lot of our personal views and ideas. We do not claim that this study has any statistical significance, and readily admit many methodological mistakes that confound many of the investigated…

Read article →
ACL Anthology 2026-11-02 00:00 UTC Score 7.0 AI-079-20261102-research-pap-f3cc39fe

Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Ningyu Zhang and Derry Wijaya in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Read article →
ACL Anthology 2026-11-02 00:00 UTC Score 10.0 AI-079-20261102-research-pap-ae249a18

Pollux: A Compact Judge and Workflow for Criteria-Based LLM Evaluation

Ulyana Isaeva, Maslennikov Egor, Alisa Glushko, Nikita Martynov, Anastasia Mordasheva, Gorbetskiy Dmitriy, Elina Basyrova, Sergey Skachkov, Berestova Victoria, Alena Fenogenova and Danil Astafurov in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Read article →
ACL Anthology 2026-11-02 00:00 UTC Score 18.0 AI-079-20261102-research-pap-07c46984

Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing

Lecheng Yan, Yichong Zhang, Xiantao Xu, Jianze Lin, Ben Pan, Xiaoyu Zheng, Jiawei Qian, Anqi Wu, Jiahui Geng, Ruizhe Li, Fengyu Cai, Jingcheng Niu, Raymond Li, Wenxi Li and Chenyang Lyu in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Read article →
ACL Anthology 2026-11-02 00:00 UTC Score 10.0 AI-079-20261102-research-pap-8b4dda2d

Cinematic Compiler: A Typed Intermediate Representation with a Deterministic Pre-Generation Constraint Checker for Multi-Shot Keyframe Continuity

Sai Rama Akash Varma Sagi, Karthik Vemireddy and Sai Ganesh Akula in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Read article →
ACL Anthology 2026-11-02 00:00 UTC Score 12.0 AI-079-20261102-research-pap-09fd76c1

Aura 💫: A Self-Hostable Platform for Human Evaluation of NLG for Under-resourced Languages and Beyond

Rudali Huidrom, Khangembam Alex D Nelson, Nicholas Khumukcham, Hoomexsun Pangsatabam, Jesu Nazarene Roy L, Kanal Trivedi, Vikas Kumar, Pinaki Das, Kshetrimayum Boynao Singh, Tarun Bisht, Justice Konjengbam, Konjengbam Anand and Vikram Goyal in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Read article →
ACL Anthology 2026-11-02 00:00 UTC Score 24.0 AI-079-20261102-research-pap-f475adfd

AI Tour Meeting: Group Travel Planning by LLM Agents

Daisuke Kikuta in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations

Read article →
ACL Anthology 2026-10-31 00:00 UTC Score 12.0 AI-079-20261031-research-pap-d33bdcb3

A Geometric Perspective on Stabilizing Value Conflict Resolution

Saket Reddy and Andy Liu in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)

Read article →
ACL Anthology 2026-10-31 00:00 UTC Score 26.0 AI-079-20261031-research-pap-4632e835

A Benchmark for Controllable Speaking-Style Captioning in Audio-Language Models

Shreeram Suresh Chandra, Ashwyn Sharma, Liang Lu, Xiutian Zhao, Leda Sari, Björn Schuller and Berrak Sisman in Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)

Read article →
SiliconANGLE AI 2026-10-30 23:59 UTC Score 48.0 USR-0127-20261030-global-ai-ne-a35d6d56

Google X co-founder Sebastian Thrun raises $130M for AI engineering startup Dulo

Dulo Inc., a startup co-founded by prominent artificial intelligence researcher Sebastian Thrun, has raised $130 million in funding. The Information broke the news of the investment on Thursday. Dulo co-founder Adrian Aoun confirmed the report in a post on X. Before launching Dulo with Thrun (pictured) earlier this year, Aoun sold an early natural language processing […] The post Google X co-founder Sebastian Thrun raises $130M for AI engineering startup Dulo appeared first on SiliconANGLE .

Read article →
Synced 2026-10-05 10:08 UTC Score 37.0 AI-041-20261005-ai-specialis-39b7560a

Comment on Natural Language Processing In Early Education by when is christam

Topic in a simple and easy--understand way. I also recently published a related resource that your readers might find . If you have a chance, I'd appreciate it if you could take a look and consider adding it as a reference if you think it adds value. Thanks for sharing such quality content, and keep up the great work! When Is Christmas

Read article →
Synced 2026-10-02 08:20 UTC Score 62.0 AI-041-20261002-ai-specialis-b27a26cc

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by Taylor

This is a fascinating look at how Microsoft is pushing the boundaries of transformer training efficiency with their Fully Pipelined Distributed Transformer. The focus on maximizing hardware efficiency and MFU is exactly the kind of innovation that keeps LLM research moving forward. If you're also interested in practical tools for working with text and models, my site Font Generator has some useful resources like a change text font tool that might come in handy.

Read article →
Synced 2026-10-02 08:18 UTC Score 57.0 AI-041-20261002-ai-specialis-1acbfefb

Comment on CMU’s Novel ‘ReStructured Pre-training’ NLP Approach Scores 40 Points Above Student Average on a Standard English Exam by Taylor

It's impressive to see CMU's RST approach outperforming student averages on the Gaokao-English Exam with such a compact model. The idea of restructuring data for pretraining really highlights how smarter data use can beat brute-force scale. By the way, if you're into language and text tools, my site Font Generator also has some handy related content like a font changer online that you might find useful.

Read article →
RIKEN AIP News 2026-09-30 01:57 UTC Score 43.0 USR-0043-20260930-research-aca-98aaf953

9 Papers Accepted to the Main Conference of EMNLP 2026

Papers involving researchers from the RIKEN Center for Advanced Intelligence Project (AIP) have been accepted to EMNLP 2026 (Conference on Empirical Methods in Natural Language Processing), one of the leading international conferences in natural l

Read article →
Korea AI Times 2026-09-29 05:31 UTC Score 43.0 USR-0048-20260929-global-ai-ne-c3332e8c

"STT 오류 복원부터 추론 효율화까지"...롯데이노베이트·아크릴, EMNLP 논문 채택

롯데이노베이트와 아크릴 등 국내 AI 전문 기업의 연구 논문이 자연어처리 국제학술대회 ‘EMNLP 2026’에 채택됐다. 롯데이노베이트는 음성인식(STT), 아크릴은 추론 효율에 초점을 맞췄다.29일 롯데이노베이트의 발표에 따르면, 채택 논문 제목은 ‘회의 요약 과정에서 사용자 맞춤형 용어집을 활용해 음성인식 오류 전문용어를 복원하는 조건과 요약 품질 평가 연구’다. 회의 요약 과정에서 사용자나 회의 맥락에 맞춘 용어집을 활용해 음성인식 오류로 훼손된 도메인 전문용어를 복원하는 방안을 다루고 있다.기업 회의에는 산업·직무별 전문 용

Read article →
Korea AI Times 2026-09-28 05:39 UTC Score 44.0 USR-0048-20260928-global-ai-ne-3d1bf71c

셀렉트스타, EMNLP서 AI 신뢰성 연구 5편 채택

셀렉트스타(대표 김세엽)는 AI 신뢰성·안전성 관련 논문 5편이 글로벌 자연어처리(NLP) 학회 \'EMNLP 2026\'에 채택됐다고 28일 밝혔다.채택 논문은 메인 컨퍼런스 1편, 파인딩스(Findings) 1편, 워크숍 3편이다. 특히 올해 메인 컨퍼런스에 제출된 1만 8000편의 논문 중 15.4%만 채택됐다는 점에서 자체 기술력을 인정받았다는 설명이다.메인 컨퍼런스에 채택된 \'정확도 일치와 불일치 관점에서의 인간–LLM 응답 정렬 진단: 대규모 집단 수준 분석\'은 AI가 정답을 맞히는지를 넘어 틀린 경우에도 사람과 유사한 오답

Read article →
Synced 2026-09-23 16:56 UTC Score 51.0 AI-041-20260923-ai-specialis-e16ea677

Comment on Say Woof? AI in Animal Language Translation by MegaTool

This is a fascinating look at how far animal language translation has come since those early novelty translators. The Zoolingua and Conservation Metrics projects really show how deep learning can bridge the gap between species, whether it's helping pet owners understand their dogs or protecting elephants from poachers. I've been following similar AI translation and NLP developments over at MegaTool , and it's exciting to see how these technologies are evolving beyond human language to decode the natural world around us. The chicken emotion detection study is particularly clever. Thanks for the great read!

Read article →
Towards Data Science 2026-09-23 14:00 UTC Score 22.0 AI-036-20260923-ai-specialis-c9f59ebc

From Words to Vectors: What Happens in Between?

A Journey through TF-IDF, vector space, and text classification The post From Words to Vectors: What Happens in Between? appeared first on Towards Data Science .

Read article →
Synced 2026-09-14 10:54 UTC Score 65.0 AI-041-20260914-ai-specialis-01425892

Comment on Google ‘mT5’ Pretrained Text-to-Text Transformer Achieves SOTA Performance on Multilingual Benchmarks by Kate

Interesting to see how mT5 is pushing multilingual NLP forward, especially with support for so many languages. I’ve been using TextNow lately, so it’s cool to think about how better multilingual models could eventually make communication across different languages feel more seamless.

Read article →
Data Science Stack Exchange 2026-09-14 00:41 UTC Score 13.0 AI-111-20260914-social-media-a89ecaea

Can the scanned fly brain do NLP?

recently a fly brain was scanned and people have took to a million creative uses of it. Is there a fundamental reason that it wouldn’t be capable of being trained to do NLP?

Read article →
Data Science Stack Exchange 2026-09-13 20:13 UTC Score 36.0 AI-111-20260913-social-media-22e87e92

Is building a manual evaluation set the right approach when no reliable labeled ground truth exists for resume-job matching?

I'm building a resume screening/ranking system (matching resumes to job descriptions using pretrained sentence embeddings + cosine similarity, no fine-tuning at this stage) as a learning project aimed at becoming a market-ready NLP practitioner. Problem: I could not find a trustworthy, publicly available English dataset with genuine human-labeled resume-job match scores. I checked several Kaggle/HuggingFace options and found labels that were either AI-generated (e.g., GPT-4o) or fully synthetic with demographic columns (race/ethnicity/gender) tied to the match label, which raised bias concerns. Academic literature (ConFit, PJFNN papers) confirms that the only broadly public dataset for this exact task (Person-Job Fit) is the 2019 Alibaba matching competition dataset, which is Chinese-only; most published research instead uses private company-provided data. My current approach: Use two real (non-synthetic) datasets: a scraped resume corpus and a real LinkedIn job postings corpus, both verified for low duplication and cleaned of PII. Build a small manual evaluation set myself (~24 resume-job pairs, selected to cover clear matches, clear non-matches, and ambiguous cases), scoring them on a 0-3 relevance scale with a confidence flag. Use this manual set as ground truth to compute ranking metrics (Precision@K, MRR, NDCG) once the embedding-based matching pipeline is built. Question: Is this a sound methodology given the lack of reliable public ground truth, or is there a better-e…

Read article →
Data Science Stack Exchange 2026-09-12 11:21 UTC Score 18.0 AI-111-20260912-social-media-b1276bf5

Algorithms to cluster documents with similar boiler-plate inclusions

Suppose I have a large corpus of text documents (say 100-million - in a common human language - e.g. English). I suspect that various subsets of these documents are 'similar'... by this, I mean they appear to have been created using common templates (no-longer available to me - except by inference from the corpus) containing substantial boilerplate. After applying standard NLP tokenisation to 'words' - I define the common-runs of a pair of documents as the longest sequences of words found in both documents. I define valid-common-runs as the longest sequence of common-runs that occur in the same order in both documents. Hence if "Abracadabra Systems" was (only) present at the end of one document and the beginning of another, it would be a common-run... but it would only be a valid-common-run if there were no other (distinct) common-runs in the documents. My ultimate objective is to cluster the documents using valid-common-runs (perhaps weighted by a simple function ranging over the valid-common-run text). My question is about efficient ways to compute the 'valid-common-runs' and find groups of similar documents. A naive approach to finding common-runs might be to consider every pair of documents in turn... but 100-million documents would require 10-quadrillion document comparisons... which seems prohibitively resource intensive. I would like to know (before I try to re-invent wheels) are there any well-established efficient algorithms to determine anything similar to 'valid-c…

Read article →
Synced 2026-09-09 11:43 UTC Score 40.0 AI-041-20260909-ai-specialis-14ec1599

Comment on AI Analyzes Chess Commentary to Learn to Play Chess by Grace

SentiMATE presents an interesting way of combining natural language processing with chess strategy. Using human commentary to evaluate move quality shows how AI can learn from language and improve decision-making without relying entirely on traditional chess features. Regulatory awareness is important before joining any online game , since rules vary by region. Players should confirm local eligibility before depositing funds.

Read article →
iAfrica 2026-09-08 08:23 UTC Score 57.0 AI-151-20260908-regional-ai--0b3f922c

Tether Releases Open Offline Translation Models for 19 African Languages, With Peer-Reviewed Benchmarks

Tether AI Research has released open-source translation models covering 19 African languages that run entirely on smartphones and laptops without an internet connection — and, unusually for a corporate AI announcement, the underlying research has been accepted for presentation at EMNLP 2026, the leading peer-reviewed conference in natural language processing. The release comprises QVAC TranslatePsy-AfriSLM, [...]

Read article →
Towards Data Science 2026-08-29 13:00 UTC Score 36.0 AI-036-20260829-ai-specialis-6912748d

RAG Is Not the Whole Toolkit: The NLP Techniques Real Problems Still Need

Enterprise Document Intelligence [Vol.1 #B00] - Retrieval answers one kind of question. Classifying a request, matching free text to a reference list, reading a table, cleaning OCR noise: each has a cheaper method that works, and the engineering is knowing which one to reach for The post RAG Is Not the Whole Toolkit: The NLP Techniques Real Problems Still Need appeared first on Towards Data Science .

Read article →
Korea AI Times 2026-08-28 07:55 UTC Score 46.0 USR-0048-20260828-global-ai-ne-c2085060

"AI 양자화 정확하고 안전하게"...옵트에이아이, EMNLP 논문 2편 채택

옵트에이아이(대표 이재호)는 논문 2편이 자연어처리(NLP) 국제학술대회 \'EMNLP 2026\'의 메인 컨퍼런스와 인더스트리 트랙에 채택됐다고 28일 밝혔다.올해 행사에는 역대 최다인 1만7669편의 논문이 제출, 메인 컨퍼런스와 파인딩스 채택률은 각각 15.4%, 14.3%를 기록했다. 옵트에이아이는 이번 채택으로 EMNLP, ACL 등 주요 NLP 학회에 3회 연속 논문이 선정됐다.채택된 논문은 ▲LLM 양자화의 신규 기준을 제시한 \'커널퀀트\' ▲온디바이스 LLM의 안전성 공백을 최초로 규명한 연구 등 2편이다.첫 번째 \'커널퀀

Read article →
Entrackr AI 2026-08-27 04:39 UTC Score 35.0 USR-0212-20260827-regional-new-011d1094

InstaAstro raises $12 Mn in Series A round

Gurugram-based digital spiritual wellness platform InstaAstro has raised $12 million in a Series A funding round led by Singularity AMC and Artha Venture Fund, with participation from the company’s founders. The round comes more than two years after InstaAstro raised Rs 18.5 crore ($2.2 million) in a pre-Series A round led by Artha Venture Fund. In November 2021, the company had raised Rs 3.2 crore in a seed round. The fresh capital will be used to expand InstaAstro’s presence in regional and international markets and invest in AI-led product development. The company plans to deploy AI for sentiment analysis and multilingual support to enhance user consultations. Founded in 2021 by Nitin Verma, InstaAstro started as an astrology consultation app and has since expanded into a broader spiritual wellness platform offering astrology, tarot, numerology, vastu, Pooja Seva and spiritual commerce. It currently has 12 million registered users and more than 5,000 verified experts across 183 countries. InstaAstro’s revenue more than doubled to Rs 111.26 crore in FY26 from Rs 52 crore in FY25. Its international business, driven largely by the Indian diaspora, contributes around 25% of its revenue. The company’s user base has also grown nearly 400% over the past two years. The company claims its annualised revenue has now crossed Rs 200 crore. InstaAstro will also expand its Pooja Seva and spiritual commerce businesses. Over the longer term, it aims to build an integrated platform coveri…

Read article →
AI Stack Exchange 2026-08-26 13:53 UTC Score 24.0 AI-110-20260826-social-media-ba1ece8a

Prompting McGill-NLP/AfriqueLlama-8B

McGill-NLP/AfriqueLlama-8B seems to hallucinate quite a bit. It chooses conversational outputs and goes beyond what I ask it to. I have the hyperparameters below. generated_ids = model.generate( **inputs, max_new_tokens=max_new_tokens, do_sample=False, num_beams=1, pad_token_id=tokenizer.pad_token_id) Does anyone have advice to have the model strictly follow instructions, without fine-tuning (other than including few-shot examples)?

Read article →
Synced 2026-08-22 20:05 UTC Score 59.0 AI-041-20260822-ai-specialis-60ccc23e

Comment on Google’s ALBERT Is a Leaner BERT; Achieves SOTA on 3 NLP Benchmarks by John Mitchell

I found this article interesting because it shows how ALBERT improved on BERT while using parameter-sharing and other techniques to reduce model size and memory demands. The results across GLUE, SQuAD 2.0, and RACE are especially useful for understanding how NLP models can become more efficient without sacrificing performance. It also made me think about how optimization and efficiency matter in many technology-driven services today. For anyone interested in practical service providers, https://abudhabimoversandpackers.com/ is another example of a business worth exploring when reliability and organized service are important.

Read article →
SiliconANGLE AI 2026-08-18 13:00 UTC Score 56.0 USR-0127-20260818-global-ai-ne-33828247

NLPatent rebrands as Clerq, launches agentic patent research workflows

Toronto patent research startup NLPatent rebranded as Clerq today and launched its first agentic workflows. The software runs a full patentability analysis in about 10 minutes. The company says every conclusion arrives with citations. That work normally takes days or weeks, and firms usually push it down to a junior associate or out to a search […] The post NLPatent rebrands as Clerq, launches agentic patent research workflows appeared first on SiliconANGLE .

Read article →
Cross Validated 2026-08-17 21:16 UTC Score 47.0 AI-113-20260817-social-media-e4b59918

How can I reduce subjectivity when selecting posts for social media comment analysis?

I’m working on a computational social science project using comments from Weibo and Douyin (Chinese TikTok) for text and sentiment analysis. My study looks at how online discourse around a female public figure changed before and after a particular event. I first define several keywords related to the person and the event, search for relevant posts/videos, and then collect comments from selected posts. The problem I’m struggling with is: how should I systematically decide which posts to select? Taking the first 10 or 20 search results does not seem reliable, because search results on these platforms are algorithmically ranked and change between searches. The posts ranked at the top are also not necessarily the ones with the most discussion. Selecting posts purely by comment count is also problematic. Some videos have thousands of comments, but many are just emojis, users tagging friends, repeated/copied comments, or spam. Meanwhile, another video with fewer comments may contain much more substantive discussion relevant to my research. So in practice, I inevitably have to make a human judgment about whether a comment section contains enough meaningful, relevant text to analyze. But this creates the methodological problem I’m most worried about: Why these particular posts? If I selected another set of relevant posts, would I still get the same result? Even if I eventually collect millions of comments, I feel that a large number of comments does not solve the problem if I cannot…

Read article →
CIO AI 2026-08-13 20:27 UTC Score 39.0 USR-0125-20260813-global-ai-ne-0b3668ac

Manual vs. AI-powered PDF redaction: protecting sensitive data in 2026

Research shows that humans play a role in 60% of breaches that expose sensitive data. That “role” often involves an employee falling for a phishing scam or using PASSWORD for their login credentials, but data exposure can also be a result of how your business redacts sensitive and personally identifiable information (PII) in your documents. Historically, manual, “black-box” redaction was considered best-practice, but this approach only obscures data, it doesn’t permanently remove it. As regulations governing data security get stricter and AI-powered redaction solutions become more accessible, organizations—especially those in highly regulated industries—are re-evaluating their PDF redaction solutions. How manual PDF redaction is different from AI-powered PDF redaction The primary difference between manual and AI-powered PDF redaction is who (or what) you rely on to do the heavy lifting. Manual redaction defined Manual redaction is a human-driven process where individuals visually scan text, select content, and apply black boxes or remove the text before sharing or storing the file. Manual redaction is only as effective as the reviewer, which makes outcomes highly variable, especially under time pressure or high document volume. AI-powered redaction defined AI-powered redaction uses machine learning and natural language processing (NLP) to automatically detect and remove sensitive information from documents. The system is trained to recognize patterns, language cues, and cont…

Read article →
ACL Anthology 2026-08-11 00:00 UTC Score 15.0 AI-079-20260811-research-pap-c39af0f6

AMI @ EVALITA2020: Automatic Misogyny Identification

Elisabetta Fersini, Debora Nozza and Paolo Rosso in Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020)

Read article →
ACL Anthology 2026-08-11 00:00 UTC Score 15.0 AI-079-20260811-research-pap-5af4404b

Automatic Expansion of Lexicons for Multilingual Misogyny Detection

Simona Frenda, Bilal Ghanem, Estefanía Guzmán-Falcón, Manuel Montes-y-Gómez and Luis Villaseñor-Pineda in Proceedings of the Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018)

Read article →
Synced 2026-08-08 09:39 UTC Score 40.0 AI-041-20260808-ai-specialis-9447d79f

Comment on Facebook Says Its ‘Blender’ Chatbot Is the Most Humanlike Ever by paramount books

Facebook’s Blender chatbot is an impressive step toward making AI conversations feel more natural and humanlike. As AI continues to improve, it could also transform how people discover educational resources, get recommendations, and interact with online stores such as Paramount Books . It will be interesting to see how this technology develops and shapes digital communication in the future.

Read article →
CIO AI 2026-08-07 09:30 UTC Score 50.0 USR-0125-20260807-global-ai-ne-6fbe4864

How AI is changing the business analyst role for the better

AI’s impact has been felt across nearly every industry, and its rise has already started to alter several roles in tech, including that of the business analyst . While the rise of agentic AI may have some questioning whether AI will replace business analyst jobs entirely, as we’ve seen with most roles impacted by AI, it’s more likely that AI will augment the role and fundamentally change how BA’s conduct daily business. “As AI takes on more routine tasks, the human side of the role is becoming even more valuable. It’s becoming more of a hybrid role, where employers are often looking for candidates who can combine technical fluency with strong communication and problem-solving skills, along with sound business judgment,” says Megan Slabinski, district president of technology talent solutions at Robert Half. AI can save business analysts time in the long run, automating many of the tasks that are time consuming and repetitive around data processing, note taking, and documentation. While automation will impact the daily tasks of the role, business analysts will still be necessary for properly interpreting outputs, collaborating across teams, and maintaining compliance and AI workflows. AI-driven analysis and automated workflows With AI-driven analysis, BA’s can use machine learning models for pattern detection, determining risk, and for forecasting demand, while natural language processing (NLP) can be used for text-heavy inputs. AI tools can also assist analysts with decision-…

Read article →
Apple Machine Learning Research 2026-08-07 00:00 UTC Score 49.0 AI-059-20260807-official-ai--3d79e525

Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models

Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs. While these models have achieved high accuracy across a range of downstream tasks, they exhibit low arithmetic intensity due to the inherent sequential dependency in next-token prediction. Recently, Diffusion Language Models (DLMs) have emerged as a promising…

Read article →
ACL Anthology 2026-07-31 00:00 UTC Score 13.0 AI-079-20260731-research-pap-82652bf0

CSULoRA: Closest Safe Update Low-Rank Adaptation

Oleksandr Marchenko, Adelaide Danilov, Aria Nourbakhsh and Salima Lamsiyah in Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security

Read article →
Synced 2026-07-29 14:17 UTC Score 37.0 AI-041-20260729-ai-specialis-a8632e0f

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by juego copero

Great article on FPDT! The memory hierarchy optimization is really impressive. As someone who runs a gaming site, I can appreciate efficiency improvements—smooth gameplay depends heavily on resource management. Check out https://juegocopero.com/ for more on that!

Read article →
LessWrong AI 2026-07-25 01:19 UTC Score 68.0 USR-0152-20260725-community-fo-03cc5338

Linear probes tell you where quantization will hurt

Epistemic status: I have only tested one encoder family (BERT-base and its relatives) and one decoder LLM ( Qwen2.5 -3B), one seed, token-level tasks, and post-training weight quantization. I trust the results because I am not inventing anything; I am just connecting quantization with a very general idea from mech interp: the model does the easy syntax work first, the semantic work in later layers, and the prompt-relevant task work in the latest layers. That's the main idea I lean on in this project. What I'm not sure of is whether it holds when the task is diffuse, like general-knowledge QA (probably not). TL;DR. First I train a linear probe on each layer and try to see where a signal lives in a model. I only probe for classical NLP or CV tasks, like checking for depth or vision for CV or named entity recognition, parts of speech, or chunking for NLP. Then using that information to create a map of where the important work is happening and quantizing the less important layers for a task. This method is cheap; it's only one RidgeClassifier per layer. And it kind of works; a map built once on CoNLL news keeps 99–100% of full-precision accuracy at a 5-bit average on three unseen datasets, whereas Uniform kept 16–41% and opposite/anti layers kept 2–41%. This method is perfect for sharp use cases where the transformer needs specialized knowledge and can compromise on general knowledge. The itch So this all started during my 2nd semester at Northeastern, where I took Applied Progr…

Read article →
Synced 2026-07-19 01:55 UTC Score 46.0 AI-041-20260719-ai-specialis-c7be6d26

Comment on Peking U & Microsoft’s Knowledge Attribution Method Enables Editing Factual Knowledge in Pretrained Transformers Without Fine-Tuning by Markdown to Doc

Impressive progress in editing factual knowledge without fine-tuning! The idea of targeting "knowledge neurons" is fascinating. For a deeper dive into transformer mechanics, check out this Markdown to Doc resource—it helps analyze such groundbreaking NLP innovations efficiently.

Read article →
Data Science Stack Exchange 2026-07-16 17:56 UTC Score 29.0 AI-111-20260716-social-media-49c70074

Improving short‑text classification accuracy with overlapping classes and imbalanced data

I’m working on a multi‑class text classification problem where the input consists of very short descriptions (often only a few words) and the goal is to predict the correct category. I’m currently using an XGBoost classifier for the final prediction layer. for embeddings I am using e5 large model. The main challenges I’m facing are: The descriptions are extremely short and many classes share similar vocabulary. Because of this, the model sometimes produces high‑confidence but incorrect predictions when common words appear across multiple classes. Description is the only feature that supports the category classification. The dataset is imbalanced: some classes have significantly more training samples than others, but the distribution of the prediction data is different. This causes the model to over‑predict certain classes. I experimented with TF‑IDF features, embeddings from a generative AI model and a combination of TF‑IDF + embeddings. However, combining both actually reduced accuracy. I also tried downsampling majority class but it not really make any difference. Current model accuracy is around 30%. I’m looking for advice on: How to improve classification when different classes share highly overlapping vocabulary. How to reduce high‑confidence wrong predictions. What techniques work well for class imbalance and distribution shift between training and prediction data.

Read article →
Machine Learning Mastery 2026-07-15 12:00 UTC Score 27.0 AI-039-20260715-ai-specialis-f38e483d

Scikit-Ollama for Scikit-LLM/Ollama Integration

In this article, you will learn how scikit-ollama bridges the scikit-learn interface with locally running Ollama models to perform zero-shot text classification; no cloud API...

Read article →
Synced 2026-07-11 04:11 UTC Score 40.0 AI-041-20260711-ai-specialis-71f860d6

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by taichiwalk.org

Fascinating to see how Microsoft is pushing hardware efficiency for longer sequence processing—definitely a leap for AI scalability. All that computational intensity makes me think about the other side of the coin: finding calm after a deep tech session. Lately I’ve been exploring tai chi walking as a low-impact way to reset focus and improve balance, especially since sitting at a desk for hours can take a toll. There’s a beginner-friendly guide at taichiwalk.org that offers free routines and even a guided coach—no login needed, which I appreciate. Anyone else here use movement or gentle exercise to counterbalance screen time?

Read article →
Synced 2026-07-07 07:10 UTC Score 51.0 AI-041-20260707-ai-specialis-3f6bf0e5

Comment on Interview with Tencent AI Lab- Four Core Research Fields:Computer Vision, Speech Recognition, Natural Language Processing and Machine Learning by Owen Parker

I've seen examples where clear tracking information greatly improves the user experience. If you're interested in how a modern tracking system presents shipment updates and delivery statuses, this guide is worth a look: https://anposttracking.org/ . It's a helpful reference for anyone building customer-focused applications

Read article →
Transactions on Machine Learning Research 2026-07-06 00:00 UTC Score 49.0 AI-084-20260706-research-pap-960a167b

Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selection

Attention mechanisms have revolutionized several domains of artificial intelligence, such as natural language processing and computer vision, by enabling models to selectively focus on relevant parts of the input data. While recent work has characterized the optimization dynamics of gradient descent (GD) in attention-based models and the structural properties of its preferred solutions, less is known about more general optimization algorithms such as mirror descent (MD). In this paper, we investigate the convergence properties and implicit biases of a family of MD algorithms tailored for softmax attention mechanisms, with the potential function chosen as the $p$-th power of the $\ell_p$-norm. Specifically, we show that these algorithms converge in direction to a generalized hard-margin SVM with an $\ell_p$-norm objective when applied to a classification problem using a softmax attention model. Notably, our theoretical results reveal that the convergence rate is comparable to that of traditional GD in simpler models, despite the highly nonlinear and nonconvex nature of the present problem. Additionally, we delve into the joint optimization dynamics of the key-query matrix and the decoder, establishing conditions under which this complex joint optimization converges to their respective hard-margin SVM solutions. Lastly, our numerical experiments on real data demonstrate that MD algorithms improve generalization over standard GD and excel in optimal token selection.

Read article →
Transactions on Machine Learning Research 2026-07-06 00:00 UTC Score 40.0 AI-084-20260706-research-pap-bedebbba

Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective

The Transformer model is widely used in various application areas of machine learning, such as natural language processing. This paper investigates the approximation of the Hölder continuous function class $\mathcal{H}_{Q}^{\beta}\left([0,1]^{d\times n},\mathbb{R}^{d\times n}\right)$ by Transformers and constructs several Transformers that can overcome the curse of dimensionality. These Transformers consist of one self-attention layer with one head and the softmax function as the activation function, along with several feedforward layers. For example, to achieve an approximation accuracy of $\epsilon$, if the activation functions of the feedforward layers in the Transformer are ReLU and floor, only $\mathcal{O}\left(\log\frac{1}{\epsilon}\right)$ layers of feedforward layers are needed, with widths of these layers not exceeding $\mathcal{O}\left(\frac{1}{\epsilon^{2/\beta}}\log\frac{1}{\epsilon}\right)$. If other activation functions are allowed in the feedforward layers, the width of the feedforward layers can be further reduced to a constant. These results demonstrate that Transformers have a strong expressive capability. The construction in this paper is based on the Kolmogorov-Arnold Superposition Theorem and does not require the concept of contextual mapping, hence our proof is more intuitively clear compared to previous Transformer approximation works. Additionally, the translation technique proposed in this paper helps to apply the previous approximation results of fe…

Read article →
Machine Learning Mastery 2026-06-16 12:00 UTC Score 27.0 AI-039-20260616-ai-specialis-e9483392

Building an End-to-End Sentiment Analysis Pipeline with Scikit-LLM

Traditional machine learning pipelines for predictive tasks like text classification usually rely on extracting structured, numerical features from raw text — for instance, TF-IDF frequencies or token embeddings — to feed into classical models such as logistic regression, ensembles, or support vector machines.

Read article →
Machine Learning Mastery 2026-06-11 12:00 UTC Score 18.0 AI-039-20260611-ai-specialis-824e0fa0

Multi-Label Text Classification with Scikit-LLM

Text classification typically boils down to scenarios where a product review is "positive" or "negative", or a customer inquiry belongs to one category or another.

Read article →
AI Stack Exchange 2026-03-13 09:45 UTC Score 26.0 AI-110-20260313-social-media-d58701e5

Why do Transformers handle long-range dependencies better than LSTMs despite lacking explicit recurrence?

Recurrent architectures such as LSTMs and GRUs were originally designed to address the vanishing gradient problem and capture long-range dependencies in sequential data. However, in recent years Transformer-based architectures have largely replaced RNN-based models in many domains such as natural language processing, time-series modeling, and even reinforcement learning. One commonly cited explanation is that Transformers allow parallel computation and avoid sequential processing, which improves training efficiency. However, this does not fully explain why they often outperform LSTMs in modeling long-range relationships. From a modeling perspective, I am trying to understand the following points: In LSTMs, the cell state is explicitly designed to propagate information across time steps. In Transformers, there is no recurrence or persistent state between tokens. Why does the self-attention mechanism still capture long-range dependencies more effectively? Is the improvement mainly due to the attention mechanism allowing direct connections between distant tokens, or are there additional factors such as representation capacity and optimization dynamics? Are there known theoretical explanations or empirical studies comparing the ability of Transformers and LSTMs to capture long-range dependencies? Are there scenarios (for example streaming data or low-resource environments) where recurrent architectures still outperform Transformers? I would appreciate references to research pape…

Read article →
AI Stack Exchange 2024-06-25 00:58 UTC Score 21.0 AI-110-20240625-social-media-6dfd5a00

How do transformer models handle negation in sentiment analysis

I'm trying to understand how transformer models, such as BERT or GPT, handle negation in sentiment analysis. Specifically, I'm curious about how these models manage to correctly interpret sentences where negation changes the sentiment, such as "The movie is not good." A simple model using word embeddings + global averaging fails to handle negation properly. Intuitively, for example, if "good" has a positive sentiment score and "bad" has a negative sentiment score, a model might misinterpret "not good" by simply averaging the scores of "not" and "good". Example Without Negation Consider the following sentences with sentiment words: "The movie is good." "The movie is awesome." "The movie is terrible." Suppose we have the following word embeddings representing sentiment scores: "good" = [10] "awesome" = [12] "terrible" = [-10] Neutral words (assuming embeddings around 0): "the" = [0] "movie" = [0] "is" = [0] For these sentences, a simple global average of the sentiment scores works well: "The movie is good" = average([0, 0, 0, 10]) = 10 / 4 = 2.5 (positive sentiment) "The movie is awesome" = average([0, 0, 0, 12]) = 12 / 4 = 3 (positive sentiment) "The movie is terrible" = average([0, 0, 0, -10]) = -10 / 4 = -2.5 (negative sentiment) Example With Negation Now, consider the sentence "The movie is not good." In this case, the sentiment should be negative due to the presence of "not." However, averaging the scores naively might not handle this correctly. For example: "The movie is…

Read article →
AI Stack Exchange 2024-05-30 06:02 UTC Score 29.0 AI-110-20240530-social-media-da3d22c6

Probability interpretation of attention mechanism in Seq2Seq

I have ready many explanations of the seq2seq model. In my opinion, however, it is really like a robot that might say something correctly, but doesn't really understand it, just as is true with an LLM generally. In my opinion, the correct way to describe Seq2Seq and similar NLP models should start from a probability view. My probability view is very simple; the output of the encoder is a representation of the probability distribution of the next word. In each step of the Decoder, it just modifies the distribution based on each word it predicted from the distribution and outputs the modified distribution. It then does this repeatedly. Assuming this probability view is correct, how could we explain the attention mechanism used in Seq2seq?

Read article →
Cross Validated 2023-01-16 07:40 UTC Score 15.0 AI-113-20230116-social-media-752a4f12

How to calculate dimension of weight matrix for a vanilla RNN

Sorry if this is kind of a dumb question but I'm taking the NLP course from Andrew Ng on Coursera and I don't understand how to arrive at the correct answer for a question pertaining to calculating the dimensions for the weight matrix of a simple RNN.

Read article →
AI Stack Exchange 2022-11-26 20:53 UTC Score 24.0 AI-110-20221126-social-media-05252ea3

What is the difference between prompt tuning and prefix tuning?

I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really? Prompt Tuning: https://aclanthology.org/2021.emnlp-main.243/ Prefix-Tuning: https://arxiv.org/abs/2101.00190

Read article →
Data Science Stack Exchange 2022-08-11 00:20 UTC Score 29.0 AI-111-20220811-social-media-485f9224

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2 ). The concept seems to be that we want the neural network to focus on or pay more attention to certain features, and has demonstrated some empirical success in NLP and related sequential models. When I look at some code examples such as this one , adding an Attention layer intuitively makes sense and seems to improve performance of the LSTM model. However it looks very much like a regular fully-connected layer. In that link (and with some slight change of notation), the Attention layer outputs $$ c(x) = \tanh(\mathbf{W}x + \mathbf{b} ) $$ $$ \beta(x) = \frac{e^{c(x_j)}}{\sum_{j} e^{c(x_j)}} $$ $$ f_{Attention}(x) = x\beta $$ where $W,b$ are weights/biases, $x$ is layer input, and $f(.)$ is the layer output. In contrast, a regular fully connected layer: $$ f_{Dense} = \sigma(\mathbf{W}x + \mathbf{b}) $$ for some activation function $\sigma(.)$ . My interpretation of the Attention implementation above is that it is pretty much the same thing as a standard fully connected layer, but with a $\tanh$ activation (why?), followed by a $\text{softmax}$ (okay, so that the "attention weights" $\beta$ sum to 1), followed by a linear dot product. How does this architecture allow the model to have "attention"? I do not see how it is fundamentally different or more expressive from just adding a standard fully-connected layer. Am I misunderstanding something here? Edit/My…

Read article →
Stanford AI Lab Blog 2022-05-31 07:00 UTC Score 47.0 USR-0006-20220531-research-aca-a57ebba7

LinkBERT: Improving Language Model Training with Document Link

Language Model Pretraining Language models (LMs), like BERT 1 and the GPT series 2 , achieve remarkable performance on many natural language processing (NLP) tasks. They are now the foundation of today’s NLP systems. 3 These models serve important roles in products and tools that we use every day, such as search engines like Google 4 and personal assistants like Alexa 5 . These LMs are powerful because they can be pretrained via self-supervised learning on massive amounts of text data on the web without the need for labels, after which the pretrained models can be quickly adapted to a wide range of new tasks without much task-specific finetuning. For instance, BERT is pretrained to predict randomly masked words in original text (masked language modeling), e.g. predicting the masked word “dog” from “My __ is fetching the ball”. GPTs are pretrained to predict the next word given a previous sequence of text (causal language modeling), e.g. predicting the next word “ball” from “My dog is fetching the”. In either cases, through pretraining, LMs learn to encode various knowledge from a text corpus that helps to perform downstream applications involving language understanding or generation. In particular, LMs can learn world knowledge (associations between concepts like “dog”, “fetch”, “ball”) from training text where the concepts appear together, and help for knowledge-intensive applications like question answering. 6 Challenges. A challenge with most common LM pretraining strateg…

Read article →
Stanford AI Lab Blog 2022-05-25 07:00 UTC Score 47.0 USR-0006-20220525-research-aca-2eecb290

Stanford AI Lab Papers and Talks at ACL 2022

The 60th Annual Meeting of the Association for Computational Linguistics (ACL) 2022 is taking place May 22nd - May 27th. We’re excited to share all the work from SAIL that’s being presented, and you’ll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that’s happening at Stanford! List of Accepted Papers LinkBERT: Pretraining Language Models with Document Links Authors : Michihiro Yasunaga, Jure Leskovec*, Percy Liang* Contact : myasu@cs.stanford.edu Links: Paper | Website Keywords : language model, pretraining, knowledge, hyperlink, bionlp When classifying grammatical role, BERT doesn’t care about word order… except when it matters Authors : Isabel Papadimitriou, Richard Futrell, Kyle Mahowald Contact : isabelvp@stanford.edu Links: Paper Keywords : large language models, analysis, word order, order invariance, grammatical role, syntax, semantics Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words Authors : Kaitlyn Zhou, Kawin Ethayarajh, Dallas Card, Dan Jurafsky Contact : katezhou@stanford.edu Keywords : cosine similarity, training data frequency, model analysis Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization Authors : Faisal Ladhak, Esin Durmus, He He, Claire Cardie, Kathleen McKeown Contact : esdurmus@stanford.edu Links: Paper Keywords : text summarization, text generation, evaluation, faithfulness Sp…

Read article →
AI Stack Exchange 2022-04-05 17:07 UTC Score 24.0 AI-110-20220405-social-media-ec190fe9

What are examples of simple gradient based NLP models?

I am looking for a simple example of gradient-based methods for NLP. More specifically I am looking for post-hoc local explanations gradient-based methods, that is to say, which explain a single prediction by performing additional operations (after the model has emitted a prediction). Here is an example of a gradient-based method for "Finance"? (I'm not very sure of what domain it is as it is not NLP, nor Computer Vision, nor time series ...): Let us consider a minimal example, where a linear regression is used to estimate the future capital asset $y_{c}$ , based on two investments $x_{1}$ and $x_{2}$ . Let assume the assumptions above are met and the model parameters are estimated as follows: $$\mathbb{E}\left[y_{c} \mid x_{1}, x_{2}\right]=1.05 x_{1}+1.50 x_{2},$$ We can derive immediately a global interpretation of this model. Every dollar invested in fund $x_{1}$ will produce capital of $1.05 \\\$$ , while every dollar invested in $x_{2}$ will produce a capital of $1.50 \\\$$ , independently of the values $x_{1}$ and $x_{2}$ might assume in a concrete scenario. Notice that this explanation is purely based on the learned coefficient $w_{1}=1.05$ and $w_{2}=1.50$ . These, sometimes called partial regression coefficients, are themselves candidate attribution values to explain the influence of the independent variables of the target variable: $$R_{1}(x)=1.05 \quad R_{2}(x)=1.50$$ Notice also that the coefficients are the partial derivatives of the target variable with respec…

Read article →
AI Stack Exchange 2022-03-15 17:05 UTC Score 29.0 AI-110-20220315-social-media-22338a14

Having the negative cases in the same batch vs. shuffling the dataset

I am working on a model for an NLP task. The model encodes the text and has a regression output layer. In this task, from each instance (positive), I create several negative cases using a specific technique and I merge them with their positive corresponding ones in a data split (training/val/test). After that, I shuffle the data split. I was thinking of the following: Isn't better to keep the negative instances with their corresponding positive ones in the same batch instead of shuffling the data? Is there an answer to this question? does it depend on the task?

Read article →
Stanford AI Lab Blog 2022-01-21 08:00 UTC Score 58.0 USR-0006-20220121-research-aca-1e3c1829

Reward Isn't Free: Supervising Robot Learning with Language and Video from the Web

This work was conducted as part of SAIL and CRFM . Deep learning has enabled improvements in the capabilities of robots on a range of problems such as grasping 1 and locomotion 2 in recent years. However, building the quintessential home robot that can perform a range of interactive tasks, from cooking to cleaning, in novel environments has remained elusive. While a number of hardware and software challenges remain, a necessary component is robots that can generalize their prior knowledge to new environments, tasks, and objects in a zero or few shot manner. For example, a home robot tasked with setting the dining table cannot afford lengthy re-training for every new dish, piece of cutlery, or dining room it may need to interact with. A natural way to enable such generalization in our robots is to train them on rich data sources that contain a wide range of different environments, tasks, and objects. Indeed, this recipe of massive, diverse datasets combined with scalable offline learning algorithms (e.g. self-supervised or cheaply supervised learning) has been the backbone of the many recent successes of foundation models 3 in NLP 4 5 6 7 8 9 and vision 10 11 12 . Replicating these impressive generalization and adaptation capabilities in robot learning algorithms would certainly be a step toward robots that can be used in unstructured, real world environments. However, directly extending this recipe to robotics is nontrivial, as we neither have sufficiently large and diverse…

Read article →
AI Stack Exchange 2021-11-23 15:44 UTC Score 9.0 AI-110-20211123-social-media-2131c0fb

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of thousands of different merchants. Some questions that I have: How much labelling do I need to do given the number of merchants I need to extract? How many different instances of the same merchant I need to label to get decent results?

Read article →
Stanford AI Lab Blog 2021-11-05 07:00 UTC Score 52.0 USR-0006-20211105-research-aca-02e23852

Stanford AI Lab Papers at EMNLP/CoNLL 2021

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) will take place next week, colocated with CoNLL 2021. We’re excited to share all the work from SAIL that will be presented, and you’ll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that’s happening at Stanford! List of Accepted Papers Calibrate your listeners! Robust communication-based training for pragmatic speakers Authors : Rose E. Wang, Julia White, Jesse Mu, Noah D. Goodman Contact : rewang@stanford.edu Links: Paper | Video Keywords : language generation, pragmatics, communication-based training, calibration, uncertainty Cross-Domain Data Integration for Named Entity Disambiguation in Biomedical Text Authors : Maya Varma, Laurel Orr, Sen Wu, Megan Leszczynski, Xiao Ling, Christopher Ré Contact : mvarma2@stanford.edu Links: Paper | Video Keywords : named entity disambiguation, biomedical text, rare entities, data integration ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts Authors : Yuta Koreeda, Christopher D. Manning Contact : koreeda@stanford.edu Links: Paper | Website Keywords : natural language inference, contract, law, legal, dataset Venue : The Findings of EMNLP 2021 The Emergence of the Shape Bias Results from Communicative Efficiency Authors : Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche Contact : portelan@stanford.edu Links…

Read article →
Stanford AI Lab Blog 2021-10-13 07:00 UTC Score 52.0 USR-0006-20211013-research-aca-a7eb5ef1

Selective Classification Can Magnify Disparities Across Groups

Selective classification, where models are allowed to “abstain” when they are uncertain about a prediction, is a useful approach for deploying models in settings where errors are costly. For example, in medicine, model errors can have life-or-death ramifications, but abstentions can be easily handled by backing off to a doctor, who then makes a diagnosis. Across a range of applications from vision 1 2 3 and NLP 4 5 , even simple selective classifiers, relying only on model logits, routinely and often dramatically improve accuracy by abstaining. This makes selective classification a compelling tool for ML practitioners 6 7 . However, in our recent ICLR paper, we find that despite reliably improving average accuracy, selective classification can fail to improve and even hurt the accuracy over certain subpopulations of the data . As a motivating example, consider the task of diagnosing pleural effusion, or fluid in the lungs, from chest X-rays. Pleural effusion is often treated with a chest drain, so many pleural effusion cases also have chest drains, while most cases without pleural effusion do not have chest drains 8 . While selective classification improves average accuracy for this task, we find that it does not appreciably improve accuracy on the most clinically relevant subgroup, or subpopulation, of the data: those that have pleural effusion but don’t yet have a chest drain, i.e. those that have pleural effusion but have not yet been treated for it. Practitioners, thus,…

Read article →