Companies Home Search Profile

Statistical Significance Testing for Natural Language Processing (Synthesis Lectures on Human Language Technologies)

Statistical Significance Testing for Natural Language Processing (Synthesis Lectures on Human Language Technologies)

Download pdf

Statistical Significance Testing for Natural Language Processing (Synthesis Lectures on Human Language Technologies)

Statistical Significance Testing for Natural Language Processing (Synthesis Lectures on Human Language Technologies)

Category

Artificial Intelligence

Natural Language Processing

Statistics

Author

Rotem Dror

Lotem Peled-Cohen

Segev Shlomov

Publication

Morgan & Claypool Publishers

0 View

Data-driven experimental analysis has become the main evaluation tool of Natural Language Processing (NLP) algorithms. In fact, in the last decade, it has become rare to see an NLP paper, particularly one that proposes a new algorithm, that does not include extensive experimental analysis, and the number of involved tasks, datasets, domains, and languages is constantly growing. This emphasis on empirical results highlights the role of statistical significance testing in NLP research: If we, as a community, rely on empirical evaluation to validate our hypotheses and reveal the correct language processing mechanisms, we better be sure that our results are not coincidental.

The goal of this book is to discuss the main aspects of statistical significance testing in NLP. Our guiding assumption throughout the book is that the basic question NLP researchers and engineers deal with is whether or not one algorithm can be considered better than another one. This question drives the field forward as it allows the constant progress of developing better technology for language processing challenges. In practice, researchers and engineers would like to draw the right conclusion from a limited set of experiments, and this conclusion should hold for other experiments with datasets they do not have at their disposal or that they cannot perform due to limited time and resources. The book hence discusses the opportunities and challenges in using statistical significance testing in NLP, from the point of view of experimental comparison between two algorithms. We cover topics such as choosing an appropriate significance test for the major NLP tasks, dealing with the unique aspects of significance testing for non-convex deep neural networks, accounting for a large number of comparisons between two NLP algorithms in a statistically valid manner (multiple hypothesis testing), and, finally, the unique challenges yielded by the nature of the data and practices of the field.

'

Download pdf

ISBN-10

1681737973

ISBN-13

978-1681737973

Publisher

Morgan & Claypool Publishers

Price

69.95

File Type

PDF

Page No.

118

Rotem Dror

Lotem Peled-Cohen

Lotem Peled-Cohen

Segev Shlomov

Similar Books

Artificial Intelligence and Machine Learning Methods in COVID-19 and Related Health Diseases (Studies in Computational Intelligence Book 1023)

Artificial Intelligence and Machine Learning Methods in COVID-19 and Related Health Diseases (Studies in Computational Intelligence Book 1023)

Victor Chang

Machine Learning

Applied Deep Learning: Tools, Techniques, and Implementation (Computational Intelligence Methods and Applications)

Applied Deep Learning: Tools, Techniques, and Implementation (Computational Intelligence Methods and Applications)

Paul Fergus

Artificial Intelligence

Intelligence - Theories and Applications

Intelligence - Theories and Applications

Rainer M. Holm-Hadulla

Artificial Intelligence

Regulation of Innovative Technologies: Blockchain, Artificial Intelligence and Quantum Computing

Regulation of Innovative Technologies: Blockchain, Artificial Intelligence and Quantum Computing

Rosario Girasa

Machine Learning and Non-volatile Memories

Machine Learning and Non-volatile Memories

Rino Micheloni

Machine Learning

Artificial Intelligence in Music, Sound, Art and Design: 11th International Conference, EvoMUSART 2022, Held as Part of EvoStar 2022, Madrid, Spain, April ... Notes in Computer Science Book 13221)

Artificial Intelligence in Music, Sound, Art and Design: 11th International Conference, EvoMUSART 2022, Held as Part of EvoStar 2022, Madrid, Spain, April ... Notes in Computer Science Book 13221)

Tiago Martins

Advances in Intelligent Data Analysis XX: 20th International Symposium on Intelligent Data Analysis, IDA 2022, Rennes, France, April 20–22, 2022, Proceedings (Lecture Notes in Computer Science, 13205)

Advances in Intelligent Data Analysis XX: 20th International Symposium on Intelligent Data Analysis, IDA 2022, Rennes, France, April 20–22, 2022, Proceedings (Lecture Notes in Computer Science, 13205)

Tassadit Bouadi

Artificial Intelligence

Artificial Intelligence/Machine Learning in Nuclear Medicine and Hybrid Imaging

Artificial Intelligence/Machine Learning in Nuclear Medicine and Hybrid Imaging

Patrick Veit-Haibach

Machine Learning

Security and Artificial Intelligence: A Crossdisciplinary Approach (Lecture Notes in Computer Science, 13049)

Security and Artificial Intelligence: A Crossdisciplinary Approach (Lecture Notes in Computer Science, 13049)

Lejla Batina

Artificial Intelligence

RoboCup 2021: Robot World Cup XXIV (Lecture Notes in Computer Science)

RoboCup 2021: Robot World Cup XXIV (Lecture Notes in Computer Science)

Oliver Obst

Artificial Intelligence

Language, Logic, and Computation: 13th International Tbilisi Symposium, TbiLLC 2019, Batumi, Georgia, September 16–20, 2019, Revised Selected Papers (Lecture Notes in Computer Science Book 13206)

Language, Logic, and Computation: 13th International Tbilisi Symposium, TbiLLC 2019, Batumi, Georgia, September 16–20, 2019, Revised Selected Papers (Lecture Notes in Computer Science Book 13206)

Aybüke Özgün

Artificial Intelligence

Advanced Artificial Intelligence and Robo-Justice

Advanced Artificial Intelligence and Robo-Justice

Georgios I. Zekos

Artificial Intelligence

Enjoying Natural Computing: Essays Dedicated to Mario de Jesús Pérez-Jiménez on the Occasion of His 70th Birthday (Lecture Notes in Computer Science, 11270)

Enjoying Natural Computing: Essays Dedicated to Mario de Jesús Pérez-Jiménez on the Occasion of His 70th Birthday (Lecture Notes in Computer Science, 11270)

Carmen Graciani

Natural Language Processing

Python Natural Language Processing Cookbook: Over 50 recipes to understand, analyze, and generate text for implementing language processing tasks

Python Natural Language Processing Cookbook: Over 50 recipes to understand, analyze, and generate text for implementing language processing tasks

Zhenya Antic

Natural Language Processing

Natural Language Processing with TensorFlow: The definitive NLP book to implement the most sought-after machine learning models and tasks, 2nd Edition

Natural Language Processing with TensorFlow: The definitive NLP book to implement the most sought-after machine learning models and tasks, 2nd Edition

Thushan Ganegedara

Machine Learning

Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, and GPT-3, 2nd Edition

Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, and GPT-3, 2nd Edition

Denis Rothman

Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python

Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python

Yuxi (Hayden) Liu

Machine Learning

The Kaggle Book: Data analysis and machine learning for competitive data science

The Kaggle Book: Data analysis and machine learning for competitive data science

Luca Massaron

Machine Learning

Deep Natural Language Processing and Ai Applications for Industry 5.0 (Advances in Computational Intelligence and Robotics)

Deep Natural Language Processing and Ai Applications for Industry 5.0 (Advances in Computational Intelligence and Robotics)

Poonam Tanwar

Artificial Intelligence

Natural Language Processing and Computational Linguistics: A practical guide to text analysis with Python, Gensim, spaCy, and Keras

Natural Language Processing and Computational Linguistics: A practical guide to text analysis with Python, Gensim, spaCy, and Keras

Bhargav Srinivasa-Desikan

Hands-On Transfer Learning with Python: Implement advanced deep learning and neural network models using TensorFlow and Keras

Hands-On Transfer Learning with Python: Implement advanced deep learning and neural network models using TensorFlow and Keras

Dipanjan Sarkar

Statistics for Data Scientists: An Introduction to Probability, Statistics, and Data Analysis (Undergraduate Topics in Computer Science)

Statistics for Data Scientists: An Introduction to Probability, Statistics, and Data Analysis (Undergraduate Topics in Computer Science)

Maurits Kaptein

Software Testing

Training Systems Using Python Statistical Modeling: Explore popular techniques for modeling your data in Python

Training Systems Using Python Statistical Modeling: Explore popular techniques for modeling your data in Python

Curtis Miller

R Statistics Cookbook: Over 100 recipes for performing complex statistical operations with R 3.5

R Statistics Cookbook: Over 100 recipes for performing complex statistical operations with R 3.5

Francisco Juretig

The COVID-19 Pandemic: A Multidisciplinary Review of Diagnosis, Prevention, and Treatment

The COVID-19 Pandemic: A Multidisciplinary Review of Diagnosis, Prevention, and Treatment

Sagheer Ahmed

Program Evaluation: Forms and approaches

Program Evaluation: Forms and approaches

John M. Owen

BeginneR Introductory Statistics Using R

BeginneR Introductory Statistics Using R

Darrin Thomas

Straightforward Statistics with Excel

Straightforward Statistics with Excel

Chieh-Chen Bowen

Statistics and Data Visualization Using R: The Art and Practice of Data Analysis

Statistics and Data Visualization Using R: The Art and Practice of Data Analysis

David S. Brown

Think Bayes: Bayesian Statistics in Python (O'reilly)

Think Bayes: Bayesian Statistics in Python (O'reilly)

Allen Downey

Practical Time Series Analysis: Prediction with Statistics and Machine Learning

Practical Time Series Analysis: Prediction with Statistics and Machine Learning

Aileen Nielsen

Machine Learning

An Introductory Guide to R: Easing the Learning Curve

An Introductory Guide to R: Easing the Learning Curve

Eric L. Einspruch

Other Authors' Books

Other Publishing Books

Quantum Computing for Computer Architects

Quantum Computing for Computer Architects

Tzvetan S. Metodi

Computer Architecture

Transactional Memory (Synthesis Lectures on Computer Architecture, 2)

Transactional Memory (Synthesis Lectures on Computer Architecture, 2)

James Larus

Computer Architecture

Link Reversal Algorithms (Synthesis Lectures on Distributed Computing Theory, 8)

Link Reversal Algorithms (Synthesis Lectures on Distributed Computing Theory, 8)

Jennifer E. Walter

Embedded Systems Design with the Atmel AVR Microcontroller (Synthesis Lectures on Digital Circuits and Systems, 24)

Embedded Systems Design with the Atmel AVR Microcontroller (Synthesis Lectures on Digital Circuits and Systems, 24)

Steven Barrett

PIC Microcontroller

Advanced Metasearch Engine Technology (Synthessis Lectures on Data Management)

Advanced Metasearch Engine Technology (Synthessis Lectures on Data Management)

Weiyi Meng

Software Engineering

A Guide to Visual Multi-Level Interface Design From Synthesis of Empirical Study (Synthesis Lectures on Visualization)

A Guide to Visual Multi-Level Interface Design From Synthesis of Empirical Study (Synthesis Lectures on Visualization)

Tamara Munzner

Data Visualization

Analysis of the MPEG-1 Layer III (MP3) Algorithm Using MATLAB (Synthesis Lectures on Algorithms and Software in Engineering)

Analysis of the MPEG-1 Layer III (MP3) Algorithm Using MATLAB (Synthesis Lectures on Algorithms and Software in Engineering)

Jayaraman J. Thiagarajan

High-level Structures in Quantum Computing

High-level Structures in Quantum Computing

Jaroslaw Adam Miszczak

Visual Information Retrieval using Java and LIRE (Synthesis Lectures on Information Concepts, Retrieval, and Services, 25)

Visual Information Retrieval using Java and LIRE (Synthesis Lectures on Information Concepts, Retrieval, and Services, 25)

Mathias Lux

Signal Processing for Solar Array Monitoring, Fault Detection, and Optimization (Synthesis Lectures on Power Electronics, 4)

Signal Processing for Solar Array Monitoring, Fault Detection, and Optimization (Synthesis Lectures on Power Electronics, 4)

Andreas Spanias

Electrical Engineering

A Primer on Hardware Prefetching (Synthesis Lectures on Computer Architecture)

A Primer on Hardware Prefetching (Synthesis Lectures on Computer Architecture)

Babak Falsafi

Computer Architecture

Distributed Graph Coloring: Fundamentals and Recent Developments (Synthesis Lectures on Digital Computing Theory)

Distributed Graph Coloring: Fundamentals and Recent Developments (Synthesis Lectures on Digital Computing Theory)

Leonid Barenboim

CFA Chartered Financial Analyst

User Reviews

Log in first to post a comment

Rating

0

0

0

0

0

average 0

Total votes0